[openib-general] Multicast Group Routing Question

[Hal Rosenstock]

On Thu, 2006-12-07 at 07:08, Sean Hubbell wrote:
> Hal Rosenstock wrote:
> > On Wed, 2006-12-06 at 16:06, Sean Hubbell wrote:
> >   
> >> Hal Rosenstock wrote:
> >>     
> >>> On Wed, 2006-12-06 at 13:48, Sean Hubbell wrote:
> >>>   
> >>>       
> >>>> Hal Rosenstock wrote:
> >>>>     
> >>>>         
> >>>>> Hi Sean,
> >>>>>
> >>>>> On Wed, 2006-12-06 at 12:52, Sean Hubbell wrote:
> >>>>>   
> >>>>>       
> >>>>>           
> >>>>>> Hello,
> >>>>>>
> >>>>>>   I was testing our code and noticed that when I send data using 
> >>>>>> multicast over our ib0 interface, all of the infiniband switches route 
> >>>>>> the data to each switch and each node instead of a node that has an 
> >>>>>> application listening to the interface like Ethernet. Is this by design?
> >>>>>>     
> >>>>>>         
> >>>>>>             
> >>>>> It depends on what multicast group is being used and which end nodes
> >>>>> have registered for that group as to where the data is routed.
> >>>>>
> >>>>> -- Hal
> >>>>>   
> >>>>>       
> >>>>>           
> >>>> Hey Hal,
> >>>>
> >>>>   The multicast group I am sending data to is 224.10.10.x (not 
> >>>> 224.0.0.x) and I have no clients / nodes listening but the data is still 
> >>>> being sent.
> >>>>     
> >>>>         
> >>> Yes, if there is only a sender, the data should not be routed anywhere.
> >>>
> >>>   
> >>>       
> >>>>  I am using wwtop from warewulf to view the network load for 
> >>>> each node.
> >>>>     
> >>>>         
> >>> I'm not familiar with those tools.
> >>>
> >>>   
> >>>       
> >>>>  Does this make sense?
> >>>>     
> >>>>         
> >>> Nope. To state the obvious, something is not as it seems...
> >>>
> >>> Can you state which SM you are using ?
> >>>
> >>> Also, can you do the following:
> >>> saquery -g
> >>> saquery -m
> >>> and send me the output.
> >>>
> >>> I may have some more experiments once I get that level of info.
> >>>
> >>> -- Hal
> >>>   
> >>>       
> >> We have a Voltaire HW subnet manager. I do not have the saquery command. 
> >> I'll have to find this and install it.
> >>     
> >
> > What is running on your end nodes ? Is it OpenIB/OFED or something else
> > ? If it is OpenIB/OFED, saquery should be there. I think OFED 1.2
> > supports the options I mentioned.
> >
> >   
> >>  Would the web interface help?
> >>     
> >
> > Not sure whether there is anything there for this.
> >
> > -- Hal
> >
> >   
> >> Sean
> >>     
> >
> >
> >   
> Hal,
> 
>   Here are the results:
> 
> The result of saquery -g on our head node:
> 
> [root at neptune ~]# saquery -g
> 
> MCMemberRecord group dump:
>                                 
> MGID....................0xff12401bffff0000 : 0x00000000ffffffff
>                                 Mlid....................0xC000
>                                 Mtu.....................0x4
>                                 pkey....................0xFFFF
>                                 Rate....................0x3
> 
> MCMemberRecord group dump:
>                                 
> MGID....................0xff12401bffff0000 : 0x0000000000000001
>                                 Mlid....................0xC001
>                                 Mtu.....................0x4
>                                 pkey....................0xFFFF
>                                 Rate....................0x3

I don't see the mgrp for 224.10.10.x here.

> The result of saquery -m on our root node:
> 
> Query SA failed: IB_TIMEOUT

This failure can be valid and is SM dependent.

-- Hal

> Running package openib-diags-1.1.0-0
> 
> Sean