[openib-general] IPoIB broadcast MC group membership

[Fabian Tillier]

On 2/21/06, Greg Lindahl <lindahl at pathscale.com> wrote:
> On Tue, Feb 21, 2006 at 05:21:33PM -0800, Roland Dreier wrote:
>
> > No, but an IB multicast group has a speed associated to it.  This is
> > to allow, say, a 4X port sending multicast packets to use the right
> > static rate to avoid overrunning a 1X port that is also a member of
> > the group.
>
> Thanks for the clarification, Roland, I see that the previous
> discussion had successfully confused me. So is it the case that the
> create and the joins to a multicast group have to specify the correct
> speed?

The node joining or creating the multicast group doesn't need to
specify the rate - the SA can figure out the rate to use based on the
requestor (for creation), or validate that the requestor supports the
existing group's rate (for joining).

The problem is that OpenSM does not enforce this, allowing a rate
mismatch between members of a multicast group.

> And the problem then would be that a IB host at boot doesn't
> know the right speed?

A host can always figure out its rate by looking at the local port
attributes - the link width active and link speed active would give it
all the information necessary.

> Am I getting colder or warmer?

Colder.

The problem, assuming OpenSM will be fixed to check rates, is for
IPoIB to detect that the join failed due to a rate mismatch so that it
can log an intelligent message to the system log.  Currently, for
Linux it will retry forever, albeit at a decaying interval.  The
algorithm in Windows is a little different, as the driver has been
designed to function properly with SMs that don't pre-create the
broadcast group, and in this case, the potential races in creation and
joining the group coupled with the lack of detailed error status from
the SA upon failure make it impossible to differentiate between a true
error or just a timing race.

Hope that helps!

- Fab