[openib-general] IPoIB broadcast MC group membership

[Fabian Tillier]

On 22 Feb 2006 17:01:43 -0500, Hal Rosenstock <halr at voltaire.com> wrote:
> On Wed, 2006-02-22 at 12:53, Fabian Tillier wrote:
> > On 2/22/06, Greg Lindahl <lindahl at pathscale.com> wrote:
> > >
> > > On Tue, Feb 21, 2006 at 11:40:53PM -0800, Fabian Tillier wrote:
> > >
> > > > You'd have to make the group 1X.  Note that the group being 1X doesn't
> > > > limit unicast traffic to 1X rates, since the rate for unicast traffic
> > > > would be set based on the rate reported in the path records for the
> > > > various endpoints.
> > > >
> > > > So 4X SDR and 4X DDR nodes would have to set their inter-packet delay
> > > > for the broadcast group to end up with a 1X packet injection rate.
> > >
> > > So, basically, MVAPICH doesn't have code that does either the group
> > > creation properly when there is a mixture of HCA bandwidths, or limit
> > > the packet injection rate. And IPoIB could violate this rule depending
> > > on how user programs use it, e.g. if I did a lot of broadcasting, I
> > > could easily exceed 1X's bandwidth.
> > >
> > > So this is more than just a "fix OpenSM" issue. It's more of a "fix
> > > the spec" issue, if I'm understanding it correctly.
> >
> > No, the spec is fine.  This is a "fix the SW" issue.  If OpenSM
> > rejected join requests of nodes for which the MC group is unrealizable
> > (that is, some setting of the requestor conflict with the existing
> > group, such as the rate), such nodes would not be able to join the
> > broadcast group and thus not have IPoIB connectivity.
>
> I thought there were 2 issues here:
> 1. OpenSM not checking the realizability of the join request
> and
> 2. The spec issue with discerning the reason for SA refusal of the joing
> request

That's correct, but neither of the above indicates a fatal flaw in
multicast handling in the IB specification.  Though I guess I
overstated things when I said "the spec is fine".  The spec could be
better, but it's not broken.  How's that? ;)

- Fab