[openib-general] IPv6 and IPoIB scalability issue

[Hal Rosenstock]

On Fri, 2006-12-01 at 20:20, Jason Gunthorpe wrote:
> On Fri, Dec 01, 2006 at 06:47:38PM -0500, Hal Rosenstock wrote:
> 
> > > It isn't in the IB spec, but what would really help here is to be able
> > > to join a multicast prefix (more than 1 group with a single entry).
> > > 
> > > Todd's option 1 optimization is then easially realized by having all
> > > IPv6 nodes join FF02::1:FF00:0/104 (all 2**24 multicast entries)
> > 
> > These are IPmc groups not IB mc groups though. I suppose you are asking
> > for the equivalent function in IB. When that subscribe is done,
> > would it
> 
> Right, it looks like the MGID would be close to
> FF1E:601B:xxxx::1FF00:0/104 for IPv6 SN multicast (RFC4391).
> 
> My thinking was to add a new 8 bit field to MCMemberRecord called
> prefixLen. Broadly (without considering how to manage compatability)
> joins like we have today would set prefixLen to 128. To do this
> suggestion we'd set prefixLen to 104.
> 
> prefixLen of 104 means 2**24 MGID addresses are matched by the join
> and the node is subscribed to them all. The existing MGID field is
> used to encode the prefix bits, only the first 104 bits are used.
> 
> Off hand I don't see a way to indicate the length using the existing
> record fields.

Another 8 bit field could do this if this were needed.

> If we call a MCMemberRecord with a prefixLen != 128 a prefix join..

And hence the backward compatibility issue. One way to handle this would
be an exception (if component mask does not specify PrefixLength rather
than being wildcarded, it assumes PrefixLength of 128. There may be
others.

> MLID mapping is a little tricky in this scheme since once a prefix
> join is registered you have to start unioning membership lists with
> other joins to get the right spans. (ie joins to FF1E::1000/120 and
> FF1E::1001/128 may have different MLIDs but they would both reach a
> unioned membership)
> 
> That would mean that a send-only prefix join MLID would effectively be
> a broadcast MLID so if you use it with option 2 you reduce the SM
> query rate and subscription load, but you are just broadcasting ND
> packets. I guess the sensible use would be with option 1 where both
> the send-only and and full-membership join are a /104 prefix join.
> [Basically, it ends up using broadcasting like Todd sugested, but in
>  a way where the SM can properly integrate IPoIB stacks that
>  don't use prefix joins.]
> 
> I don't know if this is worth persuing.. Certainly if the main issue
> is just MLID usage then option 2 is much simpler. Something like this
> might be part of improving IPv6 ND scalability but that is a different
> problem entirely (and does anyone care?)..

Is that only an IPoIB issue though or is it more generic and apply to
other networks ?

-- Hal

> Jason