[openib-general] IPv6 and IPoIB scalability issue

[Jason Gunthorpe]

On Fri, Dec 01, 2006 at 06:47:38PM -0500, Hal Rosenstock wrote:

> > It isn't in the IB spec, but what would really help here is to be able
> > to join a multicast prefix (more than 1 group with a single entry).
> > 
> > Todd's option 1 optimization is then easially realized by having all
> > IPv6 nodes join FF02::1:FF00:0/104 (all 2**24 multicast entries)
> 
> These are IPmc groups not IB mc groups though. I suppose you are asking
> for the equivalent function in IB. When that subscribe is done,
> would it

Right, it looks like the MGID would be close to
FF1E:601B:xxxx::1FF00:0/104 for IPv6 SN multicast (RFC4391).

My thinking was to add a new 8 bit field to MCMemberRecord called
prefixLen. Broadly (without considering how to manage compatability)
joins like we have today would set prefixLen to 128. To do this
suggestion we'd set prefixLen to 104.

prefixLen of 104 means 2**24 MGID addresses are matched by the join
and the node is subscribed to them all. The existing MGID field is
used to encode the prefix bits, only the first 104 bits are used.

Off hand I don't see a way to indicate the length using the existing
record fields.

If we call a MCMemberRecord with a prefixLen != 128 a prefix join..

MLID mapping is a little tricky in this scheme since once a prefix
join is registered you have to start unioning membership lists with
other joins to get the right spans. (ie joins to FF1E::1000/120 and
FF1E::1001/128 may have different MLIDs but they would both reach a
unioned membership)

That would mean that a send-only prefix join MLID would effectively be
a broadcast MLID so if you use it with option 2 you reduce the SM
query rate and subscription load, but you are just broadcasting ND
packets. I guess the sensible use would be with option 1 where both
the send-only and and full-membership join are a /104 prefix join.
[Basically, it ends up using broadcasting like Todd sugested, but in
 a way where the SM can properly integrate IPoIB stacks that
 don't use prefix joins.]

I don't know if this is worth persuing.. Certainly if the main issue
is just MLID usage then option 2 is much simpler. Something like this
might be part of improving IPv6 ND scalability but that is a different
problem entirely (and does anyone care?)..

Jason