[openib-general] IPv6 and IPoIB scalability issue

[Jason Gunthorpe]

On Thu, Nov 30, 2006 at 05:29:16PM -0500, Hal Rosenstock wrote:

> > IPV6 defines that each node will have a Solicited Node Multicast
> > address.  This address is unique per node and is constructed from the
> > IPV6 unicast address of the node.  (see RFC 2373 for more details).
> > 
> > IP over IB defines that IPV6 multicast addresses map to IB multicast
> > GIDs in a one to one manner.
> > 
> > IB defines a multicast address space limit of 4095 LIDs.
> 
> actually it is 16K-1

For IPv6 only the lower 24 bits of each assigned IPv6 address are
used to construct a solicited node multicast in the range 
FF02::1:FF00:0/104. The Solicited Node Multicast address it not
expected to be uniquely subscribed.

> MGIDs are different from MLIDs. Multiple MGIDs can be mapped onto a
> single MLID if the characteristics are the same. Is that the case for
> the IPv6 groups ?

The solicited node multicast feature is intended for scalability by
having the switching core prune ND queries. It is OK if the multicast
goes to more nodes than subscribe to it (this happens on cheap
ethernet switch gear without multicast support anyhow).

I think the thing to do here is for the SM to have an option to
compress a particular MGID range (using a hash of some kind). Ie
configure so that all of IPv6 FF02::1:FF00:0/104 will use at most 16
MLIDs.

That way the site can select that some MGID's get mapped directly to
MLIDs and others get shared to save LID space.

Then if you still run out it can randomly combine MGIDs into MLIDs.

Jason