[openib-general] IPv6 and IPoIB scalability issue

[Hal Rosenstock]

On Thu, 2006-11-30 at 18:01, Jason Gunthorpe wrote:
> On Thu, Nov 30, 2006 at 05:29:16PM -0500, Hal Rosenstock wrote:
> 
> > > IPV6 defines that each node will have a Solicited Node Multicast
> > > address.  This address is unique per node and is constructed from the
> > > IPV6 unicast address of the node.  (see RFC 2373 for more details).
> > > 
> > > IP over IB defines that IPV6 multicast addresses map to IB multicast
> > > GIDs in a one to one manner.
> > > 
> > > IB defines a multicast address space limit of 4095 LIDs.
> > 
> > actually it is 16K-1
> 
> For IPv6 only the lower 24 bits of each assigned IPv6 address are
> used to construct a solicited node multicast in the range 
> FF02::1:FF00:0/104. The Solicited Node Multicast address it not
> expected to be uniquely subscribed.

Any idea on how many would subscribe ? What does this depend on ?

> > MGIDs are different from MLIDs. Multiple MGIDs can be mapped onto a
> > single MLID if the characteristics are the same. Is that the case for
> > the IPv6 groups ?
> 
> The solicited node multicast feature is intended for scalability by
> having the switching core prune ND queries. It is OK if the multicast
> goes to more nodes than subscribe to it (this happens on cheap
> ethernet switch gear without multicast support anyhow).

And a similar thing is accomodated within IB. With limited MFT space,
the collapse of multiple (similar) MGRPs (MGIDs) on a single MLID is
seems important (and reduces some of the scalability issues Todd
mentioned in terms of IPv6).

> I think the thing to do here is for the SM to have an option to
> compress a particular MGID range (using a hash of some kind). Ie
> configure so that all of IPv6 FF02::1:FF00:0/104 will use at most 16
> MLIDs.

Yes, that is one strategy which seems reasonable to me.

> That way the site can select that some MGID's get mapped directly to
> MLIDs and others get shared to save LID space.
> 
> Then if you still run out it can randomly combine MGIDs into MLIDs.

Yes, that's another wrinkle.

-- Hal

> Jason