[ofa-general] Re: [RFC] OpenSM and IPv6 Scalability Proposal

[Hal Rosenstock]

Hi Sasha,

On Mon, 2008-06-16 at 09:39 +0300, Sasha Khapyorsky wrote:
> Hi Hal,
> 
> On 06:03 Sat 07 Jun     , Hal Rosenstock wrote:
> > 
> > 1. Change the current underlying multicast tree from being MLID based 
> > to MGID based. This involves using fleximap rather than qmap. The
> > downside of this is that MLID lookups will be slower as now they
> > are not as "direct" as the MLID will no longer be the key in the map.
> > Rather than searching by MLID key, the tree will need to be scanned
> > entry by entry for MLID matches.
> 
> Actually it is not necessary to slow down there. Instead of mgrp_mlid_tbl
> and in addition to MGID map we can just keep array of pointers to
> existing multicast groups indexed by MLID offset from 0xc000. It will be
> always limited in size and should be even faster than original "quick
> map".

Is a 16K array of pointers preferable to some map ?

> > It's unclear how much this will slow down
> > MLID searches but it is thought that none of these searches are
> > time critical (and shouldn't cause any existing timeouts to "pop").
> 
> Some searches are on a fast paths. I think it is important to not slow
> down there.

The comment was at a fine not coarse level. There was some minor
additions thought to be in the code path.

> > 2. Add in support for overloading MLIDs. On the configuration side, a
> > number of additional options would be added to consolidate_ipv6_snm_req.
> > These include the number of MLIDs to compress down to (default 16),
> > a multicast group (MGID) base address and (full MGID) mask. this would
> > default to 0xff1Z601bXXXX0000 : 0x00000001ffYYYYYY where Z is the scope,
> > XXXX is the P_Key, and YYYYYY is the last 24 bits of the port guid (
> > the YYYYYY bits would be masked out by default). This is what the
> > current workaround uses for collapsing the multicast groups.
> > 
> > The criteria for overloading MLIDs includes any group parameters that 
> > need to be in common (e.g. rate. MTU, perhaps PKey (see below), etc.).
> > 
> > Aside from changing the underlying implementation of MLID searches,
> > multicast group deletion wll need another check when there are no ports
> > left in a group. If that group is on a compressed MLID (this part of the
> > check is an optimization), then the multicast group tree needs to be checked
> > to ensure there are no other groups sharing that MLID.
> 
> If we will have MLID indexed array with pointers to multicast groups
> kept in a list it should be easy to do. Something like:
> 
>  i: (MLID = 0xc000 + i) -> MGRP MGID1 -> MGRP MGID2 -> ...

That's how I envisioned the mgrp_mlid_tbl changing. It would point at an
MLID and a MGID "chain" rather than an individual MGID.

> > IBA 1.2.1 v1 p.151 4.1.3 Local Identifiers item 10) states:
> > "When a multicast LID is overloaded, the multicast groups
> > sharing the same MLID must have the same P_Key. This simplification
> > is required to allow switches and routers that implement optional
> > P_Key enforcement for multicast operations."
> > This is part of the C4-5 compliance.
> >  
> > OPEN ISSUE
> > As PKey is part of the MGID, does this need to be addressed (and if
> > so) how ?
> 
> Since it is required by spec

It's unclear the spec is "right" in this.

> and we want to care about another common
> multicast parameters (mtu, rate, etc.) pkey can be handled in similar
> manner.

Yes, it's potentially just another parameter for the sharing
eligibility.

> In worst case we can disable/enable it with command line option
> or so.

I was hoping that isn't needed.

-- Hal

> > If the approach above seems reasonable, I will work on such a set of patches.
> 
> Great!
> 
> Sasha
> _______________________________________________
> general mailing list
> general at lists.openfabrics.org
> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
> 
> To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general