[openib-general] IPoIB and lid change

[Hal Rosenstock]

Hi Eitan,

On Mon, 2006-02-13 at 10:23, Eitan Zahavi wrote:
> Hi,
> 
> I had a long discussion today with Michael, Yael and Tziporet regarding
> this issue.
> We have got to the following conclusions/proposal:
> 
> 1. As we use only GID[0] (that can not change) and a QP that is reserved
> for the interface even if it is down we actually "never" change IPoIB
> MAC (unless you download the module).

remove the module

> So unsolicited ARP reply is not a must. The MAC (which is GID,QP in IPoIB) 
> is kept even if the LID changes.
> 
> 2. When a new SM is brought up it optionally can send
> ClientReRegistration which should cause the entire UD AV cache (and the
> new SA Path Cache) to be flushed.

Would this be an SM requirement ? The entire UD AV cache includes both
unicast and multicast, right ?

> 3. When subnets are merged there is currently no way to tell if there
> were any LID changes. The SM does not need to do any "Set(PortInfo)" on
> a node if no change is required so the remote node (that did not change
> LID) will not know about any such change. There are several solutions
> for this problem. The one we believe should be promoted is the concept
> of "UnPath" Notice:

UnPath will be needed for QoS too. It has been discussed in that context
as well.

> a. In IB any class manager (SM) can generate Traps carrying a Notice
> attribute. We  
>     propose a new Notice of trap number = YYY which will mean UnPath
> message.
>     The notice will carry several fields of the path record as its
> DataDetails field and also 
>     a component mask. (the DataDetails field is 432 bits long).
>     So the following fields are to be included in the DataDetails:
>     SLID, DLID, P_KEY, TCLASS, SL, compMask.
>     The component mask will allow to wildcard the fields values such
> that:
>     * an UnPath Notice including a component mask = 0 will UnPath all
> paths.
>     * an UnPath Notice including a component mask = 1 will UnPath all
> paths from the   
>       LID = the SLID included in the Notice.
> 
> b. The IPoIB will have to register with the SA to receive these notices.
> 
> c. The SM needs to be smart with the way the notices are being built:
>     The SM should be able to coalesce multiple change events to one
> notice by using the 
>     wildcard (zero compmask) as appropriate for the case inspected by
> the SM.
> 
> With this kind of coalescing the SM does not need to generate O(N^2)
> Reports but only O(N) Reports. As you might guess any discovery and
> setting of the fabric require O(N^2/32) just for LFTs setting

Or perhaps these are distributed on some MC group reserved for this
(unpath) ?

> s so our
> problem of distributing these N reports is not so big.
> 
>  My plan is to bring this UnPath Notice to the IBTA MgtWG for
> discussion. 

Sounds good; (That's where this will need to be standardized.)

-- Hal

> Thanks
> 
> Eitan
>