[ofa-general] [PATCH 2/2] IB/IPoIB: Separate IB events to groups and handle each according to level of severity

[Hal Rosenstock]

On Mon, 2008-05-19 at 18:47 +0300, Olga Shern (Voltaire) wrote:
> 
> 
> On 5/19/08, Hal Rosenstock <hrosenstock at xsigo.com> wrote: 
>         On Sun, 2008-05-18 at 18:10 +0300, Moni Shoua wrote:
>         > Hal Rosenstock wrote:
>         > > On Sun, 2008-05-18 at 15:36 +0300, Moni Shoua wrote:
>         > >> The purpose of this patch is to make the events that are
>         related to SM change
>         > >> (namely CLIENT_REREGISTER event and SM_CHANGE event) less
>         disruptive.
>         > >> When SM related events are handled, it is not necessary
>         to flush unicast
>         > >> info from device but only multicast info.
>         > >
>         > > How is unicast invalidation handled on these changes ? On
>         a local LID
>         > > change event, how does an end port know/determine what
>         else (e.g. other
>         > > LIDs, paths) the SM might have changed (that specifically
>         might affect
>         > > IPoIB since this is limited to IPoIB) ?
>         > I'm not sure I understand the question but local LID change
>         would be handled as before
>         > with a LID_CHANGE event. For this type of event, there is
>         not change in what IPoIB does to cope.
>         
>         It's SM change which I'm not sure about. I'm unaware of an IBA
>         spec
>         guarantee on preservation of paths on SM failover. Can you
>         point me at
>         this ?
>         
>         Also, as many routing protocols are dependent on where they
>         are run in
>         the subnet (location of SM node in the topology), I don't
>         think all path
>         parameters can be maintained when in a heterogeneous subnet
>         and hence
>         would need refreshing (or flushing to cause this) on an SM
>         change event.
>         
>         So while it may work in a homogeneous subnet, I don't think
>         this is the
>         general case.
>  
> You are rigth there is no IBA spec request to preserve LIDs but all
> SMs that we are familiar with, 
> are doing so.

It's more than LID preservation though; it's also routing preservation
(and rate, etc.) if a path is rerouted in a heterogenous subnet on SM
failover.

In terms of SM LID preservation though, in the case of OpenSM there are
two additional scenarios where LID preservation on SM failover is not a
valid assumption:
1. If the guid2lid files are not sync'd with OpenSM instances.
2. If reassign LIDs is used.

Also, since it's not a spec requirement, I don't see how this can be
relied upon. Maybe some option (mod param) for this being the case in
some configuration with the default being that LID preservation is not
assumed ?

Also, what happens when that assumption is not valid ? I'm referring to
the case where it's treated as a less disruptive event but it really
needed to be treated as a more disruptive one ?

> You are refering to the case where there is remote LID change but not
> local LID change,

Yes, in addition to the changing the SM change event handling issues
above.

> but also without this patch this case is not taken care of.

True.

>   We should think about solution for this case in the future.

Indeed.

-- Hal

>         > > Also, wouldn't there be similar issues with other ULPs ?
>         > There might be but the purpose of this one is to make things
>         better  for IPoIB
>         
>         Understood; just trying to widen the scope. IMO other ULPs
>         should at
>         least be inspected for the same issues. The multicast issue is
>         IPoIB
>         specific but local LID, client reregister (maybe only events
>         for other
>         ULPs as multicast and service records may not apply (perhaps
>         except DAPL
>         but this may be old implementation)) and SM changes apply to
>         all.
>         
>         -- Hal
>         
>         > > -- Hal
>         > >
>         > _______________________________________________
>         > general mailing list
>         > general at lists.openfabrics.org
>         > http://lists.openfabrics.org/cgi-
>         bin/mailman/listinfo/general
>         >
>         > To unsubscribe, please visit
>         http://openib.org/mailman/listinfo/openib-general
>         
>         _______________________________________________
>         general mailing list
>         general at lists.openfabrics.org
>         http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
>         
>         To unsubscribe, please visit
>         http://openib.org/mailman/listinfo/openib-general
>