[openib-general] Problem is routing CM REQ

[Michael Krause]

At 02:47 PM 2/12/2007, Sean Hefty wrote:
> > 1) What does the TClass and FlowLabel returned from SGID=local
> >    DGID=remote mean?
> >    Do you use it in the Node1 -> Node2 direction or the Node2 -> Node1 
> direction
> >    or both?
>
>Maybe it would help if we can agree on a set of expectations.  These are 
>what I
>am thinking:
>
>1. An SA should be able to respond to a valid PR query if at least one of the
>GIDs in the path record is local.
>
>2. The LIDs in a PR are relative to the SA's subnet that returned the record.
>
>3. An IB router should not failover transparently to QPs sending traffic 
>through
>that router.

There is no reason for such a restriction.  APM can work with routers and 
the IB protocol will recover from any out of order packet processing just fine.


>4. A PR from the local SA with reversible=1 indicates that data sent from the
>remote GID to the local GID using the PR TC and FL will route locally 
>using the
>specified LID pair.  This holds whether the PR SGID is local or remote.
>
>5. A PR from a remote SA with reversible=1 indicates that data sent from the
>local GID to the remote GID using the PR TC and FL will route remotely 
>using the
>specified LID pair.  This holds whether the PR SGID is local or remote.
>
>6. A PR with reversible=0 is relative to SA's subnet.  The SGID->DGID data 
>flow
>over the PR TC and FL indicates the SLID->DLID mapping for that subnet.
>
>Do your expectations differ from these?
>
>The use of reversible between subnets is what's concerning me.  It may be 
>that
>an SA could not return any paths as reversible between two subnets without 
>using
>some trick like what you mentioned.
>
>These add a requirement on the SA that they must be aware of the routes 
>packets
>take between two GIDs using a given TC and FL, but I don't believe that this
>necessarily forces SA to SA communication.  The SA may only need to exchange
>information with a router...?

It should not force SA to SA communication.   Such communication is overly 
complex and will be a major issue to control and manage in the end. 
Further, security concerns, partition management, etc. start to complex 
enough as it is without adding more fuel to the fire.

> > Implicit in this are five IBA affecting things:
> >  - that PRs with SGID=non-local mean something specific
>
>I don't think that we're changing any of the meanings of the fields though.
>
> >  - Routers do the SLID spoofing you outlined.
>
>I'm not sure this is something that we do want now.  APM should really handle
>path failover.
>
> > There is alot of complex work in the router and SA side to make this
> > kind of topology work, but it is critical that the clients use path
> > queries that can provide enough data to the SA and return enough data
> > to the client to support this.
>
>I'm still deciding if the existing path record attribute is sufficient.

Our original IB router work I believe drove some of what is in the current 
records so I suspect they are fine as is.

Mike