[openib-general] Problem is routing CM REQ

[Sean Hefty]

>>4. A PR from the local SA with reversible=1 indicates that data sent from 
>>the remote GID to the local GID using the PR TC and FL will route locally 
>>using the specified LID pair.  This holds whether the PR SGID is local or 
>>remote.
> 
>>5. A PR from a remote SA with reversible=1 indicates that data sent from 
>>the local GID to the remote GID using the PR TC and FL will route remotely 
>>using the specified LID pair.  This holds whether the PR SGID is local or 
>>remote.
> 
> I can't think how to actually implement these restrictions in the
> general case without SLID spoofing and the general method I outlined
> in my prior email.

But you agree with the expectations, and what reversible indicates?  Or are you 
claiming that reversible paths between different subnets is undefined, or means 
something different than specified in 13.5.4?  (E.g. reversible applies only at 
the network level if global routing is used.)

> Think about this - it is backwards for the UD case. You have specified
> that the SGID->DGID direction uses the returned SLID/DLID which are
> ensured by the flowlabel in the GRH. But the local side only controls
> what it sends. How does this GRH get to the remote side? In UD the
> returned GRH from the PR controls the selection of LID on the DGID's
> subnet. That is how it must be.

I'm not following you here.  For UD, query the local SA, then direct the send to 
the router LID.  I would only query the remote SA for RC, in order to get the 
remote LID information to put into the CM REQ.

> The major problem is that there are multiple router paths that a given
> GRH can take that are only fully disambiguated by the router lid at
> the sender.

But doesn't 19.2.4.1 imply that once a router selects a path, it will continue 
to use that same path for similar packets?  So, if we inject a GRH into the 
internetwork from the source router, then isn't a single path followed to the 
remote endpoint?

Relaxing 9.6.1.5 seems like a nice solution to most of the problems, but it also 
seems like one that would fail to work with any existing HCAs.

- Sean