[openib-general] Problem is routing CM REQ

[Sean Hefty]

>So basically what you are saying is that the TClass and FlowLabel act
>as some kind of global dis-ambiguation that lets all SAs know that the
>tuple <SGID,DGID,TClass,FlowLabel> MUST be matched with <LRH_A,LRH_B>
>on each side.

Sort of...  My reasoning is that if you look at a packet traveling from the
source QP to the destination QP, and examine the packet in some intermediate
subnet (say between two routers), then the only information that it carries is
the <SGID, DGID, TClass, FlowLabel> tuple.  This information must be sufficient
to direct the routing at the endpoints.

I don't see that all SAs need to know this information.  An SA would:

1. Given local and remote GIDs, need to know which router the packet will arrive
on.
2. Know the SLID/DLID mapping used by that router.

It shouldn't need information about the paths used by packets on the remote
subnet.  If a subnet has multiple routers into it, they can forward packets to
the correct router if needed.  (Could the routers just forward to the end node
and insert the expected SLID?)

If the path is reversible (with reversible defined relative to SLID/DLID that is
returned in the path record), then the active node would only need two SA
queries - one to each subnet.  For non-reversible paths, 4 queries may be
needed.

>I've been thinking that the <SGID,DGID,TClass,FlowLabel> tuple would
>only reflect 2 of the 4 lids (ie the ones the router chooses on entry
>to the final subnet).

This was my thinking as well, which is why I think 2 path record queries are
needed.  Each path would specify 2 of the 4 LIDs that we need.  The local path
record gives us the local QP information, and the remote path record is used to
fill in the SLID/DLID in the CM REQ.

>The very simple reversible paths only solution still seems best to me
>only because it involves the least work and only requires that IBA
>specify routed reversible paths.

I'm still trying to find a solution that doesn't violate the architecture as
defined.  I don't see why my idea wouldn't work yet.  It just requires some
unspecified coordination between the local SA and local routers.

- Sean