[openib-general] Problem is routing CM REQ

[Sean Hefty]

> Ah, I think I missed the key step in your scheme.. You plan to query
> the local SM for SGID=remote DGID=local? (ie reversed from 'normal'. I
> was thinking only about the SGID=local DGID=remote query direction)

I'm not sure that the query needs the GIDs reversed, as long as the path is 
reversible.  So, the local query would be:

SGID=local, DGID=remote, reversible=1

And the remote query would be:

SGID=local, DGID=remote, reversible=1,
TClass & FlowLabel=from previous query response

Use of reversible indicates that the remote side can send a packet back, and it 
will be received successfully at the local side.  This seems to imply 
information about the local routing tables and GID to LID mappings.  That is, 
packets traveling from the SGID->DGID and DGID->SGID use the same local LID pair.

>    SA                                                      SA'
> Node1 --> (LID 1) Router A -------  Router A' (LID A) ---> Node2
>       |-> (LID 2) Router A                              |
>       |-> (LID 3) Router B -------  Router B' (LID B) --|
> 
> Router A and Router B are independent redundant devices, not a route
> cloud of some sort. B -> A' is not a possible path.

Since A' and B' connect to the same subnet, B -> A' should be a valid path.

> So your idea is to do:
>   PR0: Node 1 asks SA for Node1 -> Node2 reversable path.
>        SA returns SLID=Node1 DLID=1, FlowLabel=Magic Reversable
>        indicator. This path is used for CM GMPs, or for the
>        normal non-routed CM.
>   PR1: Detecting a routed situation from PR0, 
>        Node 1 asks SA for Node2 -> Node1. SA returns SLID=1
>        DLID=Node1 and a GRH that configures Router A to use SLID=1
>        You reverse the local LIDS from that path to get the QP
>        configuration.

I think PR0 and PR1 could be the same.

> I can think of the following downsides:
>  1) Re-reading Michael Krause's email makes me think that defeating
>     the QP SLID check is contrary to the spirit of IBA

I don't think we need to defeat the QP SLID check if we want extra routing, but 
having redundant routers use the same link layer address isn't necessarily a bad 
thing.

>  4) Some means of remote SA communication needs to be decided
>     pre-standardization :< (I agree that a magic GID seems best)

I think this is the first thing that must be solved, regardless of other 
details.  We should see if we can at least get agreement on this, and if there 
are any issues.

> But this has turned into such a complex problem it seems really hard
> to predict what will pass through to standardization.. That is the
> main benifit I see of the small change to the passive side. No matter
> what is standardized it can be accomidated in the resulting
> standard, wheras defining a PR with SGID==offsubnet to mean one thing
> or another seems much more risky.

I think the only thing we're asking for so far is a magic GID, unless I'm 
reading too much into what a reversible path indicates.

- Sean