[openib-general] IB routing discussion summary

[Michael Krause]

At 11:39 AM 2/15/2007, Sean Hefty wrote:
>>Ideas were presented around trying to construct an 'inter-subnet path record'
>>that contained the following:
>>    - Side A GRH.SGID = active side's Port GID
>>    - Side A GRH.DGID = passive side's Port GID
>>    - Side A LRH.SLID = any active side's port LID
>>    - Side A LRH.DLID = A subnet router
>>    - Side A LRH.SL   = SL to A subnet router
>>    - Side B GRH.SGID = Side A GRH.DGID
>>    - Side B GRH.DGID = Side A GRH.SGID
>>    - Side B LRH.SLID = any passive side's port LID
>>    - Side B LRH.DLID = B subnet router
>>    - Side B LRH.SL   = SL to B subnet router
>
>Until I can become convinced that the above isn't needed, I've been trying 
>to brainstorm of ways to obtain this information.

Is this first an IBTA problem to solve if you believe there is a 
problem?   I believe the track you are on is incorrect and any attempt to 
surface subnet local information across subnets will create unnecessary 
complexity and therefore makes such solutions less practical to execute 
within the industry.   I've tried to illustrate the role of the router, how 
the flows work, etc.  I believe these to be correct and are reflected not 
only in the existing specifications but also the prior router specification 
work and thinking.   They also parallel the IP world quite nicely which 
should also lend credence that subnet-local information does not need to be 
exchanged between subnets.   I contend CM does not require anything that is 
subnet local other than to target a given router port which should be 
derived from local SM/SA only information.  I will further state that SA-SA 
communication sans perhaps a P_Key / Q_Key service lookup should be avoided 
wherever possible.

I strongly urge you to take this problem to the IBTA where any issues 
regarding specification interpretation can be sorted out and an official 
position taken.   This will yield a faster and more successful 
investigation into whether there is a problem and if so, how best to solve it.

Mike


>0. Have the SA return pairs of PathRecords for inter-subnet queries.
>
>But, since this simply punts the problem to the SA, my other thought is to 
>define the following:
>
>1. Inter-subnet PathRecord/MultiPathRecord Get/GetTable requests require 
>both an SGID and DGID, one of which must be subnet local to the processing SA.
>2. PathRecord/MultiPathRecord Get/GetTable request fields are relative to
>the subnet specified by the SGID.
>3. PathRecord GetResp/GetTableResp response fields are relative to the
>subnet local to the processing SA.
>4. SAs are addressable by a well-known GID suffix.
>
>I think this may allow establishing inter-subnet connections.  As an 
>example of
>its usage:
>
>a. Active side issues a PathRecord query to the local SA with SGID=local,
>DGID=remote.
>b. SA responds with PathRecord(s).
>c. Active side selects local PathRecord P1.
>d. Active side issues a PathRecord query to the remote SA using PathRecord 
>P1 to
>format the request: SGID, DGID, SLID, DLID, TC, FL, SL, etc.
>e. The remote SA responds with PathRecord(s).  The SA must ensure that 
>packets injected into the internetwork using P1 will route to the returned 
>records.
>f. Active side selects remote PathRecord P2.
>g. Active side validates that remote packets injected using P2 route to P1.
>
>At this point, the active side should have path information that can be 
>used to
>configure the QPs for a connection.
>
>Assuming that this will work, what I don't like about it is the validation 
>at step g.  This adds a third query that I don't see a way to 
>eliminate.  If the check fails, the client restarts at step c.
>
>- Sean