[openib-general] IB routing discussion summary

[Michael Krause]

At 11:49 AM 2/21/2007, Sean Hefty wrote:
>I sent a message on this topic to the IBTA several days ago, but I am still
>awaiting details (likely early next week).

Unclear if that will occur.  I just responded to some e-mail in the IBTA on 
the router subject as well.    Given that discussion, I suspect it will be 
some time coming to fully answer the router dilemma.


> >It should not be carried in the CM REQ.  The SLID / DLID of the router
> >ports should be derived through local subnet SA / SM query.  When a CM REQ
> >traverses one or more subnets there will be potentially many SLID / DLID
> >involved in the communication.   Each router should be populating its
> >routing tables in order to build the new LRH attached to the GRH / CM REQ
> >that it is forwarding to the next hop.
>
>I'm referring to configuration of the QP, not the operation of the routers.
>
>To establish a connection, the passive side QP needs to transition from 
>Init to
>RTR.  As part of that transition, the modify QP verb needs as input the
>Destination LID of its local router.  It sounds like you expect the 
>passive side
>to perform an SA query to obtain its own local routing information, which 
>would
>essentially invalidate the data carried in the primary and alternate path 
>fields
>in the CM REQ.

The source always queries to obtain a subnet-local router Port.   A sink 
can simply reflect back the LRH with source / destination LID reversed 
assuming it had such information or it can query to find the optimal / 
preferred subnet-local router Port.


> >From reading 12.7.11, 13.5.1, and 17.4, I do not believe that such a 
> requirement
>was expected to be placed on the passive side of a connection.  The initial
>response I received agreed with this.
>
> >I'd need to go back but the architecture is predicated that the SM and SA
> >are strictly local and for security purposes their communication should
> >remain local.  Higher level management entities built to communicate with
> >SM and SA are responsible for cross subnet communications without exposing
> >the SA or SM to direct interaction.  P_Key and Q_Key management across
> >subnets is an example of such communication across subnets that would not
> >be exposed to the SA and SM.
>
>My initial thoughts are that this sounds like a good idea.  It's not 
>eliminating
>the need for interacting with a remote SA, so much as it abstracts it to 
>another
>entity.
>
>My hope is that we can reach an agreement on the CM REQ.  Depending on 
>that, it
>still needs to determine if the existing SA attributes are sufficient to allow
>forming inter-subnet connections, and if they are, can such attributes be
>obtained.

A lot of discussion will be required within the IBTA to nail anything 
down.   As I noted above, I just provided answers to a number of questions 
posed as well as opened up perhaps a few more.   I am not aware of a TTM to 
complete this work but clearly some amount of standardization is required 
and it will take a bit to define the scope so that the specification does 
not become so large that it will take significant amount of time to develop 
and more importantly, significant resources and time to validate that the 
routing protocol is solid.   Routing protocols are not as simple as some 
may think - they vary as a function of the functional robustness and 
scalability provided.

For now, I'll assume this discussion is on hold until the IBTA gets its act 
together.

Mike