[ofa-general] RE: [RFC] host stack IB-to-IB router support

[Sean Hefty]

>Hmm. If the goal is enable router development and experimentation then
>it would be best if the 'ib_remote_sa' server was in user space, delt
>with all 4 path records in one query and was centralized so it could
>be made to store routing topology and configuration to solve the
>multipath problems. Otherwise I think you are better to just talk
>directly to the SA.

Unfortunately, at least opensm cannot respond to SA queries issued from a remote
subnet.  I'm not sure how much work this would take to fix, or if other SAs have
this issue.  Hal briefly looked at the problems, and I do plan on trying to fix
them.  But that still leaves trying to find the remote SA, handling SA failover,
etc.  This is why I'm bouncing queries through an intermediary.  

I see two separate pieces that are needed: an interface to query for the path
info, and a mechanism to provide it.  At least the former is needed in the
kernel, and I can at least envision that the implementation of this piece could
evolve into some final solution.  But at this point, the query response
mechanism seems like throw-away code.

>Maybe the best thing here is to have a simple ib_remote_sa client
>module that just consults a list of servers and makes a normal SA
>query. People working on multipath router support could then extend
>that to specify a non-SA server and a new 4 path query type.
>
>A list something like:
>2001::/64 2001:1 SA
>2001::/64 2001:2 SA
>2002::/64 2000:1 not-SA   <-- On the local subnet.. new 4 PR format
>
>Set via netlink or sysfs..
>
>To start with no ib_remote_sa server would be needed, just a boot
>script to set the expected SA addresses. You could define the MAD
>format for a new 4 PR query but not implement a server to handle it.

Hmm... let me give this more thought.

>Do you have any idea what the PathForward program expects to do here?

not really...

- Sean