[ofa-general] RE: [RFC] host stack IB-to-IB router support

[Hal Rosenstock]

On Mon, 2007-03-19 at 22:26, Sean Hefty wrote:
> >Hmm. If the goal is enable router development and experimentation then
> >it would be best if the 'ib_remote_sa' server was in user space, delt
> >with all 4 path records in one query and was centralized so it could
> >be made to store routing topology and configuration to solve the
> >multipath problems. Otherwise I think you are better to just talk
> >directly to the SA.
> 
> Unfortunately, at least opensm cannot respond to SA queries issued from a remote
> subnet.  I'm not sure how much work this would take to fix, or if other SAs have
> this issue.  Hal briefly looked at the problems,

FWIW, I'll be looking some more at these again.

>  and I do plan on trying to fix
> them.  But that still leaves trying to find the remote SA,

Yes, that is one primary obstacle to solve one way or the other that
seems like a pretty basic need.

>  handling SA failover,

This would be a bonus rather than an initial requirement (for
experimentation in connecting more than one IB subnet) IMO.

-- Hal

> etc.  This is why I'm bouncing queries through an intermediary.  
> 
> I see two separate pieces that are needed: an interface to query for the path
> info, and a mechanism to provide it.  At least the former is needed in the
> kernel, and I can at least envision that the implementation of this piece could
> evolve into some final solution.  But at this point, the query response
> mechanism seems like throw-away code.
> 
> >Maybe the best thing here is to have a simple ib_remote_sa client
> >module that just consults a list of servers and makes a normal SA
> >query. People working on multipath router support could then extend
> >that to specify a non-SA server and a new 4 path query type.
> >
> >A list something like:
> >2001::/64 2001:1 SA
> >2001::/64 2001:2 SA
> >2002::/64 2000:1 not-SA   <-- On the local subnet.. new 4 PR format
> >
> >Set via netlink or sysfs..
> >
> >To start with no ib_remote_sa server would be needed, just a boot
> >script to set the expected SA addresses. You could define the MAD
> >format for a new 4 PR query but not implement a server to handle it.
> 
> Hmm... let me give this more thought.
> 
> >Do you have any idea what the PathForward program expects to do here?
> 
> not really...
> 
> - Sean