[openib-general] re RDS missing features

[Or Gerlitz]

On 5/2/06, Ranjit Pandit <rpandit at silverstorm.com> wrote:
> On 5/2/06, Or Gerlitz <ogerlitz at voltaire.com> wrote:
> > Ranjit Pandit wrote:

> Loopback connections can be optimized by not going to the HCA.
> In b-copy mode we can directly copy sends into destination sockets on
> the same node.

So its an optimization missing but the functionality is there, that is
currently its possible to run netperf/crload/oracle over RDS when both
client and server are on the same node?


>> I see. Can you remind me ... where is the location of the reference gen1
>>   RDS code? does it support failover?

> Yes, Rds reference implementation implements failover across HCAs.
> It was checked into contrib/silverstorm/rds.
> r3471 was the first checkin and then a few more updates were made with
> bug fixes.

sorry, but i don't find it, are you reffering to the code under
https://openib.org/svn/trunk/contrib/silverstorm/rds/ ? it does not
seems to me your GEN1 code, am i wrong? please send me a pointer.

> Keep it simple ie., apply the same failover scheme between two ports
> whether on same HCA or not.

>> Are you aware to something in the openib infrastructure which is missing
>> for the failover design of RDS? if you specify the design/requirements i
>> am sure people on this list can quickly say if something is missing...

> For failover Rds need support for the following:
> 1. Ability to assign single IP address to multiple IB ports
> 2. Address resolution mechanism should return multiple paths for the
> same destination IP address.

Cool, i like that.

> On SilverStorm stack a single IP address can be assigned to two ports
> in the system.
> When a path fails, RDS can re-establish connection to the same
> destination IP address...ipoib_path( dst_ip) returns all possible
> paths to the destination ip.

I see.

> Does the CMA handle multiple paths to a destination IP?
> It does not need to return multiple paths to Rds. For now, even if it
> picks the first available path that should be sufficient.

I guess by "should be sufficient" you mean for everything but failover?

Sean, what will the current ib_addr module implementation does if it
being asked to send ARP and there are > 1 network interfaces for the
subnet over which the resolution is needed? will it send ARP on the
first match or over all the devices? It seems that if there's no issue
with setting two network/ipoib devices with the same address the RDS
failover scheme should work over the CMA if the callback following
rdma_resolve_addr would just be able to return N > 1 SRC / DST GIDs
and IB devices, does it make sense to you?

And if i have (say) two <src,dst> GID couples, then i would need to
just call rdma_resolve_route twice and then rdma_connect twice,
correct?

Or.