[openib-general] [PATCH] librdmacm: updated librdmacm to work with proposed 2.6.20 kernel CMA

[Or Gerlitz]

Sean Hefty wrote:
>> 1) librdmacm does not get built against libibverbs-1.0 (see below) so 
>> i am using libibverbs (ie the non released yet libibverbs1.1)

> I need to think about what we can do here.  The librdmacm uses 
> functionality not found in libibverbs-1.0.

Have you looked on that? from the compilation failure against 
libibverbs-1.0 the gap seem pretty small. If indeed this is the case, 
since libibverbs-1.1 is in development lets check with Roland if it 
makes sense for him to support these small-gap-features in 
libibverbs-1.0.X, i guess what matters here is ABI versions...

If it is not possible, maybe we can somehow instrument the code of 
librdmacm to do well with libibverbs-1.0.Y

If this is not possible as well, i guess the way to use librdmacm for 
the time being is against a devel drop of libibverbs-1.1 as i am doing now.

>> 2) the cma rdma multicast does not let a consumer to join as send-only

> This would require some sort of change to the API and ABI, so if this is 
> needed, I'd like to incorporate this now.  (Adding it could be done by 
> specifying join parameters.)  Do we need/want this level of control in 
> the librdmacm, or should users go to a direct IB interface for this?

I think we do want it. The rdma cm provide the means to offload ip 
multicast to ib multicast though registration (join/leave etc) with the 
ib_sa module. IP Multicast does use the send-only feature and hence IP 
Multicast offloading apps need it as well. The rdma cm framework fits 
very well for such apps and the ib_usa (which does not exist now, and i 
am not sure needs to exist... it was a project of a summer student with 
open-mpi that required that...) not.

Currently, librdmacm does not have the means to distinguish between 
sender and receiver, so it joins the sender as full member and attaches 
its qp to this group mgid, this hurts performance, first and second 
might cause this sender CQ to receive the posts as well (i am not sure 
here) which can get it go crazy...

Or.