[ofa-general] Question about RDMA CM

[Jeff Squyres]

On Sep 17, 2008, at 3:54 AM, Or Gerlitz wrote:

>> Don't do that.  Assume if rdmacm provides an interface for doing  
>> something, then there is likely a reason.  In this case, when you  
>> call rdma_create_qp(), it does more than just call ibv_create_qp()  
>> and ibv_modify_qp() on your behalf.  It also pipes information  
>> about the state changes in the qp to the kernel rdma_cm module (by  
>> writing the commands in rdma_cm format to id->channel->fd, which is  
>> the rdma_cm fd not the qp fd, in places like rdma_init_qp_attr()).
> As far as I remember, unlike the kernel rdma-cm, librdmacm doesn't  
> support a mode where the app creates the qp and maintain the qp  
> state transitions. However, looking for example on rdma_accept, you  
> can see that it calls ucma_modify_qp_rtr unconditionally and the  
> latter caused a return of -EINVAL if id->qp is NULL, on the other  
> hand what the accept code does next is to branch based on whethere  
> id->qp is NULL or not...


Right.  I agree that it's a bit weird that we create the QP manually  
-- it's just what the original author chose to do.  But we never did  
any of the QP state transitions -- user RDMACM does that for us.  All  
I've done in my tests is verify that the state transitions were  
occurring properly (but putting a little testing code in librdmacm,  
actually).

I'll keep digging...

-- 
Jeff Squyres
Cisco Systems