[ofa-general] Question about RDMA CM

[Or Gerlitz]

Doug Ledford wrote:
>> No; we are using ibv_create_qp, and then assigning id->qp afterwards.
> Don't do that.  Assume if rdmacm provides an interface for doing something, then there is likely a reason.  In this case, when you call rdma_create_qp(), it does more than just call ibv_create_qp() and ibv_modify_qp() on your behalf.  It also pipes information about the state changes in the qp to the kernel rdma_cm module (by writing the commands in rdma_cm format to id->channel->fd, which is the rdma_cm fd not the qp fd, in places like rdma_init_qp_attr()).
As far as I remember, unlike the kernel rdma-cm, librdmacm doesn't 
support a mode where the app creates the qp and maintain the qp state 
transitions. However, looking for example on rdma_accept, you can see 
that it calls ucma_modify_qp_rtr unconditionally and the latter caused a 
return of -EINVAL if id->qp is NULL, on the other hand what the accept 
code does next is to branch based on whethere id->qp is NULL or not...

Or.