[ofa-general] rping / librdmacm deadlock question

[Steve Wise]

Sean Hefty wrote:
>>> I wonder if it would be better if the iWarp CM acquired/released the 
>>> QP reference on a per call basis, rather than holding a reference 
>>> throughout the entire connection.
>>>
>>
>> The design assume the iwcm can hold this reference and cache the qp 
>> ptr.    In the iwarp design, the cm_id (connection) and qp are tighly 
>> bound once the connection is transitioned into rdma mode.  This is 
>> different than infiniband.
> 
> I don't know if this tight binding is necessary in the implementation. 
> The cm_id could store the qpn, rather than a pointer to the structure. 
> When necessary, the qp pointer could be acquired using the qpn, then 
> released at the end of the function call.  I don't think we need to hold 
> the reference on the qp structure for the entire connection.
> 
> I'm just tossing this out as an idea.  I'm not familiar enough with the 
> details to claim that it's a better approach over what's currently done.

Maybe, but I'm not gonna change this code now.  It was too painful to 
get working... ;-)

> 
>> I still don't see the deadlock?
> 
> What happens if a user calls destroy qp immediately after connecting it?
> 
> 

See my reply to clark.  The iwarp provider _must_ disassociate the 
endpoint/cm_id from the qp in destroy_qp()...  This involves aborting or 
closing the connection and passing a CLOSE event to the iwcm which 
removes its reference.


Steve.