[openib-general] Completion callback /teardown race

[Or Gerlitz]

Eric Barton wrote:
> I create 1 CQ just for receive completions on each of my QPs.  When I tear down
> the QP, I rdma_disconnect(), change the QP state to IB_QPS_ERR and then wait
> for all currently posted receives to complete.

I understand your driver is a CMA consumer whose QP state transitions 
are carried out by the CMA. So you need ***not*** modify the QP state to 
error, as the CMA does it for you in rdma_disconnect() before sending 
the DREQ or DREP.

Please note that you need to call rdma_disconnect() in both sides, the 
one that initiates the disconnection but also on the side that gets the 
DREQ, that is suddenly gets RDMA_CM_EVENT_DISCONNECTED event (note that 
also the disconnection initiator would get this event and if you call 
there again to rdma_disconnect() its not going to break anything, i think).

Is it possible that manual QP modify to error in your code actually 
covered the latter case where you did not call rdma_disconnect()?

Or.