[ofa-general] CQ destory

[Roland Dreier]

 > There are a couple of cq-s registerd with the same completion channel.
 > 
 > ibv_cq_get_event is done in a thread. When this thread is woken up by
 > one or more completion events, ibv_poll_cq is done to extract each of
 > the completion  entry.
 > If the completion status is bad the cq is destroyed as part of processing this
 > completion entry. The question is will destroy of cq flush all the
 > completion events
 > for this cq? If not what is the way to get around this problem?

I think the current code is OK.  Destroying a CQ does remove all the
events associated with that CQ from its completion channel.  However
there is an unavoidable race if you call ibv_destroy_cq() from one
thread but then retrieve a completion event for the same CQ from
another thread before the destroy operation actually takes place.  To
deal with this, the destroy CQ operation will not actually return
until you have acked all the completion events returned with
ibv_ack_cq_events().

 - R.