[openib-general] Re: Re: ibv_get_async_event

[Michael S. Tsirkin]

Quoting r. Roland Dreier <rolandd at cisco.com>:
> Subject: Re: Re: ibv_get_async_event
> 
>     Sean> I was actually thinking about the destruction of the CQ, not
>     Sean> the QP.  Thinking about it more, it seems unlikely, but
>     Sean> couldn't a user destroy a QP followed by the CQ before
>     Sean> ibv_get_cq_event() returns?
> 
> I guess so.  I was actually also confusing myself between CQ events
> and CQ entries.  Yeah, we probably need a ibv_put_cq_event() call as
> well.
> 
>  - R.

If I'm not mistaken, the idea is that ibv_put_cq_event will release
a cq reference and allow it to be destroyed?
My problem with this approach is that it adds overhead
to the data path cq event instead of non-datapath cq destroy.

If thats what we are trying to do, I'd like to propose another idea:
when cq is destroyed, and after all cq events are queued to user,
put a special "cq destroyed" event into the event queue.
When a user calls destroy_cq, and after releasing kernel/hardware resources,
move the cq structure to a special cleanup list, userspace library will
only destroy the userspace cq structure when it gets this event.

Instead of reference counting, we require the user to poll for this event
after destroying the cq.
Since all events are polled from a single thread, when the library sees this
event it knows the user does not keep a pointer to the cq any more.

Let me know if this makes sense I can code up a patch.

Hope this helps,
MST

-- 
MST