[ofw] [PATCH v2] libibverbs/device: destroy completion channel when closing device

[Sean Hefty]

>There is a general problem with closing a handle that one might be waiting on:
>Consider a single handle g_handle.
>
>Thread A is calling waitforsingleobject(g_handle)
>
>Thread B that wants to free thread A is calling closehandle on g_handle.
>If Thread A was indeed able to wait on the handle than it will be freed. But it
>is also possibale that for some reason thread a was not scheduled and never
>reached the wait code. Further more, it is possibale that after thread b has
>closed the g_handle this handle will be reused. In that case thread A might
>wait for ever.
>
>Are you sure that your checkin doesn't introduce this kind of bug?

In short, this is a user issue.  A user cannot access the device structure, for
whatever reason - including waiting on a handle associated with the device, at
the same time that they are trying to destroy the device.

The actual bug is that: memory is freed that contains an active critical
section.  This destroys the critical section before freeing the memory.  Even if
the critical section is not destroyed, the memory is still freed, and any
attempt to access the critical section is fatal.

With this change, application verifier now reports success for rdma_cmatose,
which is what I used to trigger the error.

- Sean