[openib-general] Re: RFD: uverbs and hotplug

[Roland Dreier]

    Michael> The application will get a completion with error, and it
    Michael> just needs a way to know that its because of the HCA
    Michael> going away, so it seems to me you have to signal the
    Michael> event before closing the QPs, not after that.

It won't get any completion at all: we just destroy the QPs (== move
them to RESET) and CQs, so at a certain point the HCA stops processing
work requests.  The CQ buffers are in user space, so the contents stay
valid and the app can continue polling until it has seen all CQEs.

It's possible that a request may complete remotely but not generate a
completion locally, but this is possible anyway, since the HCA might
be removed before an ACK is received.  And certainly the remote side
of the communication will see its requests fail with a "retries
exceeded" status.

    Michael> Hmm, it also seems that since posting work request on a
    Michael> closed qp does not produce a new work request, a post
    Michael> will need to check a state flag so that post on such a qp
    Michael> fails.  Thats probably why you have /* XXX check that
    Michael> state is OK to post send */ at the top of mthca post
    Michael> send?  But I dont see how to set the state field without
    Michael> a race?

I don't see why this is necessary.  The doorbell page no longer points
at the HW, so any work requests will be silently ignored.

The comment in the post send routines are just there because
technically the spec requires the work request posting verbs to fail
immediately if the QP state is wrong.

 - R.