[ofa-general] post_recv question

[Caitlin Bestler]

On Thu, Feb 21, 2008 at 7:41 AM, Steve Wise <swise at opengridcomputing.com> wrote:
> Hey all:
>
>  I have a question regarding exactly _when_ a posted recv buffer is
>  available for the HW to use:
>

None of the verbs specifications will be explicit about this for the
simple reason
that none wants to specify exactly what an RQ or SRQ actually is. This is how
good specifications are written.

But, wherever this queue (RQ) or pool (SRQ) of receive WQEs live, successfully
posting a receive WQE should mean exactly that -- it was successfully posted.

And if a receive WQE was successfully posted I cannot see a justification for
an RDMA device raising a "no buffer available" subsequent to that point.

Now if a buffer was received prior to the recv wqe post completing, the error
may have already been raised, and the exception might not be delivered to
the host until after the recv wqe call successfully completed. So applications
SHOULD NOT be written to assume that they will win a "tie", but rather to
ensure that the recv WQE is posted *before* the message arrives. Sending
the request after posting the recv WQE for the reply should be mor than
adequate for that purpose. Specific implementations should take whatever
steps are required to ensure that the hardware will not declare  "no buffer"
after the user has posted a buffer, whether that is through a doorbell or
by rechecking the queue after any check of cached contents comes up
empty.