[ofa-general] post_recv question

[Caitlin Bestler]

On Thu, Feb 21, 2008 at 11:40 AM, Sean Hefty <sean.hefty at intel.com> wrote:
> >I'm asking this from a device driver developer's perspective.  I'm not
>  >writing an application.  I'm trying to understand and define exactly
>  >what must be guaranteed by the device/driver up returning from
>  >post_recv().
>
>  At least from IB's view for post receive (from spec):
>
>  Control returns to the Consumer immediately after the WQEs have been submitted
>  to the Receive Queue or the SRQ and the HCA has been notified that one or more
>  WQEs are ready to process.
>
>  - Sean
>

Would you agree that if the WQEs have already successfully been
"submitted to the Receive Queue or the SRQ and the HCA has been notified"
that the HCA would be incorrect in subsequently raising an error
stating that the
buffers were not available?

iWARP does not convey send credits in the RDMA protocol, but I believe both
iWARP and IB are in agreement that declaring "no buffer available" and causing
the reliable connection to be torn down is a serious step. The HCA/RNIC is not
free to be sloppy in making this determination.

There are other places in both specifications where the RDMA device is given
latitude to asynchronously implement a request. For example, it is clear that
a window is *not* necessarily bound when the bind call completes. But in
all those cases there is an explicit completion to allow the consumer to
unambiguously know when it is safe to proceed.

the application is never expected to rely on knowledge of specific HCAs
or RNICs, or to "guess" what might be good enough. There are only
two feedbacks from posting a Receive WQE: the call completion
and the CQE being returned by cq_poll().

There are only two states for the Recv WQE between those two events:
available for allocation and allocated. And the application does not
need to know about the difference between those two states on a
per-WQE basis.

If there were a third state, then there would have to be a mechanism
to make that information available. There is none, so such a third state
must not exist  (at least in any observable form).