[dat-discussions] [openib-general] [RFC] DAT 2.0immediatedataproposal

[Sean Hefty]

>But the remote side does posts Recv. Since it anticipate that
>this Recv will be matched against the RDMA Write with immediate
>it posts the recv buffer which fits. Yes, there is an issue
>for Transport-independent ULP that it does needs a buffer.
>For IB it is possible to post 0-size buffer. But if this is the case
>Recv end Consumer DOES know that it will be macthed against RDMA
>Write so ULP DOES know what it will be matched against.
>So in the worst case Consumer does have to pay the price of creating
>LMR to handle 4 byte buffer to match RDMA Write Immediate data.

How does the remote ULP know this?  A DAPL implementation has no idea what a
receive will match up against.  You're pushing a requirement on the ordering of
sends/writes to the application that was not there before.

>> It just dawned on me that the immediate data must be in
>> registered memory to be sent in a message.  This means the
>> API must be amended to pass an LMR or, even worse, the
>> provider would have to register memory in the speed path or
>> create and manipulate its own queue of "immediate"
>> data buffers/LMRs.  Of course, LMRs are not needed and an
>> overhead for transports that provide true immediate data.
>
>No registration on the speed path. It is Consumer responsibility
>to provide Recv Buffer of the right size.
>Yes for IB only ULP this can be avoided.
>But ULP can be written to the proposed API to take full
>advantage of IB performance but that code will not be transport
>independent.

The immediate data needs to be registered before being sent.  This will need to
be hidden from the user.

>But this API allows to write transport independent code
>albeit with certain price attached.

What good does it do to have "transport independent" code, when the feature
being invoked is "transport dependent"?  There's no requirement that immediate
data be supported.  Why define an API so that it can be emulated?  Define the
right API, and let transports that don't support immediate data indicate so.  A
"transport independent" application can check this bit and take whatever action
is necessary.  They need to do so anyway, since the bit may or may not be set.

- Sean