[dat-discussions] [openib-general] [RFC] DAT 2.0 immediatedataproposal

[Caitlin Bestler]

Larsen, Roy K wrote:
> If it is up to the ULP to separate out "normal" receive data
> from that associated with a write immediate, how is this
> different from the ULP doing a write followed by a send?  If
> there is no difference, then what we're really talking about
> is a convenience to the initiating ULP.
> 
> Perhaps what would be best is to construct an API that allows
> the ULP to perform standard write/send operations into one
> call which the underlying provider could optimize into one
> transaction with the associated interconnect interface.
> Better yet, a general request combining interface would have
> even more value, but calling this write/send "immediate" data
> is a stretch, if not downright silly.  Some transports have
> true immediate data that provides unique value.  There is
> nothing unique in a write/send sequence - ULPs do it all the time...
> 

The data provided is to identify the completion notification
that completes the RDMA Write to the data sink. So, yes, it
is not really an "immediate" value. We could consider a better
name for it, much as we renamed QP to something better.

But the meaning is "the tag value associated with a specific
RDMA Message". It is delivered in order, after that RDMA
Message has fully completed.

What varies by transport is *how* it is is delivered. We
are considering identifying it as a single work request
so that transport-specific contraction to  a single
wire message is enabled.

But we don't want to change any of the semantics vs.
the application doing Write then Send. The new call
enables an optimization, but should not change the
overall semantics. That could extend as far as having
the the receiver recognize the alternate reception.