[dat-discussions] [openib-general] [RFC] DAT 2.0immediatedataproposal

[Caitlin Bestler]

Sean Hefty wrote:
>> I am not clear what you are proposing?
>> A transport specific API?
>> 
>> The current proposal provides on sending side:
>> single post, and single completion in the error free case.
>> This is commonality that simplify ULP.
> 
> App 1 - transport aware:
> 
> if (transport == IB)
> 	Do something
> else
> 	Do something different
> 
> App 2 - transport independent:
> 
> if (immediate data flag set)
> 	if (DTO == 1)
> 		Do something
> 	else
> 		do something else
> else
> 	do something different
> 

Two points:

	First it is more like 

		the_data = (DTO == 1) ? event->immediate : dto->buf;

	And further it is only on the receiving side.
	And only if the receiving side cares about the data
		(sometimes it only needs the notification).

The more critical issue to me is whether this justifies all
the text required to explain proper sizing of EPs and EVDs.

I'd love to hear from some app developers as whether this
is too complex to use, or a nice simplification. It isn't
that hard for an iWARP Provider to do this, but any extra
work is best avoided if the feature won't get used because
the associated "caveats" text is too long.


> All you've done is add flags in order to call the API
> "transport neutral".  The result to the application is the
> same, except that the interface is more complex than it needs
> to be, and causes confusion on the receiving side.  And on
> the sending side, the application still needs to check the
> flag to see if immediate data is supported.
> 
> A true transport neutral API wouldn't need flags that specify
> the actual differences between the transports.
> 
> The requirement is to provide an API that supports RDMA
> writes with immediate data.  A send that follows an RDMA
> write is not immediate data, and the API should not be
> constructed around trying to make it so.
> 
> If you want to add a new requirement to the API to support
> posting multiple work requests with a single call, that is a
> different requirement. 
> 
> 

The attempt is to define a composite work request that can
reduce the number of actual work requests required for
some providers, without requiring different work flows
dependent on whether the "immediate" feature was present.

If this is not possible then DAT should simply not support
the feature through anything other than a provider dependent
extension function. In particular we should not be carving
any exceptions to normal DAT ordering rules. In other words
it is important that the delivery that confirms the RDMA
Write to the Data Sink be ordered exactly as if an RDMA Send
had been used to confirm delivery.