[dat-discussions] [openib-general] [RFC] DAT 2.0immediatedataproposal

[Kanevsky, Arkady]

It is much simplier to handle immediate data as DAT extension.
Spec changes are minimal. One extra field for DTO completion and for
DAT_DTOS. One fix in redirection.
The rest is up to a provider to define in dat_providername_extensions.

How each provider defines analogous features are outside the scope of
DAT. This includes versioning and feature discovery.

Why would any Consumer hook itself on "proprietary" features and
APIs is a different question.

The other possibility is to define Immed data as DAT API with pure IB
semantic.
Providers for other transports will report feature not supported.
If in the future there transport will define analogous features with
different semantic we will need to redefine APIs to be transport
independent.
But we may have the same problem with the current proposal also.

Arkady Kanevsky                       email: arkady at netapp.com
Network Appliance Inc.               phone: 781-768-5395
1601 Trapelo Rd. - Suite 16.        Fax: 781-895-1195
Waltham, MA 02451                   central phone: 781-768-5300
 

> -----Original Message-----
> From: Caitlin Bestler [mailto:caitlinb at broadcom.com] 
> Sent: Tuesday, February 07, 2006 9:31 AM
> To: Sean Hefty; Kanevsky, Arkady; Larsen, Roy K; 
> dat-discussions at yahoogroups.com; Sean Hefty
> Cc: openib-general at openib.org
> Subject: RE: [dat-discussions] [openib-general] [RFC] DAT 
> 2.0immediatedataproposal
> 
> Sean Hefty wrote:
> >> I am not clear what you are proposing?
> >> A transport specific API?
> >> 
> >> The current proposal provides on sending side:
> >> single post, and single completion in the error free case.
> >> This is commonality that simplify ULP.
> > 
> > App 1 - transport aware:
> > 
> > if (transport == IB)
> > 	Do something
> > else
> > 	Do something different
> > 
> > App 2 - transport independent:
> > 
> > if (immediate data flag set)
> > 	if (DTO == 1)
> > 		Do something
> > 	else
> > 		do something else
> > else
> > 	do something different
> > 
> 
> Two points:
> 
> 	First it is more like 
> 
> 		the_data = (DTO == 1) ? event->immediate : dto->buf;
> 
> 	And further it is only on the receiving side.
> 	And only if the receiving side cares about the data
> 		(sometimes it only needs the notification).
> 
> The more critical issue to me is whether this justifies all 
> the text required to explain proper sizing of EPs and EVDs.
> 
> I'd love to hear from some app developers as whether this is 
> too complex to use, or a nice simplification. It isn't that 
> hard for an iWARP Provider to do this, but any extra work is 
> best avoided if the feature won't get used because the 
> associated "caveats" text is too long.
> 
> 
> > All you've done is add flags in order to call the API "transport 
> > neutral".  The result to the application is the same, 
> except that the 
> > interface is more complex than it needs to be, and causes 
> confusion on 
> > the receiving side.  And on the sending side, the application still 
> > needs to check the flag to see if immediate data is supported.
> > 
> > A true transport neutral API wouldn't need flags that specify the 
> > actual differences between the transports.
> > 
> > The requirement is to provide an API that supports RDMA writes with 
> > immediate data.  A send that follows an RDMA write is not immediate 
> > data, and the API should not be constructed around trying 
> to make it 
> > so.
> > 
> > If you want to add a new requirement to the API to support posting 
> > multiple work requests with a single call, that is a different 
> > requirement.
> > 
> > 
> 
> The attempt is to define a composite work request that can 
> reduce the number of actual work requests required for some 
> providers, without requiring different work flows dependent 
> on whether the "immediate" feature was present.
> 
> If this is not possible then DAT should simply not support 
> the feature through anything other than a provider dependent 
> extension function. In particular we should not be carving 
> any exceptions to normal DAT ordering rules. In other words 
> it is important that the delivery that confirms the RDMA 
> Write to the Data Sink be ordered exactly as if an RDMA Send 
> had been used to confirm delivery.
> 
> 
>