[dat-discussions] [openib-general] [RFC]DAT2.0immediatedataproposal

[Kanevsky, Arkady]

Roy,
and if tomorrow iWARP decides to support Immediate data with variable
length. API does not changes. Semantic does not changes and IB
will not be able to support it.

I am trying to define the semantic and API which will not have to be
modified for each rev of the transport.

Arkady Kanevsky                       email: arkady at netapp.com
Network Appliance Inc.               phone: 781-768-5395
1601 Trapelo Rd. - Suite 16.        Fax: 781-895-1195
Waltham, MA 02451                   central phone: 781-768-5300
 

> -----Original Message-----
> From: Larsen, Roy K [mailto:roy.k.larsen at intel.com] 
> Sent: Thursday, February 09, 2006 3:32 PM
> To: dat-discussions at yahoogroups.com; Arlin Davis; Roland Dreier
> Cc: openib-general at openib.org
> Subject: RE: [dat-discussions] [openib-general] 
> [RFC]DAT2.0immediatedataproposal
> 
> >>> Hmm.  Can you put a number on how much better RDMA write with 
> >>> immediate is on current HCA hardware?  How does using the 
> underlying 
> >>> OpenIB verbs ability to post a list of work requests compare (ie 
> >>> posting an RDMA write followed by a send in one verbs call)?
> >>> Maybe "post multiple" is a better direction for DAT.
> >>>
> >>>
> >> With post multiple, unlike immediate data, you don't have 
> the ability 
> >> to distinguish between a normal receive and a rdma write 
> completion 
> >> indication on the other end. This is the uniqueness of the service 
> >> that cannot be provided by the post multiple. Yes, post multiple 
> >> would be a nice option for DAT it is just a different service. It 
> >> would also be required to conform to the semantics rules of the 
> >> bundled operations so you could not do any optimization 
> tricks under 
> >> the covers with an IB rdma_write_immediate operation.
> >>
> >
> >A post_multiple also requires defining a single "DTO" data 
> structure. 
> >If the post multiple is atomic (meaning all make it or none 
> do) then it 
> >requires an intermediate data structure to have been 
> created. If it is 
> >not atomic there really isn't reason for it to not just be a utility 
> >function layered above DAT.
> 
> That is very good point.  And since the emulated immediate 
> data service can't make the atomic guarantee it is the killer 
> argument for just making the service plain - a potentially 
> more efficient write/send.
> 
> >
> >What I'm not seeing with the immediate is this urgent need by the 
> >application to be able to use the same 32-bit value for both an 
> >immediate and a 4 byte message that requires an entire 
> additional API 
> >just to support it.  Why can't the application just add a 
> bool to the 
> >send message?
> >Or encode the 32-bits so that they come from disjoint domains?
> 
> Some applications can do as you suggest.  Some applications 
> can make good use of unambiguous indications where the buffer 
> size, content, or arrival timing is not constrained.  Some 
> don't need write notification at all.  What's your point?
> 
> >
> >There seems to be agreement that a consolidated write-and-send call 
> >would enable the application to get the benefits of rdma write with 
> >immediate whenever the application could distinguish the two.
> 
> Well, I think there is agreement that *some* applications can 
> use write-and-send in a beneficial way.  But then again, 
> nothing prevents them from doing that now.  They do not need 
> an additional API.  But again, I don't have an issue with 
> defining a helper function.  I do have an issue with defining 
> an API and semantic that says the target side needs to be 
> coded in a way to always deal with both "true" immediate data 
> and emulation.  Just define a write/send helper API and the 
> UPL can be coded in a consistent manner if that is a 
> beneficial service.  If a true unambiguous indication service 
> is more beneficial or required, it can use the extension and 
> accept the extra complexity.  To demand extra complexity in 
> applications that obviously don't need the true immediate 
> data semantic is just wrong in my option.
> 
> >
> >I cannot see why doing this is almost free for virtually all 
> >applications, and trivial for the remainder. Adding and 
> documenting an 
> >extra call to deal with such an extreme corner case that is being 
> >presented only in the abstract is just not justified. This extra 
> >capability has to have enough functionality for enough 
> applications to 
> >justify keeping it on the books, writing test cases for it, etc.
> 
> All we're asking is that a write/send combined API not be 
> called immediate data unless it fits the semantics of 
> immediate data.  I am puzzled at the resistance this is 
> getting.  There is a standards body specification for 
> immediate data.  If it is not followed, don't call it 
> immediate data.  It's that simple.  For those transports that 
> can provide the service, the UPL may be able to gain access 
> to it through an extension.
> 
> Roy
> _______________________________________________
> openib-general mailing list
> openib-general at openib.org
> http://openib.org/mailman/listinfo/openib-general
> 
> To unsubscribe, please visit 
> http://openib.org/mailman/listinfo/openib-general
>