[openib-general] RE: [RFC] DAT 2.0 immediate data proposal

[Kanevsky, Arkady]

Arlin,
I am not convinced we need a new recv for immediate data.
But what is needed is change in normative text in many places.
Recv, RDMA Write, DTO completion events, error behavior.
Sure you can define immed data in extension but it still effects
behavior of the normative part of the spec.
This is why my preference is to put it into the main spec.

The xfer_size is minor thing. We just need to define it meaning
with respect to immed_data. Defining it either way is fine.

Handling extra space on CQ can be handled by Provider.
We can add a new EVD attribute for the use for handling RDMA_write with
immed
data and Provider can automatically add extra space on CQ.
Provider is already responsible to handing user a single completion.
SO it will only be used for error handling.

Error handling takes maost of the new write up anyhow.
Regardless where it is done in the spec or in extension.

Question on do we want to support Send with immed_data have to be
decided.
Ditto remote RMR invalidation with new post(s) for immed_data.
Just because IB supports all possible correlation under one Send post
does not mean that uDAPL should follow that too.

Arkady

Arkady Kanevsky                       email: arkady at netapp.com
Network Appliance Inc.               phone: 781-768-5395
1601 Trapelo Rd. - Suite 16.        Fax: 781-895-1195
Waltham, MA 02451                   central phone: 781-768-5300
 

> -----Original Message-----
> From: Arlin Davis [mailto:arlin.r.davis at intel.com] 
> Sent: Thursday, January 26, 2006 3:02 PM
> To: Kanevsky, Arkady; Arlin Davis; Caitlin Bestler
> Cc: Lentini, James; dat-discussions at yahoogroups.com; 
> openib-general at openib.org
> Subject: RE: [openib-general] RE: [RFC] DAT 2.0 immediate 
> data proposal
> 
> 
> >But this penalizes user which need to deal with 2 way to 
> deal with post 
> >calls and completions.
> >
> >I do not think we are not to far from consensus.
> >Transport independent App will allocate 4 bytes extra for 
> buffers that 
> >can match immediate data.
> >Completion data will return where the immediate data is return 
> >(Consumer can not request it on posting), and 4 bytes for immediate 
> >data in completion event.
> >The rest are ironing details for complete specification.
> >This is no different than for any other new functionality proposed.
> >And except for wasting 4 bytes per buffer or completion I do not see 
> >how it penalizes IB. Moreover if Apps knows that Provider returns 
> >immediate data in completion event it can avoid any penalty.
> 
> There is no penalty to the user if you just provide native 
> features via extensions. Your extension
> will provide the best possible interface for your native 
> capabilities.   
>  
> I think we are further from consensus then we first thought:
> 
> Right now we have a new post recv, different delivery 
> mechanisms, and a requirement to allocate an extra 4 bytes of 
> user data. 
> 
> The only requirement to support immediate data on IB, is a 
> new post send and write immediate data calls and a new event 
> data construct. The normal post_recv can be used unchanged 
> and can already process normal and immediate data. No 
> requirement on the user to allocate and manage an extra 4 
> bytes in the receive buffer. In fact, you can post receive 
> with no buffer.
> 
> In order to support immediate data via iWARP, you now have a 
> requirement to use a special new receive post, new user 
> buffer constructs to place the data, and new delivery method 
> that has to be checked via provider attributes or at event time. 
> 
> Is there anyway to get this closer? If not, I would recommend 
> going back to an extension interface for immediate data. 
> 
> -arlin
> 
>