[dat-discussions] [openib-general] [RFC] DAT2.0immediatedataproposal

[Kanevsky, Arkady]

One more issue to discuss.
Does Completion of Recv that matches RDMA Write with Immediate Data
automatically sync local memory or Consumer still need to do
lmr_sync_rdma_write prior to accessing RDMAed data.

Arkady Kanevsky                       email: arkady at netapp.com
Network Appliance Inc.               phone: 781-768-5395
1601 Trapelo Rd. - Suite 16.        Fax: 781-895-1195
Waltham, MA 02451                   central phone: 781-768-5300
 

> -----Original Message-----
> From: Caitlin Bestler [mailto:caitlinb at broadcom.com] 
> Sent: Tuesday, February 07, 2006 7:40 PM
> To: dat-discussions at yahoogroups.com; Larsen, Roy K; Arlin 
> Davis; Hefty, Sean
> Cc: openib-general at openib.org
> Subject: RE: [dat-discussions] [openib-general] [RFC] 
> DAT2.0immediatedataproposal
> 
> dat-discussions at yahoogroups.com wrote:
> > We have problem no matter which option we choose.
> > The current Transport Level Requirement state:
> > 
> > There is a one-to-one correspondence between send operation on one 
> > Endpoint of the Connection and recv operations on the other 
> Endpoint 
> > of the Connection.
> > There is no correspondence between RDMA operations on one 
> Endpoint of 
> > the Connection and recv or send data transfer operation on 
> the other 
> > Endpoint of the Connection.
> > Receive operations on a Connection must be completed in the 
> order of 
> > posting of their corresponding sends.
> > 
> > The Immediate data and Atomic ops violate these 
> requirements including 
> > ordering rules.
> > 
> > I had started updating these rules when I generated the 
> first draft of 
> > the requirements. They are included in the enclosed pdf file.
> > But they do not cover Atomic ops that also impact transport 
> > requirements. This chapter of the spec have not been changed since 
> > DAPL 1.0 and I am very concern with any changes to it.
> > 
> > Arkady
> > 
> 
> If "RDMA Write with Immediate" is viewed as being the 
> equivalent of doing RDMA Write and then an RDMA Send the 
> correspondence rule is maintained. But *only* if the "rdma 
> write with immediate"
> has all of the semantics of a Send.
> 
> Atomics do not violate the rules if you view them as being a 
> variation on an RDMA Read. They are an RDMA Read with modify.
> The real question is whether it makes sense to put it in the 
> RDMA device. It is also not subject to emulation at a highe layer. 
> 
> With send with invalidate we know how InfiniBand *will* 
> support it, because of the IB 1.2 verbs. We do not know that 
> for atomics over iWARP. We do not know whether it will be 
> added, more importantly we do not know *how* it would be 
> added if it were added. That makes coming up with a transport 
> neutral definition very premature.
> In particular, if atomics were added to iWARP there is a 
> distinct design option where it would *not* be the same work 
> queue as RDMA Reads (adding atomics through Queue ID 3 would 
> make layering on top of a current implementation much easier. 
> But it would mean that atomic credits would be distinct from 
> read credits. This is a very strong reason to defer 
> attempting to define RDMA Atomics in a transport neutral fashion.
> 
>  
> 
> 
> 
> 
>  
> Yahoo! Groups Links
> 
> <*> To visit your group on the web, go to:
>     http://groups.yahoo.com/group/dat-discussions/
> 
> <*> To unsubscribe from this group, send an email to:
>     dat-discussions-unsubscribe at yahoogroups.com
> 
> <*> Your use of Yahoo! Groups is subject to:
>     http://docs.yahoo.com/info/terms/
>  
> 
>