[dat-discussions] [openib-general][RFC] DAT2.0immediatedataproposal

[Kanevsky, Arkady]

Mike,
but then the combined operation can as easily be handle by a "multiple
post operation".
What is the need specific transport-independent RDMA Write with
immediate data.
 
I am still concern over the need of Consumer Recv side to separate recv
of Immediate Data
from "regular" Recv. Consumer "knows" what it expect to match the posted
Recv.
There is one to one mapping between non-pure RDMA transfer ops of one
side with Recv
of another. Sure ULP may use the same size buffers for all. But how many
ULPs mix the Immediate Data size messages ( 4 bytes on IB ) with normal
Sends of the same exact size.
 
Arkady
 

Arkady Kanevsky                       email: arkady at netapp.com

Network Appliance Inc.               phone: 781-768-5395

1601 Trapelo Rd. - Suite 16.        Fax: 781-895-1195

Waltham, MA 02451                   central phone: 781-768-5300

 


________________________________

	From: Michael Krause [mailto:krause at cup.hp.com] 
	Sent: Thursday, February 09, 2006 3:25 PM
	To: Arlin Davis
	Cc: dat-discussions at yahoogroups.com; openib-general at openib.org
	Subject: Re: [dat-discussions] [openib-general][RFC]
DAT2.0immediatedataproposal
	
	
	At 03:36 PM 2/8/2006, Arlin Davis wrote:
	

		Roland Dreier wrote:
		
		

			   Michael> So, here we have a long discussion
on attempting to
			   Michael> perpetuate a concept that is not
universal across
			   Michael> transports and was deemed to have
minimal value that most
			   Michael> wanted to see removed from the
architecture.
			
			But this discussion is being driven by an
application developer who
			does see value in immediate data.
			
			Arlin, can you quantify the benefit you see from
RDMA write with
			immediate vs. RDMA write followed by a send?
			
			 
			

		We need speed and simplicity.
		
		A very latency sensitive application that requires
immediate notification of RDMA write completion on the remote node
without ANY latency penalties associated with combining operations, HCA
priority rules across QPs, wire congestion, etc. An application that has
no requirement for messaging outside of remote rdma write completion
notifications. The application would not have to register and manage
additional message buffers on either side, we can just size the queues
accordingly and post zero byte messages. We need something that would be
equivelent to setting there polling on the last byte of inbound data.
But, since data ordering within an operation is not guaranteed that is
not an option. So, rdma with immediate data is the most optimal and
simplistic method for indication of RDMA-write completion that we have
available today. In fact, I would like to see it increased in size to
make it even more useful.


	RDMA Write with Immediate is part of the IB Extended Transport
Header.  It is a fixed-sized quantity and not one subject to change,
i.e. increasing its size.
	
	Your argument above reinforces that the particular application
need is IB-specific and thus should not be part of a general API but a
transport-specific API.   If the application will only operate optimally
using immediate data, then it is only suitable for an IB fabric.  This
reinforces the need for a transport-specific API.
	
	Those applications that simply want to enable completion
notification when a RDMA Write has occurred can use a general purpose
API that is interconnect independent and whose code is predicated upon a
RDMA Write - Send set of operations.  This will enable application
portability across all interconnect types.
	
	Mike 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/general/attachments/20060209/1ae21682/attachment.html>