<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<META http-equiv=Content-Type content="text/html; charset=us-ascii">
<META content="MSHTML 6.00.2900.2769" name=GENERATOR></HEAD>
<BODY>
<DIV dir=ltr align=left><FONT face=Arial color=#0000ff size=2><SPAN
class=947594820-09022006>Mike,</SPAN></FONT></DIV>
<DIV dir=ltr align=left><FONT face=Arial color=#0000ff size=2><SPAN
class=947594820-09022006>but then the combined operation can as easily be handle
by a "multiple post operation".</SPAN></FONT></DIV>
<DIV dir=ltr align=left><FONT face=Arial color=#0000ff size=2><SPAN
class=947594820-09022006>What is the need specific transport-independent RDMA
Write with immediate data.</SPAN></FONT></DIV>
<DIV dir=ltr align=left><FONT face=Arial color=#0000ff size=2><SPAN
class=947594820-09022006></SPAN></FONT> </DIV>
<DIV dir=ltr align=left><FONT face=Arial color=#0000ff size=2><SPAN
class=947594820-09022006>I am still concern over the need of Consumer Recv side
to separate recv of Immediate Data</SPAN></FONT></DIV>
<DIV dir=ltr align=left><FONT face=Arial color=#0000ff size=2><SPAN
class=947594820-09022006>from "regular" Recv. Consumer "knows" what it expect to
match the posted Recv.</SPAN></FONT></DIV>
<DIV dir=ltr align=left><FONT face=Arial color=#0000ff size=2><SPAN
class=947594820-09022006>There is one to one mapping between non-pure RDMA
transfer ops of one side with Recv</SPAN></FONT></DIV>
<DIV dir=ltr align=left><FONT face=Arial color=#0000ff size=2><SPAN
class=947594820-09022006>of another. Sure ULP may use the same size buffers for
all. But how many</SPAN></FONT></DIV>
<DIV dir=ltr align=left><FONT face=Arial color=#0000ff size=2><SPAN
class=947594820-09022006>ULPs mix the Immediate Data size messages ( 4 bytes on
IB ) with normal</SPAN></FONT></DIV>
<DIV dir=ltr align=left><FONT face=Arial color=#0000ff size=2><SPAN
class=947594820-09022006>Sends of the same exact size.</SPAN></FONT></DIV>
<DIV dir=ltr align=left><FONT face=Arial color=#0000ff size=2><SPAN
class=947594820-09022006></SPAN></FONT> </DIV>
<DIV dir=ltr align=left><FONT face=Arial color=#0000ff size=2><SPAN
class=947594820-09022006>Arkady</SPAN></FONT></DIV>
<DIV> </DIV><?xml:namespace prefix = o ns =
"urn:schemas-microsoft-com:office:office" /><o:SmartTagType name="Street"
namespaceuri="urn:schemas-microsoft-com:office:smarttags"></o:SmartTagType><o:SmartTagType
name="address"
namespaceuri="urn:schemas-microsoft-com:office:smarttags"></o:SmartTagType>
<STYLE>st1\:* {
BEHAVIOR: url(#ieooui)
}
</STYLE>
<STYLE>@page Section1 {size: 8.5in 11.0in; margin: 1.0in 1.25in 1.0in 1.25in; mso-header-margin: .5in; mso-footer-margin: .5in; mso-paper-source: 0; }
P.MsoNormal {
FONT-SIZE: 12pt; MARGIN: 0in 0in 0pt; FONT-FAMILY: "Times New Roman"; mso-style-parent: ""; mso-pagination: widow-orphan; mso-fareast-font-family: "Times New Roman"
}
LI.MsoNormal {
FONT-SIZE: 12pt; MARGIN: 0in 0in 0pt; FONT-FAMILY: "Times New Roman"; mso-style-parent: ""; mso-pagination: widow-orphan; mso-fareast-font-family: "Times New Roman"
}
DIV.MsoNormal {
FONT-SIZE: 12pt; MARGIN: 0in 0in 0pt; FONT-FAMILY: "Times New Roman"; mso-style-parent: ""; mso-pagination: widow-orphan; mso-fareast-font-family: "Times New Roman"
}
A:link {
COLOR: blue; TEXT-DECORATION: underline; text-underline: single
}
SPAN.MsoHyperlink {
COLOR: blue; TEXT-DECORATION: underline; text-underline: single
}
A:visited {
COLOR: purple; TEXT-DECORATION: underline; text-underline: single
}
SPAN.MsoHyperlinkFollowed {
COLOR: purple; TEXT-DECORATION: underline; text-underline: single
}
SPAN.SpellE {
mso-style-name: ""; mso-spl-e: yes
}
SPAN.GramE {
mso-style-name: ""; mso-gram-e: yes
}
DIV.Section1 {
page: Section1
}
</STYLE>
<DIV class=Section1>
<P class=MsoNormal align=left><SPAN class=SpellE><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">Arkady</SPAN></SPAN><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: Arial"> <SPAN
class=SpellE>Kanevsky</SPAN><SPAN
style="mso-tab-count: 2">
</SPAN>email: <A
href="mailto:arkady@netapp.com">arkady@netapp.com</A><o:p></o:p></SPAN></P>
<P class=MsoNormal><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">Network
Appliance Inc.<SPAN
style="mso-tab-count: 2">
</SPAN><SPAN class=GramE>phone</SPAN>: 781-768-5395<o:p></o:p></SPAN></P>
<P class=MsoNormal><?xml:namespace prefix = st1 ns =
"urn:schemas-microsoft-com:office:smarttags" /><st1:Street
w:st="on"><st1:address
style="BACKGROUND-POSITION: left bottom; BACKGROUND-IMAGE: url(res://ietag.dll/#34/#1001); BACKGROUND-REPEAT: repeat-x"
tabIndex=0 w:st="on"><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">1601
Trapelo Rd. - Suite 16.</SPAN></st1:address></st1:Street><SPAN
style="mso-tab-count: 2"> </SPAN><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">Fax:
781-895-1195<o:p></o:p></SPAN></P>
<P class=MsoNormal><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">Waltham, MA
02451
central phone: 781-768-5300</SPAN></P></DIV>
<DIV> </DIV><BR>
<BLOCKQUOTE
style="PADDING-LEFT: 5px; MARGIN-LEFT: 5px; BORDER-LEFT: #0000ff 2px solid; MARGIN-RIGHT: 0px">
<DIV class=OutlookMessageHeader lang=en-us dir=ltr align=left>
<HR tabIndex=-1>
<FONT face=Tahoma size=2><B>From:</B> Michael Krause
[mailto:krause@cup.hp.com] <BR><B>Sent:</B> Thursday, February 09, 2006 3:25
PM<BR><B>To:</B> Arlin Davis<BR><B>Cc:</B> dat-discussions@yahoogroups.com;
openib-general@openib.org<BR><B>Subject:</B> Re: [dat-discussions]
[openib-general][RFC] DAT2.0immediatedataproposal<BR></FONT><BR></DIV>
<DIV></DIV><FONT size=3>At 03:36 PM 2/8/2006, Arlin Davis wrote:<BR>
<BLOCKQUOTE class=cite cite="" type="cite">Roland Dreier wrote:<BR><BR>
<BLOCKQUOTE class=cite cite="" type="cite"> Michael> So,
here we have a long discussion on attempting to<BR>
Michael> perpetuate a concept that is not universal
across<BR> Michael> transports and was deemed to have
minimal value that most<BR> Michael> wanted to see removed
from the architecture.<BR><BR>But this discussion is being driven by an
application developer who<BR>does see value in immediate
data.<BR><BR>Arlin, can you quantify the benefit you see from RDMA write
with<BR>immediate vs. RDMA write followed by a
send?<BR><BR> <BR></BLOCKQUOTE>We need speed and simplicity.<BR><BR>A
very latency sensitive application that requires immediate notification of
RDMA write completion on the remote node without ANY latency penalties
associated with combining operations, HCA priority rules across QPs, wire
congestion, etc. An application that has no requirement for messaging
outside of remote rdma write completion notifications. The application would
not have to register and manage additional message buffers on either side,
we can just size the queues accordingly and post zero byte messages. We need
something that would be equivelent to setting there polling on the last byte
of inbound data. But, since data ordering within an operation is not
guaranteed that is not an option. So, rdma with immediate data is the most
optimal and simplistic method for indication of RDMA-write completion that
we have available today. In fact, I would like to see it increased in size
to make it even more useful.</BLOCKQUOTE><BR>RDMA Write with Immediate is part
of the IB Extended Transport Header. It is a fixed-sized quantity and
not one subject to change, i.e. increasing its size.<BR><BR>Your argument
above reinforces that the particular application need is IB-specific and thus
should not be part of a general API but a transport-specific API.
If the application will only operate optimally using immediate data, then it
is only suitable for an IB fabric. This reinforces the need for a
transport-specific API.<BR><BR>Those applications that simply want to enable
completion notification when a RDMA Write has occurred can use a general
purpose API that is interconnect independent and whose code is predicated upon
a RDMA Write - Send set of operations. This will enable application
portability across all interconnect types.<BR><BR>Mike</FONT>
</BLOCKQUOTE></BODY></HTML>