<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML xmlns="http://www.w3.org/TR/REC-html40" xmlns:v =
"urn:schemas-microsoft-com:vml" xmlns:o =
"urn:schemas-microsoft-com:office:office" xmlns:w =
"urn:schemas-microsoft-com:office:word"><HEAD>
<META http-equiv=Content-Type content="text/html; charset=us-ascii">
<META content="MSHTML 6.00.2900.2769" name=GENERATOR><!--[if !mso]>
<STYLE>v\:* {
BEHAVIOR: url(#default#VML)
}
o\:* {
BEHAVIOR: url(#default#VML)
}
w\:* {
BEHAVIOR: url(#default#VML)
}
.shape {
BEHAVIOR: url(#default#VML)
}
</STYLE>
<![endif]-->
<STYLE>@font-face {
font-family: Tahoma;
}
@page Section1 {size: 8.5in 11.0in; margin: 1.0in 1.25in 1.0in 1.25in; }
st1\:* {
BEHAVIOR: url(#ieooui)
}
P.MsoNormal {
FONT-SIZE: 12pt; MARGIN: 0in 0in 0pt; FONT-FAMILY: "Times New Roman"
}
LI.MsoNormal {
FONT-SIZE: 12pt; MARGIN: 0in 0in 0pt; FONT-FAMILY: "Times New Roman"
}
DIV.MsoNormal {
FONT-SIZE: 12pt; MARGIN: 0in 0in 0pt; FONT-FAMILY: "Times New Roman"
}
A:link {
COLOR: blue; TEXT-DECORATION: underline
}
SPAN.MsoHyperlink {
COLOR: blue; TEXT-DECORATION: underline
}
A:visited {
COLOR: purple; TEXT-DECORATION: underline
}
SPAN.MsoHyperlinkFollowed {
COLOR: purple; TEXT-DECORATION: underline
}
SPAN.EmailStyle17 {
COLOR: windowtext; FONT-FAMILY: Arial; mso-style-type: personal
}
SPAN.EmailStyle18 {
COLOR: navy; FONT-FAMILY: Arial; mso-style-type: personal-reply
}
DIV.Section1 {
page: Section1
}
</STYLE>
<!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></HEAD>
<BODY lang=EN-US vLink=purple link=blue>
<DIV dir=ltr align=left><SPAN class=414341403-24012006><FONT face=Arial
color=#0000ff size=2>Arlin,</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=414341403-24012006><FONT face=Arial
color=#0000ff size=2>comments inline.</FONT></SPAN></DIV>
<DIV> </DIV><o:SmartTagType name="Street"
namespaceuri="urn:schemas-microsoft-com:office:smarttags"></o:SmartTagType><o:SmartTagType
name="address"
namespaceuri="urn:schemas-microsoft-com:office:smarttags"></o:SmartTagType>
<STYLE>st1\:* {
BEHAVIOR: url(#ieooui)
}
</STYLE>
<STYLE>@page Section1 {size: 8.5in 11.0in; margin: 1.0in 1.25in 1.0in 1.25in; mso-header-margin: .5in; mso-footer-margin: .5in; mso-paper-source: 0; }
P.MsoNormal {
FONT-SIZE: 12pt; MARGIN: 0in 0in 0pt; FONT-FAMILY: "Times New Roman"; mso-style-parent: ""; mso-pagination: widow-orphan; mso-fareast-font-family: "Times New Roman"
}
LI.MsoNormal {
FONT-SIZE: 12pt; MARGIN: 0in 0in 0pt; FONT-FAMILY: "Times New Roman"; mso-style-parent: ""; mso-pagination: widow-orphan; mso-fareast-font-family: "Times New Roman"
}
DIV.MsoNormal {
FONT-SIZE: 12pt; MARGIN: 0in 0in 0pt; FONT-FAMILY: "Times New Roman"; mso-style-parent: ""; mso-pagination: widow-orphan; mso-fareast-font-family: "Times New Roman"
}
A:link {
COLOR: blue; TEXT-DECORATION: underline; text-underline: single
}
SPAN.MsoHyperlink {
COLOR: blue; TEXT-DECORATION: underline; text-underline: single
}
A:visited {
COLOR: purple; TEXT-DECORATION: underline; text-underline: single
}
SPAN.MsoHyperlinkFollowed {
COLOR: purple; TEXT-DECORATION: underline; text-underline: single
}
SPAN.SpellE {
mso-style-name: ""; mso-spl-e: yes
}
SPAN.GramE {
mso-style-name: ""; mso-gram-e: yes
}
DIV.Section1 {
page: Section1
}
</STYLE>
<DIV class=Section1>
<P class=MsoNormal align=left><SPAN class=SpellE><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">Arkady</SPAN></SPAN><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: Arial"> <SPAN
class=SpellE>Kanevsky</SPAN><SPAN
style="mso-tab-count: 2">
</SPAN>email: <A
href="mailto:arkady@netapp.com">arkady@netapp.com</A><o:p></o:p></SPAN></P>
<P class=MsoNormal><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">Network
Appliance Inc.<SPAN
style="mso-tab-count: 2">
</SPAN><SPAN class=GramE>phone</SPAN>: 781-768-5395<o:p></o:p></SPAN></P>
<P class=MsoNormal><?xml:namespace prefix = st1 ns =
"urn:schemas-microsoft-com:office:smarttags" /><st1:Street
w:st="on"><st1:address
style="BACKGROUND-POSITION: left bottom; BACKGROUND-IMAGE: url(res://ietag.dll/#34/#1001); BACKGROUND-REPEAT: repeat-x"
tabIndex=0 w:st="on"><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">1601
Trapelo Rd. - Suite 16.</SPAN></st1:address></st1:Street><SPAN
style="mso-tab-count: 2"> </SPAN><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">Fax:
781-895-1195<o:p></o:p></SPAN></P>
<P class=MsoNormal><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">Waltham, MA
02451
central phone: 781-768-5300</SPAN></P></DIV>
<DIV> </DIV><BR>
<BLOCKQUOTE dir=ltr
style="PADDING-LEFT: 5px; MARGIN-LEFT: 5px; BORDER-LEFT: #0000ff 2px solid; MARGIN-RIGHT: 0px">
<DIV class=OutlookMessageHeader lang=en-us dir=ltr align=left>
<HR tabIndex=-1>
<FONT face=Tahoma size=2><B>From:</B> Davis, Arlin R
[mailto:arlin.r.davis@intel.com] <BR><B>Sent:</B> Monday, January 23, 2006
7:15 PM<BR><B>To:</B> Kanevsky, Arkady; Lentini, James<BR><B>Cc:</B>
openib-general@openib.org; dat-discussions@yahoogroups.com<BR><B>Subject:</B>
RE: [RFC] DAT 2.0 immediate data proposal<BR></FONT><BR></DIV>
<DIV></DIV>
<DIV class=Section1>
<P class=MsoNormal><B><FONT face=Arial color=navy size=2><SPAN
style="FONT-WEIGHT: bold; FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial">Arkady,<o:p></o:p></SPAN></FONT></B></P>
<P class=MsoNormal><B><FONT face=Arial color=navy size=2><SPAN
style="FONT-WEIGHT: bold; FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial"><o:p> </o:p></SPAN></FONT></B></P>
<P class=MsoNormal><B><FONT face=Arial color=navy size=2><SPAN
style="FONT-WEIGHT: bold; FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial">Response
inline…<o:p></o:p></SPAN></FONT></B></P>
<P class=MsoNormal><FONT face=Arial color=navy size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial"><o:p> </o:p></SPAN></FONT></P>
<DIV
style="BORDER-RIGHT: medium none; PADDING-RIGHT: 0in; BORDER-TOP: medium none; PADDING-LEFT: 4pt; PADDING-BOTTOM: 0in; BORDER-LEFT: blue 1.5pt solid; PADDING-TOP: 0in; BORDER-BOTTOM: medium none">
<DIV>
<DIV class=MsoNormal style="TEXT-ALIGN: center" align=center><FONT
face="Times New Roman" size=3><SPAN style="FONT-SIZE: 12pt">
<HR tabIndex=-1 align=center width="100%" SIZE=2>
</SPAN></FONT></DIV>
<P class=MsoNormal><B><FONT face=Tahoma size=2><SPAN
style="FONT-WEIGHT: bold; FONT-SIZE: 10pt; FONT-FAMILY: Tahoma">From:</SPAN></FONT></B><FONT
face=Tahoma size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Tahoma">
Kanevsky, Arkady [mailto:Arkady.Kanevsky@netapp.com] <BR><B><SPAN
style="FONT-WEIGHT: bold">Sent:</SPAN></B> Tuesday, January 17, 2006 7:16
AM<BR><B><SPAN style="FONT-WEIGHT: bold">To:</SPAN></B> Davis, Arlin R;
Lentini, James<BR><B><SPAN style="FONT-WEIGHT: bold">Cc:</SPAN></B>
dat-discussions@yahoogroups.com; openib-general@openib.org<BR><B><SPAN
style="FONT-WEIGHT: bold">Subject:</SPAN></B> RE: [RFC] DAT 2.0 immediate data
proposal</SPAN></FONT><o:p></o:p></P></DIV>
<P class=MsoNormal><FONT face="Times New Roman" size=3><SPAN
style="FONT-SIZE: 12pt"><o:p> </o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial color=blue size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: Arial">Arlin,</SPAN></FONT><o:p></o:p></P>
<P class=MsoNormal><FONT face=Arial color=blue size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: Arial">a few things need to
be addressed.</SPAN></FONT><o:p></o:p></P>
<P class=MsoNormal><FONT face="Times New Roman" size=3><SPAN
style="FONT-SIZE: 12pt"> <o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial color=blue size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: Arial">1. correlation with
local and remote invalidate</SPAN></FONT><o:p></o:p></P>
<P class=MsoNormal><FONT face=Arial color=blue size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: Arial">This potentially
effects both DAT_DTOs and post operations</SPAN></FONT><FONT face=Arial
size=2><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: Arial"><o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><B><I><FONT face=Arial color=navy size=2><SPAN
style="FONT-WEIGHT: bold; FONT-SIZE: 10pt; COLOR: navy; FONT-STYLE: italic; FONT-FAMILY: Arial"><o:p> </o:p></SPAN></FONT></I></B></P>
<P class=MsoNormal><B><I><FONT face=Arial color=navy size=2><SPAN
style="FONT-WEIGHT: bold; FONT-SIZE: 10pt; COLOR: navy; FONT-STYLE: italic; FONT-FAMILY: Arial"><o:p> </o:p></SPAN></FONT></I></B></P>
<P class=MsoNormal><FONT color=navy><SPAN
style="FONT-WEIGHT: bold; FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial">How
does this differ from normal sends or writes?<BR><FONT color=#0000ff><SPAN
class=414341403-24012006>[AK] We had added a new Send_with_Invalidate.
The completion also states</SPAN></FONT></SPAN></FONT></P>
<P class=MsoNormal><FONT color=navy><SPAN
style="FONT-WEIGHT: bold; FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial"><FONT
color=#0000ff><SPAN class=414341403-24012006>whether RMR was invalidated and
which one. But the text for interaction</SPAN></FONT></SPAN></FONT></P>
<P class=MsoNormal><FONT color=navy><SPAN
style="FONT-WEIGHT: bold; FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial"><FONT
color=#0000ff><SPAN class=414341403-24012006>is added through out the
completion and post operations.</SPAN></FONT></SPAN></FONT></P>
<P class=MsoNormal><FONT color=navy><SPAN
style="FONT-WEIGHT: bold; FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial"><FONT
color=#0000ff><SPAN class=414341403-24012006>See the latest draft of uDAPL and
kDAPL 1.3 specs on the DAT reflector.</SPAN></FONT></SPAN></FONT></P>
<P class=MsoNormal><B><I><FONT face=Arial color=navy size=2><SPAN
style="FONT-WEIGHT: bold; FONT-SIZE: 10pt; COLOR: navy; FONT-STYLE: italic; FONT-FAMILY: Arial"> </SPAN></FONT></I></B><FONT
face=Arial color=navy size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial"><o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face="Times New Roman" size=3><SPAN
style="FONT-SIZE: 12pt"> <o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial color=blue size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: Arial">2. Need a precise
defintion for CONFIRM_FLAG definition in a transport independent
fashion.</SPAN></FONT><o:p></o:p></P>
<P class=MsoNormal><FONT face=Arial color=blue size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: Arial">What guarantees DAT
Provider "provides" on successful local
completion?</SPAN></FONT><o:p></o:p></P>
<P class=MsoNormal><FONT face=Arial color=blue size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: Arial">Remote end
guarantee?</SPAN></FONT><FONT face=Arial size=2><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: Arial"><o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><B><I><FONT face=Arial color=navy size=2><SPAN
style="FONT-WEIGHT: bold; FONT-SIZE: 10pt; COLOR: navy; FONT-STYLE: italic; FONT-FAMILY: Arial"><o:p> </o:p></SPAN></FONT></I></B></P>
<P class=MsoNormal><FONT face=Arial color=blue size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: Arial">My understanding what
you are trying to do is create 2 models one IB and one for
iWARP.</SPAN></FONT><o:p></o:p></P>
<P class=MsoNormal><FONT face=Arial color=blue size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: Arial">So for IB Consumers
will use CONFIRM_FLAG and for iWARP IMMED_FLAG.</SPAN></FONT><o:p></o:p></P>
<P class=MsoNormal><FONT face=Arial color=blue size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: Arial">Provider will
indicate in Provider_attr which model it
supports.</SPAN></FONT><o:p></o:p></P>
<P class=MsoNormal><FONT face="Times New Roman" size=3><SPAN
style="FONT-SIZE: 12pt"> <o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial color=blue size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: Arial">The issue I have with
it is that I do not see a model that Consumer can use to
create</SPAN></FONT><o:p></o:p></P>
<P class=MsoNormal><FONT face=Arial color=blue size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: Arial">a transport
independent code.</SPAN></FONT><o:p></o:p></P>
<P class=MsoNormal><FONT face=Arial color=blue size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: Arial">It looks like
Immed_flag can be made transport independent. But with "sender"
specifying</SPAN></FONT><o:p></o:p></P>
<P class=MsoNormal><FONT face=Arial color=blue size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: Arial">the behavior a
protocol extension is needed for IB. IB will always deliver Immediate
data</SPAN></FONT><o:p></o:p></P>
<P class=MsoNormal><FONT face=Arial color=blue size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: Arial">in the header not a
payload and remote Provider can control how it is delivered to a
Consumer.</SPAN></FONT><o:p></o:p></P>
<P class=MsoNormal><FONT face=Arial color=blue size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: Arial">But this means that
there is no need for DTO_flags for Send side. Instead it can
be</SPAN></FONT><o:p></o:p></P>
<P class=MsoNormal><FONT face=Arial color=blue size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: Arial">used for Recv side or
controlled purely by Provider.</SPAN></FONT><o:p></o:p></P>
<P class=MsoNormal><FONT face="Times New Roman" size=3><SPAN
style="FONT-SIZE: 12pt"> <o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><B><FONT face=Arial color=navy size=2><SPAN
style="FONT-WEIGHT: bold; FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial">Maybe
we need to just go back to one model and always deliver via the event? With
the post_recv_immed requirements, other transports have a mechanism to emulate
and create the necessary resources on the recv side to place idata and copy to
event when operation is completed. Would this work for
iWARP?<o:p></o:p></SPAN></FONT></B></P>
<P class=MsoNormal><B><FONT face=Arial color=navy size=2><SPAN
style="FONT-WEIGHT: bold; FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial"><o:p> </o:p></SPAN></FONT></B></P>
<P class=MsoNormal><FONT color=navy><SPAN
style="FONT-WEIGHT: bold; FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial">Two
different models for receiving idata should be avoided if at all
possible.<BR><FONT color=#0000ff><SPAN
class=414341403-24012006>[AK] Caitlin already responded to
this. </SPAN><o:p></o:p></FONT></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial color=navy size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial"><o:p> </o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial color=blue size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: Arial">3. Need to define
error behavior. for new operations, async errors, EP
behavior.</SPAN></FONT><FONT face=Arial size=2><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: Arial"><o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial color=navy size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial"><o:p> </o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT color=navy><SPAN
style="FONT-WEIGHT: bold; FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial">I
will work on updating the draft. post_send_immed will look much like post_send
and post_rdma_write_immed will look a lot like post_rdma_write with some
additional errors based on the post receive buffer
requirement. <BR><SPAN class=414341403-24012006><FONT
color=#0000ff>[AK] Also consider if you want to add remote
invalidate to the new operation. </FONT></SPAN></SPAN></FONT><FONT
color=navy><SPAN
style="FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial"><o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face="Times New Roman" size=3><SPAN
style="FONT-SIZE: 12pt"> <o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial color=blue size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: Arial">4. Need to define
DAT_Provider attributes for immediate data and dto_flags
behavior</SPAN></FONT><FONT face=Arial size=2><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: Arial"><o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial color=navy size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial"><o:p> </o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face="Times New Roman" size=3><SPAN
style="FONT-SIZE: 12pt"> <o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial color=blue size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: Arial">5. Does
Solicited_wait completion_flag value now applicable for RDMA_write for
immediate data?</SPAN></FONT><o:p></o:p></P>
<P class=MsoNormal><FONT face="Times New Roman" color=navy size=3><SPAN
style="FONT-SIZE: 12pt; COLOR: navy"><o:p> </o:p></SPAN></FONT></P>
<P class=MsoNormal><B><FONT face=Arial color=navy size=2><SPAN
style="FONT-WEIGHT: bold; FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial">yes,
applicable to send, send_immed, and
write_immed<o:p></o:p></SPAN></FONT></B></P>
<P class=MsoNormal><FONT face="Times New Roman" size=3><SPAN
style="FONT-SIZE: 12pt"> <o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial color=blue size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: Arial">6. Is
dto_completion_data xfer_length include immediate_data size or
not?</SPAN></FONT><FONT face=Arial size=2><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: Arial"><o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><B><I><FONT face=Arial color=navy size=2><SPAN
style="FONT-WEIGHT: bold; FONT-SIZE: 10pt; COLOR: navy; FONT-STYLE: italic; FONT-FAMILY: Arial"><o:p> </o:p></SPAN></FONT></I></B></P>
<P class=MsoNormal><FONT color=navy><SPAN
style="FONT-WEIGHT: bold; FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial">no<BR><SPAN
class=414341403-24012006><FONT color=#0000ff>[AK] It can work both ways.
Either we include 4 extra bytes for immediate data or
not.</FONT></SPAN></SPAN></FONT></P>
<P class=MsoNormal><FONT color=#0000ff><SPAN
style="FONT-WEIGHT: bold; FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial"><SPAN
class=414341403-24012006>Consumer just have to know. The real data
always starts at 4 byte boundary into the buffer</SPAN></SPAN></FONT></P>
<P class=MsoNormal><FONT color=#0000ff><SPAN
style="FONT-WEIGHT: bold; FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial"><SPAN
class=414341403-24012006>is immediate data is returned inline. We need to
state how immediate data is positioned</SPAN></SPAN></FONT></P>
<P class=MsoNormal><FONT color=#0000ff><SPAN
style="FONT-WEIGHT: bold; FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial"><SPAN
class=414341403-24012006>if it is smaller than 4
bytes.</SPAN></SPAN></FONT><FONT color=navy><SPAN
style="FONT-WEIGHT: bold; FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial"><SPAN
class=414341403-24012006> </SPAN><BR></SPAN></FONT><FONT
face="Times New Roman" size=3><SPAN
style="FONT-SIZE: 12pt"> <o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial color=blue size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: Arial">7. what memory
privilages needed for a recv buffer for immediate data?</SPAN></FONT><FONT
face=Arial size=2><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: Arial"><o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial color=navy size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial"><o:p> </o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT color=navy><SPAN
style="FONT-WEIGHT: bold; FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial">Based
on the operation… write_immed would require write privileges and send_immed
would require recv privileges.<FONT color=#0000ff><SPAN
class=414341403-24012006> </SPAN><o:p></o:p></FONT></SPAN></FONT></P>
<P class=MsoNormal><FONT face="Times New Roman" size=3><SPAN
style="FONT-SIZE: 12pt"> <o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial color=blue size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: Arial">8. SRQ
interaction?</SPAN></FONT><FONT face=Arial size=2><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: Arial"><o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><B><I><FONT face=Arial color=navy size=2><SPAN
style="FONT-WEIGHT: bold; FONT-SIZE: 10pt; COLOR: navy; FONT-STYLE: italic; FONT-FAMILY: Arial"><o:p> </o:p></SPAN></FONT></I></B></P>
<P class=MsoNormal><FONT color=navy><SPAN
style="FONT-WEIGHT: bold; FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial">Good
question. all post_recv_immed or all post_recv? <BR><FONT
color=#0000ff><SPAN class=414341403-24012006>[AK] Will this work for the
user model? Not supporting handling immediate recv and regular recv with
potential immediate data on one
SRQ. </SPAN><o:p></o:p></FONT></SPAN></FONT></P>
<P class=MsoNormal><FONT face="Times New Roman" size=3><SPAN
style="FONT-SIZE: 12pt"> <o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial color=blue size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: Arial">9. What happens of
buffer for recv operation NOT recv_immed is matched for incomming
recv/rdma_write op?</SPAN></FONT><FONT face=Arial size=2><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: Arial"><o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><B><I><FONT face=Arial color=navy size=2><SPAN
style="FONT-WEIGHT: bold; FONT-SIZE: 10pt; COLOR: navy; FONT-STYLE: italic; FONT-FAMILY: Arial"><o:p> </o:p></SPAN></FONT></I></B></P>
<P class=MsoNormal><B><FONT face=Arial color=navy size=2><SPAN
style="FONT-WEIGHT: bold; FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial">The
rules should be:<o:p></o:p></SPAN></FONT></B></P>
<P class=MsoNormal><B><FONT face=Arial color=navy size=2><SPAN
style="FONT-WEIGHT: bold; FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial">Can
receive a send, send_immed, or write_immed with recv_immed.
<o:p></o:p></SPAN></FONT></B></P>
<P class=MsoNormal><B><FONT face=Arial color=navy size=2><SPAN
style="FONT-WEIGHT: bold; FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial">Cannot
receive send_immed or write_immed on a recv.<o:p></o:p></SPAN></FONT></B></P>
<P class=MsoNormal><B><FONT face=Arial color=navy size=2><SPAN
style="FONT-WEIGHT: bold; FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial"><o:p> </o:p></SPAN></FONT></B></P>
<P class=MsoNormal><FONT color=navy><SPAN
style="FONT-WEIGHT: bold; FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial">However,
I am not sure how you would enforce this on IB (DTO error on the receiving
side?) since the idata is delivered via CQ and does not require a special
receive post descriptor. <BR><FONT
color=#0000ff><SPAN class=414341403-24012006>[AK] We can make
this Provider attribute. Or we can state that if immed data is return in
event</SPAN></FONT></SPAN></FONT></P>
<P class=MsoNormal><FONT color=navy><SPAN
style="FONT-WEIGHT: bold; FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial"><FONT
color=#0000ff><SPAN class=414341403-24012006>then there is no error for
recv. </SPAN><o:p></o:p></FONT></SPAN></FONT></P>
<P class=MsoNormal><FONT face="Times New Roman" size=3><SPAN
style="FONT-SIZE: 12pt"> <o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial color=blue size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: Arial">10. Change
dat_ep_post_write_immed to dat_ep_post_rdma_write_immed to be consistent with
current</SPAN></FONT><o:p></o:p></P>
<P class=MsoNormal><FONT face=Arial color=blue size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: Arial">terminology.</SPAN></FONT><o:p></o:p></P>
<P class=MsoNormal><FONT face="Times New Roman" size=3><SPAN
style="FONT-SIZE: 12pt"> <o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><B><FONT face=Arial color=navy size=2><SPAN
style="FONT-WEIGHT: bold; FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial">Ok<o:p></o:p></SPAN></FONT></B></P>
<P class=MsoNormal><FONT face=Arial color=navy size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial"><o:p> </o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial color=blue size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: Arial">11. Need to cleanup
operation description to make it clear that Send|RDMA_write and immediate data
part</SPAN></FONT><o:p></o:p></P>
<P class=MsoNormal><FONT face=Arial color=blue size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: Arial">is a single atomic
operation. The current "followed by" language is
misleading.</SPAN></FONT><o:p></o:p></P>
<P class=MsoNormal><FONT face=Arial color=blue size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: Arial">Make it explicit that
there is a single local DTO completion and single remote DTO
completion.</SPAN></FONT><FONT face=Arial size=2><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: Arial"><o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><B><I><FONT face=Arial color=navy size=2><SPAN
style="FONT-WEIGHT: bold; FONT-SIZE: 10pt; COLOR: navy; FONT-STYLE: italic; FONT-FAMILY: Arial"><o:p> </o:p></SPAN></FONT></I></B></P>
<P class=MsoNormal><B><FONT face=Arial color=navy size=2><SPAN
style="FONT-WEIGHT: bold; FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial">Ok,
I will clean that up<o:p></o:p></SPAN></FONT></B></P>
<P class=MsoNormal><FONT face="Times New Roman" size=3><SPAN
style="FONT-SIZE: 12pt"> <o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial color=blue size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: Arial">12. Is your intension
that post_recv_immed can ONLY except immediate data and is
not</SPAN></FONT><o:p></o:p></P>
<P class=MsoNormal><FONT face=Arial color=blue size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: Arial">capable to recv any
message?</SPAN></FONT><FONT face=Arial size=2><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: Arial"><o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><B><I><FONT face=Arial color=navy size=2><SPAN
style="FONT-WEIGHT: bold; FONT-SIZE: 10pt; COLOR: navy; FONT-STYLE: italic; FONT-FAMILY: Arial"><o:p> </o:p></SPAN></FONT></I></B></P>
<P class=MsoNormal><B><FONT face=Arial color=navy size=2><SPAN
style="FONT-WEIGHT: bold; FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial">No,
the intention is to extend the post_recv to handle 32bit idata which may
arrive with or without other send or rdma_write
data.<o:p></o:p></SPAN></FONT></B></P>
<P class=MsoNormal><B><FONT face=Arial color=navy size=2><SPAN
style="FONT-WEIGHT: bold; FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial"><o:p> </o:p></SPAN></FONT></B></P>
<P class=MsoNormal><B><FONT face=Arial color=navy size=2><SPAN
style="FONT-WEIGHT: bold; FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial">Does
it make more sense to add a dto_flags to the existing
post_recv?<o:p></o:p></SPAN></FONT></B></P>
<P class=MsoNormal><FONT face="Times New Roman" size=3><SPAN
style="FONT-SIZE: 12pt"> <o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial color=blue size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: Arial">13. size should be
num_segments for dat_ep_post_recv_immed()</SPAN></FONT><FONT face=Arial
size=2><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: Arial"><o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><B><I><FONT face=Arial color=navy size=2><SPAN
style="FONT-WEIGHT: bold; FONT-SIZE: 10pt; COLOR: navy; FONT-STYLE: italic; FONT-FAMILY: Arial"><o:p> </o:p></SPAN></FONT></I></B></P>
<P class=MsoNormal><B><FONT face=Arial color=navy size=2><SPAN
style="FONT-WEIGHT: bold; FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial">ok</SPAN></FONT></B><FONT
face=Arial color=navy size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial"><o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face="Times New Roman" size=3><SPAN
style="FONT-SIZE: 12pt"> <o:p></o:p></SPAN></FONT></P></DIV></DIV></BLOCKQUOTE></BODY></HTML>