<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML xmlns="http://www.w3.org/TR/REC-html40" xmlns:v =
"urn:schemas-microsoft-com:vml" xmlns:o =
"urn:schemas-microsoft-com:office:office" xmlns:w =
"urn:schemas-microsoft-com:office:word" xmlns:st1 =
"urn:schemas-microsoft-com:office:smarttags"><HEAD><TITLE>Re: [openfabrics-ewg] We are planning to test this stack - what is itsstatus ?</TITLE>
<META http-equiv=Content-Type content="text/html; charset=us-ascii">
<META content="MSHTML 6.00.2900.2180" name=GENERATOR><!--[if !mso]>
<STYLE>v\:* {
BEHAVIOR: url(#default#VML)
}
o\:* {
BEHAVIOR: url(#default#VML)
}
w\:* {
BEHAVIOR: url(#default#VML)
}
.shape {
BEHAVIOR: url(#default#VML)
}
</STYLE>
<![endif]--><o:SmartTagType downloadurl="http://www.microsoft.com"
name="PersonName"
namespaceuri="urn:schemas-microsoft-com:office:smarttags"></o:SmartTagType><!--[if !mso]>
<STYLE>st1\:* {
BEHAVIOR: url(#default#ieooui)
}
</STYLE>
<![endif]-->
<STYLE>@font-face {
font-family: Tahoma;
}
@page Section1 {size: 8.5in 11.0in; margin: 1.0in 1.25in 1.0in 1.25in; }
P.MsoNormal {
FONT-SIZE: 12pt; MARGIN: 0in 0in 0pt; FONT-FAMILY: "Times New Roman"
}
LI.MsoNormal {
FONT-SIZE: 12pt; MARGIN: 0in 0in 0pt; FONT-FAMILY: "Times New Roman"
}
DIV.MsoNormal {
FONT-SIZE: 12pt; MARGIN: 0in 0in 0pt; FONT-FAMILY: "Times New Roman"
}
A:link {
COLOR: blue; TEXT-DECORATION: underline
}
SPAN.MsoHyperlink {
COLOR: blue; TEXT-DECORATION: underline
}
A:visited {
COLOR: blue; TEXT-DECORATION: underline
}
SPAN.MsoHyperlinkFollowed {
COLOR: blue; TEXT-DECORATION: underline
}
SPAN.emailstyle17 {
COLOR: navy; FONT-FAMILY: Arial
}
SPAN.emailstyle18 {
COLOR: navy; FONT-FAMILY: Arial
}
SPAN.EmailStyle19 {
COLOR: navy; FONT-FAMILY: Arial; mso-style-type: personal
}
SPAN.EmailStyle20 {
COLOR: navy; FONT-FAMILY: Arial; mso-style-type: personal-reply
}
DIV.Section1 {
page: Section1
}
</STYLE>
<!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></HEAD>
<BODY lang=EN-US vLink=blue link=blue>
<DIV dir=ltr align=left><SPAN class=574471921-03052006><FONT face=Arial size=2>I
was using Intel MPI incorrectly, I was running the two procs on the same node (I
ran mpdboot incorrectly).</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=574471921-03052006><FONT face=Arial
size=2></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN class=574471921-03052006><FONT face=Arial size=2>I
am now getting 4.61 usec latency, with Cheetah 1.0.800 firmware. I tried
with –genv I_MPI_USE_RENEDEZVOUS_RDMA_WRITE 1, too, but that didn't
help.</FONT></SPAN></DIV>
<DIV> </DIV>
<DIV align=left><FONT face=Arial size=2>Scott Weitzenkamp</FONT></DIV>
<DIV align=left><FONT face=Arial size=2>SQA and Release Manager</FONT></DIV>
<DIV align=left><FONT face=Arial size=2>Server Virtualization Business
Unit</FONT></DIV>
<DIV align=left><FONT face=Arial size=2>Cisco Systems</FONT></DIV>
<DIV> </DIV><BR>
<BLOCKQUOTE dir=ltr
style="PADDING-LEFT: 5px; MARGIN-LEFT: 5px; BORDER-LEFT: #0000ff 2px solid; MARGIN-RIGHT: 0px">
<DIV class=OutlookMessageHeader lang=en-us dir=ltr align=left>
<HR tabIndex=-1>
<FONT face=Tahoma size=2><B>From:</B> Davis, Arlin R
[mailto:arlin.r.davis@intel.com] <BR><B>Sent:</B> Wednesday, May 03, 2006 2:10
PM<BR><B>To:</B> Scott Weitzenkamp (sweitzen); Woodruff, Robert J; Aviram
Gutman; Tziporet Koren; Amit Mehrotra (amehrotr);
openfabrics-ewg@openib.org<BR><B>Cc:</B> Alexander Smirnov<BR><B>Subject:</B>
RE: [openfabrics-ewg] rc3 installation issues.<BR></FONT><BR></DIV>
<DIV></DIV>
<DIV class=Section1>
<P class=MsoNormal><FONT face=Arial color=navy size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial">Scott,<o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial color=navy size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial"><o:p> </o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial color=navy size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial">What is your HCA
firmware revision? We saw some RDMA read performance issues with older
firmware.<o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial color=navy size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial"><o:p> </o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial color=navy size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial">You can try “ –genv
I_MPI_USE_RENEDEZVOUS_RDMA_WRITE 1” to use rdma writes instead of
reads.<o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial color=navy size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial"><o:p> </o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial color=navy size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial">-arlin<o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial color=navy size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial"><o:p> </o:p></SPAN></FONT></P>
<DIV
style="BORDER-RIGHT: medium none; PADDING-RIGHT: 0in; BORDER-TOP: medium none; PADDING-LEFT: 4pt; PADDING-BOTTOM: 0in; BORDER-LEFT: blue 1.5pt solid; PADDING-TOP: 0in; BORDER-BOTTOM: medium none">
<DIV>
<DIV class=MsoNormal style="TEXT-ALIGN: center" align=center><FONT
face="Times New Roman" size=3><SPAN style="FONT-SIZE: 12pt">
<HR tabIndex=-1 align=center width="100%" SIZE=2>
</SPAN></FONT></DIV>
<P class=MsoNormal><B><FONT face=Tahoma size=2><SPAN
style="FONT-WEIGHT: bold; FONT-SIZE: 10pt; FONT-FAMILY: Tahoma">From:</SPAN></FONT></B><FONT
face=Tahoma size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Tahoma"> Scott
Weitzenkamp (sweitzen) [mailto:sweitzen@cisco.com] <BR><B><SPAN
style="FONT-WEIGHT: bold">Sent:</SPAN></B> Monday, May 01, 2006 12:15
PM<BR><B><SPAN style="FONT-WEIGHT: bold">To:</SPAN></B> Davis, Arlin R;
<st1:PersonName w:st="on">Woodruff, Robert J</st1:PersonName>; Aviram Gutman;
Tziporet Koren; Amit Mehrotra (amehrotr);
openfabrics-ewg@openib.org<BR><B><SPAN
style="FONT-WEIGHT: bold">Cc:</SPAN></B> Alexander Smirnov<BR><B><SPAN
style="FONT-WEIGHT: bold">Subject:</SPAN></B> RE: [openfabrics-ewg] rc3
installation issues.</SPAN></FONT><o:p></o:p></P></DIV>
<P class=MsoNormal><FONT face="Times New Roman" size=3><SPAN
style="FONT-SIZE: 12pt"><o:p> </o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">We are using Intel MPI 2.0
(plan to upgrade to 2.0.1 soon) with "mpiexec -genv I_MPI_DEBUG 3 -genv
I_MPI_DEVICE rdma". Are there any tunable parameters we should be
setting?</SPAN></FONT><o:p></o:p></P>
<P class=MsoNormal><FONT face="Times New Roman" size=3><SPAN
style="FONT-SIZE: 12pt"> <o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">###TEST-D: MPI home dir is
/data/software/qa/MPI/intel_mpi/intelmpi-2.0-`uname
-<BR>m`.</SPAN></FONT><o:p></o:p></P>
<DIV>
<P class=MsoNormal><FONT face="Times New Roman" size=3><SPAN
style="FONT-SIZE: 12pt"> <o:p></o:p></SPAN></FONT></P></DIV>
<P class=MsoNormal><FONT face=Arial size=2><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">###TEST-D: Intel-MPI uDAPL
osu_latency test on hosts svbu-qa1850-1 svbu-qa1850-2<BR># OSU MPI Latency
Test (Version 2.1)<BR>#
Size Latency
(us)<BR>0
6.89<BR>1
7.25<BR>2
7.43<BR>4
7.38<BR>8
7.25<BR>16
7.62<BR>32
7.45<BR>64
7.86<BR>128
8.46<BR>256
9.53<BR>512
6.55<BR>1024
7.29<BR>2048
8.57<BR>4096
11.46<BR>8192
17.19<BR>16384
29.26<BR>32768
52.75<BR>65536
82.67<BR>131072
159.05<BR>262144
282.31<BR>524288
527.08<BR>1048576
1019.57<BR>2097152
1980.07<BR>4194304
3921.89<BR>###TEST-D: Intel-MPI uDAPL osu_bw test on hosts svbu-qa1850-1
svbu-qa1850-2<BR># OSU MPI Bandwidth Test (Version 2.1)<BR>#
Size Bandwidth
(MB/s)<BR>1
0.426379<BR>2
0.852988<BR>4
1.711797<BR>8
3.370452<BR>16
6.725704<BR>32
13.475464<BR>64
25.936968<BR>128
48.790506<BR>256
93.297867<BR>512
170.188285<BR>1024
290.329228<BR>2048
452.958568<BR>4096
579.709399<BR>8192
627.702319<BR>16384
643.040963<BR>32768
652.009193<BR>65536
861.314972<BR>131072
995.337153<BR>262144
1112.127322<BR>524288
1184.367387<BR>1048576
1223.328478<BR>2097152
1245.594018<BR>4194304
1256.059934<BR>###TEST-D: Intel-MPI uDAPL osu_bibw test on hosts svbu-qa1850-1
svbu-qa1850-2<BR># OSU MPI Bidirectional Bandwidth Test (Version 2.1)<BR>#
Size Bi-Bandwidth
(MB/s)<BR>1
0.531761<BR>2
1.063121<BR>4
2.147913<BR>8
4.280755<BR>16
8.523452<BR>32
16.981693<BR>64
33.874987<BR>128
64.956544<BR>256
122.497886<BR>512
226.783855<BR>1024
367.798527<BR>2048
531.763593<BR>4096
613.404163<BR>8192
638.258306<BR>16384
645.862115<BR>32768
955.762031<BR>65536
1089.402315<BR>131072
1134.346706<BR>262144
1191.923405<BR>524288
1227.324266<BR>1048576
1245.633844<BR>2097152
1254.414692<BR>4194304
1261.847862</SPAN></FONT><o:p></o:p></P>
<DIV>
<P class=MsoNormal><FONT face="Times New Roman" size=3><SPAN
style="FONT-SIZE: 12pt"> <o:p></o:p></SPAN></FONT></P></DIV>
<P class=MsoNormal><FONT face=Arial size=2><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">Scott
Weitzenkamp</SPAN></FONT><o:p></o:p></P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">SQA and Release
Manager</SPAN></FONT><o:p></o:p></P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">Server Virtualization Business
Unit</SPAN></FONT><o:p></o:p></P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">Cisco
Systems</SPAN></FONT><o:p></o:p></P>
<DIV>
<P class=MsoNormal><FONT face="Times New Roman" size=3><SPAN
style="FONT-SIZE: 12pt"> <o:p></o:p></SPAN></FONT></P></DIV>
<BLOCKQUOTE
style="BORDER-RIGHT: medium none; PADDING-RIGHT: 0in; BORDER-TOP: medium none; PADDING-LEFT: 4pt; PADDING-BOTTOM: 0in; MARGIN: 5pt 0in 5pt 3.75pt; BORDER-LEFT: blue 1.5pt solid; PADDING-TOP: 0in; BORDER-BOTTOM: medium none">
<P class=MsoNormal><FONT face="Times New Roman" size=3><SPAN
style="FONT-SIZE: 12pt"><o:p> </o:p></SPAN></FONT></P>
<DIV class=MsoNormal style="TEXT-ALIGN: center" align=center><FONT
face="Times New Roman" size=3><SPAN style="FONT-SIZE: 12pt">
<HR tabIndex=-1 align=center width="100%" SIZE=2>
</SPAN></FONT></DIV>
<P class=MsoNormal style="MARGIN-BOTTOM: 12pt"><B><FONT face=Tahoma
size=2><SPAN
style="FONT-WEIGHT: bold; FONT-SIZE: 10pt; FONT-FAMILY: Tahoma">From:</SPAN></FONT></B><FONT
face=Tahoma size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Tahoma">
Davis, Arlin R [mailto:arlin.r.davis@intel.com] <BR><B><SPAN
style="FONT-WEIGHT: bold">Sent:</SPAN></B> Monday, May 01, 2006 11:29
AM<BR><B><SPAN style="FONT-WEIGHT: bold">To:</SPAN></B> Scott Weitzenkamp
(sweitzen); <st1:PersonName w:st="on">Woodruff, Robert J</st1:PersonName>;
Aviram Gutman; Tziporet Koren; Amit Mehrotra (amehrotr);
openfabrics-ewg@openib.org<BR><B><SPAN
style="FONT-WEIGHT: bold">Cc:</SPAN></B> Alexander Smirnov<BR><B><SPAN
style="FONT-WEIGHT: bold">Subject:</SPAN></B> RE: [openfabrics-ewg] rc3
installation issues.</SPAN></FONT><o:p></o:p></P>
<P class=MsoNormal><FONT face=Arial color=navy size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial">Scott,<o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial color=navy size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial"><o:p> </o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial color=navy size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial">Thanks for testing
Intel MPI on OFED 1.0 rc3. Yes, the socket CM uDAPL provider will work fine
without uCMA and IPoIB installed. Any idea why your latency is so high? It
should be around 3-4us.<o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial color=navy size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial"><o:p> </o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial color=navy size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial">-arlin<o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial color=navy size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial"><o:p> </o:p></SPAN></FONT></P>
<DIV
style="BORDER-RIGHT: medium none; PADDING-RIGHT: 0in; BORDER-TOP: medium none; PADDING-LEFT: 4pt; PADDING-BOTTOM: 0in; BORDER-LEFT: blue 1.5pt solid; PADDING-TOP: 0in; BORDER-BOTTOM: medium none">
<DIV>
<DIV class=MsoNormal style="TEXT-ALIGN: center" align=center><FONT
face="Times New Roman" size=3><SPAN style="FONT-SIZE: 12pt">
<HR tabIndex=-1 align=center width="100%" SIZE=2>
</SPAN></FONT></DIV>
<P class=MsoNormal><B><FONT face=Tahoma size=2><SPAN
style="FONT-WEIGHT: bold; FONT-SIZE: 10pt; FONT-FAMILY: Tahoma">From:</SPAN></FONT></B><FONT
face=Tahoma size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Tahoma"> Scott
Weitzenkamp (sweitzen) [mailto:sweitzen@cisco.com] <BR><B><SPAN
style="FONT-WEIGHT: bold">Sent:</SPAN></B> Sunday, April 30, 2006 10:32
PM<BR><B><SPAN style="FONT-WEIGHT: bold">To:</SPAN></B> <st1:PersonName
w:st="on">Woodruff, Robert J</st1:PersonName>; Aviram Gutman; Tziporet
Koren; Amit Mehrotra (amehrotr); openfabrics-ewg@openib.org<BR><B><SPAN
style="FONT-WEIGHT: bold">Cc:</SPAN></B> Alexander Smirnov; Davis, Arlin
R<BR><B><SPAN style="FONT-WEIGHT: bold">Subject:</SPAN></B> RE:
[openfabrics-ewg] rc3 installation
issues.</SPAN></FONT><o:p></o:p></P></DIV>
<P class=MsoNormal><FONT face="Times New Roman" size=3><SPAN
style="FONT-SIZE: 12pt"><o:p> </o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">Intel MPI 2.0 does work with
OFED 1.0 rc3 on RHEL4 U3 despite the lack of uCMA, here's example output on
x86_64 w/PCI-E:</SPAN></FONT><o:p></o:p></P>
<P class=MsoNormal><FONT face="Times New Roman" size=3><SPAN
style="FONT-SIZE: 12pt"> <o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">.../osu_latency.x<BR>CMA: unable
to open /dev/infiniband/rdma_cm<BR>CMA: unable to open
/dev/infiniband/rdma_cm<BR>CMA: unable to open
/dev/infiniband/rdma_cm<BR>CMA: unable to open
/dev/infiniband/rdma_cm<BR>I_MPI: [0] set_up_devices(): will use device:
libmpi.rdma.so<BR>I_MPI: [0] set_up_devices(): will use DAPL provider:
OpenIB-scm1<BR>CMA: unable to open /dev/infiniband/rdma_cm<BR>CMA: unable to
open /dev/infiniband/rdma_cm<BR>CMA: unable to open
/dev/infiniband/rdma_cm<BR>CMA: unable to open
/dev/infiniband/rdma_cm<BR>I_MPI: [0] set_up_devices(): will use device:
libmpi.rdma.so<BR>I_MPI: [0] set_up_devices(): will use DAPL provider:
OpenIB-scm1<BR># OSU MPI Latency Test (Version 2.1)<BR>#
Size Latency (us)
<BR>0
7.86<BR>1
7.53<BR>2
7.37<BR>4
7.43<BR>8
7.92<BR>16
7.29<BR>32
7.36<BR>64
7.50</SPAN></FONT><o:p></o:p></P>
<DIV>
<P class=MsoNormal><FONT face="Times New Roman" size=3><SPAN
style="FONT-SIZE: 12pt"> <o:p></o:p></SPAN></FONT></P></DIV>
<P class=MsoNormal><FONT face=Arial size=2><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">Scott
Weitzenkamp</SPAN></FONT><o:p></o:p></P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">SQA and Release
Manager</SPAN></FONT><o:p></o:p></P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">Server Virtualization Business
Unit</SPAN></FONT><o:p></o:p></P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">Cisco
Systems</SPAN></FONT><o:p></o:p></P>
<DIV>
<P class=MsoNormal><FONT face="Times New Roman" size=3><SPAN
style="FONT-SIZE: 12pt"> <o:p></o:p></SPAN></FONT></P></DIV>
<BLOCKQUOTE
style="BORDER-RIGHT: medium none; PADDING-RIGHT: 0in; BORDER-TOP: medium none; PADDING-LEFT: 4pt; PADDING-BOTTOM: 0in; MARGIN: 5pt 0in 5pt 3.75pt; BORDER-LEFT: blue 1.5pt solid; PADDING-TOP: 0in; BORDER-BOTTOM: medium none">
<P class=MsoNormal><FONT face="Times New Roman" size=3><SPAN
style="FONT-SIZE: 12pt"><o:p> </o:p></SPAN></FONT></P>
<DIV class=MsoNormal style="TEXT-ALIGN: center" align=center><FONT
face="Times New Roman" size=3><SPAN style="FONT-SIZE: 12pt">
<HR tabIndex=-1 align=center width="100%" SIZE=2>
</SPAN></FONT></DIV>
<P class=MsoNormal style="MARGIN-BOTTOM: 12pt"><B><FONT face=Tahoma
size=2><SPAN
style="FONT-WEIGHT: bold; FONT-SIZE: 10pt; FONT-FAMILY: Tahoma">From:</SPAN></FONT></B><FONT
face=Tahoma size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Tahoma">
openfabrics-ewg-bounces@openib.org
[mailto:openfabrics-ewg-bounces@openib.org] <B><SPAN
style="FONT-WEIGHT: bold">On Behalf Of </SPAN></B><st1:PersonName
w:st="on">Woodruff, Robert J</st1:PersonName><BR><B><SPAN
style="FONT-WEIGHT: bold">Sent:</SPAN></B> Tuesday, April 25, 2006 9:16
AM<BR><B><SPAN style="FONT-WEIGHT: bold">To:</SPAN></B> Aviram Gutman;
Tziporet Koren; Amit Mehrotra (amehrotr);
openfabrics-ewg@openib.org<BR><B><SPAN
style="FONT-WEIGHT: bold">Cc:</SPAN></B> Alexander Smirnov; Davis, Arlin
R<BR><B><SPAN style="FONT-WEIGHT: bold">Subject:</SPAN></B> RE:
[openfabrics-ewg] rc3 installation issues.</SPAN></FONT><o:p></o:p></P>
<P class=MsoNormal><FONT face=Arial color=blue size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: Arial">Intel MPI needs a
stable uDAPL and supporting usermode verbs/uCMA and
such.</SPAN></FONT><o:p></o:p></P>
<P class=MsoNormal><FONT face=Arial color=blue size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: Arial">If you have that,
then Intel MPI should work. If you want to get a copy of Intel
MPI</SPAN></FONT><o:p></o:p></P>
<P class=MsoNormal><FONT face=Arial color=blue size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: Arial">to use in your
testing of OFED, I can put you in contact with our Intel MPI lead and he
can</SPAN></FONT><o:p></o:p></P>
<P class=MsoNormal><FONT face=Arial color=blue size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: Arial">provide you with
the code and a license. </SPAN></FONT><o:p></o:p></P>
<P class=MsoNormal><FONT face="Times New Roman" size=3><SPAN
style="FONT-SIZE: 12pt"><o:p> </o:p></SPAN></FONT></P>
<DIV class=MsoNormal style="TEXT-ALIGN: center" align=center><FONT
face="Times New Roman" size=3><SPAN style="FONT-SIZE: 12pt">
<HR tabIndex=-1 align=center width="100%" SIZE=2>
</SPAN></FONT></DIV>
<P class=MsoNormal style="MARGIN-BOTTOM: 12pt"><B><FONT face=Tahoma
size=2><SPAN
style="FONT-WEIGHT: bold; FONT-SIZE: 10pt; FONT-FAMILY: Tahoma">From:</SPAN></FONT></B><FONT
face=Tahoma size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Tahoma">
Aviram Gutman [mailto:aviram@mellanox.co.il] <BR><B><SPAN
style="FONT-WEIGHT: bold">Sent:</SPAN></B> Tuesday, April 25, 2006 9:09
AM<BR><B><SPAN style="FONT-WEIGHT: bold">To:</SPAN></B> <st1:PersonName
w:st="on">Woodruff, Robert J</st1:PersonName>; Tziporet Koren; Amit
Mehrotra (amehrotr); openfabrics-ewg@openib.org<BR><B><SPAN
style="FONT-WEIGHT: bold">Cc:</SPAN></B> Alexander Smirnov; Davis, Arlin
R<BR><B><SPAN style="FONT-WEIGHT: bold">Subject:</SPAN></B> RE:
[openfabrics-ewg] rc3 installation issues.</SPAN></FONT><o:p></o:p></P>
<DIV>
<P class=MsoNormal><FONT face=Tahoma color=maroon size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: maroon; FONT-FAMILY: Tahoma">Please work
with Vlad to make sure you have what you need in
rc4.</SPAN></FONT><o:p></o:p></P></DIV>
<DIV>
<P class=MsoNormal><FONT face="Times New Roman" size=3><SPAN
style="FONT-SIZE: 12pt"> <o:p></o:p></SPAN></FONT></P></DIV>
<P class=MsoNormal><FONT face=Arial size=2><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">Regards,</SPAN></FONT><o:p></o:p></P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">
Aviram</SPAN></FONT><o:p></o:p></P>
<DIV>
<P class=MsoNormal><FONT face="Times New Roman" size=3><SPAN
style="FONT-SIZE: 12pt"> <o:p></o:p></SPAN></FONT></P></DIV>
<P class=MsoNormal><FONT face="Times New Roman" size=3><SPAN
style="FONT-SIZE: 12pt"><o:p> </o:p></SPAN></FONT></P>
<DIV class=MsoNormal style="TEXT-ALIGN: center" align=center><FONT
face="Times New Roman" size=3><SPAN style="FONT-SIZE: 12pt">
<HR tabIndex=-1 align=center width="100%" SIZE=2>
</SPAN></FONT></DIV>
<P class=MsoNormal style="MARGIN-BOTTOM: 12pt"><B><FONT face=Tahoma
size=2><SPAN
style="FONT-WEIGHT: bold; FONT-SIZE: 10pt; FONT-FAMILY: Tahoma">From:</SPAN></FONT></B><FONT
face=Tahoma size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Tahoma">
<st1:PersonName w:st="on">Woodruff, Robert J</st1:PersonName>
[mailto:<st1:PersonName
w:st="on">rob</st1:PersonName>ert.j.woodruff@intel.com] <BR><B><SPAN
style="FONT-WEIGHT: bold">Sent:</SPAN></B> Tuesday, April 25, 2006 6:56
PM<BR><B><SPAN style="FONT-WEIGHT: bold">To:</SPAN></B> Aviram Gutman;
Tziporet Koren; Amit Mehrotra (amehrotr);
openfabrics-ewg@openib.org<BR><B><SPAN
style="FONT-WEIGHT: bold">Cc:</SPAN></B> Alexander Smirnov; Davis, Arlin
R<BR><B><SPAN style="FONT-WEIGHT: bold">Subject:</SPAN></B> RE:
[openfabrics-ewg] rc3 installation issues.</SPAN></FONT><o:p></o:p></P>
<P class=MsoNormal><FONT face=Arial color=blue size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: Arial">Not yet. Last I
heard, there was no uDAPL in OFED that is neeed by Intel MPI. Maybe RC4.
</SPAN></FONT><o:p></o:p></P>
<DIV>
<P class=MsoNormal><FONT face="Times New Roman" size=3><SPAN
style="FONT-SIZE: 12pt"> <o:p></o:p></SPAN></FONT></P></DIV>
<DIV>
<P class=MsoNormal><FONT face="Times New Roman" size=3><SPAN
style="FONT-SIZE: 12pt"> <o:p></o:p></SPAN></FONT></P></DIV>
<DIV>
<P class=MsoNormal style="MARGIN-LEFT: 0.5in"><FONT face=Arial color=blue
size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: Arial"> Tziporet
wrote,</SPAN></FONT><o:p></o:p></P>
<P class=MsoNormal style="MARGIN-LEFT: 0.5in"><FONT face=Arial color=blue
size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: Arial">> </SPAN></FONT><FONT
face=Arial color=navy size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial">Regarding uDAPL
and RHEL4 (up2 & 3): Since we have not completed the uCMA backport for
kernel 2.6.9 we eliminated uDAPL </SPAN></FONT><FONT face=Arial
color=blue size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: Arial"> </SPAN></FONT><o:p></o:p></P></DIV>
<DIV>
<P class=MsoNormal><FONT face="Times New Roman" size=3><SPAN
style="FONT-SIZE: 12pt"><o:p> </o:p></SPAN></FONT></P></DIV>
<DIV class=MsoNormal style="TEXT-ALIGN: center" align=center><FONT
face="Times New Roman" size=3><SPAN style="FONT-SIZE: 12pt">
<HR tabIndex=-1 align=center width="100%" SIZE=2>
</SPAN></FONT></DIV>
<P class=MsoNormal style="MARGIN-BOTTOM: 12pt"><B><FONT face=Tahoma
size=2><SPAN
style="FONT-WEIGHT: bold; FONT-SIZE: 10pt; FONT-FAMILY: Tahoma">From:</SPAN></FONT></B><FONT
face=Tahoma size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Tahoma">
Aviram Gutman [mailto:aviram@mellanox.co.il] <BR><B><SPAN
style="FONT-WEIGHT: bold">Sent:</SPAN></B> Tuesday, April 25, 2006 8:48
AM<BR><B><SPAN style="FONT-WEIGHT: bold">To:</SPAN></B> <st1:PersonName
w:st="on">Woodruff, Robert J</st1:PersonName>; Tziporet Koren; Amit
Mehrotra (amehrotr); openfabrics-ewg@openib.org<BR><B><SPAN
style="FONT-WEIGHT: bold">Cc:</SPAN></B> Alexander Smirnov; Davis, Arlin
R<BR><B><SPAN style="FONT-WEIGHT: bold">Subject:</SPAN></B> RE:
[openfabrics-ewg] rc3 installation issues.</SPAN></FONT><o:p></o:p></P>
<DIV>
<P class=MsoNormal><FONT face=Tahoma color=maroon size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: maroon; FONT-FAMILY: Tahoma">Woody,</SPAN></FONT><o:p></o:p></P></DIV>
<DIV>
<P class=MsoNormal><FONT face="Times New Roman" size=3><SPAN
style="FONT-SIZE: 12pt"> <o:p></o:p></SPAN></FONT></P></DIV>
<DIV>
<P class=MsoNormal><FONT face=Tahoma color=maroon size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: maroon; FONT-FAMILY: Tahoma">Are you
testing Intel MPI on top of OFED?</SPAN></FONT><o:p></o:p></P></DIV>
<DIV>
<P class=MsoNormal><FONT face="Times New Roman" size=3><SPAN
style="FONT-SIZE: 12pt"> <o:p></o:p></SPAN></FONT></P></DIV>
<P class=MsoNormal><FONT face=Arial size=2><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">Regards,</SPAN></FONT><o:p></o:p></P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">
Aviram</SPAN></FONT><o:p></o:p></P>
<DIV>
<P class=MsoNormal><FONT face="Times New Roman" size=3><SPAN
style="FONT-SIZE: 12pt"> <o:p></o:p></SPAN></FONT></P></DIV>
<P class=MsoNormal><FONT face="Times New Roman" size=3><SPAN
style="FONT-SIZE: 12pt"><o:p> </o:p></SPAN></FONT></P>
<DIV class=MsoNormal style="TEXT-ALIGN: center" align=center><FONT
face="Times New Roman" size=3><SPAN style="FONT-SIZE: 12pt">
<HR tabIndex=-1 align=center width="100%" SIZE=2>
</SPAN></FONT></DIV>
<P class=MsoNormal style="MARGIN-BOTTOM: 12pt"><B><FONT face=Tahoma
size=2><SPAN
style="FONT-WEIGHT: bold; FONT-SIZE: 10pt; FONT-FAMILY: Tahoma">From:</SPAN></FONT></B><FONT
face=Tahoma size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Tahoma">
openfabrics-ewg-bounces@openib.org
[mailto:openfabrics-ewg-bounces@openib.org] <B><SPAN
style="FONT-WEIGHT: bold">On Behalf Of </SPAN></B><st1:PersonName
w:st="on">Woodruff, Robert J</st1:PersonName><BR><B><SPAN
style="FONT-WEIGHT: bold">Sent:</SPAN></B> Monday, April 24, 2006 11:33
PM<BR><B><SPAN style="FONT-WEIGHT: bold">To:</SPAN></B> Tziporet Koren;
Amit Mehrotra (amehrotr); openfabrics-ewg@openib.org<BR><B><SPAN
style="FONT-WEIGHT: bold">Cc:</SPAN></B> Alexander Smirnov; Davis, Arlin
R<BR><B><SPAN style="FONT-WEIGHT: bold">Subject:</SPAN></B> RE:
[openfabrics-ewg] rc3 installation issues.</SPAN></FONT><o:p></o:p></P>
<P class=MsoNormal><FONT face="Times New Roman" size=3><SPAN
style="FONT-SIZE: 12pt"> <o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial color=blue size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: Arial">The latest
patches I have is for SVN 6509 on the trunk. </SPAN></FONT><o:p></o:p></P>
<P class=MsoNormal><FONT face=Arial color=blue size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: Arial"><A
href="https://openib.org/svn/gen2/branches/backport-to-2.6.9/">https://openib.org/svn/gen2/branches/backport-to-2.6.9/</A></SPAN></FONT><o:p></o:p></P>
<P class=MsoNormal><FONT face=Arial color=blue size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: Arial">infiniband-backport-svn6509-to-2.6.9-34.EL-kernel-fixups-00.diff<BR>infiniband-backport-svn6509-to-2.6.9-34.EL-kernel-fixups-01.diff<BR>infiniband-backport-svn6509-to-2.6.9-openib-drivers-02.diff<BR>infiniband-backport-svn6509-to-2.6.9-openib-fixups-03.diff</SPAN></FONT><o:p></o:p></P>
<P class=MsoNormal><FONT face=Arial color=blue size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: Arial">I don't split the
patches by component, but rather have one to strip out the old
IB</SPAN></FONT><o:p></o:p></P>
<P class=MsoNormal><FONT face=Arial color=blue size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: Arial">code in RedHat
EL4.0, one to apply needed kernel changes to the RedHat
kernel,</SPAN></FONT><o:p></o:p></P>
<P class=MsoNormal><FONT face=Arial color=blue size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: Arial">one for the
openib drivers, and one for openib fixups that are
needed.</SPAN></FONT><o:p></o:p></P>
<P class=MsoNormal><FONT face="Times New Roman" size=3><SPAN
style="FONT-SIZE: 12pt"> <o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face="Times New Roman" size=3><SPAN
style="FONT-SIZE: 12pt"> <o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face="Times New Roman" size=3><SPAN
style="FONT-SIZE: 12pt"> <o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face="Times New Roman" size=3><SPAN
style="FONT-SIZE: 12pt"> <o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face="Times New Roman" size=3><SPAN
style="FONT-SIZE: 12pt"><o:p> </o:p></SPAN></FONT></P>
<DIV class=MsoNormal style="TEXT-ALIGN: center" align=center><FONT
face="Times New Roman" size=3><SPAN style="FONT-SIZE: 12pt">
<HR tabIndex=-1 align=center width="100%" SIZE=2>
</SPAN></FONT></DIV>
<P class=MsoNormal style="MARGIN-BOTTOM: 12pt"><B><FONT face=Tahoma
size=2><SPAN
style="FONT-WEIGHT: bold; FONT-SIZE: 10pt; FONT-FAMILY: Tahoma">From:</SPAN></FONT></B><FONT
face=Tahoma size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Tahoma">
Tziporet Koren [mailto:tziporet@mellanox.co.il] <BR><B><SPAN
style="FONT-WEIGHT: bold">Sent:</SPAN></B> Monday, April 24, 2006 11:43
AM<BR><B><SPAN style="FONT-WEIGHT: bold">To:</SPAN></B> <st1:PersonName
w:st="on">Woodruff, Robert J</st1:PersonName>; Amit Mehrotra (amehrotr);
openfabrics-ewg@openib.org<BR><B><SPAN
style="FONT-WEIGHT: bold">Cc:</SPAN></B> Alexander Smirnov; Davis, Arlin
R<BR><B><SPAN style="FONT-WEIGHT: bold">Subject:</SPAN></B> RE:
[openfabrics-ewg] rc3 installation issues.</SPAN></FONT><o:p></o:p></P>
<P class=MsoNormal><FONT face=Arial color=navy size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial">We plan to do a
backport of uCMA for RC4.</SPAN></FONT><o:p></o:p></P>
<P class=MsoNormal><FONT face=Arial color=navy size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial">Can you point me
to your backport of uCMA for 2.6.9-34EL</SPAN></FONT><o:p></o:p></P>
<P class=MsoNormal><FONT face="Times New Roman" size=3><SPAN
style="FONT-SIZE: 12pt"> <o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial color=navy size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial">Thanks,</SPAN></FONT><o:p></o:p></P>
<P class=MsoNormal><FONT face=Arial color=navy size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial">Tziporet</SPAN></FONT><o:p></o:p></P>
<P class=MsoNormal><FONT face="Times New Roman" size=3><SPAN
style="FONT-SIZE: 12pt"> <o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal style="MARGIN-LEFT: 0.5in"><FONT face=Tahoma
size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Tahoma">-----Original
Message-----<BR><B><SPAN style="FONT-WEIGHT: bold">From:</SPAN></B> Bob
Woodruff [mailto:<st1:PersonName
w:st="on">rob</st1:PersonName>ert.j.woodruff@intel.com] <BR><B><SPAN
style="FONT-WEIGHT: bold">Sent:</SPAN></B> Monday, April 24, 2006 6:56
PM<BR><B><SPAN style="FONT-WEIGHT: bold">To:</SPAN></B> Tziporet Koren;
Amit Mehrotra (amehrotr); openfabrics-ewg@openib.org<BR><B><SPAN
style="FONT-WEIGHT: bold">Cc:</SPAN></B> Alexander Smirnov; Davis, Arlin
R<BR><B><SPAN style="FONT-WEIGHT: bold">Subject:</SPAN></B> RE:
[openfabrics-ewg] rc3 installation issues.</SPAN></FONT><o:p></o:p></P>
<P class=MsoNormal style="MARGIN-LEFT: 0.5in"><FONT face="Times New Roman"
size=3><SPAN style="FONT-SIZE: 12pt"> <o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal style="MARGIN-LEFT: 0.5in"><FONT face=Arial color=blue
size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: Arial"> Tziporet
wrote,</SPAN></FONT><o:p></o:p></P>
<P class=MsoNormal style="MARGIN-LEFT: 0.5in"><FONT face=Arial color=blue
size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: Arial">> </SPAN></FONT><FONT
face=Arial color=navy size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial">Regarding uDAPL
and RHEL4 (up2 & 3): Since we have not completed the uCMA backport for
kernel 2.6.9 we eliminated uDAPL </SPAN></FONT><FONT face=Arial
color=blue size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: Arial"> </SPAN></FONT><o:p></o:p></P>
<P class=MsoNormal style="MARGIN-LEFT: 0.5in"><FONT face=Arial color=blue
size=2><SPAN style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: Arial">>
</SPAN></FONT><FONT face=Arial color=navy size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial"> installation
in case of this kernel. We also indicated this in the package limitation
mail.</SPAN></FONT><FONT face=Arial color=blue size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: Arial"> </SPAN></FONT><o:p></o:p></P>
<P class=MsoNormal style="MARGIN-LEFT: 0.5in"><FONT face="Times New Roman"
size=3><SPAN style="FONT-SIZE: 12pt"> <o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal style="MARGIN-LEFT: 0.5in"><FONT face=Arial color=blue
size=2><SPAN style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: Arial">Note
that there are two uDAPL providers for OpenIB. One uses standard
sockets for connections (socket CM)</SPAN></FONT><o:p></o:p></P>
<P class=MsoNormal style="MARGIN-LEFT: 0.5in"><FONT face=Arial color=blue
size=2><SPAN style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: Arial">and
the other uses CMA. I would suggest that you should include the
socket CM uDAPL provider even</SPAN></FONT><o:p></o:p></P>
<P class=MsoNormal style="MARGIN-LEFT: 0.5in"><FONT face=Arial color=blue
size=2><SPAN style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: Arial">if
you do not have a backport for uCMA.</SPAN></FONT><o:p></o:p></P>
<P class=MsoNormal style="MARGIN-LEFT: 0.5in"><FONT face=Arial color=blue
size=2><SPAN style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: Arial">BTW.
- My backport patches contain backport for uCMA for 2.6.9-34-EL if
you need an example.</SPAN></FONT><o:p></o:p></P>
<P class=MsoNormal style="MARGIN-LEFT: 0.5in"><FONT face="Times New Roman"
size=3><SPAN style="FONT-SIZE: 12pt"> <o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal style="MARGIN-LEFT: 0.5in"><FONT face=Arial color=blue
size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: Arial">woody</SPAN></FONT><o:p></o:p></P>
<P class=MsoNormal style="MARGIN-LEFT: 0.5in"><FONT face="Times New Roman"
size=3><SPAN style="FONT-SIZE: 12pt"> <o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal style="MARGIN-LEFT: 0.5in"><FONT face="Times New Roman"
size=3><SPAN
style="FONT-SIZE: 12pt"> <o:p></o:p></SPAN></FONT></P></BLOCKQUOTE></DIV></BLOCKQUOTE></DIV></DIV></BLOCKQUOTE></BODY></HTML>