<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:st1="urn:schemas-microsoft-com:office:smarttags" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv=Content-Type content="text/html; charset=us-ascii">
<meta name=Generator content="Microsoft Word 11 (filtered medium)">
<!--[if !mso]>
<style>
v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style>
<![endif]-->
<title>Re: [openfabrics-ewg] We are planning to test this stack - what is
itsstatus ?</title>
<o:SmartTagType namespaceuri="urn:schemas-microsoft-com:office:smarttags"
name="PersonName" downloadurl="http://www.microsoft.com"/>
<!--[if !mso]>
<style>
st1\:*{behavior:url(#default#ieooui) }
</style>
<![endif]-->
<style>
<!--
/* Font Definitions */
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman";}
a:link, span.MsoHyperlink
{color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{color:blue;
text-decoration:underline;}
span.emailstyle17
{font-family:Arial;
color:navy;}
span.emailstyle18
{font-family:Arial;
color:navy;}
span.EmailStyle19
{mso-style-type:personal;
font-family:Arial;
color:navy;}
span.EmailStyle20
{mso-style-type:personal-reply;
font-family:Arial;
color:navy;}
@page Section1
{size:8.5in 11.0in;
margin:1.0in 1.25in 1.0in 1.25in;}
div.Section1
{page:Section1;}
-->
</style>
<!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang=EN-US link=blue vlink=blue>
<div class=Section1>
<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'>Scott,<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'>What is your HCA firmware revision? We saw
some RDMA read performance issues with older firmware.<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'>You can try “ –genv I_MPI_USE_RENEDEZVOUS_RDMA_WRITE
1” to use rdma writes instead of reads.<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'>-arlin<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'><o:p> </o:p></span></font></p>
<div style='border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt'>
<div>
<div class=MsoNormal align=center style='text-align:center'><font size=3
face="Times New Roman"><span style='font-size:12.0pt'>
<hr size=2 width="100%" align=center tabindex=-1>
</span></font></div>
<p class=MsoNormal><b><font size=2 face=Tahoma><span style='font-size:10.0pt;
font-family:Tahoma;font-weight:bold'>From:</span></font></b><font size=2
face=Tahoma><span style='font-size:10.0pt;font-family:Tahoma'> Scott
Weitzenkamp (sweitzen) [mailto:sweitzen@cisco.com] <br>
<b><span style='font-weight:bold'>Sent:</span></b> Monday, May 01, 2006 12:15
PM<br>
<b><span style='font-weight:bold'>To:</span></b> Davis, Arlin R; <st1:PersonName
w:st="on">Woodruff, Robert J</st1:PersonName>; Aviram Gutman; Tziporet Koren;
Amit Mehrotra (amehrotr); openfabrics-ewg@openib.org<br>
<b><span style='font-weight:bold'>Cc:</span></b> Alexander Smirnov<br>
<b><span style='font-weight:bold'>Subject:</span></b> RE: [openfabrics-ewg] rc3
installation issues.</span></font><o:p></o:p></p>
</div>
<p class=MsoNormal><font size=3 face="Times New Roman"><span style='font-size:
12.0pt'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>We are using Intel MPI 2.0 (plan to upgrade to 2.0.1
soon) with "mpiexec -genv I_MPI_DEBUG 3 -genv I_MPI_DEVICE
rdma". Are there any tunable parameters we should be setting?</span></font><o:p></o:p></p>
<p class=MsoNormal><font size=3 face="Times New Roman"><span style='font-size:
12.0pt'> <o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>###TEST-D: MPI home dir is /data/software/qa/MPI/intel_mpi/intelmpi-2.0-`uname
-<br>
m`.</span></font><o:p></o:p></p>
<div>
<p class=MsoNormal><font size=3 face="Times New Roman"><span style='font-size:
12.0pt'> <o:p></o:p></span></font></p>
</div>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>###TEST-D: Intel-MPI uDAPL osu_latency test on hosts svbu-qa1850-1
svbu-qa1850-2<br>
# OSU MPI Latency Test (Version 2.1)<br>
# Size Latency (us)<br>
0
6.89<br>
1
7.25<br>
2
7.43<br>
4
7.38<br>
8
7.25<br>
16
7.62<br>
32
7.45<br>
64
7.86<br>
128
8.46<br>
256
9.53<br>
512
6.55<br>
1024 7.29<br>
2048 8.57<br>
4096 11.46<br>
8192 17.19<br>
16384 29.26<br>
32768 52.75<br>
65536 82.67<br>
131072 159.05<br>
262144 282.31<br>
524288 527.08<br>
1048576 1019.57<br>
2097152 1980.07<br>
4194304 3921.89<br>
###TEST-D: Intel-MPI uDAPL osu_bw test on hosts svbu-qa1850-1 svbu-qa1850-2<br>
# OSU MPI Bandwidth Test (Version 2.1)<br>
# Size Bandwidth (MB/s)<br>
1
0.426379<br>
2
0.852988<br>
4
1.711797<br>
8
3.370452<br>
16
6.725704<br>
32
13.475464<br>
64
25.936968<br>
128
48.790506<br>
256
93.297867<br>
512
170.188285<br>
1024
290.329228<br>
2048
452.958568<br>
4096
579.709399<br>
8192
627.702319<br>
16384 643.040963<br>
32768 652.009193<br>
65536 861.314972<br>
131072 995.337153<br>
262144 1112.127322<br>
524288 1184.367387<br>
1048576 1223.328478<br>
2097152 1245.594018<br>
4194304 1256.059934<br>
###TEST-D: Intel-MPI uDAPL osu_bibw test on hosts svbu-qa1850-1 svbu-qa1850-2<br>
# OSU MPI Bidirectional Bandwidth Test (Version 2.1)<br>
# Size Bi-Bandwidth
(MB/s)<br>
1
0.531761<br>
2
1.063121<br>
4
2.147913<br>
8
4.280755<br>
16
8.523452<br>
32
16.981693<br>
64
33.874987<br>
128
64.956544<br>
256
122.497886<br>
512 226.783855<br>
1024
367.798527<br>
2048
531.763593<br>
4096
613.404163<br>
8192
638.258306<br>
16384 645.862115<br>
32768 955.762031<br>
65536 1089.402315<br>
131072 1134.346706<br>
262144 1191.923405<br>
524288 1227.324266<br>
1048576 1245.633844<br>
2097152 1254.414692<br>
4194304 1261.847862</span></font><o:p></o:p></p>
<div>
<p class=MsoNormal><font size=3 face="Times New Roman"><span style='font-size:
12.0pt'> <o:p></o:p></span></font></p>
</div>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>Scott Weitzenkamp</span></font><o:p></o:p></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>SQA and Release Manager</span></font><o:p></o:p></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>Server Virtualization Business Unit</span></font><o:p></o:p></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>Cisco Systems</span></font><o:p></o:p></p>
<div>
<p class=MsoNormal><font size=3 face="Times New Roman"><span style='font-size:
12.0pt'> <o:p></o:p></span></font></p>
</div>
<blockquote style='border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt;
margin-left:3.75pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt'>
<p class=MsoNormal><font size=3 face="Times New Roman"><span style='font-size:
12.0pt'><o:p> </o:p></span></font></p>
<div class=MsoNormal align=center style='text-align:center'><font size=3
face="Times New Roman"><span style='font-size:12.0pt'>
<hr size=2 width="100%" align=center tabIndex=-1>
</span></font></div>
<p class=MsoNormal style='margin-bottom:12.0pt'><b><font size=2 face=Tahoma><span
style='font-size:10.0pt;font-family:Tahoma;font-weight:bold'>From:</span></font></b><font
size=2 face=Tahoma><span style='font-size:10.0pt;font-family:Tahoma'> Davis,
Arlin R [mailto:arlin.r.davis@intel.com] <br>
<b><span style='font-weight:bold'>Sent:</span></b> Monday, May 01, 2006 11:29
AM<br>
<b><span style='font-weight:bold'>To:</span></b> Scott Weitzenkamp (sweitzen); <st1:PersonName
w:st="on">Woodruff, Robert J</st1:PersonName>; Aviram Gutman; Tziporet Koren;
Amit Mehrotra (amehrotr); openfabrics-ewg@openib.org<br>
<b><span style='font-weight:bold'>Cc:</span></b> Alexander Smirnov<br>
<b><span style='font-weight:bold'>Subject:</span></b> RE: [openfabrics-ewg] rc3
installation issues.</span></font><o:p></o:p></p>
<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'>Scott,<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'>Thanks for testing Intel MPI on OFED 1.0
rc3. Yes, the socket CM uDAPL provider will work fine without uCMA and IPoIB
installed. Any idea why your latency is so high? It should be around 3-4us.<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'>-arlin<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'><o:p> </o:p></span></font></p>
<div style='border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt'>
<div>
<div class=MsoNormal align=center style='text-align:center'><font size=3
face="Times New Roman"><span style='font-size:12.0pt'>
<hr size=2 width="100%" align=center tabIndex=-1>
</span></font></div>
<p class=MsoNormal><b><font size=2 face=Tahoma><span style='font-size:10.0pt;
font-family:Tahoma;font-weight:bold'>From:</span></font></b><font size=2
face=Tahoma><span style='font-size:10.0pt;font-family:Tahoma'> Scott Weitzenkamp
(sweitzen) [mailto:sweitzen@cisco.com] <br>
<b><span style='font-weight:bold'>Sent:</span></b> Sunday, April 30, 2006 10:32
PM<br>
<b><span style='font-weight:bold'>To:</span></b> <st1:PersonName w:st="on">Woodruff,
Robert J</st1:PersonName>; Aviram Gutman; Tziporet Koren; Amit Mehrotra
(amehrotr); openfabrics-ewg@openib.org<br>
<b><span style='font-weight:bold'>Cc:</span></b> Alexander Smirnov; Davis,
Arlin R<br>
<b><span style='font-weight:bold'>Subject:</span></b> RE: [openfabrics-ewg] rc3
installation issues.</span></font><o:p></o:p></p>
</div>
<p class=MsoNormal><font size=3 face="Times New Roman"><span style='font-size:
12.0pt'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>Intel MPI 2.0 does work with OFED 1.0 rc3 on RHEL4 U3
despite the lack of uCMA, here's example output on x86_64 w/PCI-E:</span></font><o:p></o:p></p>
<p class=MsoNormal><font size=3 face="Times New Roman"><span style='font-size:
12.0pt'> <o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>.../osu_latency.x<br>
CMA: unable to open /dev/infiniband/rdma_cm<br>
CMA: unable to open /dev/infiniband/rdma_cm<br>
CMA: unable to open /dev/infiniband/rdma_cm<br>
CMA: unable to open /dev/infiniband/rdma_cm<br>
I_MPI: [0] set_up_devices(): will use device: libmpi.rdma.so<br>
I_MPI: [0] set_up_devices(): will use DAPL provider: OpenIB-scm1<br>
CMA: unable to open /dev/infiniband/rdma_cm<br>
CMA: unable to open /dev/infiniband/rdma_cm<br>
CMA: unable to open /dev/infiniband/rdma_cm<br>
CMA: unable to open /dev/infiniband/rdma_cm<br>
I_MPI: [0] set_up_devices(): will use device: libmpi.rdma.so<br>
I_MPI: [0] set_up_devices(): will use DAPL provider: OpenIB-scm1<br>
# OSU MPI Latency Test (Version 2.1)<br>
# Size Latency (us) <br>
0
7.86<br>
1
7.53<br>
2
7.37<br>
4
7.43<br>
8
7.92<br>
16
7.29<br>
32
7.36<br>
64
7.50</span></font><o:p></o:p></p>
<div>
<p class=MsoNormal><font size=3 face="Times New Roman"><span style='font-size:
12.0pt'> <o:p></o:p></span></font></p>
</div>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>Scott Weitzenkamp</span></font><o:p></o:p></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>SQA and Release Manager</span></font><o:p></o:p></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>Server Virtualization Business Unit</span></font><o:p></o:p></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>Cisco Systems</span></font><o:p></o:p></p>
<div>
<p class=MsoNormal><font size=3 face="Times New Roman"><span style='font-size:
12.0pt'> <o:p></o:p></span></font></p>
</div>
<blockquote style='border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt;
margin-left:3.75pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt'>
<p class=MsoNormal><font size=3 face="Times New Roman"><span style='font-size:
12.0pt'><o:p> </o:p></span></font></p>
<div class=MsoNormal align=center style='text-align:center'><font size=3
face="Times New Roman"><span style='font-size:12.0pt'>
<hr size=2 width="100%" align=center tabIndex=-1>
</span></font></div>
<p class=MsoNormal style='margin-bottom:12.0pt'><b><font size=2 face=Tahoma><span
style='font-size:10.0pt;font-family:Tahoma;font-weight:bold'>From:</span></font></b><font
size=2 face=Tahoma><span style='font-size:10.0pt;font-family:Tahoma'>
openfabrics-ewg-bounces@openib.org [mailto:openfabrics-ewg-bounces@openib.org] <b><span
style='font-weight:bold'>On Behalf Of </span></b><st1:PersonName w:st="on">Woodruff,
Robert J</st1:PersonName><br>
<b><span style='font-weight:bold'>Sent:</span></b> Tuesday, April 25, 2006 9:16
AM<br>
<b><span style='font-weight:bold'>To:</span></b> Aviram Gutman; Tziporet Koren;
Amit Mehrotra (amehrotr); openfabrics-ewg@openib.org<br>
<b><span style='font-weight:bold'>Cc:</span></b> Alexander Smirnov; Davis,
Arlin R<br>
<b><span style='font-weight:bold'>Subject:</span></b> RE: [openfabrics-ewg] rc3
installation issues.</span></font><o:p></o:p></p>
<p class=MsoNormal><font size=2 color=blue face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:blue'>Intel MPI needs a stable uDAPL and
supporting usermode verbs/uCMA and such.</span></font><o:p></o:p></p>
<p class=MsoNormal><font size=2 color=blue face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:blue'>If you have that, then Intel MPI should
work. If you want to get a copy of Intel MPI</span></font><o:p></o:p></p>
<p class=MsoNormal><font size=2 color=blue face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:blue'>to use in your testing of OFED, I can put
you in contact with our Intel MPI lead and he can</span></font><o:p></o:p></p>
<p class=MsoNormal><font size=2 color=blue face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:blue'>provide you with the code and a license. </span></font><o:p></o:p></p>
<p class=MsoNormal><font size=3 face="Times New Roman"><span style='font-size:
12.0pt'><o:p> </o:p></span></font></p>
<div class=MsoNormal align=center style='text-align:center'><font size=3
face="Times New Roman"><span style='font-size:12.0pt'>
<hr size=2 width="100%" align=center tabIndex=-1>
</span></font></div>
<p class=MsoNormal style='margin-bottom:12.0pt'><b><font size=2 face=Tahoma><span
style='font-size:10.0pt;font-family:Tahoma;font-weight:bold'>From:</span></font></b><font
size=2 face=Tahoma><span style='font-size:10.0pt;font-family:Tahoma'> Aviram
Gutman [mailto:aviram@mellanox.co.il] <br>
<b><span style='font-weight:bold'>Sent:</span></b> Tuesday, April 25, 2006 9:09
AM<br>
<b><span style='font-weight:bold'>To:</span></b> <st1:PersonName w:st="on">Woodruff,
Robert J</st1:PersonName>; Tziporet Koren; Amit Mehrotra (amehrotr);
openfabrics-ewg@openib.org<br>
<b><span style='font-weight:bold'>Cc:</span></b> Alexander Smirnov; Davis,
Arlin R<br>
<b><span style='font-weight:bold'>Subject:</span></b> RE: [openfabrics-ewg] rc3
installation issues.</span></font><o:p></o:p></p>
<div>
<p class=MsoNormal><font size=2 color=maroon face=Tahoma><span
style='font-size:10.0pt;font-family:Tahoma;color:maroon'>Please work with Vlad
to make sure you have what you need in rc4.</span></font><o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><font size=3 face="Times New Roman"><span style='font-size:
12.0pt'> <o:p></o:p></span></font></p>
</div>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>Regards,</span></font><o:p></o:p></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'> Aviram</span></font><o:p></o:p></p>
<div>
<p class=MsoNormal><font size=3 face="Times New Roman"><span style='font-size:
12.0pt'> <o:p></o:p></span></font></p>
</div>
<p class=MsoNormal><font size=3 face="Times New Roman"><span style='font-size:
12.0pt'><o:p> </o:p></span></font></p>
<div class=MsoNormal align=center style='text-align:center'><font size=3
face="Times New Roman"><span style='font-size:12.0pt'>
<hr size=2 width="100%" align=center tabIndex=-1>
</span></font></div>
<p class=MsoNormal style='margin-bottom:12.0pt'><b><font size=2 face=Tahoma><span
style='font-size:10.0pt;font-family:Tahoma;font-weight:bold'>From:</span></font></b><font
size=2 face=Tahoma><span style='font-size:10.0pt;font-family:Tahoma'> <st1:PersonName
w:st="on">Woodruff, Robert J</st1:PersonName> [mailto:<st1:PersonName w:st="on">rob</st1:PersonName>ert.j.woodruff@intel.com]
<br>
<b><span style='font-weight:bold'>Sent:</span></b> Tuesday, April 25, 2006 6:56
PM<br>
<b><span style='font-weight:bold'>To:</span></b> Aviram Gutman; Tziporet Koren;
Amit Mehrotra (amehrotr); openfabrics-ewg@openib.org<br>
<b><span style='font-weight:bold'>Cc:</span></b> Alexander Smirnov; Davis,
Arlin R<br>
<b><span style='font-weight:bold'>Subject:</span></b> RE: [openfabrics-ewg] rc3
installation issues.</span></font><o:p></o:p></p>
<p class=MsoNormal><font size=2 color=blue face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:blue'>Not yet. Last I heard, there was no uDAPL
in OFED that is neeed by Intel MPI. Maybe RC4. </span></font><o:p></o:p></p>
<div>
<p class=MsoNormal><font size=3 face="Times New Roman"><span style='font-size:
12.0pt'> <o:p></o:p></span></font></p>
</div>
<div>
<p class=MsoNormal><font size=3 face="Times New Roman"><span style='font-size:
12.0pt'> <o:p></o:p></span></font></p>
</div>
<div>
<p class=MsoNormal style='margin-left:.5in'><font size=2 color=blue face=Arial><span
style='font-size:10.0pt;font-family:Arial;color:blue'> Tziporet wrote,</span></font><o:p></o:p></p>
<p class=MsoNormal style='margin-left:.5in'><font size=2 color=blue face=Arial><span
style='font-size:10.0pt;font-family:Arial;color:blue'>> </span></font><font
size=2 color=navy face=Arial><span style='font-size:10.0pt;font-family:Arial;
color:navy'>Regarding uDAPL and RHEL4 (up2 & 3): Since we have not
completed the uCMA backport for kernel 2.6.9 we eliminated uDAPL </span></font><font
size=2 color=blue face=Arial><span style='font-size:10.0pt;font-family:Arial;
color:blue'> </span></font><o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><font size=3 face="Times New Roman"><span style='font-size:
12.0pt'><o:p> </o:p></span></font></p>
</div>
<div class=MsoNormal align=center style='text-align:center'><font size=3
face="Times New Roman"><span style='font-size:12.0pt'>
<hr size=2 width="100%" align=center tabIndex=-1>
</span></font></div>
<p class=MsoNormal style='margin-bottom:12.0pt'><b><font size=2 face=Tahoma><span
style='font-size:10.0pt;font-family:Tahoma;font-weight:bold'>From:</span></font></b><font
size=2 face=Tahoma><span style='font-size:10.0pt;font-family:Tahoma'> Aviram
Gutman [mailto:aviram@mellanox.co.il] <br>
<b><span style='font-weight:bold'>Sent:</span></b> Tuesday, April 25, 2006 8:48
AM<br>
<b><span style='font-weight:bold'>To:</span></b> <st1:PersonName w:st="on">Woodruff,
Robert J</st1:PersonName>; Tziporet Koren; Amit Mehrotra (amehrotr);
openfabrics-ewg@openib.org<br>
<b><span style='font-weight:bold'>Cc:</span></b> Alexander Smirnov; Davis,
Arlin R<br>
<b><span style='font-weight:bold'>Subject:</span></b> RE: [openfabrics-ewg] rc3
installation issues.</span></font><o:p></o:p></p>
<div>
<p class=MsoNormal><font size=2 color=maroon face=Tahoma><span
style='font-size:10.0pt;font-family:Tahoma;color:maroon'>Woody,</span></font><o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><font size=3 face="Times New Roman"><span style='font-size:
12.0pt'> <o:p></o:p></span></font></p>
</div>
<div>
<p class=MsoNormal><font size=2 color=maroon face=Tahoma><span
style='font-size:10.0pt;font-family:Tahoma;color:maroon'>Are you testing Intel
MPI on top of OFED?</span></font><o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><font size=3 face="Times New Roman"><span style='font-size:
12.0pt'> <o:p></o:p></span></font></p>
</div>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>Regards,</span></font><o:p></o:p></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'> Aviram</span></font><o:p></o:p></p>
<div>
<p class=MsoNormal><font size=3 face="Times New Roman"><span style='font-size:
12.0pt'> <o:p></o:p></span></font></p>
</div>
<p class=MsoNormal><font size=3 face="Times New Roman"><span style='font-size:
12.0pt'><o:p> </o:p></span></font></p>
<div class=MsoNormal align=center style='text-align:center'><font size=3
face="Times New Roman"><span style='font-size:12.0pt'>
<hr size=2 width="100%" align=center tabIndex=-1>
</span></font></div>
<p class=MsoNormal style='margin-bottom:12.0pt'><b><font size=2 face=Tahoma><span
style='font-size:10.0pt;font-family:Tahoma;font-weight:bold'>From:</span></font></b><font
size=2 face=Tahoma><span style='font-size:10.0pt;font-family:Tahoma'>
openfabrics-ewg-bounces@openib.org [mailto:openfabrics-ewg-bounces@openib.org] <b><span
style='font-weight:bold'>On Behalf Of </span></b><st1:PersonName w:st="on">Woodruff,
Robert J</st1:PersonName><br>
<b><span style='font-weight:bold'>Sent:</span></b> Monday, April 24, 2006 11:33
PM<br>
<b><span style='font-weight:bold'>To:</span></b> Tziporet Koren; Amit Mehrotra
(amehrotr); openfabrics-ewg@openib.org<br>
<b><span style='font-weight:bold'>Cc:</span></b> Alexander Smirnov; Davis,
Arlin R<br>
<b><span style='font-weight:bold'>Subject:</span></b> RE: [openfabrics-ewg] rc3
installation issues.</span></font><o:p></o:p></p>
<p class=MsoNormal><font size=3 face="Times New Roman"><span style='font-size:
12.0pt'> <o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 color=blue face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:blue'>The latest patches I have is for SVN 6509
on the trunk. </span></font><o:p></o:p></p>
<p class=MsoNormal><font size=2 color=blue face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:blue'><a
href="https://openib.org/svn/gen2/branches/backport-to-2.6.9/">https://openib.org/svn/gen2/branches/backport-to-2.6.9/</a></span></font><o:p></o:p></p>
<p class=MsoNormal><font size=2 color=blue face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:blue'>infiniband-backport-svn6509-to-2.6.9-34.EL-kernel-fixups-00.diff<br>
infiniband-backport-svn6509-to-2.6.9-34.EL-kernel-fixups-01.diff<br>
infiniband-backport-svn6509-to-2.6.9-openib-drivers-02.diff<br>
infiniband-backport-svn6509-to-2.6.9-openib-fixups-03.diff</span></font><o:p></o:p></p>
<p class=MsoNormal><font size=2 color=blue face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:blue'>I don't split the patches by component,
but rather have one to strip out the old IB</span></font><o:p></o:p></p>
<p class=MsoNormal><font size=2 color=blue face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:blue'>code in RedHat EL4.0, one to apply needed
kernel changes to the RedHat kernel,</span></font><o:p></o:p></p>
<p class=MsoNormal><font size=2 color=blue face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:blue'>one for the openib drivers, and one for
openib fixups that are needed.</span></font><o:p></o:p></p>
<p class=MsoNormal><font size=3 face="Times New Roman"><span style='font-size:
12.0pt'> <o:p></o:p></span></font></p>
<p class=MsoNormal><font size=3 face="Times New Roman"><span style='font-size:
12.0pt'> <o:p></o:p></span></font></p>
<p class=MsoNormal><font size=3 face="Times New Roman"><span style='font-size:
12.0pt'> <o:p></o:p></span></font></p>
<p class=MsoNormal><font size=3 face="Times New Roman"><span style='font-size:
12.0pt'> <o:p></o:p></span></font></p>
<p class=MsoNormal><font size=3 face="Times New Roman"><span style='font-size:
12.0pt'><o:p> </o:p></span></font></p>
<div class=MsoNormal align=center style='text-align:center'><font size=3
face="Times New Roman"><span style='font-size:12.0pt'>
<hr size=2 width="100%" align=center tabIndex=-1>
</span></font></div>
<p class=MsoNormal style='margin-bottom:12.0pt'><b><font size=2 face=Tahoma><span
style='font-size:10.0pt;font-family:Tahoma;font-weight:bold'>From:</span></font></b><font
size=2 face=Tahoma><span style='font-size:10.0pt;font-family:Tahoma'> Tziporet
Koren [mailto:tziporet@mellanox.co.il] <br>
<b><span style='font-weight:bold'>Sent:</span></b> Monday, April 24, 2006 11:43
AM<br>
<b><span style='font-weight:bold'>To:</span></b> <st1:PersonName w:st="on">Woodruff,
Robert J</st1:PersonName>; Amit Mehrotra (amehrotr);
openfabrics-ewg@openib.org<br>
<b><span style='font-weight:bold'>Cc:</span></b> Alexander Smirnov; Davis,
Arlin R<br>
<b><span style='font-weight:bold'>Subject:</span></b> RE: [openfabrics-ewg] rc3
installation issues.</span></font><o:p></o:p></p>
<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'>We plan to do a backport of uCMA for RC4.</span></font><o:p></o:p></p>
<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'>Can you point me to your backport of uCMA
for 2.6.9-34EL</span></font><o:p></o:p></p>
<p class=MsoNormal><font size=3 face="Times New Roman"><span style='font-size:
12.0pt'> <o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'>Thanks,</span></font><o:p></o:p></p>
<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'>Tziporet</span></font><o:p></o:p></p>
<p class=MsoNormal><font size=3 face="Times New Roman"><span style='font-size:
12.0pt'> <o:p></o:p></span></font></p>
<p class=MsoNormal style='margin-left:.5in'><font size=2 face=Tahoma><span
style='font-size:10.0pt;font-family:Tahoma'>-----Original Message-----<br>
<b><span style='font-weight:bold'>From:</span></b> Bob Woodruff [mailto:<st1:PersonName
w:st="on">rob</st1:PersonName>ert.j.woodruff@intel.com] <br>
<b><span style='font-weight:bold'>Sent:</span></b> Monday, April 24, 2006 6:56
PM<br>
<b><span style='font-weight:bold'>To:</span></b> Tziporet Koren; Amit Mehrotra
(amehrotr); openfabrics-ewg@openib.org<br>
<b><span style='font-weight:bold'>Cc:</span></b> Alexander Smirnov; Davis,
Arlin R<br>
<b><span style='font-weight:bold'>Subject:</span></b> RE: [openfabrics-ewg] rc3
installation issues.</span></font><o:p></o:p></p>
<p class=MsoNormal style='margin-left:.5in'><font size=3 face="Times New Roman"><span
style='font-size:12.0pt'> <o:p></o:p></span></font></p>
<p class=MsoNormal style='margin-left:.5in'><font size=2 color=blue face=Arial><span
style='font-size:10.0pt;font-family:Arial;color:blue'> Tziporet wrote,</span></font><o:p></o:p></p>
<p class=MsoNormal style='margin-left:.5in'><font size=2 color=blue face=Arial><span
style='font-size:10.0pt;font-family:Arial;color:blue'>> </span></font><font
size=2 color=navy face=Arial><span style='font-size:10.0pt;font-family:Arial;
color:navy'>Regarding uDAPL and RHEL4 (up2 & 3): Since we have not
completed the uCMA backport for kernel 2.6.9 we eliminated uDAPL </span></font><font
size=2 color=blue face=Arial><span style='font-size:10.0pt;font-family:Arial;
color:blue'> </span></font><o:p></o:p></p>
<p class=MsoNormal style='margin-left:.5in'><font size=2 color=blue face=Arial><span
style='font-size:10.0pt;font-family:Arial;color:blue'>> </span></font><font
size=2 color=navy face=Arial><span style='font-size:10.0pt;font-family:Arial;
color:navy'> installation in case of this kernel. We also indicated this
in the package limitation mail.</span></font><font size=2 color=blue
face=Arial><span style='font-size:10.0pt;font-family:Arial;color:blue'> </span></font><o:p></o:p></p>
<p class=MsoNormal style='margin-left:.5in'><font size=3 face="Times New Roman"><span
style='font-size:12.0pt'> <o:p></o:p></span></font></p>
<p class=MsoNormal style='margin-left:.5in'><font size=2 color=blue face=Arial><span
style='font-size:10.0pt;font-family:Arial;color:blue'>Note that there are two
uDAPL providers for OpenIB. One uses standard sockets for connections
(socket CM)</span></font><o:p></o:p></p>
<p class=MsoNormal style='margin-left:.5in'><font size=2 color=blue face=Arial><span
style='font-size:10.0pt;font-family:Arial;color:blue'>and the other uses CMA. I
would suggest that you should include the socket CM uDAPL provider even</span></font><o:p></o:p></p>
<p class=MsoNormal style='margin-left:.5in'><font size=2 color=blue face=Arial><span
style='font-size:10.0pt;font-family:Arial;color:blue'>if you do not have a
backport for uCMA.</span></font><o:p></o:p></p>
<p class=MsoNormal style='margin-left:.5in'><font size=2 color=blue face=Arial><span
style='font-size:10.0pt;font-family:Arial;color:blue'>BTW. - My backport
patches contain backport for uCMA for 2.6.9-34-EL if you need an example.</span></font><o:p></o:p></p>
<p class=MsoNormal style='margin-left:.5in'><font size=3 face="Times New Roman"><span
style='font-size:12.0pt'> <o:p></o:p></span></font></p>
<p class=MsoNormal style='margin-left:.5in'><font size=2 color=blue face=Arial><span
style='font-size:10.0pt;font-family:Arial;color:blue'>woody</span></font><o:p></o:p></p>
<p class=MsoNormal style='margin-left:.5in'><font size=3 face="Times New Roman"><span
style='font-size:12.0pt'> <o:p></o:p></span></font></p>
<p class=MsoNormal style='margin-left:.5in'><font size=3 face="Times New Roman"><span
style='font-size:12.0pt'> <o:p></o:p></span></font></p>
</blockquote>
</div>
</blockquote>
</div>
</div>
</body>
</html>