<HTML dir=ltr><HEAD>
<META http-equiv=Content-Type content="text/html; charset=unicode">
<META content="MSHTML 6.00.2900.2873" name=GENERATOR>
<STYLE>
<!--
font-face
{font-family:Wingdings;}
font-face
{font-family:"Bookman Old Style";}
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman";}
a:link, span.MsoHyperlink
{color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{color:purple;
text-decoration:underline;}
span.emailstyle17
{font-family:Arial;
color:windowtext;}
span.EmailStyle18
{font-family:Arial;
color:navy;}
div.Section1
{page:Section1;}
ol
{margin-bottom:0in;}
ul
{margin-bottom:0in;}
-->
</STYLE>
<BASE href=http://mail.silverstorm.com/exchange/ksharma/Drafts/RE:%20%5Bopenfabrics-ewg%5D%20OFED-1.0-rc6%20is%20available.EML/1_text.htm></HEAD>
<BODY lang=EN-US vLink=purple link=blue>
<DIV id=idOWAReplyText69609 dir=ltr>
<DIV dir=ltr><FONT face=Arial color=#000000 size=2>Hi Tziporet:</FONT></DIV>
<DIV dir=ltr><FONT face=Arial size=2></FONT> </DIV>
<DIV dir=ltr><FONT face=Arial size=2>At the end of install script, we specify IP addresses for ib0 and ib1 interfaces. Then i reboot the system. After the system comes back, i observed that ip address of ib0 interface (ifconfig ib0) is missing. We need to configure again the ip address of ib0 and ib1 interfaces. Please let me know if this is a bug. Why do we need to specify same ip address twice? I can see the IP address which i configure during install time in ifcfg-ib0 file.</FONT></DIV>
<DIV dir=ltr><FONT face=Arial size=2></FONT> </DIV>
<DIV dir=ltr><FONT face=Arial size=2>I have seen this behaviour on RHEL4 up2 and FC4 servers. Below are some logs.</FONT></DIV>
<DIV dir=ltr><FONT face=Arial size=2></FONT> </DIV>
<DIV dir=ltr><FONT face=Arial size=2>Thanks</FONT></DIV>
<DIV dir=ltr><FONT face=Arial size=2>Karun</FONT></DIV>
<DIV dir=ltr> </DIV>
<DIV dir=ltr><FONT face=Arial size=2>################################################################</FONT></DIV>
<DIV dir=ltr>
<DIV><FONT face=Arial color=#000000 size=2></FONT> </DIV>
<DIV><FONT face=Arial color=#000000 size=2>[root@ss23 ~]# /etc/init.d/opensmd status<BR>opensm (pid 3216) is running...<BR>[root@ss23 ~]# cat /etc/sysconfig/network-scripts/ifcfg-ib0<BR>DEVICE=ib0<BR>BOOTPROTO=static<BR>IPADDR=172.20.51.223<BR>NETMASK=255.255.255.0<BR>NETWORK=172.20.0.0<BR>BROADCAST=172.20.51.255<BR>ONBOOT=yes<BR>[root@ss23 ~]# ifconfig ib0<BR>ib0 Link encap:InfiniBand HWaddr 00:00:00:14:FE:80:00:00:00:00:00:00:00:00:00:00:00:00:00:00 <BR> inet6 addr: fe80::206:6a00:a000:537/64 Scope:Link<BR> UP BROADCAST RUNNING MULTICAST MTU:2044 Metric:1<BR> RX packets:0 errors:0 dropped:0 overruns:0 frame:0<BR> TX packets:0 errors:0 dropped:0 overruns:0 carrier:0<BR> collisions:0 txqueuelen:128 <BR> RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)</FONT></DIV>
<DIV><FONT face=Arial color=#000000 size=2>[root@ss23 ~]# <BR></FONT></DIV></DIV>
<DIV dir=ltr><FONT face=Arial size=2>##############################################################################</FONT></DIV>
<DIV dir=ltr><FONT face=Arial size=2></FONT> </DIV>
<DIV dir=ltr><FONT face=Arial size=2></FONT> </DIV></DIV>
<DIV dir=ltr><BR>
<HR tabIndex=-1>
<FONT face=Tahoma size=2><B>From:</B> openfabrics-ewg-bounces@openib.org on behalf of Tziporet Koren<BR><B>Sent:</B> Wed 6/7/2006 10:58 AM<BR><B>To:</B> Tziporet Koren; openfabrics-ewg@openib.org<BR><B>Cc:</B> openib-general<BR><B>Subject:</B> [openfabrics-ewg] OFED-1.0-rc6 is available<BR></FONT><BR></DIV>
<DIV>
<DIV class=Section1>
<P class=MsoNormal><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">Hi All,</SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial"></SPAN></FONT> </P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">We have prepared OFED 1.0 RC6.</SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">Release location: https://openib.org/svn/gen2/branches/1.0/ofed/releases</SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">File: OFED-1.0-rc6.tgz</SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial"></SPAN></FONT> </P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">Note: This release is the code freeze release for OFED 1.0. Only showstopper bugs will be fixed.</SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial"></SPAN></FONT> </P>
<P class=MsoNormal><B><FONT face=Arial size=2><SPAN style="FONT-WEIGHT: bold; FONT-SIZE: 10pt; FONT-FAMILY: Arial">BUILD_ID:</SPAN></FONT></B></P>
<P class=MsoNormal style="MARGIN-LEFT: 0.5in"><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">OFED-1.0-rc6</SPAN></FONT></P>
<P class=MsoNormal style="MARGIN-LEFT: 0.5in"><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial"></SPAN></FONT> </P>
<P class=MsoNormal style="MARGIN-LEFT: 0.5in"><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">openib-1.0 (REV=7772)</SPAN></FONT></P>
<P class=MsoNormal style="MARGIN-LEFT: 0.5in"><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial"># User space</SPAN></FONT></P>
<P class=MsoNormal style="MARGIN-LEFT: 0.5in"><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">https://openib.org/svn/gen2/branches/1.0/src/userspace</SPAN></FONT></P>
<P class=MsoNormal style="MARGIN-LEFT: 0.5in"><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial"># Kernel space</SPAN></FONT></P>
<P class=MsoNormal style="MARGIN-LEFT: 0.5in"><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">https://openib.org/svn/gen2/branches/1.0/ofed/tags/rc6/linux-kernel</SPAN></FONT></P>
<P class=MsoNormal style="MARGIN-LEFT: 0.5in"><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">Git:</SPAN></FONT></P>
<P class=MsoNormal style="MARGIN-LEFT: 0.5in"><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">ref: refs/heads/for-2.6.17</SPAN></FONT></P>
<P class=MsoNormal style="MARGIN-LEFT: 0.5in"><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">commit d9ec5ad24ce80b7ef69a0717363db661d13aada5</SPAN></FONT></P>
<P class=MsoNormal style="MARGIN-LEFT: 0.5in"><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial"></SPAN></FONT> </P>
<P class=MsoNormal style="MARGIN-LEFT: 0.5in"><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial"># MPI</SPAN></FONT></P>
<P class=MsoNormal style="MARGIN-LEFT: 0.5in"><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">mpi_osu-0.9.7-mlx2.1.0.tgz</SPAN></FONT></P>
<P class=MsoNormal style="MARGIN-LEFT: 0.5in"><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">openmpi-1.1b1-1.src.rpm</SPAN></FONT></P>
<P class=MsoNormal style="MARGIN-LEFT: 0.5in"><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">mpitests-1.0-0.src.rpm </SPAN></FONT></P>
<P class=MsoNormal><B><FONT face=Arial size=2><SPAN style="FONT-WEIGHT: bold; FONT-SIZE: 10pt; FONT-FAMILY: Arial"></SPAN></FONT></B> </P>
<P class=MsoNormal><B><FONT face=Arial size=2><SPAN style="FONT-WEIGHT: bold; FONT-SIZE: 10pt; FONT-FAMILY: Arial">OSes:</SPAN></FONT></B></P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial"> * RH EL4 up2: 2.6.9-22.ELsmp</SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial"> * RH EL4 up3: 2.6.9-34.ELsmp</SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial"> * Fedora C4: 2.6.11-1.1369_FC4</SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial"> * SLES10 RC2: 2.6.16.16-1.6-smp</SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial"> * SUSE 10 Pro: 2.6.13-15-smp</SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial"> * kernel.org: 2.6.16.x</SPAN></FONT></P>
<P class=MsoNormal><B><FONT face=Arial size=2><SPAN style="FONT-WEIGHT: bold; FONT-SIZE: 10pt; FONT-FAMILY: Arial"></SPAN></FONT></B> </P>
<P class=MsoNormal><B><FONT face=Arial size=2><SPAN style="FONT-WEIGHT: bold; FONT-SIZE: 10pt; FONT-FAMILY: Arial">Systems:</SPAN></FONT></B></P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial"> * x86_64</SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial"> * x86</SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial"> * ia64</SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial"> * ppc64</SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial"></SPAN></FONT> </P>
<P class=MsoNormal><B><FONT face=Arial size=2><SPAN style="FONT-WEIGHT: bold; FONT-SIZE: 10pt; FONT-FAMILY: Arial">Main changes from RC5:</SPAN></FONT></B></P>
<P class=MsoNormal style="MARGIN-LEFT: 0.25in; TEXT-INDENT: -0.25in"><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">1.<FONT face="Times New Roman" size=1><SPAN style="FONT: 7pt 'Times New Roman'"> </SPAN></FONT></SPAN></FONT><SPAN dir=ltr><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">SDP – libsdp implementation of RFC proposed by Eitan Zahavi; bug fixes in kernel module. See details below.</SPAN></FONT></SPAN></P>
<P class=MsoNormal style="MARGIN-LEFT: 0.25in; TEXT-INDENT: -0.25in"><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">2.<FONT face="Times New Roman" size=1><SPAN style="FONT: 7pt 'Times New Roman'"> </SPAN></FONT></SPAN></FONT><SPAN dir=ltr><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">SRP – bug fixes</SPAN></FONT></SPAN></P>
<P class=MsoNormal style="MARGIN-LEFT: 0.25in; TEXT-INDENT: -0.25in"><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">3.<FONT face="Times New Roman" size=1><SPAN style="FONT: 7pt 'Times New Roman'"> </SPAN></FONT></SPAN></FONT><SPAN dir=ltr><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">Open MPI – new package based on 1.1b1-1</SPAN></FONT></SPAN></P>
<P class=MsoNormal style="MARGIN-LEFT: 0.25in; TEXT-INDENT: -0.25in"><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">4.<FONT face="Times New Roman" size=1><SPAN style="FONT: 7pt 'Times New Roman'"> </SPAN></FONT></SPAN></FONT><SPAN dir=ltr><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">OSU-MPI – See details below.</SPAN></FONT></SPAN></P>
<P class=MsoNormal style="MARGIN-LEFT: 0.25in; TEXT-INDENT: -0.25in"><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">5.<FONT face="Times New Roman" size=1><SPAN style="FONT: 7pt 'Times New Roman'"> </SPAN></FONT></SPAN></FONT><SPAN dir=ltr><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">iSER: Enhanced to support SLES 10 RC1.</SPAN></FONT></SPAN></P>
<P class=MsoNormal style="MARGIN-LEFT: 0.25in; TEXT-INDENT: -0.25in"><FONT face="Times New Roman" size=3><SPAN style="FONT-SIZE: 12pt">6.<FONT face="Times New Roman" size=1><SPAN style="FONT: 7pt 'Times New Roman'"> </SPAN></FONT></SPAN></FONT><SPAN dir=ltr><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">IPoIB default configuration changed:</SPAN></FONT></SPAN></P>
<P class=MsoNormal style="MARGIN-LEFT: 0.75in; TEXT-INDENT: -0.25in"><FONT face="Times New Roman" size=3><SPAN style="FONT-SIZE: 12pt">a.<FONT face="Times New Roman" size=1><SPAN style="FONT: 7pt 'Times New Roman'"> </SPAN></FONT></SPAN></FONT><SPAN dir=ltr><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">IPoIB configuration at install time is now optional.</SPAN></FONT></SPAN></P>
<P class=MsoNormal style="MARGIN-LEFT: 0.75in; TEXT-INDENT: -0.25in"><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">b.<FONT face="Times New Roman" size=1><SPAN style="FONT: 7pt 'Times New Roman'"> </SPAN></FONT></SPAN></FONT><SPAN dir=ltr><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">The default configuration of IPoIB interfaces (if performed at install time) is DHCP; it can be changed during interactive installation.</SPAN></FONT></SPAN></P>
<P class=MsoNormal style="MARGIN-LEFT: 0.75in; TEXT-INDENT: -0.25in"><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">c.<FONT face="Times New Roman" size=1><SPAN style="FONT: 7pt 'Times New Roman'"> </SPAN></FONT></SPAN></FONT><SPAN dir=ltr><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">For unattended installation one can give a new configuration file. See the example below.</SPAN></FONT></SPAN></P>
<P class=MsoNormal style="MARGIN-LEFT: 0.25in; TEXT-INDENT: -0.25in"><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">7.<FONT face="Times New Roman" size=1><SPAN style="FONT: 7pt 'Times New Roman'"> </SPAN></FONT></SPAN></FONT><SPAN dir=ltr><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">Bug Fixes.</SPAN></FONT></SPAN></P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial"></SPAN></FONT> </P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial"></SPAN></FONT> </P>
<P class=MsoNormal><B><FONT face=Arial size=2><SPAN style="FONT-WEIGHT: bold; FONT-SIZE: 10pt; FONT-FAMILY: Arial">Package limitations:</SPAN></FONT></B></P>
<P class=MsoNormal style="MARGIN-LEFT: 0.25in; TEXT-INDENT: -0.25in"><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">1.<FONT face="Times New Roman" size=1><SPAN style="FONT: 7pt 'Times New Roman'"> </SPAN></FONT></SPAN></FONT><SPAN dir=ltr><B><FONT face=Arial size=2><SPAN style="FONT-WEIGHT: bold; FONT-SIZE: 10pt; FONT-FAMILY: Arial">The ipath driver does not compile/load on most systems. To be fixed in final release.</SPAN></FONT></B></SPAN><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial"> <BR>Meanwhile, one must work with custom build and not choose ipath driver, or change in the conf file: ib_ipath=n.<BR>I attached a reference ofed-no_ipath.conf file. <BR>Once Qlogic fixes the backport patches I will publish them on the release page so any one interested can use them with this release.</SPAN></FONT></P>
<P class=MsoNormal style="MARGIN-LEFT: 0.25in; TEXT-INDENT: -0.25in"><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">2.<FONT face="Times New Roman" size=1><SPAN style="FONT: 7pt 'Times New Roman'"> </SPAN></FONT></SPAN></FONT><SPAN dir=ltr><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">iSER is working on SuSE SLES 10 RC1 only</SPAN></FONT></SPAN></P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial"></SPAN></FONT> </P>
<P class=MsoNormal style="MARGIN-LEFT: 0.5in"><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial"></SPAN></FONT> </P>
<P class=MsoNormal><B><FONT face=Arial size=2><SPAN style="FONT-WEIGHT: bold; FONT-SIZE: 10pt; FONT-FAMILY: Arial">IPoIB configuration file example:</SPAN></FONT></B></P>
<P class=MsoNormal style="MARGIN-LEFT: 0.5in"><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">If you are going to install OFED on a 32 node cluster and want to use static IPoIB configuration based on Ethernet device configuration follow instructions below:</SPAN></FONT></P>
<P class=MsoNormal style="MARGIN-LEFT: 0.5in"><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial"></SPAN></FONT> </P>
<P class=MsoNormal style="MARGIN-LEFT: 0.5in"><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">Assume that the Ethernet IP addresses (eth0 interfaces) of the cluster are: 10.0.0.1 - 10.0.0.32</SPAN></FONT></P>
<P class=MsoNormal style="MARGIN-LEFT: 0.5in"><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">and you want to assign to ib0 IP addresses in the range: 192.168.0.1 - 192.168.0.32</SPAN></FONT></P>
<P class=MsoNormal style="MARGIN-LEFT: 0.5in"><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">and to ib1 IP addresses in the range: 172.16.0.1 - 172.16.0.32</SPAN></FONT></P>
<P class=MsoNormal style="MARGIN-LEFT: 0.5in"><FONT face="Times New Roman" size=3><SPAN style="FONT-SIZE: 12pt"></SPAN></FONT> </P>
<P class=MsoNormal style="MARGIN-LEFT: 0.5in"><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">Then create the file ofed_net.conf with the following lines:</SPAN></FONT></P>
<P class=MsoNormal style="MARGIN-LEFT: 0.5in"><FONT face="Times New Roman" size=3><SPAN style="FONT-SIZE: 12pt"></SPAN></FONT> </P>
<P class=MsoNormal style="MARGIN-LEFT: 0.5in"><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">LAN_INTERFACE_ib0=eth0<BR>IPADDR_ib0=192.168.'*'.'*'<BR>NETMASK_ib0=255.255.0.0<BR>NETWORK_ib0=192.168.0.0<BR>BROADCAST_ib0=192.168.255.255<BR>ONBOOT_ib0=1<BR>LAN_INTERFACE_ib1=eth0<BR>IPADDR_ib1=172.16.'*'.'*'<BR>NETMASK_ib1=255.255.0.0<BR>NETWORK_ib1=172.16.0.0<BR>BROADCAST_ib1=172.16.255.255<BR>ONBOOT_ib1=1</SPAN></FONT></P>
<P class=MsoNormal style="MARGIN-LEFT: 0.5in"><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial"></SPAN></FONT> </P>
<P class=MsoNormal style="MARGIN-LEFT: 0.5in"><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">Note: ‘*’ will be replaced by the corresponding octal from the eth0 IP address.</SPAN></FONT></P>
<P class=MsoNormal style="MARGIN-LEFT: 0.5in"><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial"></SPAN></FONT> </P>
<P class=MsoNormal style="MARGIN-LEFT: 0.5in"><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">Assuming that you already have OFED configuration file (ofed.conf) with selected packages (created by running OFED-1.0/install.sh)</SPAN></FONT></P>
<P class=MsoNormal style="MARGIN-LEFT: 0.5in"><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">Run: ./install.sh -c ofed.conf -net ofed_net.conf</SPAN></FONT></P>
<P class=MsoNormal style="MARGIN-LEFT: 0.5in"><FONT face=Arial color=blue size=2><SPAN style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: Arial"></SPAN></FONT> </P>
<P class=MsoNormal style="MARGIN-LEFT: 0.5in"><FONT face=Arial color=blue size=2><SPAN style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: Arial"></SPAN></FONT> </P>
<P class=MsoNormal><FONT face=Arial color=navy size=2><SPAN style="FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial"></SPAN></FONT> </P>
<P class=MsoNormal><B><FONT face=Arial size=2><SPAN style="FONT-WEIGHT: bold; FONT-SIZE: 10pt; FONT-FAMILY: Arial">OSU MPI:</SPAN></FONT></B></P>
<P class=MsoNormal style="MARGIN-LEFT: 0.25in; TEXT-INDENT: -0.25in"><FONT face=Symbol size=3><SPAN style="FONT-SIZE: 12pt; FONT-FAMILY: Symbol">·<FONT face="Times New Roman" size=1><SPAN style="FONT: 7pt 'Times New Roman'"> </SPAN></FONT></SPAN></FONT><SPAN dir=ltr><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">Added mpi_alltoall fine tuning parameters</SPAN></FONT></SPAN></P>
<P class=MsoNormal style="MARGIN-LEFT: 0.25in; TEXT-INDENT: -0.25in"><FONT face=Symbol size=3><SPAN style="FONT-SIZE: 12pt; FONT-FAMILY: Symbol">·<FONT face="Times New Roman" size=1><SPAN style="FONT: 7pt 'Times New Roman'"> </SPAN></FONT></SPAN></FONT><SPAN dir=ltr><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">Added default configuration/documentation file $MPIHOME/etc/mvapich.conf</SPAN></FONT></SPAN></P>
<P class=MsoNormal style="MARGIN-LEFT: 0.25in; TEXT-INDENT: -0.25in"><FONT face=Symbol size=3><SPAN style="FONT-SIZE: 12pt; FONT-FAMILY: Symbol">·<FONT face="Times New Roman" size=1><SPAN style="FONT: 7pt 'Times New Roman'"> </SPAN></FONT></SPAN></FONT><SPAN dir=ltr><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">Added shell configuration files $MPIHOME/etc/mvapich.csh , $MPIHOME/etc/mvapich.csh</SPAN></FONT></SPAN></P>
<P class=MsoNormal style="MARGIN-LEFT: 0.25in; TEXT-INDENT: -0.25in"><FONT face=Symbol size=3><SPAN style="FONT-SIZE: 12pt; FONT-FAMILY: Symbol">·<FONT face="Times New Roman" size=1><SPAN style="FONT: 7pt 'Times New Roman'"> </SPAN></FONT></SPAN></FONT><SPAN dir=ltr><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">Default MTU was changed back to 2K for InfiniHost III Ex and InfiniHost III Lx HCAs. For InfiniHost card recommended value is:<BR>VIADEV_DEFAULT_MTU=MTU1024</SPAN></FONT></SPAN></P>
<P class=MsoNormal><B><FONT face=Arial color=navy size=2><SPAN style="FONT-WEIGHT: bold; FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial"></SPAN></FONT></B> </P>
<P class=MsoNormal><B><FONT face=Arial color=navy size=2><SPAN style="FONT-WEIGHT: bold; FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial"></SPAN></FONT></B> </P>
<P class=MsoNormal><B><FONT face=Arial size=2><SPAN style="FONT-WEIGHT: bold; FONT-SIZE: 10pt; FONT-FAMILY: Arial">SDP Details: </SPAN></FONT></B></P>
<P class=MsoNormal style="MARGIN-LEFT: 0.25in"><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">libsdp enhancements according to the RFC:</SPAN></FONT></P>
<OL type=1>
<LI class=MsoNormal><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">New config syntax (please see libsdp.conf)</SPAN></FONT>
<LI class=MsoNormal><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">With no config or empty config use SIMPLE_LIBSDP mode</SPAN></FONT>
<LI class=MsoNormal><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">Support listening on both tcp and sdp</SPAN></FONT>
<LI class=MsoNormal><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">Support trying both connections (first SDP then TCP)</SPAN></FONT>
<LI class=MsoNormal><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">Support IPv4 embedded in IPv6 (also convert back address)</SPAN></FONT>
<LI class=MsoNormal><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">Comprehensive verbosity logging</SPAN></FONT>
<LI class=MsoNormal><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">BNF based config parser</SPAN></FONT> </LI></OL>
<P class=MsoNormal><FONT face=Arial color=navy size=2><SPAN style="FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial"></SPAN></FONT> </P>
<P class=MsoNormal style="MARGIN-LEFT: 0.25in"><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">Current SDP limitations:</SPAN></FONT></P>
<P class=MsoNormal style="MARGIN-LEFT: 0.75in; TEXT-INDENT: -0.25in"><FONT face=Symbol size=3><SPAN style="FONT-SIZE: 12pt; FONT-FAMILY: Symbol">·<FONT face="Times New Roman" size=1><SPAN style="FONT: 7pt 'Times New Roman'"> </SPAN></FONT></SPAN></FONT><SPAN dir=ltr><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">SDP currently does not support sending/receiving out of band data (MSG_OOB).</SPAN></FONT></SPAN></P>
<P class=MsoNormal style="MARGIN-LEFT: 0.75in; TEXT-INDENT: -0.25in"><FONT face=Symbol color=navy size=3><SPAN style="FONT-SIZE: 12pt; COLOR: navy; FONT-FAMILY: Symbol">·<FONT face="Times New Roman" size=1><SPAN style="FONT: 7pt 'Times New Roman'"> </SPAN></FONT></SPAN></FONT><SPAN dir=ltr><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">Generally, SDP supports only SOL_SOCKET socket options.</SPAN></FONT></SPAN></P>
<P class=MsoNormal style="MARGIN-LEFT: 0.75in; TEXT-INDENT: -0.25in"><FONT face=Symbol color=navy size=3><SPAN style="FONT-SIZE: 12pt; COLOR: navy; FONT-FAMILY: Symbol">·<FONT face="Times New Roman" size=1><SPAN style="FONT: 7pt 'Times New Roman'"> </SPAN></FONT></SPAN></FONT><SPAN dir=ltr><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">The following options can be set but actual support is missing:</SPAN></FONT></SPAN></P>
<P class=MsoNormal style="MARGIN-LEFT: 1.25in; TEXT-INDENT: -0.25in"><FONT face="Courier New" size=3><SPAN style="FONT-SIZE: 12pt; FONT-FAMILY: 'Courier New'">o<FONT face="Times New Roman" size=1><SPAN style="FONT: 7pt 'Times New Roman'"> </SPAN></FONT></SPAN></FONT><SPAN dir=ltr><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">SO_KEEPALIVE - no keepalives are sent</SPAN></FONT></SPAN></P>
<P class=MsoNormal style="MARGIN-LEFT: 1.25in; TEXT-INDENT: -0.25in"><FONT face="Courier New" size=3><SPAN style="FONT-SIZE: 12pt; FONT-FAMILY: 'Courier New'">o<FONT face="Times New Roman" size=1><SPAN style="FONT: 7pt 'Times New Roman'"> </SPAN></FONT></SPAN></FONT><SPAN dir=ltr><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">SO_OOBINLINE - out of band data is not supported</SPAN></FONT></SPAN></P>
<P class=MsoNormal style="MARGIN-LEFT: 1.25in; TEXT-INDENT: -0.25in"><FONT face="Courier New" size=3><SPAN style="FONT-SIZE: 12pt; FONT-FAMILY: 'Courier New'">o<FONT face="Times New Roman" size=1><SPAN style="FONT: 7pt 'Times New Roman'"> </SPAN></FONT></SPAN></FONT><SPAN dir=ltr><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">SDP currently supports setting the following SOL_TCP socket options:</SPAN></FONT></SPAN></P>
<P class=MsoNormal style="MARGIN-LEFT: 1.25in; TEXT-INDENT: -0.25in"><FONT face="Courier New" size=3><SPAN style="FONT-SIZE: 12pt; FONT-FAMILY: 'Courier New'">o<FONT face="Times New Roman" size=1><SPAN style="FONT: 7pt 'Times New Roman'"> </SPAN></FONT></SPAN></FONT><SPAN dir=ltr><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">TCP_NODELAY, TCP_CORK - but actual support for these options is still missing</SPAN></FONT></SPAN></P>
<P class=MsoNormal style="MARGIN-LEFT: 0.75in; TEXT-INDENT: -0.25in"><FONT face=Symbol size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Symbol">·</SPAN></FONT><FONT size=1><SPAN style="FONT-SIZE: 7pt"> </SPAN></FONT><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">SDP currently does not handle Zcopy mode messages correctly and does not set MaxAdverts properly in HH/HAH messages.</SPAN></FONT></P>
<P class=MsoNormal style="MARGIN-LEFT: 0.25in"><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial"></SPAN></FONT> </P>
<P class=MsoNormal style="MARGIN-LEFT: 0.25in"><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial"></SPAN></FONT> </P>
<P class=MsoNormal><B><FONT face=Arial size=2><SPAN style="FONT-WEIGHT: bold; FONT-SIZE: 10pt; FONT-FAMILY: Arial">OFED components tested by Mellanox:</SPAN></FONT></B></P>
<UL type=disc>
<LI class=MsoNormal><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">Verbs over mthca</SPAN></FONT>
<LI class=MsoNormal><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">IPoIB</SPAN></FONT>
<LI class=MsoNormal><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">OpenSM</SPAN></FONT>
<LI class=MsoNormal><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">OSU-MPI</SPAN></FONT>
<LI class=MsoNormal><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">SRP</SPAN></FONT>
<LI class=MsoNormal><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">SDP</SPAN></FONT>
<LI class=MsoNormal><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">IB administration utils (ibutils)</SPAN></FONT> </LI></UL>
<P class=MsoNormal><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial"></SPAN></FONT> </P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial"></SPAN></FONT> </P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">Please send us any issues you encounter and/or test results.</SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial"></SPAN></FONT> </P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">Thanks</SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">Tziporet & Vlad</SPAN></FONT></P>
<P class=MsoNormal style="MARGIN-LEFT: 0.5in"><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial"></SPAN></FONT> </P>
<P class=MsoNormal style="MARGIN-LEFT: 0.5in"><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial"></SPAN></FONT> </P>
<P class=MsoNormal style="MARGIN-LEFT: 0.5in"><FONT face="Bookman Old Style" size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: 'Bookman Old Style'">Tziporet Koren</SPAN></FONT></P>
<P class=MsoNormal style="MARGIN-LEFT: 0.5in"><FONT face="Bookman Old Style" size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: 'Bookman Old Style'">Software Director</SPAN></FONT></P>
<P class=MsoNormal style="MARGIN-LEFT: 0.5in"><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">Mellanox Technologies</SPAN></FONT></P>
<P class=MsoNormal style="MARGIN-LEFT: 0.5in"><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">mailto: <A href="mailto:tziporet@mellanox.co.il"><FONT color=black><SPAN style="COLOR: windowtext">tziporet@mellanox.co.il</SPAN></FONT></A><BR>Tel +972-4-9097200, ext 380</SPAN></FONT></P>
<P class=MsoNormal style="MARGIN-LEFT: 0.5in"><FONT face="Times New Roman" size=3><SPAN style="FONT-SIZE: 12pt"></SPAN></FONT> </P></DIV></DIV></BODY></HTML>