<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv=Content-Type content="text/html; charset=us-ascii">
<meta name=Generator content="Microsoft Word 12 (filtered medium)">
<style>
<!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri","sans-serif";
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;}
@page Section1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.Section1
{page:Section1;}
-->
</style>
<!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang=EN-US link=blue vlink=purple>
<div class=Section1>
<p class=MsoNormal>I saw the notes about opensm not working on the 2/19 build
which is what I was using. Currently see the following behavior:<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>Have 3 blades with QDR HCAs and 2 QDR switches. The 2<sup>nd</sup>
switch only connects to the blades when nodes reboot they come up to INIT so
switch is not managed as I thought.<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>Two of the blades are installed with RHEL 5.3+OFED 1.4.1. When
the third blade is installed with OFED 1.5 the three systems all see each other
and work correctly. When the 3<sup>rd</sup> blade is installed with 1.5.1 then
it does not come ACTIVE. When you run ibnetdiscover from blade 3 you get:<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal style='margin-left:.5in'>[root@blade3 ~]# ibnetdiscover -P 2<o:p></o:p></p>
<p class=MsoNormal style='margin-left:.5in'>ibwarn: [4064] mad_rpc: _do_madrpc
failed; dport (DR path slid 0; dlid 0; 0,2)<o:p></o:p></p>
<p class=MsoNormal style='margin-left:.5in'>src/ibnetdisc.c:457; Query remote
node (DR path slid 0; dlid 0; 0,2) failed, skipping port<o:p></o:p></p>
<p class=MsoNormal style='margin-left:.5in'>#<o:p></o:p></p>
<p class=MsoNormal style='margin-left:.5in'># Topology file: generated on Sat
Feb 20 15:27:20 2010<o:p></o:p></p>
<p class=MsoNormal style='margin-left:.5in'>#<o:p></o:p></p>
<p class=MsoNormal style='margin-left:.5in'># Initiated from node
0002c9030004de20 port 0002c9030004de22<o:p></o:p></p>
<p class=MsoNormal style='margin-left:.5in'><o:p> </o:p></p>
<p class=MsoNormal style='margin-left:.5in'>vendid=0x2c9<o:p></o:p></p>
<p class=MsoNormal style='margin-left:.5in'>devid=0x673c<o:p></o:p></p>
<p class=MsoNormal style='margin-left:.5in'>sysimgguid=0x2c9030004de23<o:p></o:p></p>
<p class=MsoNormal style='margin-left:.5in'>caguid=0x2c9030004de20<o:p></o:p></p>
<p class=MsoNormal style='margin-left:.5in'>Ca 2
"H-0002c9030004de20"
# "blade3 HCA-1"<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>When you run ibnetdiscover on another blade you see:<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal style='margin-left:.5in'>[root@blade1 xrdma]# ibnetdiscover
-P 2<o:p></o:p></p>
<p class=MsoNormal style='margin-left:.5in'>ibwarn: [8307] mad_rpc: _do_madrpc
failed; dport (DR path slid 0; dlid 0; 0,2,19)<o:p></o:p></p>
<p class=MsoNormal style='margin-left:.5in'>ibwarn: [8307] handle_port:
NodeInfo on DR path slid 0; dlid 0; 0,2,19 failed, skipping port<o:p></o:p></p>
<p class=MsoNormal style='margin-left:.5in'>#<o:p></o:p></p>
<p class=MsoNormal style='margin-left:.5in'># Topology file: generated on Sat
Feb 20 15:28:19 2010<o:p></o:p></p>
<p class=MsoNormal style='margin-left:.5in'>#<o:p></o:p></p>
<p class=MsoNormal style='margin-left:.5in'># Max of 2 hops discovered<o:p></o:p></p>
<p class=MsoNormal style='margin-left:.5in'># Initiated from node
0002c9030004dc58 port 0002c9030004dc5a<o:p></o:p></p>
<p class=MsoNormal style='margin-left:.5in'><o:p> </o:p></p>
<p class=MsoNormal style='margin-left:.5in'>vendid=0x8f1<o:p></o:p></p>
<p class=MsoNormal style='margin-left:.5in'>devid=0x5a5e<o:p></o:p></p>
<p class=MsoNormal style='margin-left:.5in'>sysimgguid=0x8f1050038014d<o:p></o:p></p>
<p class=MsoNormal style='margin-left:.5in'>switchguid=0x8f1050038014c(8f1050038014c)<o:p></o:p></p>
<p class=MsoNormal style='margin-left:.5in'>Switch 36
"S-0008f1050038014c"
# "IBM HSSM" enhanced port 0 lid 8 lmc 0<o:p></o:p></p>
<p class=MsoNormal style='margin-left:.5in'>[18]
"H-0002c9030004db1c"[2](2c9030004db1e)
# "blade2 HCA-1" lid 6 4xQDR<o:p></o:p></p>
<p class=MsoNormal style='margin-left:.5in'>[17]
"H-0002c9030004dc58"[2](2c9030004dc5a)
# "blade1" lid 5 4xQDR<o:p></o:p></p>
<p class=MsoNormal style='margin-left:.5in'><o:p> </o:p></p>
<p class=MsoNormal style='margin-left:.5in'>vendid=0x2c9<o:p></o:p></p>
<p class=MsoNormal style='margin-left:.5in'>devid=0x673c<o:p></o:p></p>
<p class=MsoNormal style='margin-left:.5in'>sysimgguid=0x2c9030004db1f<o:p></o:p></p>
<p class=MsoNormal style='margin-left:.5in'>caguid=0x2c9030004db1c<o:p></o:p></p>
<p class=MsoNormal style='margin-left:.5in'>Ca 2
"H-0002c9030004db1c"
# "blade2 HCA-1"<o:p></o:p></p>
<p class=MsoNormal style='margin-left:.5in'>[2](2c9030004db1e)
"S-0008f1050038014c"[18]
# lid 6 lmc 0 "IBM HSSM" lid 8 4xQDR<o:p></o:p></p>
<p class=MsoNormal style='margin-left:.5in'><o:p> </o:p></p>
<p class=MsoNormal style='margin-left:.5in'>vendid=0x2c9<o:p></o:p></p>
<p class=MsoNormal style='margin-left:.5in'>devid=0x673c<o:p></o:p></p>
<p class=MsoNormal style='margin-left:.5in'>sysimgguid=0x2c9030004dc5b<o:p></o:p></p>
<p class=MsoNormal style='margin-left:.5in'>caguid=0x2c9030004dc58<o:p></o:p></p>
<p class=MsoNormal style='margin-left:.5in'>Ca 2
"H-0002c9030004dc58"
# "blade1"<o:p></o:p></p>
<p class=MsoNormal style='margin-left:.5in'>[2](2c9030004dc5a)
"S-0008f1050038014c"[17]
# lid 5 lmc 0 "IBM HSSM" lid 8 4xQDR<o:p></o:p></p>
<p class=MsoNormal style='margin-left:.5in'><o:p> </o:p></p>
<p class=MsoNormal>Apparently the blades do NOT work together when one is at
1.5.1 2/19 and one is earlier.<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>All of the nodes are at 2.6.818 firmware which seems to be
the most recent for IBM.<o:p></o:p></p>
</div>
</body>
</html>