<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 14 (filtered medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:Helvetica;
panose-1:2 11 6 4 2 2 2 2 2 4;}
@font-face
{font-family:Helvetica;
panose-1:2 11 6 4 2 2 2 2 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p
{mso-style-priority:99;
margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
p.MsoAcetate, li.MsoAcetate, div.MsoAcetate
{mso-style-priority:99;
mso-style-link:"Balloon Text Char";
margin:0in;
margin-bottom:.0001pt;
font-size:8.0pt;
font-family:"Tahoma","sans-serif";}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.BalloonTextChar
{mso-style-name:"Balloon Text Char";
mso-style-priority:99;
mso-style-link:"Balloon Text";
font-family:"Tahoma","sans-serif";}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Andrei,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Could you tell us what HCA/Switch models you have? I am _<i>not</i>_ the expert here but I’ll see if I can get someone to help you.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Ira<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<div style="border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt">
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> users-bounces@lists.openfabrics.org [mailto:users-bounces@lists.openfabrics.org]
<b>On Behalf Of </b>Andrei Mikhailovsky<br>
<b>Sent:</b> Friday, May 24, 2013 8:18 AM<br>
<b>To:</b> Hal Rosenstock<br>
<b>Cc:</b> users@lists.openfabrics.org<br>
<b>Subject:</b> Re: [Users] HP BLc QLogic 4X QDR IB Switch oddness<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black">It's a weird problem indeed. I've tried swapping switches and blade enclosures, as well as cards. I thought that I was going mad. Nothing really helped and HP was
no help either. That tells me what hardware vendor not to go for in your Infiniband solution. HP is usually good with support, but not this time.<br>
<br>
Andrei<o:p></o:p></span></p>
<div class="MsoNormal" align="center" style="text-align:center"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black">
<hr size="2" width="100%" align="center" id="zwchr">
</span></div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><b><span style="font-family:"Helvetica","sans-serif";color:black">From:
</span></b><span style="font-family:"Helvetica","sans-serif";color:black">"Hal Rosenstock" <<a href="mailto:hal.rosenstock@gmail.com">hal.rosenstock@gmail.com</a>><br>
<b>To: </b>"Andrei Mikhailovsky" <<a href="mailto:andrei@arhont.com">andrei@arhont.com</a>><br>
<b>Cc: </b><a href="mailto:users@lists.openfabrics.org">users@lists.openfabrics.org</a><br>
<b>Sent: </b>Friday, 24 May, 2013 4:06:17 PM<br>
<b>Subject: </b>Re: [Users] HP BLc QLogic 4X QDR IB Switch oddness<br>
<br>
Andrei,<o:p></o:p></span></p>
<div>
<p class="MsoNormal"><span style="font-family:"Helvetica","sans-serif";color:black">On Fri, May 24, 2013 at 10:13 AM, Andrei Mikhailovsky <<a href="mailto:andrei@arhont.com" target="_blank">andrei@arhont.com</a>> wrote:<o:p></o:p></span></p>
<div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black">Hal,<br>
<br>
The trouble is i've tried swapping the cards and even the IB switch, makes no difference.
<br>
<br>
The link state changes that i've mentioned before keeps cycling prior to being activated. as i've mentioned it takes various amount of time before they are being activated. However, once active they stay active for months. there are no errors or any other problems
once the link becomes active.<br>
<br>
The trouble is to become active it takes a while, which is not right )))<o:p></o:p></span></p>
</div>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Helvetica","sans-serif";color:black"> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Helvetica","sans-serif";color:black">The (physical) link state cycling is indicative of some physical connectivity issue. If swapping cards doesn't work, I'm not sure what to tell you. Maybe it's a backplane issue
but likely you can't swap the chassis.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Helvetica","sans-serif";color:black"> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Helvetica","sans-serif";color:black">-- Hal<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Helvetica","sans-serif";color:black"> <o:p></o:p></span></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black"><br>
Andrei<o:p></o:p></span></p>
<div class="MsoNormal" align="center" style="text-align:center"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black">
<hr size="2" width="100%" align="center">
</span></div>
<div>
<div>
<p class="MsoNormal"><b><span style="font-family:"Helvetica","sans-serif";color:black">From:
</span></b><span style="font-family:"Helvetica","sans-serif";color:black">"Hal Rosenstock" <<a href="mailto:hal.rosenstock@gmail.com" target="_blank">hal.rosenstock@gmail.com</a>><br>
<b>To: </b>"Andrei Mikhailovsky" <<a href="mailto:andrei@arhont.com" target="_blank">andrei@arhont.com</a>><br>
<b>Cc: </b><a href="mailto:users@lists.openfabrics.org" target="_blank">users@lists.openfabrics.org</a><o:p></o:p></span></p>
</div>
<p class="MsoNormal"><b><span style="font-family:"Helvetica","sans-serif";color:black">Sent:
</span></b><span style="font-family:"Helvetica","sans-serif";color:black">Friday, 24 May, 2013 2:47:48 PM<o:p></o:p></span></p>
<div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span style="font-family:"Helvetica","sans-serif";color:black"><br>
<b>Subject: </b>Re: [Users] HP BLc QLogic 4X QDR IB Switch oddness<br>
<br>
Andrei,<o:p></o:p></span></p>
<div>
<p class="MsoNormal"><span style="font-family:"Helvetica","sans-serif";color:black">On Fri, May 24, 2013 at 9:45 AM, Andrei Mikhailovsky <<a href="mailto:andrei@arhont.com" target="_blank">andrei@arhont.com</a>> wrote:<o:p></o:p></span></p>
<div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black">Hal,<br>
<br>
The physical state changes every few seconds and goes from DOWN/Polling to DOWN/Disabled to DOWN/(something else) don't remember exactly. It just keeps cycling through them. I've never seen the ports in INIT state, which I think it should start with when there
is a connectivity.<o:p></o:p></span></p>
</div>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Helvetica","sans-serif";color:black"> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Helvetica","sans-serif";color:black">So there's some physical connectivity issue. The link is constantly trying to negotiate and come up to LinkUp but even when it gets there it doesn't stay there for long.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Helvetica","sans-serif";color:black"> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Helvetica","sans-serif";color:black">I'd start by reseating the cards which are involved with this. Hopefully that will help to stabilize things.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Helvetica","sans-serif";color:black"> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Helvetica","sans-serif";color:black">-- Hal<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Helvetica","sans-serif";color:black"> <o:p></o:p></span></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black"><br>
Thanks<o:p></o:p></span></p>
<div class="MsoNormal" align="center" style="text-align:center"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black">
<hr size="2" width="100%" align="center">
</span></div>
<div>
<p class="MsoNormal"><b><span style="font-family:"Helvetica","sans-serif";color:black">From:
</span></b><span style="font-family:"Helvetica","sans-serif";color:black">"Hal Rosenstock" <<a href="mailto:hal.rosenstock@gmail.com" target="_blank">hal.rosenstock@gmail.com</a>><br>
<b>To: </b>"Andrei Mikhailovsky" <<a href="mailto:andrei@arhont.com" target="_blank">andrei@arhont.com</a>><br>
<b>Cc: </b><a href="mailto:users@lists.openfabrics.org" target="_blank">users@lists.openfabrics.org</a><br>
<b>Sent: </b>Friday, 24 May, 2013 2:11:24 PM<br>
<b>Subject: </b>Re: [Users] HP BLc QLogic 4X QDR IB Switch oddness<o:p></o:p></span></p>
<div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span style="font-family:"Helvetica","sans-serif";color:black"><br>
<br>
Hi Andrei,<o:p></o:p></span></p>
<div>
<p class="MsoNormal"><span style="font-family:"Helvetica","sans-serif";color:black">On Fri, May 24, 2013 at 8:51 AM, Andrei Mikhailovsky <<a href="mailto:andrei@arhont.com" target="_blank">andrei@arhont.com</a>> wrote:<o:p></o:p></span></p>
<div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black">Hello guys,<br>
<br>
I was wondering if anyone on the mailing list has experienced an odd behaviour with HP BLc QLogic 4X QDR IB Switch (505958-B21)? I've tried to get HP to solve this issue, but it resulted to HP basically saying that they do not have Infiniband specialists who
could help me.<br>
<br>
My problem is that every time I reboot the blade servers which are connected to the above blade switch the switch ports take ages to Activate. The time it takes when the servers have:<br>
<br>
State: Active<br>
Physical state: LinkUp<br>
<br>
is random and varies from several minutes to over 10 hours. I couldn't find a correlation or any consistency between the servers or switch ports. The same server could connect within minutes after a reboot, but could take hours following the next reboot.
<br>
<br>
However, the server which are connected to the same switch with a cable (not interconnected blade servers) do not have this issue and get port state Active in seconds every time.<o:p></o:p></span></p>
</div>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Helvetica","sans-serif";color:black">What is the physical state of these ports when they're not active (on both the switch and server side) ? Are they LinkUp or something else ?<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Helvetica","sans-serif";color:black"> <o:p></o:p></span></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black"><br>
I have installed the latest available firmware and I have tried chaging blade mezz cards, IB switch and the blade enclosure, different versions of OFED / opensm. It stays the same.<br>
<br>
Has anyone come across this kind of behaviour?<o:p></o:p></span></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><span style="font-family:"Helvetica","sans-serif";color:black"> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Helvetica","sans-serif";color:black">Have you tried reseating your (server) cards ?<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Helvetica","sans-serif";color:black"> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Helvetica","sans-serif";color:black">Also, would you comment on the topology ? How large a subnet is this ?<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Helvetica","sans-serif";color:black"> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Helvetica","sans-serif";color:black">-- Hal<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Helvetica","sans-serif";color:black"> <o:p></o:p></span></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black"><br>
Thanks</span><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:#888888"><br>
<br>
Andrei</span><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black"><o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span style="font-family:"Helvetica","sans-serif";color:black"><br>
_______________________________________________<br>
Users mailing list<br>
<a href="mailto:Users@lists.openfabrics.org" target="_blank">Users@lists.openfabrics.org</a><br>
<a href="http://lists.openfabrics.org/cgi-bin/mailman/listinfo/users" target="_blank">http://lists.openfabrics.org/cgi-bin/mailman/listinfo/users</a><o:p></o:p></span></p>
</blockquote>
</div>
<p class="MsoNormal"><span style="font-family:"Helvetica","sans-serif";color:black"><o:p> </o:p></span></p>
</div>
</div>
</div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black"><o:p> </o:p></span></p>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal"><span style="font-family:"Helvetica","sans-serif";color:black"><o:p> </o:p></span></p>
</div>
</div>
</div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black"><o:p> </o:p></span></p>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal"><span style="font-family:"Helvetica","sans-serif";color:black"><o:p> </o:p></span></p>
</div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black"><o:p> </o:p></span></p>
</div>
</div>
</div>
</body>
</html>