<div dir="ltr"><div class="gmail_default" style="font-family:monospace,monospace;font-size:small;color:rgb(11,83,148)">I'm noticing that the ISL port from the HP IB SW does not had the LLR state 'active' instead is said 'Inactive' and the LLR supported speeds are <span id="span_default_id">FDR10, FDR only, so I </span>try to change the LLR profile in order to support QDR but when I try to change it, it displays an error msg: "Cannot map profile hp-encl-isl to port: 1/15"<br><br></div><div class="gmail_default" style="font-family:monospace,monospace;font-size:small;color:rgb(11,83,148)">Any ideas? Maybe that's an issue and because of that the ports are not coming up? or they supposed to work anyway?<br><br></div><div class="gmail_extra"><br clear="all"><div><div class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><span><div><div dir="ltr"><p><b><span style="font-size:10.0pt;font-family:Trebuchet MS,sans-serif;color:#444444"></span></b></p><span style="color:rgb(68,68,68)"><span style="font-family:trebuchet ms,sans-serif"><font size="2"><b>German</b></font></span></span><span style="color:rgb(68,68,68)"><span style="font-family:trebuchet ms,sans-serif"><font size="2"><font size="1"><a href="mailto:ganders@despegar.com" target="_blank"></a></font></font></span></span></div></div></span></div></div></div></div></div>
<br><div class="gmail_quote">2015-10-07 12:54 GMT-03:00 Weiny, Ira <span dir="ltr"><<a href="mailto:ira.weiny@intel.com" target="_blank">ira.weiny@intel.com</a>></span>:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div link="blue" vlink="purple" lang="EN-US">
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">There are a few issues for routing in that diagram but the links should come up.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">I assume there is some backplane between the blade servers and the switch in that chassis?<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Have you gotten libipathverbs installed?<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">In ipathverbs there is a serdes tuning script.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><a href="https://github.com/01org/libipathverbs/blob/master/truescale-serdes.cmds" target="_blank">https://github.com/01org/libipathverbs/blob/master/truescale-serdes.cmds</a><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Does your libipathverbs include that file? If not try the latest from github.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Ira<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<div style="border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt">
<div>
<div style="border:none;border-top:solid #b5c4df 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> <a href="mailto:users-bounces@lists.openfabrics.org" target="_blank">users-bounces@lists.openfabrics.org</a> [mailto:<a href="mailto:users-bounces@lists.openfabrics.org" target="_blank">users-bounces@lists.openfabrics.org</a>]
<b>On Behalf Of </b>German Anders<br>
<b>Sent:</b> Wednesday, October 07, 2015 8:41 AM<br>
<b>To:</b> Hal Rosenstock<br>
<b>Cc:</b> <a href="mailto:users@lists.openfabrics.org" target="_blank">users@lists.openfabrics.org</a><br>
<b>Subject:</b> Re: [Users] IB topology config and polling state<u></u><u></u></span></p>
</div>
</div><div><div class="h5">
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span style="font-family:"Courier New";color:#0b5394">Hi Hal,<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span style="font-family:"Courier New";color:#0b5394">Thanks for the reply, I've attach a pdf with the diagram topology, I don't know if this is the best way to go or if there's another way to connect and setup
the IB network, tips and suggestions will be very appreciated, also the mezzanine cards are already installed on the blade hosts:<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span style="font-family:"Courier New";color:#0b5394"># lspci<br>
(...)<br>
41:00.0 InfiniBand: QLogic Corp. IBA7322 QDR InfiniBand HCA (rev 02)<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New";color:#0b5394"><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span style="font-family:"Courier New";color:#0b5394">Thanks in advance,<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span style="font-family:"Courier New";color:#0b5394">Cheers,<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><br clear="all">
<u></u><u></u></p>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Trebuchet MS","sans-serif";color:#444444">German</span></b><u></u><u></u></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<p class="MsoNormal">2015-10-07 11:47 GMT-03:00 Hal Rosenstock <<a href="mailto:hal.rosenstock@gmail.com" target="_blank">hal.rosenstock@gmail.com</a>>:<u></u><u></u></p>
<div>
<div>
<p class="MsoNormal">Hi again German,<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Looks like you made some progress from yesterday as the qib ports are now Polling rather than Disabled.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">But since they are Down, do you have them cabled to a switch ? That should bring the links up and the port state will be Init. That is the "starting" point.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">You will also then need to be running SM to bring the ports up to Active.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">-- Hal<u></u><u></u></p>
</div>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<div>
<div>
<p class="MsoNormal">On Wed, Oct 7, 2015 at 10:37 AM, German Anders <<a href="mailto:ganders@despegar.com" target="_blank">ganders@despegar.com</a>> wrote:<u></u><u></u></p>
</div>
</div>
<blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<div>
<div>
<div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span style="font-family:"Courier New";color:#0b5394">Hi all,<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span style="font-family:"Courier New";color:#0b5394">I don't know if this is the mailist list for this kind of topic but I'm really new to IB and I've just install two SX6036G gateways connected to each other
through two ISL ports, then I've configured a proxy-arp between both nodes (sm is disable on both gw's):<br>
<br>
GWIB01 [proxy-ha-group: master] (config) # show proxy-arp ha <br>
<br>
Load balancing algorithm: ib-base-ip<br>
Number of Proxy-Arp interfaces: 1<br>
<br>
Proxy-ARP VIP<br>
=============<br>
Pra-group name: proxy-ha-group<br>
HA VIP address: 10.xx.xx.xx/xx<br>
<br>
Active nodes:<br>
ID State IP <br>
--------------------------------------------------------------<br>
GWIB01 master 10.xx.xx.xx1 <br>
GWIB02 standby 10.xx.xx.xx2 <u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span style="font-family:"Courier New";color:#0b5394">Then I setup two SX6018F switches (<b>SWIB01</b> and
<b>SWIB02</b>), one connected to GWIB01 and the other connected to GWIB02. The SM is configured locally on both SWIB01 & SWIB02 switches. So far so good, after this config I setup a commodity server with a MLNX IB ADPT FDR to the SWIB01 & SWIB02 switches, config
the drivers, etc and then get it up & running fine. <br>
<br>
Finally I've setup a HP Enclosure with an internal IB SW (then connect port 1 of the internal SW to GWIB01 - link is up but LLR status is inactive), install one of the blades and I see the following:<br>
<br>
# ibstat<br>
CA 'qib0'<br>
CA type: InfiniPath_QMH7342<br>
Number of ports: 2<br>
Firmware version: <br>
Hardware version: 2<br>
Node GUID: 0x0011750000791fec<br>
System image GUID: 0x0011750000791fec<br>
Port 1:<br>
State: Down<br>
Physical state: Polling<br>
Rate: 40<br>
Base lid: 4660<br>
LMC: 0<br>
SM lid: 4660<br>
Capability mask: 0x0761086a<br>
Port GUID: 0x0011750000791fec<br>
Link layer: InfiniBand<br>
Port 2:<br>
State: Down<br>
Physical state: Polling<br>
Rate: 40<br>
Base lid: 4660<br>
LMC: 0<br>
SM lid: 4660<br>
Capability mask: 0x0761086a<br>
Port GUID: 0x0011750000791fed<br>
Link layer: InfiniBand<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span style="font-family:"Courier New";color:#0b5394">So I was wondering if maybe the SM is not being recognized on the Blade system and that's why is not passing the Polling state, is that possible? Or maybe
is not possible to connect an ISL between the GW and the HP internal SW so that the sm is available or maybe the inactive LLR is causing this thing, any ideas? I
</span><span style="font-family:"Courier New";color:#0b5394" lang="EN">thought about connecting the ISL of the HP IB SW to the SWIB01 or SWIB02 instead of the GW's but I don't have any available ports.</span><span style="font-family:"Courier New";color:#0b5394"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span style="font-family:"Courier New";color:#0b5394" lang="EN">Thanks in advance,</span><span style="font-family:"Courier New";color:#0b5394"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New";color:#0b5394" lang="EN">Cheers,</span><span style="font-family:"Courier New";color:#0b5394"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New";color:#0b5394"><br clear="all">
<u></u><u></u></span></p>
</div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Trebuchet MS","sans-serif";color:#444444">German</span></b><u></u><u></u></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt">_______________________________________________<br>
Users mailing list<br>
<a href="mailto:Users@lists.openfabrics.org" target="_blank">Users@lists.openfabrics.org</a><br>
<a href="http://lists.openfabrics.org/mailman/listinfo/users" target="_blank">http://lists.openfabrics.org/mailman/listinfo/users</a><u></u><u></u></p>
</blockquote>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</div>
</div></div></div>
</div>
</div>
</blockquote></div><br></div></div>