<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 14 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:"Trebuchet MS";
panose-1:2 11 6 3 2 2 2 2 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p
{mso-style-priority:99;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
pre
{mso-style-priority:99;
mso-style-link:"HTML Preformatted Char";
margin:0in;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Courier New";}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.HTMLPreformattedChar
{mso-style-name:"HTML Preformatted Char";
mso-style-priority:99;
mso-style-link:"HTML Preformatted";
font-family:"Courier New";}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri","sans-serif";}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">I don’t know if Ubuntu includes the ipath verbs packages or not. I would assume so.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">If not the ipathverbs packages are available here:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<pre><span style="color:black">libipathverbs: Mike Marciniszyn <o:p></o:p></span></pre>
<pre><span style="color:black">http://www.openfabrics.org/downloads/libipathverbs<o:p></o:p></span></pre>
<pre><span style="color:black">git://github.com/01org/libipathverbs.git master<o:p></o:p></span></pre>
<pre><span style="color:black">(infinipath@intel.com<mailto:infinipath@intel.com>) <o:p></o:p></span></pre>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Also, if your applications can use it, you will get better performance using PSM which is available here:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<pre><span style="color:black">infinipath-psm: CQ Tang<o:p></o:p></span></pre>
<pre><span style="color:black">(infinipath@intel.com<mailto:infinipath@intel.com>) <o:p></o:p></span></pre>
<pre><span style="color:black">http://www.openfabrics.org/downloads/infinipath-psm<o:p></o:p></span></pre>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Ira<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<div style="border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt">
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> users-bounces@lists.openfabrics.org [mailto:users-bounces@lists.openfabrics.org]
<b>On Behalf Of </b>German Anders<br>
<b>Sent:</b> Tuesday, October 06, 2015 12:32 PM<br>
<b>To:</b> Hal Rosenstock<br>
<b>Cc:</b> users@lists.openfabrics.org<br>
<b>Subject:</b> Re: [Users] Config errors in IBA7322 QDR InfiniBand HCA<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New";color:#0B5394">Hi Hal,<br>
<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span style="font-family:"Courier New";color:#0B5394">Thanks a lot for the quick response. I'll try those commands and let you know. Regarding the software, is there any 'best soft' to use with Ubuntu and this
kind of adapter? Qlogic<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span style="font-family:"Courier New";color:#0B5394">Thanks in advance,<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span style="font-family:"Courier New";color:#0B5394">Best regards,<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><br clear="all">
<o:p></o:p></p>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Trebuchet MS","sans-serif";color:#444444">German</span></b><o:p></o:p></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">2015-10-06 16:30 GMT-03:00 Hal Rosenstock <<a href="mailto:hal.rosenstock@gmail.com" target="_blank">hal.rosenstock@gmail.com</a>>:<o:p></o:p></p>
<div>
<div>
<p class="MsoNormal">Hi German,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">You can't get at the switches because the local ports are disabled.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Start with enabling these ports. Something like:<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">ibportstate -D 0 1 enable<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">ibportstate -D 0 2 enable<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Hopefully that should move the ports out of disabled physical state and if so try ibswitches.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">You appear to be using MOFED distribution. Is there a reason for that ? I'm not sure that it includes packages for libipathverbs or libcxgb4.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">-- Hal<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<div>
<p class="MsoNormal">On Tue, Oct 6, 2015 at 2:48 PM, German Anders <<a href="mailto:ganders@despegar.com" target="_blank">ganders@despegar.com</a>> wrote:<o:p></o:p></p>
</div>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<div>
<div>
<div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span style="font-family:"Courier New";color:#0B5394">Hi All,<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span style="font-family:"Courier New";color:#0B5394">I'm in the middle of a new Infiniband implementation but I'm getting some errors, the basic setup is a HP Enclosure with a IB SW (Infiniscale-IV Mellanox
Technologies), then I've setup a Blade with Ubuntu 14.04 and when trying to configure the IB support I'm getting some errors:<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span style="font-family:"Courier New";color:#0B5394"># lscpi<br>
(...)<br>
<b>41:00.0 InfiniBand: QLogic Corp. IBA7322 QDR InfiniBand HCA (rev 02)</b><br>
<br>
<br>
# ibstatus<br>
Infiniband device 'qib0' port 1 status:<br>
default gid: fe80:0000:0000:0000:0011:7500:0079:1ff8<br>
base lid: 0xffff<br>
sm lid: 0xffff<br>
state: 1: DOWN<br>
phys state: 3: Disabled<br>
rate: 40 Gb/sec (4X QDR)<br>
link_layer: InfiniBand<br>
<br>
Infiniband device 'qib0' port 2 status:<br>
default gid: fe80:0000:0000:0000:0011:7500:0079:1ff9<br>
base lid: 0xffff<br>
sm lid: 0xffff<br>
state: 1: DOWN<br>
phys state: 3: Disabled<br>
rate: 40 Gb/sec (4X QDR)<br>
link_layer: InfiniBand<br>
<br>
<a href="mailto:root@e60-host02:~/MLNX_OFED_LINUX-3.0-2.0.1-ubuntu14.04-x86_64#">root@e60-host02:~/MLNX_OFED_LINUX-3.0-2.0.1-ubuntu14.04-x86_64#</a> ibstat<br>
CA 'qib0'<br>
CA type: InfiniPath_QMH7342<br>
Number of ports: 2<br>
Firmware version: <br>
Hardware version: 2<br>
Node GUID: 0x0011750000791ff8<br>
System image GUID: 0x0011750000791ff8<br>
Port 1:<br>
State: Down<br>
Physical state: Disabled<br>
Rate: 40<br>
Base lid: 65535<br>
LMC: 0<br>
SM lid: 65535<br>
Capability mask: 0x07610868<br>
Port GUID: 0x0011750000791ff8<br>
Link layer: InfiniBand<br>
Port 2:<br>
State: Down<br>
Physical state: Disabled<br>
Rate: 40<br>
Base lid: 65535<br>
LMC: 0<br>
SM lid: 65535<br>
Capability mask: 0x07610868<br>
Port GUID: 0x0011750000791ff9<br>
Link layer: InfiniBand<br>
<br>
<br>
# ibstatus<br>
Infiniband device 'qib0' port 1 status:<br>
default gid: fe80:0000:0000:0000:0011:7500:0079:1ff8<br>
base lid: 0xffff<br>
sm lid: 0xffff<br>
state: 1: DOWN<br>
phys state: 3: Disabled<br>
rate: 40 Gb/sec (4X QDR)<br>
link_layer: InfiniBand<br>
<br>
Infiniband device 'qib0' port 2 status:<br>
default gid: fe80:0000:0000:0000:0011:7500:0079:1ff9<br>
base lid: 0xffff<br>
sm lid: 0xffff<br>
state: 1: DOWN<br>
phys state: 3: Disabled<br>
rate: 40 Gb/sec (4X QDR)<br>
link_layer: InfiniBand<br>
<br>
<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span style="font-family:"Courier New";color:#0B5394">Some of the errors besides that the phys state of the ports is disabled:<br>
<br>
# ibswitches<br>
src/query_smp.c:228; can't open UMAD port ((null):0)<br>
/usr/sbin/ibnetdiscover: iberror: failed: discover failed<br>
<br>
# ibv_devinfo<br>
libibverbs: Warning: couldn't load driver 'ipathverbs': libipathverbs-rdmav2.so: cannot open shared object file: No such file or directory<br>
libibverbs: Warning: couldn't load driver 'cxgb4': libcxgb4-rdmav2.so: cannot open shared object file: No such file or directory<br>
hca_id: qib0<br>
transport: InfiniBand (0)<br>
fw_ver: 0.0.0<br>
node_guid: 0011:7500:0079:1ff8<br>
sys_image_guid: 0011:7500:0079:1ff8<br>
vendor_id: 0x1175<br>
vendor_part_id: 29474<br>
hw_ver: 0x2<br>
board_id: InfiniPath_QMH7342<br>
phys_port_cnt: 2<br>
port: 1<br>
state: PORT_DOWN (1)<br>
max_mtu: 4096 (5)<br>
active_mtu: 4096 (5)<br>
sm_lid: 65535<br>
port_lid: 65535<br>
port_lmc: 0x00<br>
link_layer: InfiniBand<br>
<br>
port: 2<br>
state: PORT_DOWN (1)<br>
max_mtu: 4096 (5)<br>
active_mtu: 4096 (5)<br>
sm_lid: 65535<br>
port_lid: 65535<br>
port_lmc: 0x00<br>
link_layer: InfiniBand<br>
<br>
<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span style="font-family:"Courier New";color:#0B5394">Also, In what packages are the
<b>ipathverbs</b> and <b>cxgb4</b> contained? Also, does anyone had any experience with HP IB switches?
<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New";color:#0B5394">Thanks in advance,<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New";color:#0B5394"><br>
<br clear="all">
<o:p></o:p></span></p>
</div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Trebuchet MS","sans-serif";color:#444444">German</span></b><o:p></o:p></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt">_______________________________________________<br>
Users mailing list<br>
<a href="mailto:Users@lists.openfabrics.org" target="_blank">Users@lists.openfabrics.org</a><br>
<a href="http://lists.openfabrics.org/mailman/listinfo/users" target="_blank">http://lists.openfabrics.org/mailman/listinfo/users</a><o:p></o:p></p>
</blockquote>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
</div>
</div>
</body>
</html>