<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><meta http-equiv=Content-Type content="text/html; charset=utf-8"><meta name=Generator content="Microsoft Word 14 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:"Trebuchet MS";
panose-1:2 11 6 3 2 2 2 2 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p
{mso-style-priority:99;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
p.MsoAcetate, li.MsoAcetate, div.MsoAcetate
{mso-style-priority:99;
mso-style-link:"Balloon Text Char";
margin:0in;
margin-bottom:.0001pt;
font-size:8.0pt;
font-family:"Tahoma","sans-serif";}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Times New Roman","serif";
color:windowtext;}
span.BalloonTextChar
{mso-style-name:"Balloon Text Char";
mso-style-priority:99;
mso-style-link:"Balloon Text";
font-family:"Tahoma","sans-serif";}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></head><body lang=EN-US link=blue vlink=purple><div class=WordSection1><p class=MsoNormal>Hi German,<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>Many of the IB Utilities for RDMA (like ibnetdiscover) depend on having an SM running. So I would concentrate on trying to figure out why your ports are showing in the DOWN state. Try hooking them back to back and see if they go to the INIT state. If so, run “opensm &” on one of the HCAs and you should see the state go to ACTIVE and then your utilities should work. Then verify that your Intel HCAs will work with the Mellanox switch. Always assume the easiest thing to fix first – for example you have bad cables.<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>The OFA Software is available here: <a href="https://www.openfabrics.org/downloads/OFED/">https://www.openfabrics.org/downloads/OFED/</a><o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>Latest GA is OFED 3.18 and the next to be released is OFED 3.18-1<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>Good luck,<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>Rupert<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal><b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'>From:</span></b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'> <a href="mailto:users-bounces@lists.openfabrics.org">users-bounces@lists.openfabrics.org</a> [<a href="mailto:users-bounces@lists.openfabrics.org">mailto:users-bounces@lists.openfabrics.org</a>] <b>On Behalf Of </b>German Anders<br><b>Sent:</b> Tuesday, October 06, 2015 2:48 PM<br><b>To:</b> <a href="mailto:users@lists.openfabrics.org">users@lists.openfabrics.org</a><br><b>Subject:</b> [Users] Config errors in IBA7322 QDR InfiniBand HCA<o:p></o:p></span></p><p class=MsoNormal><o:p> </o:p></p><div><div><p class=MsoNormal style='margin-bottom:12.0pt'><span style='font-family:"Courier New";color:#0B5394'>Hi All,<o:p></o:p></span></p></div><div><p class=MsoNormal style='margin-bottom:12.0pt'><span style='font-family:"Courier New";color:#0B5394'>I'm in the middle of a new Infiniband implementation but I'm getting some errors, the basic setup is a HP Enclosure with a IB SW (Infiniscale-IV Mellanox Technologies), then I've setup a Blade with Ubuntu 14.04 and when trying to configure the IB support I'm getting some errors:<o:p></o:p></span></p></div><div><p class=MsoNormal style='margin-bottom:12.0pt'><span style='font-family:"Courier New";color:#0B5394'># lscpi<br>(...)<br><b>41:00.0 InfiniBand: QLogic Corp. IBA7322 QDR InfiniBand HCA (rev 02)</b><br><br><br># ibstatus<br>Infiniband device 'qib0' port 1 status:<br> default gid: fe80:0000:0000:0000:0011:7500:0079:1ff8<br> base lid: 0xffff<br> sm lid: 0xffff<br> state: 1: DOWN<br> phys state: 3: Disabled<br> rate: 40 Gb/sec (4X QDR)<br> link_layer: InfiniBand<br><br>Infiniband device 'qib0' port 2 status:<br> default gid: fe80:0000:0000:0000:0011:7500:0079:1ff9<br> base lid: 0xffff<br> sm lid: 0xffff<br> state: 1: DOWN<br> phys state: 3: Disabled<br> rate: 40 Gb/sec (4X QDR)<br> link_layer: InfiniBand<br><br><a href="mailto:root@e60-host02:~/MLNX_OFED_LINUX-3.0-2.0.1-ubuntu14.04-x86_64">root@e60-host02:~/MLNX_OFED_LINUX-3.0-2.0.1-ubuntu14.04-x86_64#</a> ibstat<br>CA 'qib0'<br> CA type: InfiniPath_QMH7342<br> Number of ports: 2<br> Firmware version: <br> Hardware version: 2<br> Node GUID: 0x0011750000791ff8<br> System image GUID: 0x0011750000791ff8<br> Port 1:<br> State: Down<br> Physical state: Disabled<br> Rate: 40<br> Base lid: 65535<br> LMC: 0<br> SM lid: 65535<br> Capability mask: 0x07610868<br> Port GUID: 0x0011750000791ff8<br> Link layer: InfiniBand<br> Port 2:<br> State: Down<br> Physical state: Disabled<br> Rate: 40<br> Base lid: 65535<br> LMC: 0<br> SM lid: 65535<br> Capability mask: 0x07610868<br> Port GUID: 0x0011750000791ff9<br> Link layer: InfiniBand<br><br><br># ibstatus<br>Infiniband device 'qib0' port 1 status:<br> default gid: fe80:0000:0000:0000:0011:7500:0079:1ff8<br> base lid: 0xffff<br> sm lid: 0xffff<br> state: 1: DOWN<br> phys state: 3: Disabled<br> rate: 40 Gb/sec (4X QDR)<br> link_layer: InfiniBand<br><br>Infiniband device 'qib0' port 2 status:<br> default gid: fe80:0000:0000:0000:0011:7500:0079:1ff9<br> base lid: 0xffff<br> sm lid: 0xffff<br> state: 1: DOWN<br> phys state: 3: Disabled<br> rate: 40 Gb/sec (4X QDR)<br> link_layer: InfiniBand<o:p></o:p></span></p></div><div><p class=MsoNormal style='margin-bottom:12.0pt'><span style='font-family:"Courier New";color:#0B5394'>Some of the errors besides that the phys state of the ports is disabled:<br><br># ibswitches<br>src/query_smp.c:228; can't open UMAD port ((null):0)<br>/usr/sbin/ibnetdiscover: iberror: failed: discover failed<br><br># ibv_devinfo<br>libibverbs: Warning: couldn't load driver 'ipathverbs': libipathverbs-rdmav2.so: cannot open shared object file: No such file or directory<br>libibverbs: Warning: couldn't load driver 'cxgb4': libcxgb4-rdmav2.so: cannot open shared object file: No such file or directory<br>hca_id: qib0<br> transport: InfiniBand (0)<br> fw_ver: 0.0.0<br> node_guid: 0011:7500:0079:1ff8<br> sys_image_guid: 0011:7500:0079:1ff8<br> vendor_id: 0x1175<br> vendor_part_id: 29474<br> hw_ver: 0x2<br> board_id: InfiniPath_QMH7342<br> phys_port_cnt: 2<br> port: 1<br> state: PORT_DOWN (1)<br> max_mtu: 4096 (5)<br> active_mtu: 4096 (5)<br> sm_lid: 65535<br> port_lid: 65535<br> port_lmc: 0x00<br> link_layer: InfiniBand<br><br> port: 2<br> state: PORT_DOWN (1)<br> max_mtu: 4096 (5)<br> active_mtu: 4096 (5)<br> sm_lid: 65535<br> port_lid: 65535<br> port_lmc: 0x00<br> link_layer: InfiniBand<o:p></o:p></span></p></div><div><p class=MsoNormal style='margin-bottom:12.0pt'><span style='font-family:"Courier New";color:#0B5394'>Also, In what packages are the <b>ipathverbs</b> and <b>cxgb4</b> contained? Also, does anyone had any experience with HP IB switches? <o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-family:"Courier New";color:#0B5394'>Thanks in advance,<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-family:"Courier New";color:#0B5394'><br><br clear=all><o:p></o:p></span></p></div><div><div><div><div><div><div><div><p class=MsoNormal><b><span style='font-size:10.0pt;font-family:"Trebuchet MS","sans-serif";color:#444444'>German</span></b><o:p></o:p></p></div></div></div></div></div></div></div></div></div></body></html>