<div dir="ltr"><div>Hi German,</div><div> </div><div>You can't get at the switches because the local ports are disabled.</div><div> </div><div>Start with enabling these ports. Something like:</div><div> </div><div>ibportstate -D 0 1 enable</div><div>ibportstate -D 0 2 enable</div><div> </div><div>Hopefully that should move the ports out of disabled physical state and if so try ibswitches.</div><div> </div><div>You appear to be using MOFED distribution. Is there a reason for that ? I'm not sure that it includes packages for libipathverbs or libcxgb4.</div><div> </div><div>-- Hal</div><div> </div></div><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Oct 6, 2015 at 2:48 PM, German Anders <span dir="ltr"><<a href="mailto:ganders@despegar.com" target="_blank">ganders@despegar.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_default" style="color:rgb(11,83,148);font-family:monospace,monospace;font-size:small">Hi All,<br><br></div><div class="gmail_default" style="color:rgb(11,83,148);font-family:monospace,monospace;font-size:small">I'm in the middle of a new Infiniband implementation but I'm getting some errors, the basic setup is a HP Enclosure with a IB SW (Infiniscale-IV Mellanox Technologies), then I've setup a Blade with Ubuntu 14.04 and when trying to configure the IB support I'm getting some errors:<br><br></div><div class="gmail_default" style="color:rgb(11,83,148);font-family:monospace,monospace;font-size:small"># lscpi<br>(...)<br><b>41:00.0 InfiniBand: QLogic Corp. IBA7322 QDR InfiniBand HCA (rev 02)</b><br><br><br># ibstatus<br>Infiniband device 'qib0' port 1 status:<br> default gid: fe80:0000:0000:0000:0011:7500:0079:1ff8<br> base lid: 0xffff<br> sm lid: 0xffff<br> state: 1: DOWN<br> phys state: 3: Disabled<br> rate: 40 Gb/sec (4X QDR)<br> link_layer: InfiniBand<br><br>Infiniband device 'qib0' port 2 status:<br> default gid: fe80:0000:0000:0000:0011:7500:0079:1ff9<br> base lid: 0xffff<br> sm lid: 0xffff<br> state: 1: DOWN<br> phys state: 3: Disabled<br> rate: 40 Gb/sec (4X QDR)<br> link_layer: InfiniBand<br><br>root@e60-host02:~/MLNX_OFED_LINUX-3.0-2.0.1-ubuntu14.04-x86_64# ibstat<br>CA 'qib0'<br> CA type: InfiniPath_QMH7342<br> Number of ports: 2<br> Firmware version: <br> Hardware version: 2<br> Node GUID: 0x0011750000791ff8<br> System image GUID: 0x0011750000791ff8<br> Port 1:<br> State: Down<br> Physical state: Disabled<br> Rate: 40<br> Base lid: 65535<br> LMC: 0<br> SM lid: 65535<br> Capability mask: 0x07610868<br> Port GUID: 0x0011750000791ff8<br> Link layer: InfiniBand<br> Port 2:<br> State: Down<br> Physical state: Disabled<br> Rate: 40<br> Base lid: 65535<br> LMC: 0<br> SM lid: 65535<br> Capability mask: 0x07610868<br> Port GUID: 0x0011750000791ff9<br> Link layer: InfiniBand<br><br><br># ibstatus<br>Infiniband device 'qib0' port 1 status:<br> default gid: fe80:0000:0000:0000:0011:7500:0079:1ff8<br> base lid: 0xffff<br> sm lid: 0xffff<br> state: 1: DOWN<br> phys state: 3: Disabled<br> rate: 40 Gb/sec (4X QDR)<br> link_layer: InfiniBand<br><br>Infiniband device 'qib0' port 2 status:<br> default gid: fe80:0000:0000:0000:0011:7500:0079:1ff9<br> base lid: 0xffff<br> sm lid: 0xffff<br> state: 1: DOWN<br> phys state: 3: Disabled<br> rate: 40 Gb/sec (4X QDR)<br> link_layer: InfiniBand<br><br><br></div><div class="gmail_default" style="color:rgb(11,83,148);font-family:monospace,monospace;font-size:small">Some of the errors besides that the phys state of the ports is disabled:<br><br># ibswitches<br>src/query_smp.c:228; can't open UMAD port ((null):0)<br>/usr/sbin/ibnetdiscover: iberror: failed: discover failed<br><br># ibv_devinfo<br>libibverbs: Warning: couldn't load driver 'ipathverbs': libipathverbs-rdmav2.so: cannot open shared object file: No such file or directory<br>libibverbs: Warning: couldn't load driver 'cxgb4': libcxgb4-rdmav2.so: cannot open shared object file: No such file or directory<br>hca_id: qib0<br> transport: InfiniBand (0)<br> fw_ver: 0.0.0<br> node_guid: 0011:7500:0079:1ff8<br> sys_image_guid: 0011:7500:0079:1ff8<br> vendor_id: 0x1175<br> vendor_part_id: 29474<br> hw_ver: 0x2<br> board_id: InfiniPath_QMH7342<br> phys_port_cnt: 2<br> port: 1<br> state: PORT_DOWN (1)<br> max_mtu: 4096 (5)<br> active_mtu: 4096 (5)<br> sm_lid: 65535<br> port_lid: 65535<br> port_lmc: 0x00<br> link_layer: InfiniBand<br><br> port: 2<br> state: PORT_DOWN (1)<br> max_mtu: 4096 (5)<br> active_mtu: 4096 (5)<br> sm_lid: 65535<br> port_lid: 65535<br> port_lmc: 0x00<br> link_layer: InfiniBand<br><br><br></div><div class="gmail_default" style="color:rgb(11,83,148);font-family:monospace,monospace;font-size:small">Also, In what packages are the <b>ipathverbs</b> and <b>cxgb4</b> contained? Also, does anyone had any experience with HP IB switches? <br><br></div><div class="gmail_default" style="color:rgb(11,83,148);font-family:monospace,monospace;font-size:small">Thanks in advance,<br></div><div class="gmail_default" style="color:rgb(11,83,148);font-family:monospace,monospace;font-size:small"><br><br clear="all"></div><div><div><div dir="ltr"><div><div dir="ltr"><span><div><div dir="ltr"><p><b><span style="color:rgb(68,68,68);font-family:Trebuchet MS,sans-serif;font-size:10pt"></span></b></p><span style="color:rgb(68,68,68)"><span style="font-family:trebuchet ms,sans-serif"><font size="2"><b>German</b></font></span></span><span style="color:rgb(68,68,68)"><span style="font-family:trebuchet ms,sans-serif"><font size="2"><font size="1"><a href="mailto:ganders@despegar.com" target="_blank"></a></font></font></span></span></div></div></span></div></div></div></div></div>
</div>
<br>_______________________________________________<br>
Users mailing list<br>
<a href="mailto:Users@lists.openfabrics.org">Users@lists.openfabrics.org</a><br>
<a href="http://lists.openfabrics.org/mailman/listinfo/users" target="_blank" rel="noreferrer">http://lists.openfabrics.org/mailman/listinfo/users</a><br>
<br></blockquote></div><br></div>