[Users] Config errors in IBA7322 QDR InfiniBand HCA

[Hal Rosenstock]

Hi German,

You can't get at the switches because the local ports are disabled.

Start with enabling these ports. Something like:

ibportstate -D 0 1 enable
ibportstate -D 0 2 enable

Hopefully that should move the ports out of disabled physical state and if
so try ibswitches.

You appear to be using MOFED distribution. Is there a reason for that ? I'm
not sure that it includes packages for libipathverbs or libcxgb4.

-- Hal


On Tue, Oct 6, 2015 at 2:48 PM, German Anders <ganders at despegar.com> wrote:

> Hi All,
>
> I'm in the middle of a new Infiniband implementation but I'm getting some
> errors, the basic setup is a HP Enclosure with a IB SW (Infiniscale-IV
> Mellanox Technologies), then I've setup a Blade with Ubuntu 14.04 and when
> trying to configure the IB support I'm getting some errors:
>
> # lscpi
> (...)
> *41:00.0 InfiniBand: QLogic Corp. IBA7322 QDR InfiniBand HCA (rev 02)*
>
>
> # ibstatus
> Infiniband device 'qib0' port 1 status:
>     default gid:     fe80:0000:0000:0000:0011:7500:0079:1ff8
>     base lid:     0xffff
>     sm lid:         0xffff
>     state:         1: DOWN
>     phys state:     3: Disabled
>     rate:         40 Gb/sec (4X QDR)
>     link_layer:     InfiniBand
>
> Infiniband device 'qib0' port 2 status:
>     default gid:     fe80:0000:0000:0000:0011:7500:0079:1ff9
>     base lid:     0xffff
>     sm lid:         0xffff
>     state:         1: DOWN
>     phys state:     3: Disabled
>     rate:         40 Gb/sec (4X QDR)
>     link_layer:     InfiniBand
>
> root at e60-host02:~/MLNX_OFED_LINUX-3.0-2.0.1-ubuntu14.04-x86_64# ibstat
> CA 'qib0'
>     CA type: InfiniPath_QMH7342
>     Number of ports: 2
>     Firmware version:
>     Hardware version: 2
>     Node GUID: 0x0011750000791ff8
>     System image GUID: 0x0011750000791ff8
>     Port 1:
>         State: Down
>         Physical state: Disabled
>         Rate: 40
>         Base lid: 65535
>         LMC: 0
>         SM lid: 65535
>         Capability mask: 0x07610868
>         Port GUID: 0x0011750000791ff8
>         Link layer: InfiniBand
>     Port 2:
>         State: Down
>         Physical state: Disabled
>         Rate: 40
>         Base lid: 65535
>         LMC: 0
>         SM lid: 65535
>         Capability mask: 0x07610868
>         Port GUID: 0x0011750000791ff9
>         Link layer: InfiniBand
>
>
> # ibstatus
> Infiniband device 'qib0' port 1 status:
>     default gid:     fe80:0000:0000:0000:0011:7500:0079:1ff8
>     base lid:     0xffff
>     sm lid:         0xffff
>     state:         1: DOWN
>     phys state:     3: Disabled
>     rate:         40 Gb/sec (4X QDR)
>     link_layer:     InfiniBand
>
> Infiniband device 'qib0' port 2 status:
>     default gid:     fe80:0000:0000:0000:0011:7500:0079:1ff9
>     base lid:     0xffff
>     sm lid:         0xffff
>     state:         1: DOWN
>     phys state:     3: Disabled
>     rate:         40 Gb/sec (4X QDR)
>     link_layer:     InfiniBand
>
>
> Some of the errors besides that the phys state of the ports is disabled:
>
> # ibswitches
> src/query_smp.c:228; can't open UMAD port ((null):0)
> /usr/sbin/ibnetdiscover: iberror: failed: discover failed
>
> # ibv_devinfo
> libibverbs: Warning: couldn't load driver 'ipathverbs':
> libipathverbs-rdmav2.so: cannot open shared object file: No such file or
> directory
> libibverbs: Warning: couldn't load driver 'cxgb4': libcxgb4-rdmav2.so:
> cannot open shared object file: No such file or directory
> hca_id:    qib0
>     transport:            InfiniBand (0)
>     fw_ver:                0.0.0
>     node_guid:            0011:7500:0079:1ff8
>     sys_image_guid:            0011:7500:0079:1ff8
>     vendor_id:            0x1175
>     vendor_part_id:            29474
>     hw_ver:                0x2
>     board_id:            InfiniPath_QMH7342
>     phys_port_cnt:            2
>         port:    1
>             state:            PORT_DOWN (1)
>             max_mtu:        4096 (5)
>             active_mtu:        4096 (5)
>             sm_lid:            65535
>             port_lid:        65535
>             port_lmc:        0x00
>             link_layer:        InfiniBand
>
>         port:    2
>             state:            PORT_DOWN (1)
>             max_mtu:        4096 (5)
>             active_mtu:        4096 (5)
>             sm_lid:            65535
>             port_lid:        65535
>             port_lmc:        0x00
>             link_layer:        InfiniBand
>
>
> Also, In what packages are the *ipathverbs* and *cxgb4* contained? Also,
> does anyone had any experience with HP IB switches?
>
> Thanks in advance,
>
>
> *German* <ganders at despegar.com>
>
> _______________________________________________
> Users mailing list
> Users at lists.openfabrics.org
> http://lists.openfabrics.org/mailman/listinfo/users
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/users/attachments/20151006/14999e8e/attachment.html>