[Users] Config errors in IBA7322 QDR InfiniBand HCA

[Rupert Dance]

Hi German,

 

Many of the IB Utilities for RDMA (like ibnetdiscover) depend on having an SM running. So I would concentrate on trying to figure out why your ports are showing in the DOWN state. Try hooking them back to back and see if they go to the INIT state. If so, run “opensm &” on one of the HCAs and you should see the state go to ACTIVE and then your utilities should work. Then verify that your Intel HCAs will work with the Mellanox switch. Always assume the easiest thing to fix first – for example you have bad cables.

 

The OFA Software is available here: https://www.openfabrics.org/downloads/OFED/

 

Latest GA is OFED 3.18 and the next to be released is OFED 3.18-1

 

Good luck,

 

Rupert

 

From: users-bounces at lists.openfabrics.org [mailto:users-bounces at lists.openfabrics.org] On Behalf Of German Anders
Sent: Tuesday, October 06, 2015 2:48 PM
To: users at lists.openfabrics.org
Subject: [Users] Config errors in IBA7322 QDR InfiniBand HCA

 

Hi All,

I'm in the middle of a new Infiniband implementation but I'm getting some errors, the basic setup is a HP Enclosure with a IB SW (Infiniscale-IV Mellanox Technologies), then I've setup a Blade with Ubuntu 14.04 and when trying to configure the IB support I'm getting some errors:

# lscpi
(...)
41:00.0 InfiniBand: QLogic Corp. IBA7322 QDR InfiniBand HCA (rev 02)


# ibstatus
Infiniband device 'qib0' port 1 status:
    default gid:     fe80:0000:0000:0000:0011:7500:0079:1ff8
    base lid:     0xffff
    sm lid:         0xffff
    state:         1: DOWN
    phys state:     3: Disabled
    rate:         40 Gb/sec (4X QDR)
    link_layer:     InfiniBand

Infiniband device 'qib0' port 2 status:
    default gid:     fe80:0000:0000:0000:0011:7500:0079:1ff9
    base lid:     0xffff
    sm lid:         0xffff
    state:         1: DOWN
    phys state:     3: Disabled
    rate:         40 Gb/sec (4X QDR)
    link_layer:     InfiniBand

root at e60-host02:~/MLNX_OFED_LINUX-3.0-2.0.1-ubuntu14.04-x86_64# <mailto:root at e60-host02:~/MLNX_OFED_LINUX-3.0-2.0.1-ubuntu14.04-x86_64>  ibstat
CA 'qib0'
    CA type: InfiniPath_QMH7342
    Number of ports: 2
    Firmware version: 
    Hardware version: 2
    Node GUID: 0x0011750000791ff8
    System image GUID: 0x0011750000791ff8
    Port 1:
        State: Down
        Physical state: Disabled
        Rate: 40
        Base lid: 65535
        LMC: 0
        SM lid: 65535
        Capability mask: 0x07610868
        Port GUID: 0x0011750000791ff8
        Link layer: InfiniBand
    Port 2:
        State: Down
        Physical state: Disabled
        Rate: 40
        Base lid: 65535
        LMC: 0
        SM lid: 65535
        Capability mask: 0x07610868
        Port GUID: 0x0011750000791ff9
        Link layer: InfiniBand


# ibstatus
Infiniband device 'qib0' port 1 status:
    default gid:     fe80:0000:0000:0000:0011:7500:0079:1ff8
    base lid:     0xffff
    sm lid:         0xffff
    state:         1: DOWN
    phys state:     3: Disabled
    rate:         40 Gb/sec (4X QDR)
    link_layer:     InfiniBand

Infiniband device 'qib0' port 2 status:
    default gid:     fe80:0000:0000:0000:0011:7500:0079:1ff9
    base lid:     0xffff
    sm lid:         0xffff
    state:         1: DOWN
    phys state:     3: Disabled
    rate:         40 Gb/sec (4X QDR)
    link_layer:     InfiniBand

Some of the errors besides that the phys state of the ports is disabled:

# ibswitches
src/query_smp.c:228; can't open UMAD port ((null):0)
/usr/sbin/ibnetdiscover: iberror: failed: discover failed

# ibv_devinfo
libibverbs: Warning: couldn't load driver 'ipathverbs': libipathverbs-rdmav2.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'cxgb4': libcxgb4-rdmav2.so: cannot open shared object file: No such file or directory
hca_id:    qib0
    transport:            InfiniBand (0)
    fw_ver:                0.0.0
    node_guid:            0011:7500:0079:1ff8
    sys_image_guid:            0011:7500:0079:1ff8
    vendor_id:            0x1175
    vendor_part_id:            29474
    hw_ver:                0x2
    board_id:            InfiniPath_QMH7342
    phys_port_cnt:            2
        port:    1
            state:            PORT_DOWN (1)
            max_mtu:        4096 (5)
            active_mtu:        4096 (5)
            sm_lid:            65535
            port_lid:        65535
            port_lmc:        0x00
            link_layer:        InfiniBand

        port:    2
            state:            PORT_DOWN (1)
            max_mtu:        4096 (5)
            active_mtu:        4096 (5)
            sm_lid:            65535
            port_lid:        65535
            port_lmc:        0x00
            link_layer:        InfiniBand

Also, In what packages are the ipathverbs and cxgb4 contained? Also, does anyone had any experience with HP IB switches? 

Thanks in advance,





German

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/users/attachments/20151006/f71fbe69/attachment.html>