[Users] Config errors in IBA7322 QDR InfiniBand HCA

[Weiny, Ira]

I don’t know if Ubuntu includes the ipath verbs packages or not.  I would assume so.

If not the ipathverbs packages are available here:


libipathverbs: Mike Marciniszyn

http://www.openfabrics.org/downloads/libipathverbs

git://github.com/01org/libipathverbs.git master

(infinipath at intel.com<mailto:infinipath at intel.com>)

Also, if your applications can use it, you will get better performance using PSM which is available here:


infinipath-psm: CQ Tang

(infinipath at intel.com<mailto:infinipath at intel.com>)

http://www.openfabrics.org/downloads/infinipath-psm


Ira


From: users-bounces at lists.openfabrics.org [mailto:users-bounces at lists.openfabrics.org] On Behalf Of German Anders
Sent: Tuesday, October 06, 2015 12:32 PM
To: Hal Rosenstock
Cc: users at lists.openfabrics.org
Subject: Re: [Users] Config errors in IBA7322 QDR InfiniBand HCA

Hi Hal,

Thanks a lot for the quick response. I'll try those commands and let you know. Regarding the software, is there any 'best soft' to use with Ubuntu and this kind of adapter? Qlogic
Thanks in advance,
Best regards,

German

2015-10-06 16:30 GMT-03:00 Hal Rosenstock <hal.rosenstock at gmail.com<mailto:hal.rosenstock at gmail.com>>:
Hi German,

You can't get at the switches because the local ports are disabled.

Start with enabling these ports. Something like:

ibportstate -D 0 1 enable
ibportstate -D 0 2 enable

Hopefully that should move the ports out of disabled physical state and if so try ibswitches.

You appear to be using MOFED distribution. Is there a reason for that ? I'm not sure that it includes packages for libipathverbs or libcxgb4.

-- Hal


On Tue, Oct 6, 2015 at 2:48 PM, German Anders <ganders at despegar.com<mailto:ganders at despegar.com>> wrote:
Hi All,
I'm in the middle of a new Infiniband implementation but I'm getting some errors, the basic setup is a HP Enclosure with a IB SW (Infiniscale-IV Mellanox Technologies), then I've setup a Blade with Ubuntu 14.04 and when trying to configure the IB support I'm getting some errors:
# lscpi
(...)
41:00.0 InfiniBand: QLogic Corp. IBA7322 QDR InfiniBand HCA (rev 02)


# ibstatus
Infiniband device 'qib0' port 1 status:
    default gid:     fe80:0000:0000:0000:0011:7500:0079:1ff8
    base lid:     0xffff
    sm lid:         0xffff
    state:         1: DOWN
    phys state:     3: Disabled
    rate:         40 Gb/sec (4X QDR)
    link_layer:     InfiniBand

Infiniband device 'qib0' port 2 status:
    default gid:     fe80:0000:0000:0000:0011:7500:0079:1ff9
    base lid:     0xffff
    sm lid:         0xffff
    state:         1: DOWN
    phys state:     3: Disabled
    rate:         40 Gb/sec (4X QDR)
    link_layer:     InfiniBand

root at e60-host02:~/MLNX_OFED_LINUX-3.0-2.0.1-ubuntu14.04-x86_64#<mailto:root at e60-host02:~/MLNX_OFED_LINUX-3.0-2.0.1-ubuntu14.04-x86_64#> ibstat
CA 'qib0'
    CA type: InfiniPath_QMH7342
    Number of ports: 2
    Firmware version:
    Hardware version: 2
    Node GUID: 0x0011750000791ff8
    System image GUID: 0x0011750000791ff8
    Port 1:
        State: Down
        Physical state: Disabled
        Rate: 40
        Base lid: 65535
        LMC: 0
        SM lid: 65535
        Capability mask: 0x07610868
        Port GUID: 0x0011750000791ff8
        Link layer: InfiniBand
    Port 2:
        State: Down
        Physical state: Disabled
        Rate: 40
        Base lid: 65535
        LMC: 0
        SM lid: 65535
        Capability mask: 0x07610868
        Port GUID: 0x0011750000791ff9
        Link layer: InfiniBand


# ibstatus
Infiniband device 'qib0' port 1 status:
    default gid:     fe80:0000:0000:0000:0011:7500:0079:1ff8
    base lid:     0xffff
    sm lid:         0xffff
    state:         1: DOWN
    phys state:     3: Disabled
    rate:         40 Gb/sec (4X QDR)
    link_layer:     InfiniBand

Infiniband device 'qib0' port 2 status:
    default gid:     fe80:0000:0000:0000:0011:7500:0079:1ff9
    base lid:     0xffff
    sm lid:         0xffff
    state:         1: DOWN
    phys state:     3: Disabled
    rate:         40 Gb/sec (4X QDR)
    link_layer:     InfiniBand

Some of the errors besides that the phys state of the ports is disabled:

# ibswitches
src/query_smp.c:228; can't open UMAD port ((null):0)
/usr/sbin/ibnetdiscover: iberror: failed: discover failed

# ibv_devinfo
libibverbs: Warning: couldn't load driver 'ipathverbs': libipathverbs-rdmav2.so: cannot open shared object file: No such file or directory
libibverbs: Warning: couldn't load driver 'cxgb4': libcxgb4-rdmav2.so: cannot open shared object file: No such file or directory
hca_id:    qib0
    transport:            InfiniBand (0)
    fw_ver:                0.0.0
    node_guid:            0011:7500:0079:1ff8
    sys_image_guid:            0011:7500:0079:1ff8
    vendor_id:            0x1175
    vendor_part_id:            29474
    hw_ver:                0x2
    board_id:            InfiniPath_QMH7342
    phys_port_cnt:            2
        port:    1
            state:            PORT_DOWN (1)
            max_mtu:        4096 (5)
            active_mtu:        4096 (5)
            sm_lid:            65535
            port_lid:        65535
            port_lmc:        0x00
            link_layer:        InfiniBand

        port:    2
            state:            PORT_DOWN (1)
            max_mtu:        4096 (5)
            active_mtu:        4096 (5)
            sm_lid:            65535
            port_lid:        65535
            port_lmc:        0x00
            link_layer:        InfiniBand

Also, In what packages are the ipathverbs and cxgb4 contained? Also, does anyone had any experience with HP IB switches?
Thanks in advance,


German

_______________________________________________
Users mailing list
Users at lists.openfabrics.org<mailto:Users at lists.openfabrics.org>
http://lists.openfabrics.org/mailman/listinfo/users


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/users/attachments/20151007/96a2e31e/attachment.html>