[openib-general] unable to run ibv_uc_pingpong and other example binaries

[Joe Damato]

Hello,
   
    First I would like to apologize for the long email.
   
    I am currently using the gen2 revision 2734 openib source and kernel 
2.6.11. I have compiled and installed libibverbs, libmthca and libibcm. 
I have also created the /dev/infiniband devices, and I have my card 
flashed to the latest firmware version.

[root at frodo examples]# pwd
/cluster/src/OPENIB_SRC/openib-2734/gen2/trunk/src/userspace/libibverbs/examples
[root at frodo examples]# ./ibv_uc_pingpong
libucm: Error <-1:6> couldn't open IB cm device </dev/infiniband/ucm>
libibverbs: Fatal: couldn't open sysfs class 'infiniband_verbs'.
No IB devices found
[root at frodo examples]# ./ibv_rc_pingpong
libucm: Error <-1:6> couldn't open IB cm device </dev/infiniband/ucm>
libibverbs: Fatal: couldn't open sysfs class 'infiniband_verbs'.
No IB devices found

but....

[root at frodo examples]# cd /dev/
[root at frodo dev]# ls infiniband/
issm0  issm1  ucm  umad0  umad1  uverbs0  uverbs1
[root at frodo dev]# ls -al infiniband/
total 0
drw-rw-rw-  2 root root      180 Jul 19 11:36 .
drwxr-xr-x  8 root root    13940 Jul 19 11:35 ..
crw-rw-rw-  1 root root 231,  64 Jul 19 11:35 issm0
crw-rw-rw-  1 root root 231,  65 Jul 19 11:35 issm1
crw-rw-rw-  1 root root 231, 255 Jul 19 11:36 ucm
crw-rw-rw-  1 root root 231,   0 Jul 19 11:35 umad0
crw-rw-rw-  1 root root 231,   1 Jul 19 11:35 umad1
crw-rw-rw-  1 root root 231, 192 Jul 19 11:36 uverbs0
crw-rw-rw-  1 root root 231, 193 Jul 19 11:36 uverbs1

also, I don't know if this helps but:

[root at frodo ~]# /usr/local/ib/bin/ibstat
CA 'mthca0'
        CA type: MT23108
        Number of ports: 2
        Firmware version: 3.3.2
        Hardware version: a1
        Node GUID: 0x0002c9010976c400
        System image GUID: 0x0002c9010976c403
        Port 1:
                State: Active
                Physical state: LinkUp
                Rate: 10
                Base lid: 1
                LMC: 0
                SM lid: 1
                Capability mask: 0x00500a6a
                Port GUID: 0x0002c9010976c401
        Port 2:
                State: Down
                Physical state: Polling
                Rate: 2
                Base lid: 0
                LMC: 0
                SM lid: 0
                Capability mask: 0x00500a68
                Port GUID: 0x0002c9010976c402
[root at frodo ~]#

and

[root at frodo ~]# ps aux | grep opensm
root      2929  0.0  0.0  82368  1372 pts/0    Sl   11:35   0:00 
/usr/local/ib/bin/opensm

and

[root at frodo ~]# dmesg | grep mthca
ib_mthca: Mellanox InfiniBand HCA driver v0.06 (June 23, 2005)
ib_mthca: Initializing  (0000:14:00.0)


Any suggestions on how to resolve this issue so I can test the IB set 
up? I currently have 8 other machines with IB, however when trying to 
run ibv_rc_pingpong they all report the same error  (each of the 8 
systems are running the same kernel and all cards are flashed to the 
latest firmware):

ost0# ./ibv_rc_pingpong
Couldn't get local LID

but when checking ibstat it shows that Port 2 is active

ost0# /usr/local/ib/bin/ibstat
CA 'mthca0'
        CA type: MT25208 (MT23108 compat mode)
        Number of ports: 2
        Firmware version: 4.6.2
        Hardware version: a0
        Node GUID: 0x0002c90200003f40
        System image GUID: 0x0002c90200003f43
        Port 1:
                State: Down
                Physical state: Polling
                Rate: 10
                Base lid: 0
                LMC: 0
                SM lid: 0
                Capability mask: 0x00510a68
                Port GUID: 0x0002c90200003f41
        Port 2:
                State: Active
                Physical state: LinkUp
                Rate: 10
                Base lid: 5
                LMC: 0
                SM lid: 1
                Capability mask: 0x00510a68
                Port GUID: 0x0002c90200003f42



any suggestions would be greatly appreciated.

Thanks,
Joe Damato