[ofa-general] ConnectX problems on Sles10sp1

[Pradeep Satyanarayana]

This originally started as a problem that ibv_devinfo was showing:
"No IB devices found". We are using OFED 1.2.5.

Started to dig this a little and see no entries under /sys/class/infiniband
as can be seen below.

lsmod | grep ib
mlx4_ib                74560  0
ib_addr                28704  1 rdma_cm
ib_ipoib              124200  0
ib_cm                  65904  2 rdma_cm,ib_ipoib
ib_sa                  77880  3 rdma_cm,ib_ipoib,ib_cm
ipv6                  466288  21 ib_ipoib
ib_uverbs              72440  1 rdma_ucm
ib_umad                40928  0
ib_mad                 72592  4 mlx4_ib,ib_cm,ib_sa,ib_umad
ib_core               106688  10 mlx4_ib,rdma_ucm,rdma_cm,iw_cm,ib_ipoib,ib_cm,ib_sa,ib_uverbs,ib_umad,ib_mad
mlx4_core             117904  1 mlx4_ib
libata                196100  1 ipr
scsi_mod              228216  4 sg,ipr,libata,sd_mod

modinfo mlx4_ib
filename:       /lib/modules/2.6.16.46-0.12-ppc64/updates/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko
version:        0.01
license:        Dual BSD/GPL
description:    Mellanox ConnectX HCA InfiniBand driver
author:         Roland Dreier
srcversion:     E9808B3F9850220A7A35677
depends:        mlx4_core,ib_core,ib_mad,ib_core
vermagic:       2.6.16.46-0.12-ppc64 SMP gcc-4.1


 modinfo mlx4_core
filename:       /lib/modules/2.6.16.46-0.12-ppc64/updates/kernel/drivers/net/mlx4/mlx4_core.ko
version:        0.01
license:        Dual BSD/GPL
description:    Mellanox ConnectX HCA low-level driver
author:         Roland Dreier
srcversion:     2FD23F27A2C14EE6DA1D7D7
alias:          pci:v000015B3d0000673Csv*sd*bc*sc*i*
alias:          pci:v000015B3d00006732sv*sd*bc*sc*i*
alias:          pci:v000015B3d00006354sv*sd*bc*sc*i*
alias:          pci:v000015B3d0000634Asv*sd*bc*sc*i*
alias:          pci:v000015B3d00006340sv*sd*bc*sc*i*
depends:
vermagic:       2.6.16.46-0.12-ppc64 SMP gcc-4.1
parm:           debug_level:Enable debug tracing if > 0 (int)
parm:           msi_x:attempt to use MSI-X if nonzero (int)
parm:           ierr_reset_disable:disable reset on Internal Error event if nonzero (int)

ls -l /sys/class/infiniband
total 0

My suspicion is that Installed FW version is 2.0.150, that is lower than the OFED 1.2.5 Wiki 
suggestion for ConnectX IB (fw-25408 Rev 2.2.000) maybe causing problems. However when I load the 
mlx4_ib module no errors are seen.

Additionally, even though the uverbs module is loaded, I do not see the uverbs devices (udev is running). 
Only the rdma_cm device as shown below.

ls -l /dev/infiniband/
total 0
crw-rw-rw- 1 root root 10, 62 Oct 18 17:13 rdma_cm


Any suggestions?

Pradeep