[openib-general] Error in opening IB cm device

[amith rajith mamidala]

Hi,

I couldn't find /dev/infiniband in my installation. After adding this
manually and doing mknod as you suggested the error in opening the
/dev/infiniband/ucm disappeared. However, now the program
(ibv_rc_pingpong) is not able to create QP.


I might be missing some steps in the installation.

What I have done is as follows:

1. Link kernel code in place
~> rm -fr /usr/linux-2.6.11/drivers/infiniband
~> ln -s /usr/src/roland-uverbs/src/linux-kernel/infiniband
/usr/linux-2.6.11/drivers/infiniband

2. Build kernel components
~>cd /usr/linux-2.6.11
~>make menuconfig

You are presented with a menu. Scroll to "device drivers", select it,
select Infiniband support.
Select userspace support, Mellanox HCA support.

Exit saving kernel configuration.

~>make modules modules_install
This shall build and install relevant kernel modules.

Note how low level module (mthca) is loaded automatically upon reboot, so
if you change this,
reboot or unload it as I explain later.

3. Loading kernel modules.
~>udevstart 2>&1 > /dev/null
~>modprobe ib_mthca
~>modprobe ib_umad
~>modprobe ib_uverbs


However I was not able to add the udev rules as specified in the following
instruction. I could not locate the /etc/udev/udev.rules.

---------------------------------------------------------------------
3. Adding udev rules.
You need device files for kernel modules we just built.
We could add them later, manually, but udev is more convenient.

Add this to /etc/udev/udev.rules (if the file does not exist, udev may not
be installed, you'll
have to add devices manually, I'll skip this part here, lets assume its
OK).

KERNEL="umad*", NAME="infiniband/%k"
KERNEL="issm*", NAME="infiniband/%k"
KERNEL="uverbs*", NAME="infiniband/%k", MODE="0666"
----------------------------------------------------------------------

Is this the cause for the problem ?


Thanks,
Amith




On Thu, 14 Jul 2005, Matt Leininger wrote:

> On Fri, 2005-07-15 at 01:47 -0400, amith rajith mamidala wrote:
> > Hi,
> >
> > We have installed the latest openib stack (Rev:2861) on the X86_64
> > platform. While running the pingpong tests I am encountering the
> > following error:
> >
> > libucm: Error <-1:2> couldn't open IB cm device </dev/infiniband/ucm>
> > [1] Abort: Error getting HCA context
> >
> > I am not sure why this happening ?
> >
>
> What does 'ls -l /dev/infiniband' say?  You should have:
>
> [root at n64-81 ~]# ls -l /dev/infiniband/
> total 0
> crw-rw-rw-    1 root     root     231, 255 Jul 13 10:44 ucm
> crw-rw-rw-    1 root     root     231, 192 Jul 13 10:45 uverbs0
> crw-rw-rw-    1 root     root     231, 193 Jul 13 10:45 uverbs1
>
> Try,
>
> modprobe ib_uverbs
> mknod -m 0666 /dev/infiniband/uverbs0 c 231 192
> mknod -m 0666 /dev/infiniband/uverbs1 c 231 193
>
>
> modprobe ib_ucm
> mknod /dev/infiniband/ucm c 231 255
>
>   - Matt
>
>