[openib-general] uDAPL problem

[Steve Smaldone]

Hi,

Sorry for replying to myself, but I loaded rdma_ucm and the rdma_cm 
device appears.  However, it now fails with the following:

$ ./dapltest -T S -D IB1
...
DAT Registry: dat_ia_openv (IB1,1:2,0) called
DAT Registry: IA IB1, trying to load library /usr/local/lib/libdapl.so
DAT Registry: dat_registry_add_provider (IB1,1:2,0)
libibverbs: Warning: no userspace device-specific driver found for uverbs0
        driver search path: /usr/local/lib/infiniband
libibverbs: Warning: no userspace device-specific driver found for uverbs0
        driver search path: /usr/local/lib/infiniband
DT_cs_Server: Could not open IB1 (DAT_INVALID_ADDRESS )
DT_cs_Server (IB1):  Exiting.
DAT Registry: Stopped (dat_fini)

The configuration remains the same otherwise.

Thanks,

Steve Smaldone

Steve Smaldone wrote:
> Hi,
>
> I have been trying to run dapltest with trunk rev 9717 with linux 
> kernel 2.6.18 and I get an error.  The error and configuration is 
> shown below.
> Basically, the rdma_cm device is not created under /dev/infiniband.  I 
> am wondering if this is a known problem and how to solve it.
>
> Thanks,
> Steve Smaldone
>
> $ ./dapltest -T S -D IB1
> ...
> DAT Registry: dat_ia_openv (IB1,1:2,0) called
> DAT Registry: IA IB1, trying to load library /usr/local/lib/libdapl.so
> DAT Registry: dat_registry_add_provider (IB1,1:2,0)
> librdmacm: couldn't read ABI version.
> librdmacm: assuming: 2
> libibverbs: Warning: no userspace device-specific driver found for 
> uverbs0
>        driver search path: /usr/local/lib/infiniband
> CMA: unable to open /dev/infiniband/rdma_cm
> DT_cs_Server: Could not open IB1 (DAT_INTERNAL_ERROR )
> DT_cs_Server (IB1):  Exiting.
> DAT Registry: Stopped (dat_fini)
>
> My dat.conf:
> IB1 u1.2 nonthreadsafe default /usr/local/lib/libdapl.so mv_dapl.1.2 
> "hora-1-ib0 0" ""
>
> ifconfig:
> ib0       Link encap:UNSPEC  HWaddr 
> 00-00-00-14-FE-80-00-00-00-00-00-00-00-00-00-00
>          inet addr:10.2.2.135  Bcast:10.255.255.255  Mask:255.0.0.0
>          inet6 addr: fe80::202:c901:81e:90e1/64 Scope:Link
>          UP BROADCAST RUNNING MULTICAST  MTU:2044  Metric:1
>          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
>          TX packets:1 errors:0 dropped:5 overruns:0 carrier:0
>          collisions:0 txqueuelen:128
>          RX bytes:0 (0.0 b)  TX bytes:68 (68.0 b)
>
> lsmod:
> Module                  Size  Used by
> ib_ucm                 20612  0
> ib_uverbs              40232  1 ib_ucm
> rdma_cm                33572  0
> ib_cm                  39824  2 ib_ucm,rdma_cm
> ib_addr                11524  1 rdma_cm
> ib_local_sa            15752  1 rdma_cm
> findex                  8576  1 ib_local_sa
> ib_ipoib               51144  0
> ib_multicast           15940  2 rdma_cm,ib_ipoib
> ib_sa                  20004  5 
> rdma_cm,ib_cm,ib_local_sa,ib_ipoib,ib_multicast
> ib_umad                20016  0
> ib_mthca              131236  0
> ib_mad                 42272  5 ib_cm,ib_local_sa,ib_sa,ib_umad,ib_mthca
> ib_core                52992  11 
> ib_ucm,ib_uverbs,rdma_cm,ib_cm,ib_local_sa,ib_ipoib,ib_multicast,ib_sa,ib_umad,ib_mthca,ib_mad 
>
>
> udev rules:
> KERNEL="umad*", NAME="infiniband/%k"
> KERNEL="issm*", NAME="infiniband/%k"
> KERNEL="uverbs*", NAME="infiniband/%k", MODE="0666"
> KERNEL="ucm*", NAME="infiniband/%k", MODE="0666"
> KERNEL="rdma_cm", NAME="infiniband/%k", MODE="0666"
>
> /dev/infiniband:
> issm0  issm1  ucm0  umad0  umad1  uverbs0
>
>