I am having a problem with uDAPL accessing
/dev/infiniband/{uat,ucm0}. I am running 3549, 2.6.12
kernel with backport. Here is a snippet of the uDAPL debug
messages running dtest. The dat.conf file seems to be correct,
the correclty named providers are being loaded.<br>
<br>
26248 Running as server<br>
DAT Registry: dat_ia_openv (OpenIB-ib0,1:2,0) called<br>
DAT Registry: IA OpenIB-ib0, trying to load library /usr/local/lib/libdapl.so<br>
libuat: Error <-1:6> couldn't open IB at device </dev/infiniband/uat><br>
libibcm: error <-1:6> opening device </dev/infiniband/ucm0><br>
DAPL: NOT Setting Loopback<br>
dapl_ib_init:<br>
DAT Registry: dat_registry_add_provider (OpenIB-ib0,1:2,0)<br>
dapl_ia_open (OpenIB-ib0, 8, 0x10019d40, 0x10019cc0)<br>
open_hca: mthca0 - 0x1001fdb0<br>
open_hca: Found dev mthca0 f422000002c90200<br>
open_hca: GID subnet 00000000000080fe id f522000002c90200<br>
ips_by_gid: ERR ips_by_gid -1 Bad file descriptor<br>
open_hca: ERR ib_at_ips_by_gid for mthca0<br>
dapls_ib_open_hca failed 40000<br>
dapl_ia_open () returns 0x40000<br>
26248: Error Adaptor open: DAT_INTERNAL_ERROR<br>
DAT Registry: Stopped (dat_fini)<br>
DAPL: Stopped (dapl_fini)<br>
dapl_ib_release:<br>
<br>
<br>
I am not running udev but manually create uat and ucm. Here is the list of /dev/infiniband:<br>
<br>
ls -l /dev/infiniband/<br>
total 0<br>
crw-rw-rw- 1 root root 231, 64 Sep 22 15:18 issm0<br>
crw-rw-rw- 1 root root 231, 65 Sep 22 15:18 issm1<br>
crw-rw-rw- 1 root root 231, 254 Sep 22 22:47 uat<br>
crw-rw-rw- 1 root root 231, 255 Sep 20 22:31 ucm<br>
crw-rw-rw- 1 root root 231, 255 Sep 26 20:01 ucm0<br>
crw-rw-rw- 1 root root 231, 0 Sep 22 15:18 umad0<br>
crw-rw-rw- 1 root root 231, 1 Sep 22 15:18 umad1<br>
crw-rw-rw- 1 root root 231, 192 Sep 20 22:30 uverbs0<br>
crw-rw-rw- 1 root root 231, 193 Sep 20 22:30 uverbs1<br>
<br>
<br>
And the loaded modules:<br>
<br>
kdapl_ib 82000 0<br>
kdapl
14888 1 kdapl_ib<br>
ib_uverbs 52064 0<br>
ib_ipoib 65480 0<br>
ib_ucm 32624 0<br>
ib_cm
51944 2 kdapl_ib,ib_ucm<br>
ib_uat 22168 0<br>
ib_at
34840 2 kdapl_ib,ib_uat<br>
ib_sa
25328 2 ib_ipoib,ib_at<br>
ib_mthca 160376 0<br>
ib_mad
61108 3 ib_cm,ib_sa,ib_mthca<br>
ib_core
73888 8
kdapl_ib,ib_uverbs,ib_ipoib,ib_ucm,ib_cm,ib_sa,ib_mthca,ib_mad<br>
<br>
<br>
I am sure that I am missing something simple. Can someone point me in the right direction.<br>
<br>
Thanks,<br>
Todd<br>