<br><div><span class="gmail_quote">On 27 Sep 2005 09:55:02 -0400, <b class="gmail_sendername">Hal Rosenstock</b> <<a href="mailto:halr@voltaire.com">halr@voltaire.com</a>> wrote:</span><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
On Tue, 2005-09-27 at 09:51, James Lentini wrote:<br>> On Mon, 26 Sep 2005, Hal Rosenstock wrote:<br>><br>> > On Mon, 2005-09-26 at 18:05, Todd Bowman wrote:<br>> > > I am having a problem with uDAPL accessing
<br>> > > /dev/infiniband/{uat,ucm0}. I am running 3549, 2.6.12 kernel with<br>> > > backport. Here is a snippet of the uDAPL debug messages running<br>> > > dtest. The dat.conf file seems to be correct, the correclty named
<br>> > > providers are being loaded.<br>> > ><br>> > > 26248 Running as server<br>> > > DAT Registry: dat_ia_openv (OpenIB-ib0,1:2,0) called<br>> > > DAT Registry: IA OpenIB-ib0, trying to load library
<br>> > > /usr/local/lib/libdapl.so<br>> > > libuat: Error <-1:6> couldn't open IB at device </dev/infiniband/uat><br>> > > libibcm: error <-1:6> opening device </dev/infiniband/ucm0>
<br>><br>> This means that the /dev entried are not setup correctly.<br><br>Correct. He set this up manually. Todd wrote:<br>"I am not running udev but manually create uat and ucm."</blockquote><div><br>
The correct major/minor #s fixed that problem.<br>
</div><br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">> > > DAPL: NOT Setting Loopback<br>> > > dapl_ib_init:<br>> > > DAT Registry: dat_registry_add_provider (OpenIB-ib0,1:2,0)
<br>> > > dapl_ia_open (OpenIB-ib0, 8, 0x10019d40, 0x10019cc0)<br>> > > open_hca: mthca0 - 0x1001fdb0<br>> > > open_hca: Found dev mthca0 f422000002c90200<br>> > > open_hca: GID subnet 00000000000080fe id f522000002c90200
<br>> ><br>> > These look like they need to be endianized to me.<br>><br>> This looks like a bug in the way we print these values out, but I<br>> don't think it is the real problem.<br><br>Right, it's just a cosmetic with the display.
<br><br>-- Hal<br><br>> What architecture are you using?</blockquote><div><br>
Apple G5. <br>
</div><br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">><br>> > > ips_by_gid: ERR ips_by_gid -1 Bad file descriptor<br>> > > open_hca: ERR ib_at_ips_by_gid for mthca0
<br>> > > dapls_ib_open_hca failed 40000<br>> > > dapl_ia_open () returns 0x40000<br>> > > 26248: Error Adaptor open: DAT_INTERNAL_ERROR<br>> > > DAT Registry: Stopped (dat_fini)<br>> > > DAPL: Stopped (dapl_fini)
<br>> > > dapl_ib_release:<br>> > ></blockquote><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">> > ><br>> > > I am not running udev but manually create uat and ucm. Here is the
<br>> > > list of /dev/infiniband:<br>> > ><br>> > > ls -l /dev/infiniband/<br>> > > total 0<br>> > > crw-rw-rw- 1 root root 231, 64 Sep 22 15:18 issm0<br>> > > crw-rw-rw- 1 root root 231, 65 Sep 22 15:18 issm1
<br>> > > crw-rw-rw- 1 root root 231, 254 Sep 22 22:47 uat<br>> ><br>> > uat is at 231/191.<br>> ><br>> > > crw-rw-rw- 1 root root 231, 255 Sep 20 22:31 ucm<br>> ><br>> > I don't think you need this.
<br>> ><br>> > > crw-rw-rw- 1 root root 231, 255 Sep 26 20:01 ucm0<br>> ><br>> > ucm devices start at 231/224.<br>><br>> If these changes do not fix you problem, please let us know.<br>>
<br>> > -- Hal<br>> ><br>> > > crw-rw-rw- 1 root root 231, 0 Sep 22 15:18 umad0<br>> > > crw-rw-rw- 1 root root 231, 1 Sep 22 15:18 umad1<br>> > > crw-rw-rw- 1 root root 231, 192 Sep 20 22:30 uverbs0
<br>> > > crw-rw-rw- 1 root root 231, 193 Sep 20 22:30 uverbs1<br>> > ><br>> > ><br>> > > And the loaded modules:<br>> > ><br>> > > kdapl_ib 82000 0<br>
>
> >
kdapl 14888 1
kdapl_ib<br>> > > ib_uverbs 52064 0<br>> > > ib_ipoib 65480 0<br>>
> >
ib_ucm
32624 0<br>> > >
ib_cm 51944 2
kdapl_ib,ib_ucm<br>> > >
ib_uat
22168 0<br>> > >
ib_at 34840 2
kdapl_ib,ib_uat<br>> > >
ib_sa 25328 2
ib_ipoib,ib_at<br>> > > ib_mthca 160376 0<br>>
> >
ib_mad
61108 3 ib_cm,ib_sa,ib_mthca<br>> > >
ib_core 73888 8<br>> > > kdapl_ib,ib_uverbs,ib_ipoib,ib_ucm,ib_cm,ib_sa,ib_mthca,ib_mad<br>> > ><br>> > ><br>> > > I am sure that I am missing something simple. Can someone point me in
<br>> > > the right direction.<br>> > ><br>> > > Thanks,<br>> > > Todd<br><br></blockquote></div>I am having a different problem in ips_by_gid:<br>
<br>
open_hca: Found dev mthca0 f422000002c90200<br>
open_hca: GID subnet 00000000000080fe id f522000002c90200<br>
ips_by_gid: ERR ips_by_gid -1 No such device<br>
open_hca: ERR ib_at_ips_by_gid for mthca0<br>
dapls_ib_open_hca failed 40000<br>
dapl_ia_open () returns 0x40000<br>
DT_cs_Server: Could not open OpenIB-ib0 (DAT_INTERNAL_ERROR )<br>
<br>
Thanks,<br>
Todd<br>