[openib-general] OpenIB 1.0 RC + PathScale problem

[Tim Miller]

On Thu, 18 May 2006, Di Domenico, Michael wrote:

> I'm certainly no expert, but I came across different but similar issues,
> where my applications where picking up another set of libraries, that I
> wasn't aware were on the system...  I was getting the same 'undefined
> symbols' errors.  You might want to check for ib libs that might be in
> your path.

Hi Michael,

Thanks to Bryan, Jack, and yourself for responding. I suspect it is a 
library issue, but I'm having some trouble tracking down the exact source 
of the problem. The first odd thing that doing an nm of 
/usr/local/lib/infiniband/ipathverbs.so shows ibv_cmd_poll_cq is indeed 
undefined (the symbol is defined in libibverbs.so). Here's the ldd output 
for ipathverbs.so:

tim at o8:/usr/local/lib 165$ ldd infiniband/ipathverbs.so
         libc.so.6 => /lib64/tls/libc.so.6 (0x00002b01d0d00000)
         /lib64/ld-linux-x86-64.so.2 (0x0000555555554000)

Oddly, it doesn't seem to depend on libibverbs. Is that normal? If not, I 
must be doing something wrong in building the libraries. I tried 
uninstalling all the userspace stuff (via make uninstall) and 
reconfiguring/remaking after a make clean in the source dir. I built, in 
order, libibverbs, libipathvers, and libmthca. Is that the correct way to 
do things (it's what I got from the quickstart Wiki entry)?

FWIW, this is on SuSE 9.3 with kernel 2.6.16.16 custom compiled, but this 
seems to be entirely an issue with the userspace libs. For completeness, 
here is the ldd output for libibverbs:

tim at o8:/usr/local/lib 166$ ldd libibverbs.so
         libsysfs.so.1 => /lib64/libsysfs.so.1 (0x00002ac94dbd4000)
         libpthread.so.0 => /lib64/tls/libpthread.so.0 (0x00002ac94dce0000)
         libdl.so.2 => /lib64/libdl.so.2 (0x00002ac94ddf5000)
         libc.so.6 => /lib64/tls/libc.so.6 (0x00002ac94def9000)
         /lib64/ld-linux-x86-64.so.2 (0x0000555555554000)

The nm output is pretty long, but let me know if you want to see it too.

Thanks,
Tim

-- 
Tim Miller
System Administrator -- Laboratory of Computational Biology
National Institutes of Health   --   Bldg. 50 Rm. 3309    --     301-402-0618