[openib-general] Re: unable to run ibv_uc_pingpong and other example binaries

[Michael S. Tsirkin]

Quoting r. Hal Rosenstock <halr at voltaire.com>:
> Subject: Re: unable to run ibv_uc_pingpong and other example binaries
> 
> On Tue, 2005-07-19 at 12:49, Michael S. Tsirkin wrote:
> > Quoting r. Joe Damato <damato at psc.edu>:
> > > Subject: Re: unable to run ibv_uc_pingpong and other example binaries
> > > OK, just tried it, same problem:
> > > [root at frodo examples]# pwd
> > > /cluster/src/OPENIB_SRC/openib-2734/gen2/trunk/src/userspace/libibverbs/examples
> > > [root at frodo examples]# .libs/ibv_uc_pingpong
> > > libucm: Error <-1:6> couldn't open IB cm device </dev/infiniband/ucm>
> > 
> > OK, this one is because you installed libibcm into 
> > /usr/local/lib/infiniband which is a plugin directory instead if
> > /usr/local/lib, so it goes.
> > 
> > > open("/usr/local/lib/infiniband/libibcm.so", O_RDONLY) = 3
> > 
> > Simply remove /usr/local/lib/infiniband/libucm.so
> > and it will go away.
> 
> It appears to be invoking libucm just fine (the location is not the
> issue). The issue is ucm module is not started.
> 
> > Longer term I think libucm config script must be fixed: libraries
> > go to /usr/local/lib by default. Hal?
> 
> As far as I know, the current agreed upon default is
> /usr/local/xxx/infiniband where xxx is lib and include.

Hal, libibverbs looks for libraries under /usr/local/lib/infiniband
and assumes these are low-level driver plugins.
We *dont* want libibverbs to load libucm - its not a low-level driver plugin.

Look at libibverbs, libs are in /usr/local/lib, includes in
/usr/local/include/infiniband.
This is because with includes you can do

#include <infiniband/foo.h>

but this doesnt work with libraries.

> > Hal, I also think it may not be such a good idea for a library
> > to start opening files and print things out to standard error
> > when its loaded.
> > Rather, a library should have some sane init/cleanup functions.
> 
> Due to hot swap or other issues ?

Not necessarily. Errors during library loading are much harder
to isolate and debug.

> I think all the user space libraries
> do this with perhaps the exception of libibverbs.

What else is there :)? Management doesnt count, most applications
wont use it.

-- 
MST