[openib-general] mvapich2 ofed 1.2 problem

[Doug Ledford]

On Wed, 2007-02-14 at 23:31 -0500, Shaun Rowland wrote:

> It is not clear to me why the difference of either linking libibverbs
> into libsrqtest.so or not doing so causes the IBVERBS 1.1 ABI to be used
> or not. I looked at the libibverbs code, and the 1.1 ABI is the default.
> The libsrqtest.so file in the above case seems to have lost this
> information:
> 
> [rowland at z1 ibverbs-examples]$ nm libsrqtest.so |grep ibv |head
>                   U ibv_ack_cq_events
>                   U ibv_alloc_pd
>                   U ibv_close_device
>                   U ibv_create_comp_channel
>                   U ibv_create_cq
>                   U ibv_create_qp
>                   U ibv_create_srq
>                   U ibv_dealloc_pd
>                   U ibv_dereg_mr
>                   U ibv_destroy_comp_channel

It didn't loose the information, it never had it.  When you link both
libs against the application binary, the linker is resolving linkups and
writing that into the resulting application binary output, but unless
it's allowed to write into the libsrqtest.so binary and modify *it's*
link table, that particular versioning information can't be written.
Obviously, if every gcc compile that touched a shared library as a
source object file also attempted to write back to that source object
file, people would be very surprised when their attempt to link an
application failed due to permission problems on the shared library.

> I've never had to deal with an ABI issue like this in shared library
> linking/usage. Does it make sense for this to be the case? I think
> perhaps it does, but I wanted to ask.

Yes.  If you want symbol information in a shared lib that uses other
shared libs, then they have to be linked at .so creation time, not at
application creation time.

> I've placed my test code here if it helps:
> 
> http://www.cse.ohio-state.edu/~rowland/ibverbs-examples.tar.gz
> 
> I have a fix for our code that I am testing now. It seems to work and
> solve the observed problems, but more testing will be required to be
> sure there are no issues. This will require a new SRPM if the fix is
> required, which it seems at this point.
-- 
Doug Ledford <dledford at redhat.com>
              GPG KeyID: CFBFF194
              http://people.redhat.com/dledford

Infiniband specific RPMs available at
              http://people.redhat.com/dledford/Infiniband
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
URL: <http://lists.openfabrics.org/pipermail/general/attachments/20070215/0df393c2/attachment.sig>