[openib-general] [PATCH] use mmiowb after doorbell ring

[Jason Gunthorpe]

On Tue, Oct 17, 2006 at 08:44:34PM -0700, Roland Dreier wrote:
>     Jason> I think the typical way this is done would be to use
>     Jason> ld.so's 'hwcap' handling and stick an optimized library in
>     Jason> /usr/lib/sse2.
 
> It's a good suggestion, but the problem is that the CPU-dependent code
> is in the mthca.so driver-dependent plugin, which libibverbs dlopen()s
> at runtime.  Do you know how to use the hwcap stuff with dlopen()?
> I'm not thrilled about creating an sse2 special case in libibverbs
> just to handle libmthca on i386.

It is automatic, I just doubled checked to be sure:

$ cat t.c 
#include <dlfcn.h>
int main(int argc, const char *argv[])
{
   dlopen(argv[1],RTLD_NOW);
}

$ find /usr/lib -name "libcrypto.so.0.9.8"
./i486/libcrypto.so.0.9.8
./libcrypto.so.0.9.8
./i586/libcrypto.so.0.9.8
./i686/cmov/libcrypto.so.0.9.8

$ strace ./t libcrypto.so.0.9.8
[..]
open("/usr/lib/i686/cmov/libcrypto.so.0.9.8", O_RDONLY) = 3
$ mv libcrypto.so.0.9.8 /tmp/libcrypto.so.0.9.8.x
$ strace ./t libcrypto.so.0.9.8
[.. 34 occurances of open /usr/..../libcrypto.so.0.9.8 ..]
open("/usr/lib/libcrypto.so.0.9.8", O_RDONLY) = 3
$ ldconfig
$ strace ./t libcrypto.so.0.9.8
[..]
open("/usr/lib/libcrypto.so.0.9.8", O_RDONLY) = 3

Undocumented, but it does something very close to what you'd
want.. ldconfig caches the soname mapping in /etc/ld.so.cache so you
have to be careful when experimenting. Several packages that get big
gains with specific optimizations use this already. Strace on the
above test program with a non-existing library shows the search path
and all permutations.

It is also probably worth benchmarking a full cmov+i686+sse2 build of
everything and look at always providing it if it is faster, like is
often done for glibc.

Jason