[openib-general] Re: [PATCH] rdma_lat-09 and results

[Grant Grundler]

On Wed, Jun 22, 2005 at 10:19:28AM +0300, Michael S. Tsirkin wrote:
> Here's what works for me:
> 
> cc -rdynamic -u openib_driver_init -Wall -O2 -g -D_GNU_SOURCE rdma_bw.c get_clock.c /usr/local/lib/infiniband/mthca.a /usr/local/lib/libibverbs.a  -o rdma_bw -ldl -lsysfs -lpthread

Yes - as you noted "/usr/local/lib/" needed to be "/usr/lib".
The debian packages use the latter.
I was able to build with the above command line and SVN 2670 bits.
But uverbs now seems broken with normal dynamic builds too.
More details below.

> > Or maybe a "make static" target in the Makefile....*sigh*
> 
> Problem with this is, for this I need to figure out where
> is mthca.a installed.

Understood...maybe just hard code it in the Makefile for now?
That's at least a start. 
Use something like "OPENIBLIBSDIR = /usr/local/lib/infiniband"?

I also don't know how to express
	if target == static then
		MORECFLAGS = -rdynamic...
	endif


Anyway....after rebuilding everything to deal with the ABI event
and rebooting to 2.6.12 kernel, I couldn't collect pfmon31 output:

grundler at iota:/usr/src/openib_gen2/src/userspace/perftest$ ./rdma_bw
Couldn't get context for mthca0

This error message doesn't tell me how to fix it. :^(
I'm not getting any dmesg output for this either.


I should be at SVN 2670 with everything after updating and following
the rebuild sequence below:
	cd $SRC/userspace/libibverbs
	dpkg-buildpackage -rfakeroot -uc -us
	cd ..
	sudo dpkg -i libibverbs-dev_0.1.0-1_ia64.deb \
		libibverbs1_0.1.0-1_ia64.deb

	cd libmthca
	dpkg-buildpackage -rfakeroot -uc -us
	cd ..
	sudo dpkg -i libmthca1_0.1.0-1_ia64.deb libmthca-dev_0.1.0-1_ia64.deb

	cd perftest
	make

perftest has no "install" target and doesn't need one.
Matt is there someplace appropriate to add the above recipe?

I ended up with: 
grundler at iota:/usr/src/openib_gen2/src/userspace$ ls -lrt *.deb
-rw-r--r--  1 grundler grundler  15254 Jun 22 07:54 libibverbs1_0.1.0-1_ia64.deb
-rw-r--r--  1 grundler grundler  24060 Jun 22 07:54 libibverbs-dev_0.1.0-1_ia64.deb
-rw-r--r--  1 grundler grundler  21758 Jun 22 07:54 ibverbs-examples_0.1.0-1_ia64.deb
-rw-r--r--  1 grundler grundler  18380 Jun 22 08:01 libmthca1_0.1.0-1_ia64.deb
-rw-r--r--  1 grundler grundler 179284 Jun 22 08:01 libmthca-dev_0.1.0-1_ia64.deb


google suggests I need to create /dev/infiniband nodes but I already
have the following:
grundler at iota:/usr/src/linux-2.6$ ls -l /dev/infiniband/
total 0
crw-rw-rw-  1 root root 231, 255 May  3 15:48 ucm
crw-rw-rw-  1 root root 231, 128 May  3 12:10 uverbs0
crw-rw-rw-  1 root root 231, 129 May  3 12:10 uverbs1


I'm guessing uverbs b0rkage because I can ping other HCAs:
grundler at iota:/usr/src/linux-2.6$ /sbin/ifconfig ib0
ib0       Link encap:UNSPEC  HWaddr 00-00-04-04-FE-80-00-00-00-00-00-00-00-00-00-00  
          inet addr:10.0.0.30  Bcast:10.0.0.255  Mask:255.255.255.0
	  UP BROADCAST RUNNING MULTICAST  MTU:2044  Metric:1
	  RX packets:7 errors:0 dropped:0 overruns:0 frame:0
	  TX packets:8 errors:0 dropped:0 overruns:0 carrier:0
	  collisions:0 txqueuelen:128 
	  RX bytes:532 (532.0 b)  TX bytes:620 (620.0 b)
grundler at iota:/usr/src/linux-2.6$ ping 10.0.0.51
PING 10.0.0.51 (10.0.0.51): 56 data bytes
64 bytes from 10.0.0.51: icmp_seq=0 ttl=64 time=0.5 ms
64 bytes from 10.0.0.51: icmp_seq=1 ttl=64 time=0.0 ms
...


Any clue what stupid thing I've done now?
Or is this really fallout from the ABI event?
(new bug?)

thanks,
grant