[openib-general] netperf over SDP bug

[Grant Grundler]

On Tue, Sep 27, 2005 at 06:17:00PM -0700, Grant Grundler wrote:
> Hi Michael,
> I'm trying to collect a full set of netperf TCP_STREAM over SDP for
> SVN r3547 on 2.6.13 kernel.  But some netperf runs get no throughput.

Michael,
I was able to reproduce this problem with SVN r3984.
I've posted the graphs for r3547 and r3984 on:
	http://iou.parisc-linux.org/openib-perf-2005/r3547/
	http://iou.parisc-linux.org/openib-perf-2005/r3984/

See sdpstream.png in each location.
I'll pursue collecting information you asked for a few weeks ago
as time permits.

The above data was collected with "netserver" bound to the same CPU
as the one taking IB MSI-X interrupts. This is bad for IPoIB (CPU bound)
and good for SDP (CPU cache). I'll rerun the r3984 data and bind
the netperf process as well.


BTW, in case I haven't mentioned this before, I setup a parisc-linux
box so netperf maintainer Rick Jones could manage his
releases using something better than tarballs. netperf 2.x and
netperf 4.x (under developement) source is available from:
	svn co http://www.netperf.org/svn/netperf2/
	svn co http://www.netperf.org/svn/netperf4/

thanks,
grant

> Usually when sending 1k to 4k messages.  The same netperf parameters
> sing IPoIB seem to be working fine - just alot slower of course.
> Summary of all netperf over SDP runs is appended.
> 
> Sample commandline that got < 1Mb/s throughput is:
> LD_PRELOAD=/usr/local/lib/libsdp.so /usr/local/bin/netperf -p 12866 -l 60 -H 10.0.0.30 -t TCP_STREAM -T 1 -- -m 1024 -s 16384 -S 16384
> 
> I tried with some smaller -m parameters:
> 	512   -> ~270-280 Mb/s
> 	640   -> ~200-2100 Mb/s
> 	768   -> ~30-50 Mb/s
> 	896   -> ~2-6 Mb/s
> 
> CPU is essentially idle in the above 512-896 byte cases.
...