[openib-general] ip over ib throughtput

[Grant Grundler]

On Wed, Jan 12, 2005 at 06:37:51PM -0800, Roland Dreier wrote:
> This may or may not help on your ia64 systems, but one thing I've
> found helpful for getting the highest IPoIB performance is to bind the
> HCA's interrupt handlers to one CPU and use taskset to bind the
> netperf/netserver processes to a different CPU.

Ok - thanks for the suggestion but it doesn't help.


taskset 1 /opt/netperf/netperf -l 60 -H 10.0.0.81 -t TCP_STREAM -- -m 8192 -s 262144 -S 262144

and then manually determining the netserver PID and setting it
to various masks. I got:
netperf/irq same, netserver/irq same: 1591 Mbps
netperf/irq diff, netserver/irq same: 1565 Mbps
netperf/irq same, netserver/irq diff: 1507 Mbps
netperf/irq diff, netserver/irq diff: 1510 Mbps

It's take ~5-10 seconds to locate the PID and set the mask.
The last two numbers are too close to draw any conclusions.

I didn't expect it to improve since the CPU isn't fully utilized.
And it looks like caching is playing a measurable role on
the receive side of things. Some prefetching might help...I'll
play with that later.

thanks,
grant