[openib-general] ip over ib throughtput

[Shirley Ma]

Grant Grundler <iod00d at hp.com> wrote on 05/09/2006 12:05:38 PM:

> On Tue, May 09, 2006 at 11:36:45AM -0700, Shirley Ma wrote:
> > What throughput did you get on two CPUs?
> 
> With one CPU, I get ~2.5-2.8 Gb/s. With two CPUs: 3.5-3.6 Gb/s.
> The last SVN version I tested was 2.6.15 + r4929 (several monthes ago).

I got 3.5-3.66Gb/s on UP with splitting CQ + threads supports on send/recv 

CQ polling, and didn't get any better on SMP.

> To be clear, "one CPU" means the netperf process is bound to the same
> CPU as the one handling mthca interrupts with taskset. Ditto for 
netserver
> on the other system. "Two CPU" means bind netperf/netserver processes
> to a different CPU that is NOT handling the mthca interrupt.

I tried interrupts affinity on mthca, it didn't work by changing 
/proc/irq/XXX/smp_affinity. The interrupts distruction kept bouncing 
between 
there two CPUs. 

What did you do to handle mthca interrupt on a fixed cpu?

> Also note that the "Service Demand" (CPU us/KB) goes up by
> 10-20% also. So we really only want to do this when the CPU
> handling interrupts is saturated and we know the other CPU
> is available.
> grant

Thanks
Shirley Ma
IBM Linux Technology Center
15300 SW Koll Parkway
Beaverton, OR 97006-6063
Phone(Fax): (503) 578-7638
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/general/attachments/20060509/9feddad3/attachment.html>