[openib-general] Re: openib-general Digest, Vol 22, Issue 114

[Grant Grundler]

Hi Shirley,

On Wed, Apr 19, 2006 at 11:31:32AM -0700, Shirley Ma wrote:
...
> > By moving netperf RX traffic off the CPU handling interrupts,
> > the 1.5Ghz ia64 box goes from 2.8 Gb/s to around 3.5 Gb/s.
> > But the "service demand" (CPU time per KB payload)  goes up
> > from ~2.3 usec/KB to ~3.1 usec/KB - cacheline misses go up dramatically.
> 
> Yes, netperf/netserver binding to same cpu definitely has benefit 
> cacheline. But the cpu will be the bottleneck. One cpu is not
> sufficient to drain out faster network device HCA.

I agree.
I don't think anyone is suggesting one CPU can handle TCP for a 10Gb/s
(or faster) device.

...
> IPoIB perf if important for people still use old application. We do see 
> under some workload IPoIB gain double bidirectional performance with
> splitting CQ/tune poll interval/poll more entries from WC patch.

Was this measured using ehca?
If so, the result implies at least two interrupt vectors are used.

And it seems reasonable for IPoIB to tune for that even if it
costs mthca a slight amount of overhead. Roland might be more
receptive if someone provided him with data showing perf
and "service demand" for mthca doesn't substantial degrade
because of this change.

hth,
grant