[openib-general] ip over ib throughtput

[Eric W. Biederman]

Roland Dreier <roland at topspin.com> writes:

>     Paul> Didn't see any response to Michael's numbers other than
>     Paul> another mail today from Josh England saying 'great numbers
>     Paul> till it dies'.
> 
> I've seen 225 MB/sec on my dual 2.2 GHz Xeon/PCI-X systems with NPtcp,
> and on Josh's systems I've seen 425 MB/sec.  I'm pretty sure Josh's
> problems will be solved by upgrading from FW 4.6.1 to 4.6.2.

Cool. These numbers are starting to sound reasonable.  :)

Does mthca support NAPI?  If you are interrupt limited I suspect
that will help. With the MSI-X comments it sounds like you are
interrupt limited.

Last time I was messing around I could get 600MB/s with UD packets
so it looks like there is still room from IBoIP to go.   And that has
been achieved with the Linux IP stack with 10 GigE adapters.  What
part checksum offloading and tuning for IP workloads in the hardware
design I don't know.  Still when memory bandwidth is greater than 3x
the network speed I expect it mostly because a question of how much
of your cpu the communication will utilize.

Eric