[openib-general] ip over ib throughtput

[Roland Dreier]

    Eric> Does mthca support NAPI?  If you are interrupt limited I
    Eric> suspect that will help. With the MSI-X comments it sounds
    Eric> like you are interrupt limited.

NAPI doesn't have any relevance to mthca -- it needs to be implemented
in the IPoIB driver.

I actually tried adding NAPI to the IPoIB driver a while ago and it
didn't help at all.  I'm not sure that the interrupt rate is the
limit-- IPoIB seems to be CPU-bound.

    Eric> Last time I was messing around I could get 600MB/s with UD
    Eric> packets so it looks like there is still room from IBoIP to
    Eric> go.  And that has been achieved with the Linux IP stack with
    Eric> 10 GigE adapters.  What part checksum offloading and tuning
    Eric> for IP workloads in the hardware design I don't know.  Still
    Eric> when memory bandwidth is greater than 3x the network speed I
    Eric> expect it mostly because a question of how much of your cpu
    Eric> the communication will utilize.

I know 10 gigE has shown some very high numbers, but I assume that's
doing sendfile() with 9K jumbo frames.  Do you know how well they do
in non-zero-copy workloads with a 2K MTU?

 - R.