[openib-general] How about ib_send_page() ?

[Jeff Carr]

Roland Dreier wrote:
>     Jeff> (side note: it would seem IPoIB could be re-written to
>     Jeff> dramatically improve it's performance).
> 
> Out of curiousity, what would the rewrite change to obtain better
> performance?

I'm just speculating that it could be rewritten to improve performance.
There were many complaints in the past about the speed of IPoIB; I
thought that was still the case. Maybe the performance has increased and
I have my systems misconfigured.

In any case between my two hosts with netperf:

root at delta:~# netperf -H foxtrot -- -s 105472 -r 105472
TCP STREAM TEST to foxtrot
Recv   Send    Send
Socket Socket  Message  Elapsed
Size   Size    Size     Time     Throughput
bytes  bytes   bytes    secs.    10^6bits/sec

  87380 210944 210944    10.00    2335.21

with vmstat on the server showing:

-----memory---------- ---swap-- -----io---- --system-- ----cpu----
  swpd   free   buff  cache   si   so    bi    bo   in    cs us sy id wa
   0 1115720 440988  49836    0    0     0     0 43943  8251  0 94  6  0
   0 1115720 440988  49836    0    0     0     0 43725  8384  0 94  6  0

So the CPU is throttled at around 100-200k packets/sec with 43k
interrupts / sec. Perhaps the HCA can be configured to respond/trigger
events/generate interrupts less often under higher load. Some ethernet
chips have this kind of functionality (interrupt after x packets or
after n time).

Jeff