[openib-general] Timeline of IPoIB performance
Matt Leininger
mlleinin at hpcn.ca.sandia.gov
Fri Oct 7 19:25:49 PDT 2005
I'm adding netdev to this thread to see if they can help.
I'm seeing an IPoIB (IP over InfiniBand) netperf performance drop off,
of up to 90 MB/s, when using kernels newer than 2.6.11. This doesn't
appear to be an OpenIB IPoIB issue since the older in-kernel IB for
2.6.11 and a recent svn3687 snapshot both have the same performance (464
MB/s) with 2.6.11. I used the same kernel config file as a starting
point for each of these kernel builds. Have there been any changes in
Linux that would explain these results?
Here is the hardware setup and netperf results using 'netperf -f -M -c
-C -H IPoIB_ADDRESS
All benchmarks are with RHEL4 x86_64 with HCA FW v4.7.0
dual EM64T 3.2 GHz PCIe IB HCA (memfull)
Kernel OpenIB msi_x netperf (MB/s)
2.6.14-rc3 in-kernel 1 374
2.6.13.2 svn3627 1 386
2.6.13.2 in-kernel 1 394
2.6.12.5-lustre in-kernel 1 399
2.6.12.5 in-kernel 1 402
2.6.12 in-kernel 1 406
2.6.12-rc6 in-kernel 1 407
2.6.12-rc5 in-kernel 1 405 <<<<<
2.6.12-rc4 in-kernel 1 470 <<<<<
2.6.12-rc3 in-kernel 1 466
2.6.12-rc2 in-kernel 1 469
2.6.12-rc1 in-kernel 1 466
2.6.11 in-kernel 1 464
2.6.11 svn3687 1 464
2.6.9-11.ELsmp svn3513 1 425 (Woody's results, 3.6Ghz EM64T)
Thanks,
- Matt
More information about the general
mailing list