[openib-general] ip over ib throughtput

[Roland Dreier]

    Grant> Yeah, I have mail from April 2003 discussing MMRBC issues
    Grant> with CERN.  Unfortunately, the paper doesn't describe how
    Grant> to change it and I had to look that up again: setpci -d
    Grant> 15b3:5a44 0x72 # read low byte of PCI-X command

Actually mthca_tune_pci() in mthca_main.c is already maxing out the
RBC field of the PCI-X command register (and also the read request
size for PCI Express).  So changing this shouldn't have any effect.

It might help if there is some way to tune other parts of the bus
complex (eg give the bus that the HCA is on a bigger time slice if
some hub above PCI is doing TDM...).

    Grant> MTU size helps a bit.  Using 4k or larger gives me ~1590
    Grant> Mbps vs ~1560 for 2k MTU.  But I don't see improvement for
    Grant> larger MTUs.

I think the IPoIB will refuse to set an MTU above 2044 (since the HCA
is not able to generate larger UD messages).

 - R.