[ofa-general] Re: Performance penalty of OFED 1.1 versus IBGD 1.8.2

[Roland Fehrenbacher]

>>>>> "MST" == Michael S Tsirkin <mst at mellanox.co.il> writes:

    >> Quoting Roland Fehrenbacher <rf at q-leap.de>: Subject: Re:
    >> Performance penalty of OFED 1.1 versus IBGD 1.8.2

    >> I'm migrating from IBGD 1.8.2 (kernel 2.6.15.7) to OFED 1.1,
    >> and saw some unpleasant performance drops when using OFED 1.1
    >> (kernel 2.6.20.1 with included IB drivers). The main drop is in
    >> throughput as measured by the OSU MPI bandwidth
    >> benchmark. However, the latency for large packet sizes is also
    >> worse (see results below). I tried with and without "options
    >> ib_mthca msi_x=1" (using IBGD, disabling msi_x makes a
    >> siginficant performance difference of approx. 10%). The IB card
    >> is a Mellanox MHGS18-XT (PCIe/DDR Firmware 1.2.0) running on an
    >> Opteron with nForce4 2200 Professional chipset.
    >>
    >> Does anybody have an explanation or even better a solution
    >> to this issue?

    MST> Could be a BIOS bug. Try setting tune_pci=1. If this helps,
    MST> contact your BIOS vendor: here's an explanation about what
    MST> this parameter does:

    MST> http://www.mail-archive.com/openib-general@openib.org/msg25305.html

    >> I tried this with no effect. Just to make sure the settings are
    >> in effect, is there a way I can check this after booting?

    MST> cat /sys/modules/ib_mthca/parameters/tune_pci

Ok, the settings are active, but have zero effect. Anything else I
could check?

Roland