[ofa-general] Re: Performance penalty of OFED 1.1 versus IBGD 1.8.2

[Michael S. Tsirkin]

> Quoting Roland Fehrenbacher <rf at q-leap.de>:
> Subject: Re: Performance penalty of OFED 1.1 versus IBGD 1.8.2
> 
> >>>>> "MST" == Michael S Tsirkin <mst at mellanox.co.il> writes:
> 
>     >> Quoting Roland Fehrenbacher <rf at q-leap.de>: Subject:
>     >> [ofa-general] Performance penalty of OFED 1.1 versus IBGD 1.8.2
>     >> 
>     >> Hi,
>     >> 
>     >> I'm migrating from IBGD 1.8.2 (kernel 2.6.15.7) to OFED 1.1,
>     >> and saw some unpleasant performance drops when using OFED 1.1
>     >> (kernel 2.6.20.1 with included IB drivers). The main drop is in
>     >> throughput as measured by the OSU MPI bandwidth
>     >> benchmark. However, the latency for large packet sizes is also
>     >> worse (see results below). I tried with and without "options
>     >> ib_mthca msi_x=1" (using IBGD, disabling msi_x makes a
>     >> siginficant performance difference of approx. 10%). The IB card
>     >> is a Mellanox MHGS18-XT (PCIe/DDR Firmware 1.2.0) running on an
>     >> Opteron with nForce4 2200 Professional chipset.
>     >> 
>     >> Does anybody have an explanation or even better a solution to
>     >> this issue?
> 
>     MST> Could be a BIOS bug. Try setting tune_pci=1. If this helps,
>     MST> contact your BIOS vendor: here's an explanation about what
>     MST> this parameter does:
> 
>     MST> http://www.mail-archive.com/openib-general@openib.org/msg25305.html
> 
> I tried this with no effect. Just to make sure the settings are in
> effect, is there a way I can check this after booting?

cat /sys/modules/ib_mthca/parameters/tune_pci


-- 
MST