[ofa-general] Performance penalty of OFED 1.1 versus IBGD 1.8.2

[Michael S. Tsirkin]

> Quoting Roland Fehrenbacher <rf at q-leap.de>:
> Subject: [ofa-general] Performance penalty of OFED 1.1 versus IBGD 1.8.2
> 
> Content-Description: message body text
> Hi,
> 
> I'm migrating from IBGD 1.8.2 (kernel 2.6.15.7) to OFED 1.1, and saw
> some unpleasant performance drops when using OFED 1.1 (kernel 2.6.20.1
> with included IB drivers). The main drop is in throughput as measured
> by the OSU MPI bandwidth benchmark. However, the latency for large
> packet sizes is also worse (see results below). I tried with and
> without "options ib_mthca msi_x=1" (using IBGD, disabling msi_x makes
> a siginficant performance difference of approx. 10%). The IB card is a
> Mellanox MHGS18-XT (PCIe/DDR Firmware 1.2.0) running on an Opteron
> with nForce4 2200 Professional chipset.
> 
> Does anybody have an explanation or even better a solution to this
> issue?

Could be a BIOS bug. Try setting tune_pci=1. If this helps, contact your BIOS
vendor: here's an explanation about what this parameter does:

http://www.mail-archive.com/openib-general@openib.org/msg25305.html

-- 
MST