[openfabrics-ewg] Drop in performance on Mellanox MT25204 single port DDR HCA

[Michael S. Tsirkin]

Quoting r. Woodruff, Robert J <robert.j.woodruff at intel.com>:
> Subject: Drop in performance on Mellanox MT25204 single port DDR HCA
> 
>  
> Hi Roland/Michael,
> 
> One of my coworkers in Champaign is seeing a performance
> issue with the latest SVN driver and the OFED 1.1 Mellanox 
> driver on certain platforms.
> 
> On the older SVN somewhere around 7500 the Mellanox driver
> did not save and restore certain PCI registers before a reset.
> Somewhere around SVN 8000 a patch was added to save and
> restore these registers. However on our Alcolu platform 
> this patch causes the MaxReadReq to be set to 128 bytes (rather than
> 512)
> which limits bandwith to 650MBytes/sec. If I remove the
> save/restore of these registers (attached patch), the
> bandwidth is back to where we would expect it 1250 Mbytes/sec.
> 
> Is there some problem with this patch or do you think it is
> some BIOS issue in the platform ?

This is a BIOS issue - it should set the MaxReadReq register for maximum
performance and stability.  As a work-around, you can use the setpci utility to
modify MaxReadReq before loading the driver.

Unfortunately, mthca has no way to know which values are legal
and which will give the best performance, and previous behaviour was
out of spec, reportedly causing stability issues on compliant platforms.

I will look into adding this info in release notes.

-- 
MST