[ofa-general] change mtu

[Michael Di Domenico]

>> it's showing in the log file that it picked up the cached options, but
>> its still showing 2048 MTU under ibv_devinfo
>
> It wouldn't show there; only in SA PR responses.

Okay.  Here's what i've enabled so far, for those keep track....

Bios = MTRR changed from Continuous to Discrete
options ib_mthca msi_x=1 tune_pci=1
options rdma_cm tavor_quirk=1

I'm now pushing 550MB/sec, which is a remarkable improvement.