[openib-general] ip over ib throughtput

[Grant Grundler]

On Thu, Jan 13, 2005 at 08:58:37PM -0800, Roland Dreier wrote:
> It seems that the fact that Mellanox HCAs show up as a device with a
> giant 128 MB BAR behind a bridge triggers all sorts of BIOS nastiness
> (since of course no BIOS vendor would ever test such a thing!).

Yeah - except on high end graphics cards.
It's becoming more common though.

But if exposing is causing such problems, shouldn't mellanox
try to make it easier to hide that BAR like the HP firmware
does?

Ie make it a user choice and not a vendor choice.
I guess vendors make that choice and don't want extra support costs.

> I've
> definitely seen HCA interrupt routing fail on many otherwise-working
> systems.  (Things have been gradually improving but problems still
> crop up)

"gradually improving" might be (partially) a result of users being
"trained" to not use machines with broken BIOS.

> There is a reason for the slightly awkward order.  It turns out that
> there are some quirks with the Mellanox HCA and it's better not to
> touch the PCI header after the SYS_EN firmware command is executed.
> So the driver ends up having to set up MSI/MSI-X way before it's ready
> to enable interrupts.

Ok - given the complexity of the HCA firmware and the fact that
at least parts of config space are emulated by HCA firmware this
makes sense.

thanks,
grant