[openib-general] [PATCH] mthca updates (2.6.8 dependent)

[Michael S. Tsirkin]

Hello!
Quoting r. Dror Goldenberg (gdror at mellanox.co.il) "RE: [openib-general] [PATCH] mthca updates (2.6.8 dependent)":
> 
> 
> > -----Original Message-----
> > From: Roland Dreier [mailto:roland at topspin.com]
> > Sent: Monday, August 16, 2004 6:52 PM
> 
> >     Dror> Anyway, I believe that with both MSI and MSI-X you can avoid
> >     Dror> the PIO read. * If it's MSI-X, it's easy. By the IRQ# you
> >     Dror> can tell which EQ has work.  * If it's MSI, just go and peek
> >     Dror> into all available EQs. It's much more efficient than doing
> >     Dror> a PIO read.
> >
> > It seems this would apply to standard INTx mode as well.  Do
> > you know why Mellanox didn't use this in THCA?  In any case,
> > I'll have to benchmark this approach.
> >
> 
> In PCI/PCIX, the interrupt is a wire, so it is not guaranteed that by the time
> you
> got the interrupt, the EQE will be waiting in memory.

Clarification: if this happends, and you arm the EQ, you will get
an immediate interrupt again. So what you can do, is ARM all EQs
without regard to whether you did or did not find an eq entry there.

Whether this race happends a sufficient number of times to affect
performance
negatively due to an extra interrupt remains to be seen.


> This is because interrupt
> goes on a separate wire from HCA to interrupt controller, while data goes
> up the PCI bridges. Therefore it is required to perform a PIO read to flush all
> posted writes flying upstream.

So not "required" per se - required only if you want to find out
which EQ had an event.


> In PCI-Express, the interrupt is a message, so it will work. The interrupt will
> just flush the data to the memory because it maintain ordering with posted
> writes upstream.

I'm not sure you can rely on messages being ordered properly
with regard to posted writes e.g. inside the chipset.

> In the current driver, since it's PCI and PCI-Express we
> don't do it. In the new mode for Arbel we may do it.
> When you do MSI/MSI-X, then architecturally it is guaranteed that by the time
> you get the interrupt, the data already waits for you in memory.
> Dror
>