[openib-general] Re: MMIO reads for IPoIB w/MSI-X

[Michael Krause]

It is the consensus within the industry that PIO Reads (MMIO) are pure 
"evil" and should be avoided in any main line data path including the 
interrupt path.  They should be confined to configuration / status / etc. 
checks and thus are very infrequent.  Designs that rely upon PIO Reads for 
anything in the performance path tend to not succeed that well within the 
industry.   This is true across all processor architectures including x86 
itself.  The problem only gets worse (not just because of the number of 
chip crossings) but also because the amount of cycles stalled as clock 
speeds increase.  Some might contend that technology such as multi-thread 
or predicted / delayed loads can mitigate this problem but I have yet to 
see any real data to support such assertions.  In general, designs need to 
avoid using PIO Reads in the main data path period.  I believe most I/O and 
system architects from all companies will agree on this point (I also know 
that many server IHV will also agree).

Mike


At 09:52 AM 1/24/2005, Grant Grundler wrote:
>On Sun, Jan 23, 2005 at 11:06:34AM +0200, Michael S. Tsirkin wrote:
> > But, do they hurt perf on ia64 more than on x86?
>
>It does NOT depend on the CPU architecture.
>It depends on how many bridge chips are between the CPU and IO device
>and which type of bus the MMIO read has to flow through.
>
>For ZX1 it's normally two (unless we add a PCI-PCI Bridge as well).
>Most x86 chipsets only have one unless they use the north/south
>bridge designs. IIRC, x86-64 with a single CPU also has one bridge.
>
>For more discussion on HP ZX1 chips WRT MMIO reads, see:
>         http://iou.parisc-linux.org/ols2002/
>
>hth,
>grant
>_______________________________________________
>openib-general mailing list
>openib-general at openib.org
>http://openib.org/mailman/listinfo/openib-general
>
>To unsubscribe, please visit 
>http://openib.org/mailman/listinfo/openib-general
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/general/attachments/20050124/c5634c70/attachment.html>