[ofa-general] synchronize commands issued to MTHCA

[Yicheng Jia]

> The mmiowb() is definitely necessary, because without it then commands
> were getting messed up on large Altix systems.

I'm using Duo-core Xeon and I just grep the source of "mmiowb()" in kernel 
2.6.23 include/asm-x86_64 /io.h and found that this function does nothing 
on x86_64 platform, is it true?

Thanks!
Yicheng




Roland Dreier <rdreier at cisco.com> 
01/02/2008 02:52 PM

To
Yicheng Jia <YJia at tmriusa.com>
cc
general at lists.openfabrics.org, Jack Morgenstein <jackm at dev.mellanox.co.il>
Subject
Re: [ofa-general] synchronize commands issued to MTHCA






 > Could you tell me what's the difference between "wmb()" and "mmiowb()". 
I 
 > notice that ofa-1.3 has added "mmiowb()" at the end of mthca_cmd_post, 
 > since "wmb()" is already called at the end of cmd_post, is "mmiowb()" 
 > really necessary?

wmb() orders writes from the same CPU -- it prevents highly
out-of-order architectures from making writes visible in an order
different from program order.  mmiowb() orders MMIO writes between
different CPUs, and prevents systems (such as SGI Altix) where the CPU
fabric may reorder writes before they reach the IO bus.

The mmiowb() is definitely necessary, because without it then commands
were getting messed up on large Altix systems.

 - R.

_____________________________________________________________________________
Scanned by IBM Email Security Management Services powered by MessageLabs. 
For more information please visit http://www.ers.ibm.com
_____________________________________________________________________________

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/general/attachments/20080104/4e4210b8/attachment.html>