[openib-general] Race in mthca_cmd_post()

[Jason Gunthorpe]

On Fri, Oct 20, 2006 at 02:14:43PM -0700, Roland Dreier wrote:
> I just saw "the master cannot proceed" but the first few times I read
> this, I didn't see the "(if a dependency exists)."  Since no

Yes, the languange in PCI-X is actually a bit clearer (pg 80):

``As in conventional PCI, if a requester requires one non-posted
  transaction to complete before another, it must not initiate the
  second transcation until the first one completes.''

Where completion should be understood to mean that the split
completion is observed by the requestor.

I just took a quick look at asm-ia64/io.h and there is __ia64_mf_a
barriers after all non-posted IO operations (ib/outb). config write and
config read transcations have identical rules to IO transactions at
the PCI bus level.

I'm going to go out on a limb here and say that if Linux code assumes
strong ordering of IO operations then it makes sense to also assume
strong ordering on config writes. So, instead of patching mthca with
this barrier it should go in the Altix config access mechanism..

>From io.h:
/*
 * For the in/out routines, we need to do "mf.a" _after_ doing the I/O access to ensure
 * that the access has completed before executing other I/O accesses.  Since we're doing
 * the accesses through an uncachable (UC) translation, the CPU will execute them in
 * program order.  However, we still need to tell the compiler not to shuffle them around
 * during optimization, which is why we use "volatile" pointers.
 */

Does that make sense?

Jason