[openib-general] Race in mthca_cmd_post()

[Roland Dreier]

 > The posted/non-posted write stuff in the spec really only means that a
 > split completion is generated for that transaction on the bus. There
 > is no bus-level requirement that the bus halt while an outstanding
 > split is pending. In fact, the PCI-X ordering rules in this case
 > actually would allow your config read and memory read to be re-ordered
 > by a the bridge (table 8-3). ``Split requests are permitted to be
 > blocked by or pass other split requests.''
 > 
 > Most implementations block the CPU on a non-posted write which
 > provides the necessary serialization, but Altix clearly didn't..

Thanks, I think I get it now.  It does seem like this behavior skirts
right inside the boundary of what the PCI spec allows.  What was
confusing me was the section:

    Non-posted transactions reach their ultimate destination before
    completing at the originating device. The master cannot proceed
    with any other work until the transaction has completed at the
    ultimate destination (if a dependency exists).

I just saw "the master cannot proceed" but the first few times I read
this, I didn't see the "(if a dependency exists)."  Since no
dependency exists between the config write and the subsequent memory
read, the master _can_ proceed in this case.  So I do agree that this
patch looks correct and needed, although the Altix behavior is
somewhat unusual.

Thanks,
  Roland