[ofa-general] synchronize commands issued to MTHCA

[Yicheng Jia]

> I wouldn't think so, although I don't have full details of how your
> hardware behaves to know for sure.  I assume your PCI bus/memory
> controller is already smart enough to deal with HCR writes being
> interleaved with writes to a doorbell page from userspace, so it seems
> that writes to locally attached memory should be OK too, as long as
> the HCR writes are word-sized in the right order etc.

For the problem I've seen, most probably the HCR writes mess up with 
doorbell register rings. Is it possible? The FW version I'm using is 1.1.0 
without debug trace function. This problem is really hard to debug since 
it's real time and does not occur very oftem, and it's hard to hook up a 
PCIe bus analysis either since by the time the error happens, the PCIe 
transaction has been already done. All I get from the HCA is reporting bad 
parameter error. Is there any way to get more info from the HCA?

Thanks!
Yicheng




Roland Dreier <rdreier at cisco.com> 
01/02/2008 12:13 PM

To
Jack Morgenstein <jackm at dev.mellanox.co.il>
cc
general at lists.openfabrics.org, Yicheng Jia <YJia at tmriusa.com>
Subject
Re: [ofa-general] synchronize commands issued to MTHCA






 > Roland, do you think that the memcpy_toio() call might mess things up?

I wouldn't think so, although I don't have full details of how your
hardware behaves to know for sure.  I assume your PCI bus/memory
controller is already smart enough to deal with HCR writes being
interleaved with writes to a doorbell page from userspace, so it seems
that writes to locally attached memory should be OK too, as long as
the HCR writes are word-sized in the right order etc.

 > Maybe we need "wmb()" or "mmiowb()" here as well?

I don't see any reason, although I often miss things.  It seems that
the only thing that cares about the writes of the address info being
done would be posting a send WQE that uses it, and that should already
have sufficient ordering.  What would we be ordering things against?

 - R.

_____________________________________________________________________________
Scanned by IBM Email Security Management Services powered by MessageLabs. 
For more information please visit http://www.ers.ibm.com
_____________________________________________________________________________

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/general/attachments/20080102/f9176203/attachment.html>