<html>
<body>
<font size=3>At 11:55 AM 8/25/2006, Greg Lindahl wrote:<br>
<blockquote type=cite class=cite cite="">On Fri, Aug 25, 2006 at
10:00:50AM -0400, Thomas Bachman wrote:<br><br>
> Not that I have any stance on this issue, but is this is the text in
the<br>
> spec that is being debated? <br>
> <br>
> (page 269, section 9.5, Transaction Ordering):<br>
> "An application shall not depend upon the order of data writes
to<br>
> memory within a message. For example, if an application sets up<br>
> data buffers that overlap, for separate data segments within a<br>
> message, it is not guaranteed that the last sent data will
always<br>
> overwrite the earlier."<br><br>
No. The case we're talking about is different from the example.<br>
There's text elsewhere which says, basically, that you can't access<br>
the data buffer until seeing the completion.</blockquote><br><br>
<blockquote type=cite class=cite cite="">> I'm assuming that the spec
authors had reason for putting this in there, so<br>
> maybe they could provide guidance here?</font></blockquote><br>
We put that text there to accommodate differing memory controller
architectures / coherency protocol capabilities / etc. Basically,
there is no way to guarantee that the memory is in a usable and correct
state until the completion is seen. This was intended to guide
software to not peek at memory but to examine a completion queue entry so
that if memory is updated out of order, silent data corruption would not
occur. <br><br>
<br>
<blockquote type=cite class=cite cite=""><font size=3>I can't speak for
the authors, but as an implementor, this has a huge impact on
implementation.<br><br>
For example, on an architecture where you need to do work such as
flushing the cache before accessing DMAed data, that's done in the
completion. x86 in general is not such an architecture, but they exist.
IB is intended to be portable to any CPU
architecture.</font></blockquote><br>
Invalidation protocol is one concern. The other is the a completion
notification also often acts as a flush of the local I/O fabric as
well. In the case of a RDMA Write, the only way to safely determine
complete delivery was to have a RDMA Write / Send with completion
combination or a RDMA Write / RDMA Read depending upon which side
required such completion knowledge.<br><br>
<br>
<blockquote type=cite class=cite cite=""><font size=3>For iWarp, the
issue is that packets are frequently reordered.</font></blockquote><br>
Neither IP or Ethernet re-order packets that often in practice.
Same is true for packet drop rates (the real issue for packet drop is the
impact on performance and recovery times which is why IB was not designed
to work over long or diverse topologies where intermediate elements may
see what might be termed a high packet loss rate).<br><br>
Mike<br><br>
<br>
<blockquote type=cite class=cite cite=""><font size=3>-- greg<br><br>
<br>
_______________________________________________<br>
openib-general mailing list<br>
openib-general@openib.org<br>
<a href="http://openib.org/mailman/listinfo/openib-general" eudora="autourl">
http://openib.org/mailman/listinfo/openib-general</a><br><br>
To unsubscribe, please visit
<a href="http://openib.org/mailman/listinfo/openib-general" eudora="autourl">
http://openib.org/mailman/listinfo/openib-general</a>
</font></blockquote></body>
</html>