[openib-general] basic IB doubt

[Caitlin Bestler]

openib-general-bounces at openib.org wrote:
> Quoting r. john t <johnt1johnt2 at gmail.com>:
>> Subject: basic IB doubt
>> 
>> Hi
>> 
>> I have a very basic doubt. Suppose Host A is doing RDMA write (say 8
>> MB) to Host B. When data is copied into Host B's local
> buffer, is it guaranteed that data will be copied starting
> from the first location (first buffer address) to the last
> location (last buffer address)? or it could be in any order?
> 
> Once B gets a completion (e.g. of a subsequent send), data in
> its buffer matches that of A, byte for byte.

An excellent and concise answer. That is exactly what the application
should rely upon, and nothing else. With iWARP this is very explicit,
because portions of the message not only MAY be placed out of 
order, they SHOULD be when packets have been re-ordered by the
network. But for *any* RDMA adapter there is no guarantee on
what order the adapter flushes things to host memory or particularly
when old contents that may be cached are invalidated or updated.
The role of the completion is to limit the frequency with which
the RDMA adapter MUST guarantee coherency with application visible
buffers. The completion not only indicates that the entire message
was received, but that it has been entirely delivered to host memory.