[libfabric-users] fi_sockets remote CQ data ordering

[Ezra Kissel]

Two nodes, one EP each, bi-directional interleaved fi_writedata and 
fi_writes.  Sending and polling are concurrent but sending is serialized 
to the EP. I should mention I'm currently seeing the behavior on some 
rather slow arch ARMv7l boards, will have to see if I can force it on 
more capable x86_64 systems.

On 2/23/2016 4:02 PM, Jose, Jithin wrote:
> Hi Ezra,
>
> Does multiple senders imply multiple Eps or multiple threads?
>
> If TX operations are posted to the same EP, then the completions should be in order. It might be a bug if the completions are out-of-order. It would be great if we could get a reproducer for this.
>
> Thanks,
> - Jithin
>
>
>
>
>
>
> -----Original Message-----
> From: <libfabric-users-bounces at lists.openfabrics.org> on behalf of Ezra Kissel <ezkissel at indiana.edu>
> Date: Tuesday, February 23, 2016 at 12:43 PM
> To: "libfabric-users at lists.openfabrics.org" <libfabric-users at lists.openfabrics.org>
> Subject: [libfabric-users] fi_sockets remote CQ data ordering
>
>> Hi,
>>
>> I'm running into an issue where I occasionally see out-of-order
>> FI_REMOTE_CQ_DATA being popped from a CQ using sockets FI_RMA.
>> FI_ORDER_STRICT is shown in fi_getinfo() and I'm under the
>> max_order_waw_size, although I'm not sure that latter limit has an
>> impact on the completion ordering.  I've verified that my senders are
>> writing immediate data in the order expected.
>>
>> Anything I might be missing?  I can work on a fi-only reproducer but I
>> wanted to make sure I understood the completion ordering conditions first.
>>
>> thanks,
>> - ezra
>> _______________________________________________
>> Libfabric-users mailing list
>> Libfabric-users at lists.openfabrics.org
>> http://lists.openfabrics.org/mailman/listinfo/libfabric-users