[libfabric-users] fi_read questions

[Arne]

Am 20.10.20 um 20:57 schrieb Hefty, Sean:
>> First of all I think I have to explain something. In the program I build
>> one connection for message based communication and another for rma based
>> communication. Basically to be able to use different Provider for each
>> communication type later (both allocate their own fabric, domain, eq and
>> cqs, so no complications there).
> This can be fine.  But there's no ordering guarantees between different endpoints.  So if you issue an RMA on one, then send a message on another which indicates that the RMA has been transferred, it's possible for the message to arrive before the RMA has completed.


I think it is no problem in my case. Due to the the amount of keys the 
program knows how many rma-reads will be done. Sending the completion 
message for the other side will be done until |keys| will have been 
generated.

But thanks for the warning.


> The sockets provider starts with implementing reliable, unconnected communication (RDM endpoints).  It layers other endpoint types, including MSG endpoints, over that.  It's goofy.  But most of the original target applications wanted RDM endpoint semantics.
>
> The tcp provider implements only MSG endpoints over tcp sockets, which is the natural mapping.
>
> If you can switch to using tcp, that would be preferred.  But either provider should work.  The use of MSG endpoints with the tcp provider is just more widely tested than the MSG endpoint support from the socket provider.

So you have no idea either why my setup is stuck at the fi_read call 
with ENOENT when using the second (rma) endpoint?

I just double checked: The only differences between both endpoints are 
the capabilities and mode bits.

>> Long story short: Not everything is fine (there seems to be a problem
>> with transmited data), but most of the data transfer seems to work.

That is not your fault, but mine. A backend DB-entry was not cleaned 
properly.


Greetings, Arne.