[libfabric-users] no perf effect from FI_MORE with verbs; ofi_rxm provider?

[Titus, Greg]

> ... the initiator for my
> fi_writemsg() calls seems to be concluding that all of them have arrived at the target
> before that's actually happened, ...

This could be a bug in the provider implementing DELIVERY_COMPLETE correctly.  Note that delivery complete isn't supported natively by most hardware, and requires software based acks in order to meet that guarantee.

Are you using RDM endpoints?

Yes, this is over RDM endpoints.  I've learned from a co-worker that while ofi_rxm;verbs cannot provide FI_DELIVERY COMPLETE semantics for RDMA, if you put that in the hints there is a bug (in ofi_rxm) that allows that provider to appear in the list returned by fi_getinfo() nevertheless, with the unfortunate result that you silently get FI_TRANSMIT_COMPLETE semantics instead.  That's what's happening to me.  Even with this fixed, though, FI_DELIVERY_COMPLETE won't be supported by the ofi_rxm;verbs provider.  There are workarounds, which I'm starting on.

greg
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/libfabric-users/attachments/20200129/24a0806d/attachment.htm>