[libfabric-users] FI_MR_ENDPOINT question

[Welch, Steve]

Hi John,

You mention messaging (send/recv) in your question; if FI_MR_LOCAL is not required, then for those operations the registration for local access is transparent to the client and multiple EP within the same domain/VA space could transparently share the underlying DMA mapping.

For RMA (write/read/amo) where remote MR are explicitly registered, bound and enabled, there is a good discussion here:
https://github.com/ofiwg/libfabric/discussions/7321

Steve


From: Libfabric-users <libfabric-users-bounces at lists.openfabrics.org> on behalf of Biddiscombe, John A. <john.biddiscombe at cscs.ch>
Date: Thursday, September 1, 2022 at 4:22 AM
To: libfabric-users at lists.openfabrics.org <libfabric-users at lists.openfabrics.org>
Subject: [libfabric-users] FI_MR_ENDPOINT question

For machines that set the FI_MR_ENDPOINT flag - we must bind memory regions to an endpoint.



If I use a separate endpoint for sending and receiving, or worse, threadlocal endpoints (for sending), or a scalable endpoint (for sending)



Must I only use memory regions bound to the exact endpoint I'm actually using for a send  or receive operation (meaning threadlocal pools of registered memory for a threadlocal endpoint setup and separate pools of memory for send/receive).



How strict are the rules? If I bind memory to endpoint 1, and send using endpoint 2 - will it break?



thanks...



JB
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/libfabric-users/attachments/20220901/36fea3cc/attachment.htm>