[libfabric-users] FI_MR_ENDPOINT question

[Biddiscombe, John A.]

Steve


Thanks very much, that discussion is very helpful (And yes, when I said send/recv - what I really meant was initiating transfers including RMA on the send endpoint, and pre-posting rx buffers on the recv one)


Cheers


JB


________________________________
From: Welch, Steve <welch at hpe.com>
Sent: 01 September 2022 17:05:16
To: Biddiscombe, John A.; libfabric-users at lists.openfabrics.org
Subject: Re: FI_MR_ENDPOINT question

Hi John,

You mention messaging (send/recv) in your question; if FI_MR_LOCAL is not required, then for those operations the registration for local access is transparent to the client and multiple EP within the same domain/VA space could transparently share the underlying DMA mapping.

For RMA (write/read/amo) where remote MR are explicitly registered, bound and enabled, there is a good discussion here:
https://github.com/ofiwg/libfabric/discussions/7321

Steve


From: Libfabric-users <libfabric-users-bounces at lists.openfabrics.org> on behalf of Biddiscombe, John A. <john.biddiscombe at cscs.ch>
Date: Thursday, September 1, 2022 at 4:22 AM
To: libfabric-users at lists.openfabrics.org <libfabric-users at lists.openfabrics.org>
Subject: [libfabric-users] FI_MR_ENDPOINT question

For machines that set the FI_MR_ENDPOINT flag - we must bind memory regions to an endpoint.



If I use a separate endpoint for sending and receiving, or worse, threadlocal endpoints (for sending), or a scalable endpoint (for sending)



Must I only use memory regions bound to the exact endpoint I'm actually using for a send  or receive operation (meaning threadlocal pools of registered memory for a threadlocal endpoint setup and separate pools of memory for send/receive).



How strict are the rules? If I bind memory to endpoint 1, and send using endpoint 2 - will it break?



thanks...



JB
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/libfabric-users/attachments/20220901/2a2faa40/attachment-0001.htm>