[libfabric-users] Multiple ranks/instances on the same node
Biddiscombe, John A.
biddisco at cscs.ch
Thu Sep 10 14:36:12 PDT 2020
Thanks Sean, I guess assumed that I'd have to handle the case myself, however it shouldn't be too hard to enable a shared memory provider, then use a dedicated endpoint for ranks known to be on the same node. After quickly looking again at the docs, I see that I missed GNI_XPMEM_ENABLE for the GNI backend, so I'll play with that and see if it makes a difference.
From: Hefty, Sean <sean.hefty at intel.com>
Sent: 10 September 2020 19:22:06
To: Biddiscombe, John A.; libfabric-users at lists.openfabrics.org
Subject: RE: Multiple ranks/instances on the same node
> If I have multiple processes (ranks) on the same node, and they send messages to each
> other. Does libfabric auto-magically do the right thing and use some kind of shared
> memory for them. If not - can this be enabled by using one of the shared memory
> providers (in conjunction with the gni provider for example) - and if this is the case,
> does one need to use a special endpoint to communicate within node - or will that be
> handled automatically.
The answer is provider specific. For example, psm/psm2 will use shared memory. Other providers will not. (I'm not sure about gni). A generic solution is unlikely. Shared memory support must be integrated on a provider by provider basis to support proper tag matching semantics. Ideally, at least rxm and rxd would integrate shared memory support. There's just a developer resource shortage to make that happen.
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Libfabric-users