[libfabric-users] Multiple ranks/instances on the same node

[Biddiscombe, John A.]

Thanks Sean, I guess assumed that I'd have to handle the case myself, however it shouldn't be too hard to enable a shared memory provider, then use a dedicated endpoint for ranks known to be on the same node. After quickly looking again at the docs, I see that I missed GNI_XPMEM_ENABLE for the GNI backend, so I'll play with that and see if it makes a difference.


Cheers


JB

________________________________
From: Hefty, Sean <sean.hefty at intel.com>
Sent: 10 September 2020 19:22:06
To: Biddiscombe, John A.; libfabric-users at lists.openfabrics.org
Subject: RE: Multiple ranks/instances on the same node

> If I have multiple processes (ranks) on the same node, and they send messages to each
> other. Does libfabric auto-magically do the right thing and use some kind of shared
> memory for them. If not - can this be enabled by using one of the shared memory
> providers (in conjunction with the gni provider for example) - and if this is the case,
> does one need to use a special endpoint to communicate within node - or will that be
> handled automatically.

The answer is provider specific.  For example, psm/psm2 will use shared memory.  Other providers will not. (I'm not sure about gni).  A generic solution is unlikely.  Shared memory support must be integrated on a provider by provider basis to support proper tag matching semantics.  Ideally, at least rxm and rxd would integrate shared memory support.  There's just a developer resource shortage to make that happen.

- Sean
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/libfabric-users/attachments/20200910/206246ea/attachment.htm>