[ofiwg] input on intra-node implementation

[Hefty, Sean]

I want to provide an intra-node communication (i.e. loopback) utility to libfabric.  The loopback utility could be part of a stand-alone provider, or incorporated into other providers.  For this, I'm looking at selecting a single, easily maintained implementation.  These are my choices so far:

1. Control messages transfer over shared memory
   Data transfers use shared memory bounce buffers
2. Control messages transfer over shared memory
   Data transfers occur over CMA
   (small transfers go with control messages)
3. Use XPMEM in some TBD way

Some of these options are only available on Linux.  Does the portability of this solution matter?  FreeBSD and Solaris would fall back to using the network loopback device.

How much concern needs to be given to security?  Should the loopback utility enforce RMA registrations?  Do we require processes to share a certain level of access, such as ptrace capability?

I think we need input on this not just from the MPI community, but other users as well.

- Sean