[ofiwg] [libfabric-users] feature requests

[Hefty, Sean]

> "Domains usually map to a specific local network interface adapter. A
> domain may either refer to the entire NIC, a port on a multi-port NIC,
> or a virtual device exposed by a NIC. From the viewpoint of the
> application, a domain identifies a set of resources that may be used
> together." (https://github.com/ofiwg/ofi-
> guide/blob/master/OFIGuide.md)
> 
> From this, MPI libraries and the like would then need to support
> multiple domains.

The intent is that OFI could support multi-rail both above and below the interface, depending on which one the application wants.  We didn't want to restrict what the provider could do, as long as it met the API definition.

I do have concerns that implementing multi-rail below OFI has the *potential* of negatively impacting performance when multi-rail is not available or enabled.  So care is needed to avoid this.  This seems more likely to be a problem in devices that support full transport offload.

- Sean