[ofiwg] selecting a fabric for a destination

[Reese Faucette (rfaucett)]

If a program needs to send to only one remote address, selecting a fabric is simple: the program just passes the remote address to fi_getinfo and is returned an ordered list of fabrics that can communicate with the remote address.

Suppose, though, that you have an application that may need to send data to a large set of remote destinations that may only become known "later".  I see the app calling fi_getinfo() at startup to learn what suitable fabrics are available, and opening a domain and set of endpoints on each of them.

Now, later, through some out-of-band mechanism, the process learns of remote destinations it should contact.  The problem the app now faces is, "which endpoint do I use?" since not all fabrics can necessarily reach the remote address.  Calling fi_getinfo() with this newly learned address is somewhat clumsy, since fi_getinfo will malloc a whole new list of fi_infos to return, which then must be mapped by the application to actual fabric/domain/endpoint in use.

Any suggestions for how the app should learn which fabric is the best to use to reach this remote destination?  One possibility is that the program can pass in a list of fabrics and a remote destination to "fi_fabric_for_dest(fi_list, remote_address)" and be returned a pointer to the "best" one to use, "best" defined by the providers.

Another possibility is for the program itself to loop through the fabrics it knows about calling "fi_get_distance_metric(domain, remote_address)" and pick the one with the lowest (provider-defined within guidelines) metric.

The mechanism sockets uses is, IMO, unacceptable here - creating one "fabric" for all interfaces and letting the kernel (or some agent) pick the right one at send time.  Bypassing all this critical path lookup is much of the reason we want OS-bypass in the first place.

Thoughts, suggestions, flames?
-reese

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/ofiwg/attachments/20141008/bd66d39d/attachment.html>