[ofiwg] c++ Fabric - from ofiwg discussion from today

[Taylor, Christopher P CIV (US)]

Sean, Jeff, interested parties,

Hopefully, this email clarifies what was mentioned on the phone call yesterday. Currently, a *very* *early* and *rough* version of a 'fabric<libfabric>' looks something like this:

// base template case - allows fabric libraries the ability to specialize against the struct fabric interface

template<typename fabric_impl>
struct fabric {
    using impl = fabric_impl;

    // methods enumerated
    ...
};

// template specialization of a libfabric fabric - a libfabric specific implementation of a 'fabric'

template<>
struct fabric<libfabric> {
    using impl = fabric_impl;

    // current thought about where domains should live
    std::vector<fi_domain *> domains;

    // member variables
    ...

    // methods enumerated
    ...
};

Current plan is to build wrapper template specialized structures around member variables found in this structure. After re-reviewing the libfabric notion of an fi_domain, it makes sense to house fi_domains in the fabric. That said, there might be an opportunity, at this point, to seek parity with data structures found in asio.

asio has this notion of an 'asio::io_context'. It might make sense to create a wrapper type around fi_domains called a 'fabric::io_context'. In asio, the OS provides these 'asio::io_context' structures; in the case of libfabric, one might be able to consider the 'fabric' as a 'fabric::io_context' provider/maintainer.

Users would ask the fabric to create fabric::io_contexts that could then be used to perform fabric operations. These fabric::io_contexts would be light wrappers around the existing libfabric data structures (ie: fi_domains). The io_context could then house domains, queues, memory registration, etc.

Example of a client/server application that emphasizes asio::io_contexts:

https://www.boost.org/doc/libs/1_67_0/doc/html/boost_asio/example/cpp11/echo/async_tcp_echo_server.cpp

Chris

________________________________________
From: Hefty, Sean [sean.hefty at intel.com]
Sent: Tuesday, May 22, 2018 10:55 PM
To: Jeff Squyres (jsquyres)
Cc: OFIWG Mailing list; Taylor, Christopher P CIV (US)
Subject: [Non-DoD Source] RE: [ofiwg] c++ Fabric - from ofiwg discussion from today

> If I'm understanding this correctly, you're saying "class Fabric" is
> the C++ equivalent of "struct fid_domain" -- is that right?

That is my initial understanding; however, it may be closer to some merger of fid_fabric and fid_domain.  Chris called out that the names are subject to change.

> If so, that is going to be tremendously confusing.

I agree, but fixing the name isn't too hard at this point.  Maybe we call it class FabricDomain.  Anyway, I'm more concerned at the moment with figuring out if C++ needs separate Fabric and Domain classes, or if we only deal with fid_domains.  And if we go with the latter what happens with the fid_pep.

I don't quite understand how ASIO deals with connection establishment yet.

- Sean