[ofiwg] c++ Fabric - from ofiwg discussion from today

[Hefty, Sean]

> // template specialization of a libfabric fabric - a libfabric
> specific implementation of a 'fabric'
> 
> template<>
> struct fabric<libfabric> {
>     using impl = fabric_impl;
> 
>     // current thought about where domains should live
>     std::vector<fi_domain *> domains;
> 
>     // member variables
>     ...
> 
>     // methods enumerated
>     ...
> };

I don't know if this is right or wrong.  It at least seems reasonable.  But I'm still not clear how the passive endpoints fit into things.

> Current plan is to build wrapper template specialized structures
> around member variables found in this structure. After re-reviewing
> the libfabric notion of an fi_domain, it makes sense to house
> fi_domains in the fabric. That said, there might be an opportunity,
> at this point, to seek parity with data structures found in asio.

IMO, I think it would help the ofiwg community to talk about classes and their member functions, rather than templates.  That is, focus on the things an application would use.

What classes are being defined, and how could those map to OFI objects?  This would let us gage the efficiency of the mappings.

> asio has this notion of an 'asio::io_context'. It might make sense
> to create a wrapper type around fi_domains called a
> 'fabric::io_context'. In asio, the OS provides these
> 'asio::io_context' structures; in the case of libfabric, one might
> be able to consider the 'fabric' as a 'fabric::io_context'
> provider/maintainer.

Based on quick searches, it looks to me that io_context is equivalent or replaces io_service.  Is that correct?  That is, it is associated with a threading or progress domain, rather than a hardware construct.

> Users would ask the fabric to create fabric::io_contexts that could
> then be used to perform fabric operations. These fabric::io_contexts
> would be light wrappers around the existing libfabric data
> structures (ie: fi_domains). The io_context could then house
> domains, queues, memory registration, etc.

Assuming what I said above is true, I'm not sure we can easily map io_context to fid_domain and still keep the ASIO endpoint model.  We need fid_ep associated with a single fid_domain.

> Example of a client/server application that emphasizes
> asio::io_contexts:
> 
> https://www.boost.org/doc/libs/1_67_0/doc/html/boost_asio/example/cp
> p11/echo/async_tcp_echo_server.cpp

Every time I look at these examples I think python and java have strong futures.  ;)

- Sean