[ofiwg] [RFC] libfabric provider interoperability

[Patrick MacArthur]

Hi, all,

Based on the last two meetings and having given the topic of
interoperability among providers some thought, I thought I would share a
proposal for encouraging interoperability among providers that implement
the same wire protocol.

I think a decent solution to the interoperability problem would be as
follows: providers which claim to implement the same *existing* wire
protocol must interoperate for all calls that are *natively* supported
by the wire protocol.  In fi_endpoint(3) man page under the description
of a wire protocol, we should list what the natively supported features
are considered to be to avoid ambiguity.  The two emphasized words add
meaning that may not have been explicitly mentioned during the last
OFIWG call.

So, as an example, for InfiniBand RC, this would require that two
providers implement the three atomic operations that are defined in the
InfiniBand specification using the wire protocol's native support, but
for all of the other atomics that libfabric provides, all bets are
off---they might work within a single provider but not between providers.

If two vendors wished to implement the rest of the atomics in InfiniBand
in an interoperable way, then they would declare a new extended wire
protocol as we discussed in the call.  The providers would still be
responsible for interoperability of the calls that are natively
supported by InfiniBand.  This would require that we define a way to
state that a protocol is merely an extension of an existing wire
protocol.  Perhaps we could have some place in libfabric or on the
website to document these "agreements between friends" wire protocols.
And/or these ultimately may be taken to, e.g., the IBTA, if it was
decided that a real standard solution was needed.

As for the "enforcement" problem, we could make the above
interoperability requirement for all in-tree providers, perhaps
leveraging the IOL cluster and the logo program to verify
interoperability prior to major releases.  I don't have a good solution
to enforce this for out-of-tree providers.

Thoughts?

Thanks,
Patrick

-- 
Patrick MacArthur <pmacarth at iol.unh.edu>
Research and Development, High Performance Networking and Storage
UNH InterOperability Laboratory