[ofiwg] provider specific definitions

[Byrne, John (Labs)]

Sean,

For cases when a provider is defining an ops vector for fi_open(), I think option (1) is clearly the way to go.

The discussion in PR 7054 is interesting.  I think you are very close to having the right answer in your response:

--- a/include/rdma/fi_endpoint.h
+++ b/include/rdma/fi_endpoint.h
@@ -66,6 +66,7 @@ enum {
        FI_OPT_RECV_BUF_SIZE,
        FI_OPT_TX_SIZE,
        FI_OPT_RX_SIZE,
+       FI_OPT_EFA_RNR_RETRY = (1U | FI_PROV_SPECIFIC),
 };

I think in fi_endpoint.h we should  define a FI_OPT_PROV_SPECIFIC with a comment showing  a generic example, but directing that the actual definition be added to fi_ext.h or a provider-specific fi_ext_xxxx.h file. (If they are going to have a provider-specific file for ops vectors, then I'd think you'd want all the definitions in one place, but if it was just the enum, that could be in fi_ext.h)   You'd want similar additions to other enums, but that could be done on an as-needed basis.

- John Byrne

-----Original Message-----
From: ofiwg <ofiwg-bounces at lists.openfabrics.org> On Behalf Of Hefty, Sean
Sent: Tuesday, September 14, 2021 9:11 AM
To: ofiwg at lists.openfabrics.org
Subject: [ofiwg] provider specific definitions

This email is in response to a PR which adds an EFA specific extension:

https://github.com/ofiwg/libfabric/pull/7054

My question to the group is, what policy should we use for exporting provider specific definitions to applications? 

Option 1 is a provider specific header file.  For example:

https://github.com/ofiwg/libfabric/blob/main/prov/usnic/src/fi_ext_usnic.h

Given the contents, this option makes sense for usnic.

However, there are cases (e.g. PR 7054), where the extensions are minimal.  Given that provider extensions have been rare, I propose using fi_ext.h.  This would make it easier on the apps, and provide better visibility in the definitions to avoid unnecessary overlap between providers (to help avoid application coding errors).

A third alternative is for very simple extensions to be defined in one of the main header files (e.g. fi_endpoint.h).  This option would work for PR 7054.

- Sean
_______________________________________________
ofiwg mailing list
ofiwg at lists.openfabrics.org
https://lists.openfabrics.org/mailman/listinfo/ofiwg