From sean.hefty at intel.com  Tue Sep 14 09:11:14 2021
From: sean.hefty at intel.com (Hefty, Sean)
Date: Tue, 14 Sep 2021 16:11:14 +0000
Subject: [ofiwg] provider specific definitions
Message-ID: <DM6PR11MB4609D6EE0715FF71BD31FCC19EDA9@DM6PR11MB4609.namprd11.prod.outlook.com>

This email is in response to a PR which adds an EFA specific extension:

https://github.com/ofiwg/libfabric/pull/7054

My question to the group is, what policy should we use for exporting provider specific definitions to applications? 

Option 1 is a provider specific header file.  For example:

https://github.com/ofiwg/libfabric/blob/main/prov/usnic/src/fi_ext_usnic.h

Given the contents, this option makes sense for usnic.

However, there are cases (e.g. PR 7054), where the extensions are minimal.  Given that provider extensions have been rare, I propose using fi_ext.h.  This would make it easier on the apps, and provide better visibility in the definitions to avoid unnecessary overlap between providers (to help avoid application coding errors).

A third alternative is for very simple extensions to be defined in one of the main header files (e.g. fi_endpoint.h).  This option would work for PR 7054.

- Sean

From john.l.byrne at hpe.com  Tue Sep 14 11:27:55 2021
From: john.l.byrne at hpe.com (Byrne, John (Labs))
Date: Tue, 14 Sep 2021 18:27:55 +0000
Subject: [ofiwg] provider specific definitions
In-Reply-To: <DM6PR11MB4609D6EE0715FF71BD31FCC19EDA9@DM6PR11MB4609.namprd11.prod.outlook.com>
References: <DM6PR11MB4609D6EE0715FF71BD31FCC19EDA9@DM6PR11MB4609.namprd11.prod.outlook.com>
Message-ID: <TU4PR8401MB07337C311CC52056EE0818F8C5DA9@TU4PR8401MB0733.NAMPRD84.PROD.OUTLOOK.COM>

Sean,

For cases when a provider is defining an ops vector for fi_open(), I think option (1) is clearly the way to go.

The discussion in PR 7054 is interesting.  I think you are very close to having the right answer in your response:

--- a/include/rdma/fi_endpoint.h
+++ b/include/rdma/fi_endpoint.h
@@ -66,6 +66,7 @@ enum {
        FI_OPT_RECV_BUF_SIZE,
        FI_OPT_TX_SIZE,
        FI_OPT_RX_SIZE,
+       FI_OPT_EFA_RNR_RETRY = (1U | FI_PROV_SPECIFIC),
 };

I think in fi_endpoint.h we should  define a FI_OPT_PROV_SPECIFIC with a comment showing  a generic example, but directing that the actual definition be added to fi_ext.h or a provider-specific fi_ext_xxxx.h file. (If they are going to have a provider-specific file for ops vectors, then I'd think you'd want all the definitions in one place, but if it was just the enum, that could be in fi_ext.h)   You'd want similar additions to other enums, but that could be done on an as-needed basis.

- John Byrne

-----Original Message-----
From: ofiwg <ofiwg-bounces at lists.openfabrics.org> On Behalf Of Hefty, Sean
Sent: Tuesday, September 14, 2021 9:11 AM
To: ofiwg at lists.openfabrics.org
Subject: [ofiwg] provider specific definitions

This email is in response to a PR which adds an EFA specific extension:

https://github.com/ofiwg/libfabric/pull/7054

My question to the group is, what policy should we use for exporting provider specific definitions to applications? 

Option 1 is a provider specific header file.  For example:

https://github.com/ofiwg/libfabric/blob/main/prov/usnic/src/fi_ext_usnic.h

Given the contents, this option makes sense for usnic.

However, there are cases (e.g. PR 7054), where the extensions are minimal.  Given that provider extensions have been rare, I propose using fi_ext.h.  This would make it easier on the apps, and provide better visibility in the definitions to avoid unnecessary overlap between providers (to help avoid application coding errors).

A third alternative is for very simple extensions to be defined in one of the main header files (e.g. fi_endpoint.h).  This option would work for PR 7054.

- Sean
_______________________________________________
ofiwg mailing list
ofiwg at lists.openfabrics.org
https://lists.openfabrics.org/mailman/listinfo/ofiwg 

From sean.hefty at intel.com  Wed Sep 29 10:52:01 2021
From: sean.hefty at intel.com (Hefty, Sean)
Date: Wed, 29 Sep 2021 17:52:01 +0000
Subject: [ofiwg] libfabric v1.13.2 release request
Message-ID: <DM6PR11MB46090992CAFD152F296DBFA89EA99@DM6PR11MB4609.namprd11.prod.outlook.com>

I have received a request for a v1.13.2 release to pick up a handful of fixes added since the v1.13.1 release.  Maintainers, if there are fixes in master that you would like to include, please cherry-pick them this week, if possible.  I'd like to target rc1 for early next week.  I believe there are a couple of fixes still in progress that would be nice to include.

- Sean