[ofiwg] libfabric releases, stable branches, and DL providers
Hefty, Sean
sean.hefty at intel.com
Tue Aug 11 10:03:11 PDT 2020
A possible future step is to create what I'll call a core+ provider. This would be a single provider that contains both core + utility provider functionality, merged into a single provider.
Otherwise, we could end up debugging 3 or more components of different versions (e.g. libfabric, verbs, rxm, rxd), rather than just 2.
- Sean
> One potential issue: If a change in a provider requires a change in another provider
> (core and utility). I'm making an assumption that utility providers would also follow a
> similar model. It could be difficult to synchronize, but probably not terribly so.
> Issues like this create a dependency that is difficult to track when providers are
> released independently.
>
> --
> James Swaro
> P: +1 (651) 605-9000
>
> On 8/3/20, 2:50 PM, "ofiwg on behalf of Hefty, Sean" <ofiwg-
> bounces at lists.openfabrics.org on behalf of sean.hefty at intel.com> wrote:
>
> I have the following proposal for creating a provider release package for Linux.
> This should allow mostly independent provider and libfabric releases, and ease provider
> backporting.
>
>
> Detailed steps:
> 1. All code needed by the provider needs to be available under libfabric/prov/X.
> 1a.Add symbolic links for 'common_srcs':
>
> libfabric/prov/X/prov/util -> libfabric/prov/util
> libfabric/prov/X/include -> libfabric/include
> libfabric/prov/X/src/shared -> libfabric/src/shared
> additional links for libfabric/src/ files (common.c, enosys.c, mem.c, etc.)
>
> (Symbolic links are not available on all platforms, which is why
> they are not currently used.)
>
> 2. Add Makefile.am and configure.ac files for libfabric/prov/X.
> Each provider would manage their own build files.
>
> 3. Update the provider's getinfo call to handle ABI structure changes
> e.g. see src/abi_1_0.c
>
> 4. Report API version based on libfabric target version.
> Determined during configure or make process
>
>
> Restructuring the code and build files would help make this more manageable, but
> isn't required. E.g. moving shared code from src -> src/shared, adding a
> Makefile.include for prov/util, etc.
>
> The provider would build against their own copy of the headers to support multiple
> libfabric versions. API compatibility code is already built into the providers, mostly
> through prov/util. Today, the libfabric core handles ABI conversions, so that the
> provider's getinfo call only sees the latest version of the structures. Step 3
> requires DL providers to handle this translation as well.
>
> The above changes could integrate into master. Each provider could also create
> their own stable release branch(es) to manage as desired. One possible work flow would
> be:
>
>
> 1. Align the provider branch with the latest major libfabric release.
> 2. Release a new provider package to support older libfabric releases.
> 3. Cherry pick patches from master to provider branch as needed.
> 4. Release provider update packages as needed.
> 5. Restart at step 1 each major libfabric release.
>
>
> This sort of flow allows the provider to focus on mostly linear development model.
> There would be no need to backport fixes to a provider to multiple stable branches. A
> single provider release would be able to support all previous libfabric versions.
> Rebasing provider release branches to align with the latest libfabric release would
> ensure that only released APIs are ever used.
>
> There's roughly a 40% chance that I'm missing some critical item that makes this
> completely fall apart.
>
> - Sean
> _______________________________________________
> ofiwg mailing list
> ofiwg at lists.openfabrics.org
> https://urldefense.proofpoint.com/v2/url?u=https-
> 3A__lists.openfabrics.org_mailman_listinfo_ofiwg&d=DwIGaQ&c=C5b8zRQO1miGmBeVZ2LFWg&r=TG
> nc32XdTk02d6uCEdYjAGlqvSwDHPR0LsoJozLVKjE&m=Zpu9Uj3jefbVB3Ghqo8hN2xpLtwbbrjnsNHt1Yzf2js
> &s=PQ9fhhKDoTA9WZbUtj_ZSeO0k4iSny9tTi6H_HB2PpA&e=
>
>
More information about the ofiwg
mailing list