[ofiwg] thoughts on initializing libfabric structs

[Hefty, Sean]

> That sounds nice and simple, but you need to consider the effects of
> shared libraries:
> 
> main program :
>   res = fi_getinfo(API_VER = 2)
>   some_solib(res)
> 
> some_solib():
>   struct fabric_foo_rel1 = {}
>   fi_something(res,&rel1);
> 
> Oops, now it blows up, because main program and same_solib are not
> using the same libfabric ABI revision.

This seems solvable if the app conveys its FI version to the shared library.

> Look closely at everything, and decided if it *really* needs a
> structure, if the structure can be on-stack and symbol versioning will
> do the job, if the path is low speed and an approach like pthread_attr
> is more appropriate.

I think structures can be divided into 1 of 3 categories.

- Attributes: allocated by the app (likely on the stack) and used as input/output to allocate and configure objects.
- 'Msg' structures: used in generic data transfer operations (~ sendmsg)
- Objects instances: ~ C++ class, allocated by provider

The situation is a little different than traditional libraries, since this is designed as a pass-through library.  (Provider libraries may be built-in, but are not required to be.)  For each of the cases above:

- Attributes - If a version cannot be passed into some initialization call, a version (or GUID) could be added to all calls that contain an attribute structure.
- Msg - Cannot change.  New structures would require new entry points (~ sendmsg2)
- Object instances - App needs some mechanism to check for usability of a given function that does not affect speed path.

- Sean