[openib-general] User level packages make, install and release

[Eitan Zahavi]

Hi

I had the pleasure to study the autotools environment for the last 2
weeks or so in the process of making OpenSM make and install behave.

What I learned is summarized below. I propose we take these as
guidelines and work towards a standard way the entire user level code
make and install operates.

Any comments are welcome.

Eitan

A proposal for user level tree autotools install and distribution:
------------------------------------------------------------------
1. It is possible to provide a hierarchical autotools environment such
that running the autogen.sh, configure, make and make install at the top
most directory will actually run the process on the entire tree below.
(autogen.sh should be made hierarchical)

2. It is possible to support selection of which sub-packages are
installed by using the --enable-<package> or --disable-<package> flags.

3. When building a full hierarchical tree of autotools packages it is
very useful to use the configure --cache-file option to speed up the
process and avoid double checking of the dependencies at each sub project.

4. Distribution of autotools projects can be made simple by following
the procedure:
a. At distribution time check out the source tree
b. Run autogen.sh
c. Run configure
d. Run make dist - create a tar file named <package>-<version>.tgz
e. During installation untar the file
f. ./configure, make, make install
This procedure remove the dependency on autotools during installation.

One can script the above process of steps a to d and get a simple 
"distribution" system for the user level tree sources.
It might be useful to provide a "release number" <revision> to the 
entire release.
Autotools track that revision string in the VERSION variable.
It is optional to define that VERSION by provisioning it to the AC_INIT
macro. A smart distribution script can obtain the <revision> as an
input and apply it to the AC_INIT automatically.

5. Shared library versioning:

5.1 Library Versioning as used by LD:
Library versioning is different from release version since the library
primary and secondary versions are interpreted by the dynamic linker and
thus provide an API version definition rather then a release version
info. API versions should be manually tracked and provided to libtool
through the -version-info <major>:<serial number>:age option.
Where:
<major> is the major version of the API. Changing every time the API
    changes.
<serial> is the serial number (also called minor) which is advanced any
    time a change is made to the implementation of the API.
<age> is the number of major versions that the current <major> version
    is backward compatible with.
This methodology is described by the libtool manuals on the web:
http://www.gnu.org/software/libtool/manual.html#Libtool-versioning
My proposal is to use a special file named lib<libname>.ver to track
this info. The configure.in should parse this file and define the
variable to be used during the libtool invocation.
When running an automatic distribution flow as described in the
previous section one can automate updating the lib<libname>.ver based on 
the change set number.

5.2 Controlling the exported API for various major versions:
LD actualy provides even finer control of the exported symbols for each
major version of the lib. Such that during link time a library can
expose a different set of APIs based on the require API version. To use
this feature we need to provide a file defining the API to the linker
using the -version-script. I propose to name the file lib<libname>.map.
Please see the following link for more info:
http://www.gnu.org/software/binutils/manual/ld-2.9.1/html_node/ld_25.html

5.3 Tracking Release number of shared libs once the numbering is used by 
the LD API version marking.
libttol -release <revision> option should have been the best choice.
This option renames the shared library file according to the follwing
format:
lib<soname>-<revision>.so.<major>.<minor>.<sub>
while it also provides a link with no revision:
lib<soname>.so -> lib<soname>-<revision>.so.<major>
This looks very good but libtool is currently broken since it modifies
the soname to <soname>-<rev> (the name of the shared object by while LD
searches for matching libs during runtime).
The implications of this change are that any application linked to the
<soname>-<revision> lib will not be able to use any next release library
since its soname will be different. Several mails and even a patch to
libtool were posted regarding this limitation, but the common libtool
still does not fix this issue.
My proposal is to avoid the usage of -release flag and instead create
the following symbolic link which should act as a "property mark":
lib<soname>-<revision>.so.<major>->lib<soname>.so.<major>.<minor>.<sub>
Traversing the link one can answer the question "which library version
was released by in release <revision>?", and the reverse one.