[openfabrics-ewg] OFED-1.1-rc2 review comments: openib.spec

[Jeff Squyres]

On 8/27/06 1:12 PM, "Doug Ledford" <dledford at redhat.com> wrote:

> http://fedoraproject.org/wiki pages have a good deal of useful

So bookmarked; thanks!

>> One additional question for you -- how do the Linux vendors typically treat
>> multiple, equivalent packages?  For example, OFED distributes both MVAPICH
>> and Open MPI.  They have executables and libraries that are the same name.
>> What is the currently accepted practice for distributing them?  Just usr
>> /opt/openmpi and /opt/mvapich as installation bases, and let the user figure
>> out PATH, LD_LIBRARY_PATH, and MANPATH issues?
> 
> As a general rule, we try to only distribute one version of a package.
> There has to be a compelling reason to distribute more than one.  That

Unfortunately, in the HPC arena, there are those who have strong religious
preferences for one MPI or another.  There are both logical and
irrational/religious reasons for individuals' preferences.  More
specifically: whenever I've seen a large-scale HPC installation try to
insist on just one MPI, the users rebelled.

> being said, we use the alternatives package to manage this.  My OpenMPI
> package has to use this mechanism to work along side LAM (LAM was
> pre-existing).

FWIW, we're trying to deprecate LAM/MPI (I was the LAM maintainer for many
years).  LAM is great, but many of the features of Open MPI are far greater
(although there's still a small number of features where LAM is superior,
which is why we haven't actively killed it yet).  There is no OFED support
for LAM (and never will be), for example.
 
> Generally speaking, the alternatives setup is used like this:
> 
> 1. Make sure the binaries, etc. all have unique names.  If needed, the
> typical --program-transform-name= ability of the configure scripts is
> helpful.

This *may* be possible for Open MPI, but it'll require an audit.  I have a
dim recollection of one of our developers saying a long time ago that this
would be quite difficult for OMPI for some obscure reason...

I have no idea whether MVAPICH can do this or not.
 
> 2. Put the libraries (if there are any) in a directory that is not in
> the library search path (such as /usr/lib/openmpi and /usr/lib/lam in my
> case).  Ditto for devel files (/usr/include/openmpi
> and /usr/include/lam).

This is probably do-able, but *may* require a little more than --libdir and
--incdir options to configure (I *think* we did this stuff right, but I
would need to check).
 
> 3. In the post and preun sections of the spec file, add/remove the
> package to the alternatives setup.  This has to be coordinated between
> the different packages in that they need to use the same link names and
> master/slave setup.

Is there developer-level documentation about how the alternatives stuff
works somewhere?
 
> The reason we use the alternatives setup is that it allows for a default
> order of selection based on priority, but should the user ever manually
> select an alternative, that selection is kept by the system.  A little
> later today I'll make my latest lam/openmpi packages available on my
> infiniband upload area so people can look over the spec file to see what
> I'm talking about if they wish.

Thanks; that'll be useful.

Did you use the LAM and/or Open MPI specfiles at least as a starting point?
We actually have a fairly sophisticated specfile for Open MPI (can't
remember what LAM does -- it's been a year or two since I've looked at it)
based on requirements from a bunch of different groups.

-- 
Jeff Squyres
Server Virtualization Business Unit
Cisco Systems