[ewg] Re: [ofa-general] OFED 1.2 Feb-26 meeting summary
Eric W. Biederman
ebiederman at lnxi.com
Sun Mar 18 16:03:37 PDT 2007
"Jeff Squyres" <jsquyres at cisco.com> writes:
> To be totally clear, there are three issues:
> 1. *NOT AN MPI ISSUE*: base location of the stack. Doug has repeatedly
> mentioned that /usr/local/ofed is not good. This is a group issue to decide.
If you are really supporting linux you have two choices:
/usr or /opt/ofed/ (assuming you register /opt/ofed with LANNA) anything else
is wrong by definition.
How hard is this to change? If it isn't too bad this should probably be changed
as soon as possible. ABI issues suck and should be fixed as soon as you can.
The install path and config file path is an ABI issue.
> 2. *NOT AN MPI ISSUE*: how the RPMs are built is Bad(tm). Not deleting the
> buildroot is Bad; munging %build into %install is Bad; ...etc. This needs to
> change. 4 choices jump to mind:
> a. Keep the same scheme. Ick.
> b. Install while we build (i.e., the normal way to build a pile of
> interdependent RPMs)
> c. Use chroot (Red Hat does this in their internal setup, for example)
> d. Only distribute binary RPMs for supported platforms; source is available
> for those who want it.
e. Give up and building RPM's and let the distributions do it.
I think e is the most common solution and what distributions are doing now.
The only problem with it is that ofed may be evolving to fast to reasonably
expect the distributions to keep up.
Of course the ideal build scenario looks something like:
for source in *.tgz ; do rpmbuild -tb $source ; rpm -i ? ; done
Where the source tarballs are have a spec file in them that can be used
to build an rpm.
Sorting this out needs to happen but likely is something that ugly bits
can be lived with.
> 3. Doug's final point about allowing multiple MPI's to play harmoniously on a
> single system is obviously an MPI issue. The /etc/ alternatives mechanism is
> not really good enough (IMHO) for this -- / etc/alternatives is about choosing
> one implementation and making everyone use it. The problem is that when
> multiple MPI's are installed on a single system, people need all of them (some
> users prefer one over the other, but much more important, some apps are only
> certified with one MPI or another). The mpi-selector tool we introduced in
> OFED 1.2 will likely be "good enough" for this purpose, but we can also work on
> integrating the /etc/alternatives stuff if desired, particularly for those who
> only need/want one MPI implementation.
Agreed. There are a few other issues here as well.
There seems to be no agreement on a fortran ABI for linux. Even little
things like f77 and g90 are incompatible. Or was their an ABI agreement
recently and I missed it?
When providing MPI fortran bindings that is a problem because you need
to compile separately for each different installed compiler, possibly even
for different versions of the same compiler.
Which makes even an /opt solution not quite good enough because you need multiple
version of the same mpi built with different compilers.
More information about the ewg