[openfabrics-ewg] Default and per-user MPI selections

[Steffen Persvold]

Jeff,

What's so difficult with doing :

$ . /opt/scali/etc/scampivars.sh

Or 

$ . /opt/intel_mpi_10/etc/intelvars.sh


You could just as well have :

$ . /etc/openmpivars.sh

For example ??

In the HPC community, people also like to use "modules" :

$ module load mpi/scali


Kind regards,

Steffen Persvold
Technical Director Americas
tel. 508-281-7100 x401
fax. 508-281-7171

http://www.scali.com/
Scaling the Linux datacenter


> -----Original Message-----
> From: openfabrics-ewg-bounces at openib.org [mailto:openfabrics-ewg-
> bounces at openib.org] On Behalf Of Jeff Squyres
> Sent: Tuesday, August 29, 2006 9:14 AM
> To: Open Fabrics
> Subject: [openfabrics-ewg] Default and per-user MPI selections
> 
> All --
> 
> I think we need to re-visit the which-MPI-to-put-in-the-PATH issue.
> Others
> have previously expressed [strong] opinions that no MPI should be in
> the
> path; sysadmins and/or users can set their PATH/LD_LIBRARY_PATH however
> they
> want.  I disagree with this for two reasons:
> 
> 1. Editing shell startup files is trivial for IT
> professionals/developers,
> but can be a daunting task for a chemical engineer who doesn't
> know/care how
> the system works and just wants to run his/her HPC programs to get
> results.
> 
> 2. The job of the OFED installer is to make it *easier* to run HPC
> applications.  OFED does everything *except* put MPI in your path --
> yet MPI
> is the most highly visible protocol used in HPC.  It seems pretty weird
> (to
> me) that we don't go the final few steps to provide a complete
> solution.
> 
> What do the Linux vendors will feel about installing software that is
> intended for general use but is not put in the PATH/LD_LIBRARY_PATH?
> 
> I think that OFED should have the capability of setting a system-wide
> default MPI and/or setting a per-user default MPI.  During
> installation,
> OFED should ask the sysadmin if they want a system-wide default MPI,
> and if
> so, which one.  And then do the Right magic to make it so.  I also
> think
> that OFED should provide a per-user mechanism to override the system-
> default
> MPI (something better than "edit your shell startup files and change
> your
> PATH/LD_LIBRARY_PATH").
> 
> -----
> 
> Doug Ledford has proposed a solution -- that the MPI implementations
> use the
> "alternatives" system (is "alternatives" also available in SLES?).
> This
> would require:
> 
> A) the MPIs to be installed in non-default locations (e.g.,
> /opt/openmpi*
> and /opt/mvapich*)
>    --> This will require changing the builder scripts to allow
> different
> prefix values for OFED and each MPI installation (e.g., /usr for OFED,
> /opt/openmpi-gnu for Open MPI compiled by GNU, /opt/openmpi-intel for
> Open
> MPI compiled by the Intel compilers, etc.).
>    --> The installer should also prompt as to which MPI should be the
> default for the system (if any).
> 
> B) use the "alternatives" mechanism to allow users to choose which MPI
> to
> use as the system default (i.e., would show up in /usr/bin and the
> like)
>    --> This will require changing the specfile for both Open MPI and
> MVAPICH.  Both MPI implementations' specfiles must be done for this to
> be
> meaningful.
> 
> Note that the "alternatives" system does not have the inherent ability
> to
> allow users to choose which MPI to use; "alternatives" only seems to
> set a
> system-wide default.  If a user wants to use a different MPI, they can
> prepend their PATH / LD_LIBRARY_PATH / etc. to avoid the one that is
> found
> in system default paths (e.g., /usr/bin).
> 
> -----
> 
> Another option is to use something like environment modules and the
> "switcher" package from OSCAR to have both a system-wide default for
> MPI and
> allow users to et which MPI they want to use (if they want to override
> the
> system-wide default) without needing to edit their shell startup files.
> 
> The changes for using modules/switcher are similar to those for using
> "alternatives" (i.e., both options will take work to implement).  There
> are
> other systems available that can be used as well (softenv comes to
> mind; I
> think there might be others, too).
> 
> -----
> 
> Comments?
> 
> (obviously, this is not for OFED 1.1 -- this would be a 1.2 issue)
> 
> --
> Jeff Squyres
> Server Virtualization Business Unit
> Cisco Systems
> 
> _______________________________________________
> openfabrics-ewg mailing list
> openfabrics-ewg at openib.org
> http://openib.org/mailman/listinfo/openfabrics-ewg