[openfabrics-ewg] OpenMPI package

[Jeff Squyres (jsquyres)]

> - The README appears to be quite out of date; I assume you will need
> some Open MPI-specific information for it.  How should I get this to
> you?
> 
> <Aviram> Just send it and we'll incorporate it.

Is there a repository we can work with?  If the logistics can be worked
out, this might be a *lot* easier than wholesale mailing of code around
(that's what version control systems are for).  I would strongly prefer
this model; it's just better software engineering.

> - It looks like the LICENSE file is incomplete -- shouldn't it contain
> at least references to the license / copyright information of the
> sub-packages?  (e.g., MVAPICH and others)
> 
> <Aviram> That is good issue. I am not a LICENSE expert. Who can answer
> it? Marketing guys? 

This is probably a murky area -- we *are* including the license and
copyright notices from the other packages, but buried deep down,
sometimes in multiple layers of binary (tarballs).  As a good faith show
to the open source communities who developed these packages, 

I also note that some of the packages have no license and/or copyright
information (PMB, prestal).  Has anyone checked to see that we are
allowed to distribute them?  I think we all want to avoid scenarios
where someone comes back to us someday and says "you owe me money
because you distributed / re-sold my software."

> - I see a bunch of top-level *.sh scripts -- these will need to be
> expanded for Open MPI.  Who does that -- me or you?
> 
> <Aviram> Can you do it and send it over?

This will be tremendously simpler with a shared repository -- to avoid
conflicts, allow proper merging, etc.  (e.g., what if someone else is
working on the same code at the same time?)

> - I see SOURCES/mpi_osu...tgz.  That contains a bunch of 
> scripts and the
> mvapich tarball.  I'm assuming that Open MPI needs to be bundled this
> way as well...?  Is this documented anywhere?  Is there a reason why a
> tarball contains another tarball?
> 
> - The mpi_osu...tgz file contains several MPI-independent utilities
> (PMB, prestal, etc.).  Should these be moved out of the OSU 
> MPI tarball
> and into an MPI-agnostic tarball, and then compiled against each MPI
> that is installed?
> 
> <Aviram> Will get back to you on those two.

Ok.  I need these answers before I give you an Open MPI package to
integrate.

Specifically: you need more than just an RPM and/or SRPM.  It's not
entirely clear to me yet what exactly that is, but there does appear to
be a bunch of scripts and other things that are necessary.

> - What is the plan for having 2 MPIs in this distribution -- how will
> users/sysadmins choose between them?  I.e., are we going to allow both
> to be installed and make it a user-level decision which to 
> use?  Or will
> the sysadmin only pick one to install?  Or ...?
> 
> <Aviram> We need to decide on that. All, how do you view it?

Someone else replied that we should let the sysadmin choose to install
one.

I think that this is a tremendously complex issue; cluster-installation
packages (OSCAR, Rocks, Warewulf, etc.) have spent a great deal of time
wrestling with this issue over the years.  This is probably worth some
time on the next teleconference (is there a next one scheduled?).

In the *best* case scenario, there will only be one MPI installed.  I am
not familiar with everyone's customer base, but I have seen clusters
with upwards of 30 MPI implementations installed (i.e., including a
large number of variations of the same implementation -- e.g., Open MPI
compiled against different compilers -- the issue is the same).  This
brings up all kinds of practical and logistical derrivative issues.

> 2. Some more questions that I did not include in my mail last night:
> 
> - Is there a source code repository for IBED somewhere?  What is the
> model for developers to modify / test IBED?
> 
> - What version of MVAPICH is being used?  I see
> mvapich-0.9.5-mlx2.0.1.tar.gz -- does this mean it's Mellanox's v2.0.1
> of MVAPICH 0.9.5?  Are other vendors allowed to modify this?  (I ask
> because all of our MVAPICH's are slightly different -- fixed bugs
> specific to our customers, etc.)
> 
> <Aviram> We'll use 0.9.7 It will be incorporated on the next 
> rc. Yes we
> can fix and modify it.

What will be the model for vendors other than Mellanox to collaborate
and contribute?

> - There appear to be multiple levels of indirection in the 
> MVAPICH build
> scripts -- what directory --prefix is it being installed to?  (this is
> going to be influenced by the answer to the "2 MPI" question, above)
> 
> <Aviram> Will get back to you it.

Ok.  I need this answer before I can provide an Open MPI package for
you.

> 4. Gleb sent me a proposed spec file for Open MPI -- we'll 
> iterate about
> this off-list.
> 
> <Aviram> Who will send us the OpenMPI version to be integrated?

Me.  Gleb and I have already iterated a bit; we will definitely have a
new specfile for IBED.

I'm now a bit confused -- Aviram said in a later mail:

> OK. We'll integrate the current one on Sunday, unless we get a new one
> from Jeff till the end of the week. 

What, exactly, are you going to integrate?  Having a single spec file is
not enough.  Are you going to do all the other script and README work?

I thought that I understood what you needed (integration with the *sh
scripts, some kind of megga-tarball with Open MPI and some other
to-be-defined "stuff", etc.), but this statement seems to imply that all
you wanted was a spec file.  

What exactly do you need?

-- 
Jeff Squyres
Server Virtualization Business Unit
Cisco Systems