[openib-general] MVAPICH2 rpmbuild issue

[Shaun Rowland]

Vladimir Sokolovsky wrote:
> Hi Shaun,
> Please check the following issue:

Hi Vladimir. I can tell from the output what seems to have happened, but
I don't know why it happened. When I tested using the install/build
scripts you had given us originally, I tested against OFED 1.1 files to
understand how the build procedure worked. From that testing, the first
thing I found that I had to deal with was the fact that the openib
packages were built in /var/tmp/OFED and left there for other packages
to be built against. Since the openib files are in a location other than
their final destination, I created the %ofed_build_root macro to define
this location and in addition, set a %ofed_bootstarp condition in the
RPM. From the rpmbuild command below, this appears to be called how I
expect. However:

> Executing(%install): /bin/sh -e /var/tmp/rpm-tmp.84872
> + umask 022
> + cd /var/tmp/OFEDRPM/BUILD
> + cd mvapich2-0.9.8
> + export OPEN_IB_HOME=/var/tmp/OFED/usr/local/ofed
> + OPEN_IB_HOME=/var/tmp/OFED/usr/local/ofed
> + '[' -d /var/tmp/OFED/usr/local/ofed/lib ']'
> + '[' -d /var/tmp/OFED/usr/local/ofed/lib64 ']'

In the two lines above, I am setting LD_LIBRARY_PATH so that MVAPICH2
can be built. I do this because, again, the files are not in their final
destination directory, but in /var/tmp/OFED/$STACK_PREFIX/lib[64].
Above, I am testing for either the lib or lib64 directory in that path,
but neither is being found because there is no associated export of
LD_LIBRARY_PATH above. This is also why:

> + export PREFIX=/var/tmp/OFED/usr/local/ofed/mpi/gcc/mvapich2-0.9.8-1
> + PREFIX=/var/tmp/OFED/usr/local/ofed/mpi/gcc/mvapich2-0.9.8-1
> + export CC=gcc CXX=g++ F77=gfortran
> + CC=gcc
> + CXX=g++
> + F77=gfortran
> + export ROMIO=yes
> + ROMIO=yes
> + export SHARED_LIBS=yes
> + SHARED_LIBS=yes
> + ./make.mvapich2.gen2
> Could not find the OPEN_IB_HOME/lib64 or OPEN_IB_HOME/lib directory.
> Exiting.
> error: Bad exit status from /var/tmp/rpm-tmp.84872 (%install)

our make.mvapich2.gen2 script fails. It actually exists if either of
these directories cannot be found. It is basically the same check,
except in make.mvapich2.gen2 LD_LIBRARY_PATH is not set, and it also
exists if the directories are not found. It would be possible to do the
LD_LIBRARY_PATH setting in make.mvapich2.gen2 as well, but usually it
isn't necessary - so I had added the code to the spec file myself.

So my question in this case, given the error output, is what happened
to /var/tmp/OFED/usr/local/ofed/lib or
/var/tmp/OFED/usr/local/ofed/lib64? The rpmbuild is not finding those
directories, but the files should still be there for MVAPICH2 to be
built against, yes? Unless the build process has changed, it seems these
directories do not exist when I was expecting them to exist. They should
be there at that location, right?

I've not tried the new install/build scripts since you've updated them.
I think I need to make an openib SRPM for this or ? I am currently
investigating this and will attempt to use the new scripts on my own
testing system. I will also check if there are any files I can use
instead of making an SRPM if that's even necessary (it seems that it
was, so I had not done it yet).

-- 
Shaun Rowland	rowland at cse.ohio-state.edu
http://www.cse.ohio-state.edu/~rowland/