[openib-general] Open MPI rpmbuild fails in OFED-1.2

[Shaun Rowland]

Michael S. Tsirkin wrote:
>> Quoting Jeff Squyres <jsquyres at cisco.com>:

>> 2) we're trying to *use* the software when it is installed in the  
>> DESTDIR
>> --> this means that you have to put special-case in the software so  
>> that they look for support files in both the DESTDIR *and* the final  
>> installation directory

Either that, or fix your resulting package so that it will work with the
final installation directory case (not work with both), and then setup a
temporary environment that will allow it to work for the rest of the
build process. For the mpitests package being built against our RPM
result, this is the approach I took. It took me a little time to figure
out how to do this, because it is odd.

> How do you mean, use?

I assume this means linking against the libraries. In the mpitests RPM
build, it could mean using mpicc, etc. from the MPI package while it
itself is not working in its final destination directory. No one does
this sort of thing normally when building software packages from source
code.

> Hmm. I guess my question is - this works fine when I run OFED's
> configure script, why is SRPM so much more difficult?

Anyone can correct me if I am reading this wrong, but I've commented on
this at least once before - somewhat indirectly. I've built and
supported open source software at the OSU CSE department for a long time
- so I've built many different packages from source about a million
times. When you build a source code package, obviously you make sure the
necessary libraries can be found. These libraries are in some system
location - their "final installation directory". If this location is not
in the default search path, you can deal with that various ways or add
the path to the system's default. If your package builds its own
libraries and also uses them, then you deal with that yourself in your
package's build system. When you say OFED's configure script, this is
the situation I see.

Never have I purposely built the libraries required for a package in a
temporary location, built the package against those libraries, and then
moved everything to a final location. It makes things more difficult.
Take our SRPM for example. If you have OFED installed, I am mainly
concerned with the stack prefix, by default /usr/local/ofed. If I build
our code with the libraries in their final location, I don't have to
worry about subtle things like the various scripts having this path hard
coded in them. Most packages rightly make the assumption that these
paths you use are to dependencies that have already been installed, and
if there is some need to incorporate those paths into the package build
result, for whatever reason, it's all right to do that. In our SRPM, I
need to fix some things in the OFED installer case because the libraries
I am building against are not in their final location. These are things
that I do not have to do normally, however to be fair - I already have
to fix some things because of the RPM BuildRoot usage anyway because our
package is not installed in its final destination either in any RPM
building scenario. It only goes into its final destination directory
when you install the actual RPM. With RPMs, this is a good, safe way to
build _individual_ packages. In the SuSE case, the %build section is
assuming this and cleaning things up before you start, because - why
would there be anything in there BuildRoot already? Is this right or
wrong? That's a matter of opinion. There's information in various RPM
building resources that mention some of this stuff. Luckily this is not
a big deal for me to handle in our case. However, it could have been.
This is like a "bootstrap" situation, but it's not normally how one
would go and build some source code package on their system, and RPMs
are all about reproducible source code builds. Again, to be totally
fair, you wouldn't normally install your package into a BuildRoot
prefixed "prefix" location either, but that seems easier to deal with
than the location of libraries your package may depend on. And to go
even further, as in the mpitests RPM build case, would one normally
expect the RPM build result that is installed in a BuildRoot prefixed
"prefix" and just left there to even work? I would say it is absolutely
not safe to just assume that for any given RPM build. This all depends
on the source code you are trying to build and what it does exactly. Any
time you are using paths that don't reflect the final destination of
whatever dependency, you have the potential to have to deal with extra
work to fix the final result.

In the usual SRPM building case, the packages that your package depends
on would already be installed on the system in their final destination
directory. I could even require these RPM packages as build requirements
- something I am not doing in the spec file itself, yes? This would mean
that I could take a few steps out of my SRPM spec file. From what I am
reading here, this would be one reason to think of a chroot situation.
But it seems to me that this would make things potentially difficult.
Another option could be to go ahead and install the OFA packages before
the MPI packages. Either way would work for me because I've already
handled this DESTDIR situation (even for the mpitests being built
against our RPM result - because I leave it in DESTDIR after the RPM
build is done if this is all being done by the OFED installer). In
addition, if you were to follow this logic, the MPI packages would be
installed in their final location before the mpitests packages were
built against them. If this were done instead, I don't think chroot
would be required. However, it does mean the packages have to be
installed, and from what I've seen - in a 3 step process.

When I was first trying to make our SRPM work with the OFED install
scripts, this was the first thing I had to fix. And I definitely was not
expecting this type of situation. Now, perhaps I have misread this
thread and applied it to my own experience. If I have not misread it,
then I understand what the point is. This does not mean I advocate
changing it personally. From my testing, I've made this aspect of our
build work. I had no problems moving my %build section code to the
%install section. To me the question is if this is too difficult to deal
with. It depends on the package being built, but no standalone packages
I am aware of contain logic for "temporary paths" to dependencies, or
again to be fair in the RPM building scenario, a "temporary prefix".
None of what I am saying applies to a package that builds and uses its
own libraries though. In those cases, the developers obviously have to
deal with that.
-- 
Shaun Rowland	rowland at cse.ohio-state.edu
http://www.cse.ohio-state.edu/~rowland/