[ofa-general] Re: [ewg] Re: RFC OFED-1.3 installation

Jeff Squyres jsquyres at cisco.com
Tue Jul 17 04:12:01 PDT 2007


All: You may have skipped this mail because of its length.  Please  
read it; Doug lists specific guidelines in here that SRPMs will need  
to adhere to for RH to include OFED v1.3 (many of the OFED RPMs --  
including the OMPI RPM -- do not adhere to these guidelines; we all  
have work to do).


On Jul 16, 2007, at 11:29 PM, Doug Ledford wrote:

> On Mon, 2007-07-16 at 15:24 +0300, Vladimir Sokolovsky wrote:
>
> [ snip ]
>
> Most of this proposal was just about splitting the packages up.   
> That's
> good, but it doesn't warrant much comment.  It's not the existence of
> different packages that will draw my comments, it's the *content*  
> of the
> packages that will, the actual spec files themselves.
>
> However, there is this one tidbit:
>
>>      Source RPMs will be created for each userspace package in the  
>> following way:
>>
>>      git clone ...
>>      autogen.sh
>>      configure --disable-libcheck
>>      make dist
>
> This is so fundamentally broken as to be brain dead.  Yet it's what  
> has
> been done since OFED 1.0.  Can you imagine how screwed the open source
> world would be if one day Linus released kernel-2.6.24.tar.gz on
> kernel.org, only to silently update the file kernel-2.6.24.tar.gz to
> something else the next?  This is the *ONLY* open source software  
> group
> I know of that creates new tar.gz files any time they make a change  
> but
> keeps the version of the file the same.
>
> Let me copy and paste an email conversation I had with Or that
> highlights why this is broken:
>
> ------- Begin cut-n-paste
> On Mon, 2007-07-02 at 22:25 +0300, Or Gerlitz wrote:
>> [sorry for breaking the thread, I am working from home now and unable
> to use normal mailer.]
>>
>> Does this means that the OFED 1.3 effort is useless from your  
>> point of
> view?
>
> Yes and no.  The effort to get a complete set of working libraries and
> stacks pulled together and debugged is good and worthwhile.  The
> packaging has been done all wrong though.  Because the ewg has
> concentrated on supporting local compile installations, they don't
> really have the faintest clue about several important issues that crop
> up specifically when you are attempting to support binary distribution
> instead of source distribution.  That in turn has led them to make
> decisions that have proved to be very counterproductive to my end goal
> of a supportable environment for my customers.
>
> Let me give an example.  In OFED 1.0, you shipped dapl version  
> 1.2.  In
> OFED 1.1, you also shipped dapl version 1.2.  However, code inspection
> shows that between OFED 1.0 and OFED 1.1, dapl did in fact change  
> (not a
> lot, but anything is enough).  So, between OFED 1.0 and OFED 1.1, you
> have two different versions of dapl, but with exactly the same version
> number.  A person can't tell them apart.  Furthermore, unless the  
> person
> is compiling locally, they'll never get the OFED 1.1 dapl installed
> because RPM/up2date will see that they already have the current  
> version
> even when they have the OFED 1.0 version.  So, in our RPMs, I updated
> the OFED 1.1 dapl version we built to be 1.2.1.  Without doing  
> that, the
> binary upgrade process that we use would have never worked.  Then, in
> OFED 1.2, you guys update the dapl code again, and this time you  
> decide
> to use...wait for it...that's right, 1.2.1.  Great.  Now we have a
> conflict between your 1.2.1 and our 1.2.1.  How do people know  
> which is
> which?  They don't.  And, of course, in order for binary upgrades to
> work, I once again had to bump our number.  Our OFED 1.2 package now
> builds dapl 1.2.1.1 just because I had to do *something* in order to
> make upgrades work.
>
> The only reason that the OFED distribution has *ever* reliably  
> installed
> the rpms you wanted installed is because you compile things locally  
> and
> then *force* the upgrade of rpms over the top of older rpms that have
> the same version number.  And even then, you yourselves can't tell the
> difference between a customer with the OFED 1.0 or OFED 1.1 dapl
> installed by checking the RPM version, you just have to go off what  
> the
> end user *tells* you he installed and hope he's right.
>
> So, quite simply, the EWG has *chosen* to support source distribution
> and local compiles.  That's fine really.  But they've also chosen to
> bury their head in the sand about basic, non-flexible rules associated
> with any successful binary distribution and update process, even when
> I've brought those rules up multiple times.
>
> It should be no wonder then why I get all up in arms about packaging
> issues.  Everything I give my customers has to automatically and
> correctly install, upgrade, downgrade, delete, verify, etc using
> RPM/up2date/yum.  It can't require any --force options.  And I don't
> have a choice about that.
>
> And I have to *know* what software my customer is running in order to
> support them.  Because you guys have done things the way you have, I
> can't know that.  I might be able to know if I could also guarantee  
> they
> didn't download and locally compile your packages, but if they did,  
> then
> the same version number of RPM can mean two different things entirely
> depending on whether it's your RPM or mine.
>
> ------- End cut-n-paste
>
> I posted links to a wealth of valuable information on the topic of
> making a proper spec file and creating *good* packages during my  
> talk at
> Sonoma.  I gather you haven't read those or you never would have
> suggested the above for creating the RPMs.
>
> I've already reached the decision that the next release of the RDMA
> stack that Red Hat releases will adhere to much stricter guidelines  
> than
> in the past.  From now on, all packages I build based upon software  
> from
> the OpenFabrics Alliance will adhere to these guidelines:
>
> 1.  All tar.gz files will be imported once and exactly once into  
> our SCM
> repo.  At that point they will be MD5 summed and the MD5 sum will be
> checked on all subsequent builds to verify the tar.gz file has not
> changed.
>
> 2.  All fixes to released tar.gz files will be in the form of patches
> applied in the spec file during the %prep phase, or they will  
> require a
> new tar.gz file.  Under no circumstances will an existing tar.gz  
> file be
> updated to include fixes.
>
> 3.  All packages will have a version and release number appropriate to
> the tar.gz release of the software and the build of the package.
>
> 4.  All tar.gz files *must* have a publicly available URL from which
> they can be downloaded.
>
> 5.  All tar.gz files that have a home site other than openfabrics.org
> will be taken from their home site.  Eg. openmpi will come from the
> openmpi site.  No special openfabrics versions of already existing
> packages will be considered.
>
> 6.  All source repos that utilize the autoconf configure capability  
> will
> have configure run at build time.  Any configure output produced prior
> to build will not be considered usable.  On the other hand, we expect
> that autogen.sh *will* be run prior to making the tarball.
>
> If the software does not meet the above minimal guidelines, then it
> won't be considered for inclusion in our product.
>
> In addition to these rules, if you want me to consider using your spec
> files to build the packages, then these additional rules apply:
>
> 7.  All spec changes will be accompanied by a changelog entry.
>
> 8.  The spec file will be clean and readable.  Spec files cluttered up
> with multitudes of options that have no impact on a standardized
> distribution will not be included.
>
> 9.  All spec files must be Linux File Hierarchy Standards compliant.
>
> 10. All spec files must pass rpmlint tests.
>
> 11. All code must be built using the %build section of the spec file.
> The %install section is for installation *ONLY*.
>
> 12. Spec files must build debug packages.
>
> 13. Spec files must leave the default build scripts enabled.
>
> 14. Spec files must list appropriate BuildRequires entries.
>
> 15. Spec files must not list Provides entries unless the build scripts
> are unable to determine that they provide a particular item, or in  
> cases
> like the MPI packages where they can specify that they provide the
> generic mpi facility in addition to the specific mpi library provides
> that the build system will pick up automatically.
>
> There's probably more things to list, but I really don't feel like
> repeating what amounts to our standard build requirements when they  
> are
> already all written out in the guides I talked about in my Sonoma  
> talk.
>
> Hopefully, all of this will help you get a clearer picture of what I
> expect the EWG's work on cleaning up their packaging to cover.
>
> -- 
> Doug Ledford <dledford at redhat.com>
>               GPG KeyID: CFBFF194
>               http://people.redhat.com/dledford
>
> Infiniband specific RPMs available at
>               http://people.redhat.com/dledford/Infiniband
> _______________________________________________
> ewg mailing list
> ewg at lists.openfabrics.org
> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg


-- 
Jeff Squyres
Cisco Systems




More information about the general mailing list