[ewg] Re: [ofa-general] Re: RFC OFED-1.3 installation

[Michael S. Tsirkin]

> Quoting Doug Ledford <dledford at redhat.com>:
> Subject: Re: [ofa-general] Re: RFC OFED-1.3 installation
> 
> On Wed, 2007-07-18 at 05:18 +0300, Michael S. Tsirkin wrote:
> > > Quoting Doug Ledford <dledford at redhat.com>:
> > > Subject: Re: [ofa-general] Re: RFC OFED-1.3 installation
> > > 
> > > On Wed, 2007-07-18 at 00:09 +0300, Michael S. Tsirkin wrote:
> > > > > Quoting Roland Dreier <rdreier at cisco.com>:
> > > > > Subject: Re: [ofa-general] Re: RFC OFED-1.3 installation
> > > > > 
> > > > >  > I don't really think we want customers to run beta code
> > > > > 
> > > > > What's the point of a beta then??
> > > > 
> > > > Donnu.
> > > > In previous OFED releases, we had "release candidates" rather than "beta".
> > > > Openfabrics members were running RCs and reporting issues on the list and in
> > > > bugzilla. Do you really ask your customers to do this for you?
> > > 
> > > Sure, as much as possible.  I generally don't recommend using it in
> > > production, but just as close as they can get to production is fine with
> > > me.  The more issues they find while I'm still actually working on it
> > > and making new revisions, the less issues they'll find after I stupidly
> > > think I'm done.
> > 
> > So,Roland's idea of sticking a date in RPM revision willwork, won't it?
> 
> As long as you don't do two package builds on the same day.  That's why
> my script encodes both an increasing number and the date into the
> revision.
> 
> For reference, I'll attach the updated script I made for spitting out a
> buildable tarball.
> 
> Hehehe...resending because the ofa list server ate my message due to the
> script attachment :-D  I'll inline it instead.
> 
> I guess I'll also mention that this script exists in my ~/repos/upstream
> directory, and also in that directory are all the git repos that I have
> cloned from ofa (as well as other places).  So, it's one level above all
> the various git clones and spits everything out into dist/.  The easiest
> way to use this script for any given package you want to create a daily
> snapshot of is to run ./make.dist repodir daily; scp
> dist/repodir-git.tgz dist/repodir-daily.HEAD ofaserver:downloads.  That
> simple action would (assuming you create a reasonable reponame.spec.in
> file in the repos that are missing one) spit out a tarball that can be
> passed directly to rpmbuild --rebuild reponame-git.tgz and rpm will spit
> out the packages, and the repodir-daily.HEAD file shows the HEAD of the
> git repo so you know exactly what state the tarball represents and you
> can always get to it in another more recent repo by just updating to
> that commit as head of tree.

Thanks for the script.
In OFED, since we control the upstream, I think we'll try to do
as much as possible at the package level, for example make sure
that each package has a reasonable spec file.
Some ideas on how we might want to do this below.

> #!/bin/bash
> 
> usage() {
> echo "$0 repo daily | release [ signed | <key-id> ]"
> echo
> echo "	You must specify the repo to make a distribution tarball in.  This"
> echo "script will not work with complex repos like the management repo that"
> echo "builds more than one package.  It expects a repo to be a single package"
> echo "repo where the directory name and the package name are the same, and"
> echo "where a properly formatted reponame.spec.in file exists."
> echo
> echo "	You must specify either release or daily in order for this script"
> echo "to make tarballs.  If this is a daily release, the tarballs will"
> echo "be named <component>-git.tgz and will overwrite existing tarballs."
> echo "If this is a release build, then the tarball will be named"
> echo "<component>-<version>.tgz and must be a new file.  In addition,"
> echo "the script will add a new set of symbolic tags to the git repo"
> echo "that correspond to the <component>-<version> of each tarball."
> echo
> echo "	If the script detects that the tag on any component already exists,"
> echo "it will abort the release and prompt you to update the version on"
> echo "the already tagged component.  This enforces the proper behavior of"
> echo "treating any released tarball as set in stone so that in the future"
> echo "you will always be able to get to any given release tarball by"
> echo "checking out the git tag and know with certainty that it is the same"
> echo "code as released before even if you no longer have the same tarball"
> echo "around."
> echo
> echo "	As part of this process, the script will parse the <target>.spec.in"
> echo "file and output a <target>.spec file.  Since this script isn't smart"
> echo "enough to deal with other random changes that should have their own" 
> echo "checkin the script will refuse to run if the current repo state is not"
> echo "clean."
> echo
> echo "	NOTE: the script has no clue if you are tagging on the right branch,"
> echo "it will however show you the git branch output so you can confirm it"
> echo "is on the right branch before proceeding with the release."
> echo
> echo "	In addition to just tagging the git repo, whenever creating a release"
> echo "there is an optional argument of either signed or a hex gpg key-id."
> echo "If you do not pass an argument to release, then the tag will be a"
> echo "simple git annotated tag.  If you pass signed as the argument, the"
> echo "git tag operation will use your default signing key to sign the tag."
> echo "Or you can pass an actual gpg key id in hex format and git will sign"
> echo "the tag with that key."
> echo 
> }
> 
> if [ -z "$1" -o -z "$2" ]; then usage; exit 1; fi
> 
> if [ ! -d "$1" ]; then usage; exit 1; fi
> 
> TMPDIR=dist
> if [ ! -d $TMPDIR ]; then mkdir $TMPDIR; fi
> 
> if [ "$2" = "daily" -o "$2" = "release" ]; then
> 	if [ ! -f $TMPDIR/$1-$2.HEAD ]; then
> 		touch $TMPDIR/$1-$2.HEAD
> 	fi
> 	NEWHEAD=`cat $TMPDIR/$1-$2.HEAD`
> else
> 	usage
> 	exit 1
> fi
> 
> cd "$1"
> echo "Updating git repo..."
> git pull
> RESULT=$?
> HEAD=`git log --pretty=oneline -1`
> 
> if [ "$RESULT" -ne 0 ]; then
> 	echo "Failed to update the git repo cleanly, manual intervention required"
> 	exit 1
> fi

pull really will merge your local modifications with upstream.
In OFED we really want just git clone, and use upstream code unmodified.

> if [ "$HEAD" = "$NEWHEAD" ]; then
> 	echo "No new commits since last tarball creation, nothing to do."
> 	cd ..
> 	exit 0
> fi
> 
> if [ "$2" = "release" ]; then
> 	# Is the repo clean?
> 	git status | grep modified > /dev/null 2>&1
> 	if [ $? = 0 ]; then
> 		echo "There are modified files in the repo.  Please check any"
> 		echo "changes in before proceeding."
> 		exit 4
> 	fi
> 	# Since we will be tagging things, make sure we are on the right
> 	# branch
> 	git branch
> 	echo -n "Is the active branch the right one to tag this release on [y/N]? "
> 	read answer
> 	if [ "$answer" = y -o "$answer" = Y ]; then
> 		echo "Proceeding..."
> 	else
> 		echo "Please check out the right branch and run make.dist again"
> 		exit 0
> 	fi

See below on what we should do in OFED IMO.

> 	# Check versions to make sure that we can proceed
> 	VERSION=`grep "AC_INIT.*$1" configure.in | cut -f 2 -d ',' | sed -e 's/ //g'`
> 	TARBALL=$1-$VERSION.tgz
> 	if [ -f ../$TMPDIR/$TARBALL ]; then
> 		echo "Target $TARBALL already exists, please update the version of"
> 		echo "$1"
> 		exit 2
> 	fi
> 	if [ ! -z "`git tag -l $1-$VERSION`" ]; then
> 		echo "A git tag already exists for $1-$VERSION.  Please change the version"
> 		echo "of $1 so a tag replacement won't occur."
> 		exit 3
> 	fi
> # On a real release, this resets the daily release starting point, on the
> # assumption that any new daily builds will have a version number that is
> # incrementally higher than the last officially released tarball.
> 	RELEASE=1
> 	echo $RELEASE > ../$TMPDIR/$1.release
> else
> 	DATE=`date +%Y%m%d`
> 	if [ -f ../$TMPDIR/$1.release ]; then
> 		RELEASE=`cat ../$TMPDIR/$1.release`
> 		RELEASE=`expr $RELEASE + 1`
> 	else
> 		RELEASE=1
> 	fi
> 	echo $RELEASE > ../$TMPDIR/$1.release
> 	RELEASE=0.${RELEASE}.${DATE}git
> 	TARBALL=$1-git.tgz
> fi
> 
> cd ..
> cp -a $1 $1-$VERSION
> [ -f $1/$1.spec.in ] && sed -e 's/@VERSION@/'$VERSION'/;s/@RELEASE@/'$RELEASE'/;s/@TARBALL@/'$TARBALL'/' < $1/$1.spec.in > $1-$VERSION/$1.spec

This, I think, is the bit that we definitely want to reuse.

> if [ -f $1-$VERSION/autogen.sh ]; then
> 	cd $1-$VERSION
> 	./autogen.sh
> 	cd ..

I think we will want to call make dist too.

> fi
> echo "Creating $TMPDIR/$TARBALL"
> tar -czf $TMPDIR/$TARBALL --exclude=.git $1-$VERSION
> rm -rf $1-$VERSION
> echo "$HEAD" > $TMPDIR/$1-$2.HEAD
> 
> if [ $2 = release ]; then
> 	echo "Tagging release."
> 	cd $1
> 	if [ ! -z "$3" ]; then
> 		if [ $3 = "signed" ]; then
> 			git tag -s -m "Auto tag by make.dist on release tarball creation" $1-$VERSION
> 		else
> 			git tag -u "$3" -m "Auto tag by make.dist on release tarball creation" $1-$VERSION
> 		fi
> 	else
> 		git tag -a -m "Auto tag by make.dist on release tarball creation" $1-$VERSION
> 	fi
> 	cd ..
> fi

This takes whatever's at git head and then tags that.
In OFED it is the other way around: maintainers tag
the appropriate bits, release script just packages that.

-- 
MST