[ewg] Re: [ofa-general] Re: RFC OFED-1.3 installation

[Doug Ledford]

On Wed, 2007-07-18 at 07:31 +0300, Michael S. Tsirkin wrote:

> Thanks for the script.
> In OFED, since we control the upstream, I think we'll try to do
> as much as possible at the package level, for example make sure
> that each package has a reasonable spec file.
> Some ideas on how we might want to do this below.
> 
> > #!/bin/bash
> > 
> > usage() {
> > echo "$0 repo daily | release [ signed | <key-id> ]"
> > echo
> > echo "	You must specify the repo to make a distribution tarball in.  This"
> > echo "script will not work with complex repos like the management repo that"
> > echo "builds more than one package.  It expects a repo to be a single package"
> > echo "repo where the directory name and the package name are the same, and"
> > echo "where a properly formatted reponame.spec.in file exists."
> > echo
> > echo "	You must specify either release or daily in order for this script"
> > echo "to make tarballs.  If this is a daily release, the tarballs will"
> > echo "be named <component>-git.tgz and will overwrite existing tarballs."
> > echo "If this is a release build, then the tarball will be named"
> > echo "<component>-<version>.tgz and must be a new file.  In addition,"
> > echo "the script will add a new set of symbolic tags to the git repo"
> > echo "that correspond to the <component>-<version> of each tarball."
> > echo
> > echo "	If the script detects that the tag on any component already exists,"
> > echo "it will abort the release and prompt you to update the version on"
> > echo "the already tagged component.  This enforces the proper behavior of"
> > echo "treating any released tarball as set in stone so that in the future"
> > echo "you will always be able to get to any given release tarball by"
> > echo "checking out the git tag and know with certainty that it is the same"
> > echo "code as released before even if you no longer have the same tarball"
> > echo "around."
> > echo
> > echo "	As part of this process, the script will parse the <target>.spec.in"
> > echo "file and output a <target>.spec file.  Since this script isn't smart"
> > echo "enough to deal with other random changes that should have their own" 
> > echo "checkin the script will refuse to run if the current repo state is not"
> > echo "clean."
> > echo
> > echo "	NOTE: the script has no clue if you are tagging on the right branch,"
> > echo "it will however show you the git branch output so you can confirm it"
> > echo "is on the right branch before proceeding with the release."
> > echo
> > echo "	In addition to just tagging the git repo, whenever creating a release"
> > echo "there is an optional argument of either signed or a hex gpg key-id."
> > echo "If you do not pass an argument to release, then the tag will be a"
> > echo "simple git annotated tag.  If you pass signed as the argument, the"
> > echo "git tag operation will use your default signing key to sign the tag."
> > echo "Or you can pass an actual gpg key id in hex format and git will sign"
> > echo "the tag with that key."
> > echo 
> > }
> > 
> > if [ -z "$1" -o -z "$2" ]; then usage; exit 1; fi
> > 
> > if [ ! -d "$1" ]; then usage; exit 1; fi
> > 
> > TMPDIR=dist
> > if [ ! -d $TMPDIR ]; then mkdir $TMPDIR; fi
> > 
> > if [ "$2" = "daily" -o "$2" = "release" ]; then
> > 	if [ ! -f $TMPDIR/$1-$2.HEAD ]; then
> > 		touch $TMPDIR/$1-$2.HEAD
> > 	fi
> > 	NEWHEAD=`cat $TMPDIR/$1-$2.HEAD`
> > else
> > 	usage
> > 	exit 1
> > fi
> > 
> > cd "$1"
> > echo "Updating git repo..."
> > git pull
> > RESULT=$?
> > HEAD=`git log --pretty=oneline -1`
> > 
> > if [ "$RESULT" -ne 0 ]; then
> > 	echo "Failed to update the git repo cleanly, manual intervention required"
> > 	exit 1
> > fi
> 
> pull really will merge your local modifications with upstream.
> In OFED we really want just git clone, and use upstream code unmodified.

That depends on how you have your repos set up.  I keep separate repos
for tracking upstream and for doing local work.  You can run this on
either repo, and it will either give you a clean upstream copy or your
local copy merged up to date (assuming you are on a branch that merges
from upstream, otherwise if you are on a local branch, the pull will
update the repo, but not your checked out files).

This script is a little schitzophrenic at the moment because it acts
like it's both a customer of the git repo and a master of the git repo.
In truth, the release part of the script was only ever intended to be
used by a maintainer in a clean master repo.  The daily part of the
script can be used by anyone who wants to spit out quick daily builds.
But, if you are a consumer of the repo instead of the maintainer, then
for the daily builds you need to update the repo.

So, in short, the daily part is usable by anyone tracking development of
any repo and will pull from the upstream repo to keep up to date.  The
release functionality should only be used by maintainers, and then only
in their master repo.  Make more sense that way?

> > if [ "$HEAD" = "$NEWHEAD" ]; then
> > 	echo "No new commits since last tarball creation, nothing to do."
> > 	cd ..
> > 	exit 0
> > fi
> > 
> > if [ "$2" = "release" ]; then
> > 	# Is the repo clean?
> > 	git status | grep modified > /dev/null 2>&1
> > 	if [ $? = 0 ]; then
> > 		echo "There are modified files in the repo.  Please check any"
> > 		echo "changes in before proceeding."
> > 		exit 4
> > 	fi
> > 	# Since we will be tagging things, make sure we are on the right
> > 	# branch
> > 	git branch
> > 	echo -n "Is the active branch the right one to tag this release on [y/N]? "
> > 	read answer
> > 	if [ "$answer" = y -o "$answer" = Y ]; then
> > 		echo "Proceeding..."
> > 	else
> > 		echo "Please check out the right branch and run make.dist again"
> > 		exit 0
> > 	fi
> 
> See below on what we should do in OFED IMO.
> 
> > 	# Check versions to make sure that we can proceed
> > 	VERSION=`grep "AC_INIT.*$1" configure.in | cut -f 2 -d ',' | sed -e 's/ //g'`
> > 	TARBALL=$1-$VERSION.tgz
> > 	if [ -f ../$TMPDIR/$TARBALL ]; then
> > 		echo "Target $TARBALL already exists, please update the version of"
> > 		echo "$1"
> > 		exit 2
> > 	fi
> > 	if [ ! -z "`git tag -l $1-$VERSION`" ]; then
> > 		echo "A git tag already exists for $1-$VERSION.  Please change the version"
> > 		echo "of $1 so a tag replacement won't occur."
> > 		exit 3
> > 	fi
> > # On a real release, this resets the daily release starting point, on the
> > # assumption that any new daily builds will have a version number that is
> > # incrementally higher than the last officially released tarball.
> > 	RELEASE=1
> > 	echo $RELEASE > ../$TMPDIR/$1.release
> > else
> > 	DATE=`date +%Y%m%d`
> > 	if [ -f ../$TMPDIR/$1.release ]; then
> > 		RELEASE=`cat ../$TMPDIR/$1.release`
> > 		RELEASE=`expr $RELEASE + 1`
> > 	else
> > 		RELEASE=1
> > 	fi
> > 	echo $RELEASE > ../$TMPDIR/$1.release
> > 	RELEASE=0.${RELEASE}.${DATE}git
> > 	TARBALL=$1-git.tgz
> > fi
> > 
> > cd ..
> > cp -a $1 $1-$VERSION
> > [ -f $1/$1.spec.in ] && sed -e 's/@VERSION@/'$VERSION'/;s/@RELEASE@/'$RELEASE'/;s/@TARBALL@/'$TARBALL'/' < $1/$1.spec.in > $1-$VERSION/$1.spec
> 
> This, I think, is the bit that we definitely want to reuse.
> 
> > if [ -f $1-$VERSION/autogen.sh ]; then
> > 	cd $1-$VERSION
> > 	./autogen.sh
> > 	cd ..
> 
> I think we will want to call make dist too.

As long as make dist doesn't remove vital files, then sure.  Gonna have
to run configure to run make dist so you'll have to add both calls.

> > fi
> > echo "Creating $TMPDIR/$TARBALL"
> > tar -czf $TMPDIR/$TARBALL --exclude=.git $1-$VERSION
> > rm -rf $1-$VERSION
> > echo "$HEAD" > $TMPDIR/$1-$2.HEAD
> > 
> > if [ $2 = release ]; then
> > 	echo "Tagging release."
> > 	cd $1
> > 	if [ ! -z "$3" ]; then
> > 		if [ $3 = "signed" ]; then
> > 			git tag -s -m "Auto tag by make.dist on release tarball creation" $1-$VERSION
> > 		else
> > 			git tag -u "$3" -m "Auto tag by make.dist on release tarball creation" $1-$VERSION
> > 		fi
> > 	else
> > 		git tag -a -m "Auto tag by make.dist on release tarball creation" $1-$VERSION
> > 	fi
> > 	cd ..
> > fi
> 
> This takes whatever's at git head and then tags that.
> In OFED it is the other way around: maintainers tag
> the appropriate bits, release script just packages that.

Like I said earlier, the release operation is intended to only be used
by a maintainer, and in this case it just automates the process of
tagging and spitting out a tarball into the same action.  And for
clarity, it tags the head of whatever branch you are on.  So, if you've
been working in the ofed_1_2 branch, and made some changes, and are
ready for a release, it spits out the tarball and tags the ofed_1_2
branch head as the symbolic tag reponame-version.  If you want to work
on master and do a release there, then it works similarly.


-- 
Doug Ledford <dledford at redhat.com>
              GPG KeyID: CFBFF194
              http://people.redhat.com/dledford

Infiniband specific RPMs available at
              http://people.redhat.com/dledford/Infiniband
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
URL: <http://lists.openfabrics.org/pipermail/ewg/attachments/20070718/b4d26f3b/attachment.sig>