I’ve recently been uploading packages to tooltool for my work on Rust-in-Gecko and Android toolchains. The steps I usually follow are:
- Put together tarball of files.
- Call tooltool.py from build-tooltool to create a tooltool manifest.
- Upload files to tooltool with said manifest.
- Copy bits from said manifest into one of the manifest files automation uses.
- Do try push with new manifest.
- Admire my completely green try push.
That would be the ideal, anyway. What usually happens at step 4 is that I forget a comma, or I forget a field in the manifest, and so step 5 winds up going awry, and I end up taking several times as long as I would have liked.
After running into this again today, I decided to implement some minimal validation for automation manifests. I use a fork of gecko-dev for development, as I prefer Git to Mercurial. Git supports running programs when certain things occur; these programs are known as hooks and are usually implemented as shell scripts. The hook I’m interested in is the pre-commit hook, which is looked for at .git/hooks/pre-commit in any git repository. Repositories come with a sample hook for every hook supported by Git, so I started with:
cp .git/hooks/pre-commit.sample .git/hooks/pre-commit
The sample pre-commit hook checks trailing whitespace in files, which I sometimes leave around, especially when I’m editing Python, and can check for non-ASCII filenames being added. I then added the following lines to that file:
if git diff --cached --name-only | grep -q releng.manifest; then
for f in $(git diff --cached --name-only | grep releng.manifest); do
if ! python -<<EOF
import json
import sys
try:
with open("$f", 'r') as f:
json.loads(f.read())
sys.exit(0)
except:
sys.exit(1)
EOF
then
echo $f is not valid JSON
exit 1
fi
done
fi
In prose, we’re checking to see if the current commit has any releng.manifest files being changed in any way. If so, then we’ll try parsing each of those files as JSON, and throwing an error if one doesn’t parse.
There are several ways this check could be more robust:
- The check will error if a commit is removing a releng.manifest, because that file won’t exist for the script to check;
- The check could ensure that the unpack field is set for all files, as the manifest file used for the upload in step 3, above, doesn’t include that field: it needs to be added manually.
- The check could ensure that all of the digest fields are the correct length for the specified digest in use.
- …and so on.
So far, though, simple syntax errors are the greatest source of pain for me, so that’s what’s getting checked for. (Mismatched sizes have also been an issue, but I’m unsure of how to check that…)
What pre-commit hooks have you found useful in your own projects?