{"id":463,"date":"2016-07-29T13:27:44","date_gmt":"2016-07-29T17:27:44","guid":{"rendered":"http:\/\/blog.mozilla.org\/nfroyd\/?p=463"},"modified":"2016-07-29T13:27:44","modified_gmt":"2016-07-29T17:27:44","slug":"a-git-pre-commit-hook-for-tooltool-manifest-checking","status":"publish","type":"post","link":"https:\/\/blog.mozilla.org\/nfroyd\/2016\/07\/29\/a-git-pre-commit-hook-for-tooltool-manifest-checking\/","title":{"rendered":"a git pre-commit hook for tooltool manifest checking"},"content":{"rendered":"

I’ve recently been uploading packages to tooltool<\/a> for my work on Rust-in-Gecko and Android toolchains. The steps I usually follow are:<\/p>\n

    \n
  1. Put together tarball of files.<\/li>\n
  2. Call tooltool.py from build-tooltool<\/a> to create a tooltool manifest.<\/li>\n
  3. Upload files to tooltool with said manifest.<\/li>\n
  4. Copy bits from said manifest into one of the manifest files<\/a> automation uses.<\/li>\n
  5. Do try push with new manifest.<\/li>\n
  6. Admire my completely green try push.<\/li>\n<\/ol>\n

    That would be the ideal, anyway.\u00a0 What usually happens at step 4 is that I forget a comma, or I forget a field in the manifest, and so step 5 winds up going awry, and I end up taking several times as long as I would have liked.<\/p>\n

    After running into this again today, I decided to implement some minimal validation for automation manifests.\u00a0 I use a fork of gecko-dev<\/a> for development, as I prefer Git to Mercurial. Git supports running programs when certain things occur; these programs are known as hooks<\/cite> and are usually implemented as shell scripts. The hook I’m interested in is the pre-commit<\/tt> hook, which is looked for at .git\/hooks\/pre-commit<\/tt> in any git repository. Repositories come with a sample hook for every hook supported by Git, so I started with:<\/p>\n

    cp .git\/hooks\/pre-commit.sample .git\/hooks\/pre-commit\r\n<\/pre>\n
    The sample pre-commit hook checks trailing whitespace in files, which I sometimes leave around, especially when I’m editing Python, and can check for non-ASCII filenames being added.\u00a0 I then added the following lines to that file:<\/p>\n
    if git diff --cached --name-only | grep -q releng.manifest; then\r\n    for f in $(git diff --cached --name-only | grep releng.manifest); do\r\n\tif ! python -<<EOF\r\nimport json\r\nimport sys\r\ntry:\r\n    with open(\"$f\", 'r') as f:\r\n        json.loads(f.read())\r\n    sys.exit(0)\r\nexcept:\r\n    sys.exit(1)\r\nEOF\r\n\t    then\r\n\t    echo $f is not valid JSON\r\n\t    exit 1\r\n\tfi\r\n     done\r\nfi\r\n\r\n<\/pre>\n
    In prose, we’re checking to see if the current commit has any releng.manifest<\/tt> files being changed in any way. If so, then we’ll try parsing each of those files as JSON, and throwing an error if one doesn’t parse.<\/p>\n
    There are several ways this check could be more robust:<\/p>\n
      \n
    • The check will error if a commit is removing a releng.manifest<\/tt>, because that file won’t exist for the script to check;<\/li>\n
    • The check could ensure that the unpack<\/tt> field is set for all files, as the manifest file used for the upload in step 3, above, doesn’t include that field: it needs to be added manually.<\/li>\n
    • The check could ensure that all of the digest fields are the correct length for the specified digest in use.<\/li>\n
    • …and so on.<\/li>\n<\/ul>\n
      So far, though, simple syntax errors are the greatest source of pain for me, so that’s what’s getting checked for.\u00a0 (Mismatched sizes have also been an issue, but I’m unsure of how to check that…)<\/p>\n
      What pre-commit hooks have you found useful in your own projects?<\/p>\n","protected":false},"excerpt":{"rendered":"
      I’ve recently been uploading packages to tooltool for my work on Rust-in-Gecko and Android toolchains. The steps I usually follow are: Put together tarball of files. Call tooltool.py from build-tooltool to create a tooltool manifest. Upload files to tooltool with said manifest. Copy bits from said manifest into one of the manifest files automation uses. […]<\/p>\n","protected":false},"author":320,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[13429,5,706,72450],"_links":{"self":[{"href":"https:\/\/blog.mozilla.org\/nfroyd\/wp-json\/wp\/v2\/posts\/463"}],"collection":[{"href":"https:\/\/blog.mozilla.org\/nfroyd\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.mozilla.org\/nfroyd\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.mozilla.org\/nfroyd\/wp-json\/wp\/v2\/users\/320"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.mozilla.org\/nfroyd\/wp-json\/wp\/v2\/comments?post=463"}],"version-history":[{"count":0,"href":"https:\/\/blog.mozilla.org\/nfroyd\/wp-json\/wp\/v2\/posts\/463\/revisions"}],"wp:attachment":[{"href":"https:\/\/blog.mozilla.org\/nfroyd\/wp-json\/wp\/v2\/media?parent=463"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.mozilla.org\/nfroyd\/wp-json\/wp\/v2\/categories?post=463"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.mozilla.org\/nfroyd\/wp-json\/wp\/v2\/tags?post=463"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}