{"id":2374,"date":"2018-09-11T08:29:17","date_gmt":"2018-09-11T15:29:17","guid":{"rendered":"https:\/\/blog.mozilla.org\/security\/?p=2374"},"modified":"2018-09-12T08:56:59","modified_gmt":"2018-09-12T15:56:59","slug":"protecting-mozillas-github-repositories-from-malicious-modification","status":"publish","type":"post","link":"https:\/\/blog.mozilla.org\/security\/2018\/09\/11\/protecting-mozillas-github-repositories-from-malicious-modification\/","title":{"rendered":"Protecting Mozilla\u2019s GitHub Repositories from Malicious Modification"},"content":{"rendered":"<p>At Mozilla, we\u2019ve been working to ensure our repositories hosted on GitHub are protected from malicious modification. As the recent <a href=\"https:\/\/twitter.com\/gentoo\/status\/1012445466515173377\">Gentoo incident<\/a> demonstrated, such attacks are possible.<\/p>\n<p>Mozilla\u2019s original usage of GitHub was an alternative way to provide access to our source code. Similar to Gentoo, the \u201csource of truth\u201d repositories were maintained on our own infrastructure. While we still do utilize our own infrastructure for much of the Firefox browser code, Mozilla has many projects which exist only on GitHub. While some of those project are just experiments, others are used in production (e.g. <a href=\"https:\/\/github.com\/topics\/firefox-accounts\">Firefox Accounts<\/a>). We need to protect such \u201csensitive repositories\u201d against malicious modification, while also keeping the barrier to contribution as low as practical.<\/p>\n<p>This describes the mitigations we have put in place to prevent shipping (or deploying) from a compromised repository. We are sharing both our findings and <a href=\"https:\/\/github.com\/mozilla-services\/GitHub-Audit\">some tooling<\/a> to support auditing. These add the protections with minimal disruption to common GitHub workflows.<\/p>\n<p>The risk we are addressing here is the compromise of a GitHub user\u2019s account, via mechanisms unique to GitHub. As the Gentoo and other incidents show, when a user account is compromised, any resource the user has permissions to can be affected.<\/p>\n<h2>Overview<\/h2>\n<p>GitHub is a wonderful ecosystem with many extensions, or \u201capps\u201d, to make certain workflows easier. Apps obtain permission from a user to perform actions on their behalf. An app can ask for permissions including modifying or adding additional user credentials. GitHub makes these permission requests transparent, and requires the user to approve via the web interface, but not all users may be conversant with the implications of granting those permissions to an app. They also may not make the connection that approving such permissions for their personal repositories could grant the same for access to any repository across GitHub where they can make changes.<\/p>\n<p>Excessive permissions can expose repositories with sensitive information to risks, without the repository admins being aware of those risks. The best a repository admin can do is detect a fraudulent modification after it has been pushed back to GitHub. Neither GitHub nor git can be configured to prevent or highlight this sort of malicious modification; external monitoring is required.<\/p>\n<h2>Implementation<\/h2>\n<p>The following are taken from our <a href=\"https:\/\/wiki.mozilla.org\/GitHub\/Repository_Security\">approach<\/a> to addressing this concern, with Mozilla specifics removed. As much as possible, we borrow from the web\u2019s best practices, used features of the GitHub platform, and tried to avoid adding friction to the daily developer workflows.<\/p>\n<h3>Organization recommendations:<\/h3>\n<ul>\n<li>2FA must be required for all members and collaborators.<\/li>\n<li>All users, or at least those with elevated permissions:\n<ul>\n<li>Should have contact methods (email, IM) given to the org owners or repo admins. (GitHub allows Users to hide their contact info for privacy.)<\/li>\n<li>Should understand it is their responsibility to inform the org owners or repo admins if they ever suspect their account has been compromised. (E.g. laptop stolen)<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<h3>Repository recommendations:<\/h3>\n<ul>\n<li>Sensitive repositories should only be hosted in an organization that follows the recommendations above.<\/li>\n<li>Production branches should be identified and configured:\n<ul>\n<li>To not allow force pushes.<\/li>\n<li>Only give commit privileges to a small set of users.<\/li>\n<li>Enforce those restrictions on admins &amp; owners as well.<\/li>\n<li>Require all commits to be GPG signed, using keys known in advance.<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<h3>Workflow recommendations:<\/h3>\n<ul>\n<li>Deployments, releases, and other audit-worthy events, should be marked with a signed tag from a GPG key known in advance.<\/li>\n<li>Deployment and release criteria should include an audit of all signed commits and tags to ensure they are signed with the expected keys.<\/li>\n<\/ul>\n<p>There are some costs to implementing these protections &#8211; especially those around the signing of commits. We have developed some internal tooling to help with auditing the configurations, and plan to add tools for auditing commits. Those tools are available in the <a href=\"https:\/\/github.com\/mozilla-services\/GitHub-Audit\">mozilla-services\/GitHub-Audit<\/a> repository.<\/p>\n<p><a href=\"https:\/\/blog.mozilla.org\/security\/files\/2018\/09\/image1.png\"><img decoding=\"async\" loading=\"lazy\" class=\"alignnone size-medium wp-image-2375\" src=\"https:\/\/blog.mozilla.org\/security\/files\/2018\/09\/image1-252x158.png\" alt=\"Image of README contents\" width=\"252\" height=\"158\" srcset=\"https:\/\/blog.mozilla.org\/security\/files\/2018\/09\/image1-252x158.png 252w, https:\/\/blog.mozilla.org\/security\/files\/2018\/09\/image1-768x480.png 768w, https:\/\/blog.mozilla.org\/security\/files\/2018\/09\/image1-600x375.png 600w, https:\/\/blog.mozilla.org\/security\/files\/2018\/09\/image1.png 953w\" sizes=\"(max-width: 252px) 100vw, 252px\" \/><\/a><\/p>\n<p>Here\u2019s an example of using the audit tools. First we obtain a local copy of the data we\u2019ll need for the \u201cocto_org\u201d organization, and then we report on each repository:<\/p>\n<blockquote><p><code>$ .\/get_branch_protections.py octo_org<br \/>\n2018-07-06 13:52:40,584 INFO: Running as ms_octo_cat<br \/>\n2018-07-06 13:52:40,854 INFO: Gathering branch protection data. (calls remaining 4992).<br \/>\n2018-07-06 13:52:41,117 INFO: Starting on org octo_org. (calls remaining 4992).<br \/>\n2018-07-06 13:52:59,116 INFO: Finished gathering branch protection data (calls remaining 4947).<\/code><\/p><\/blockquote>\n<p>Now with the data cached locally, we can run as many reports as we\u2019d like. For example, we have written one report showing which of the above recommendations are being followed:<\/p>\n<blockquote><p><code>$ .\/report_branch_status.py --header octo_org.db.json<br \/>\nname,protected,restricted,enforcement,signed,team_used<br \/>\nocto_org\/react-starter,True,False,False,False,False<br \/>\nocto_org\/node-starter,False,False,False,False,False<br \/>\n<\/code><\/p><\/blockquote>\n<p>We can see that only \u201cocto_org\/react-starter\u201d has enabled protection against force pushes on it\u2019s production branch. The final output is in CSV format, for easy pasting into spreadsheets.<\/p>\n<h2>How you can help<\/h2>\n<p>We are still rolling out these recommendations across our teams, and learning as we go. If you think our <a href=\"https:\/\/wiki.mozilla.org\/GitHub\/Repository_Security\">Repository Security<\/a> recommendations are appropriate for your situation, please help us make implementation easier. Add your experience to the <a href=\"https:\/\/wiki.mozilla.org\/GitHub\/Repository_Security\">Tips &#8216;n Tricks<\/a> page, or <a href=\"https:\/\/github.com\/mozilla-services\/GitHub-Audit\/issues\">open issues<\/a> on our <a href=\"https:\/\/github.com\/mozilla-services\/GitHub-Audit\">GitHub-Audit<\/a> repository.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>At Mozilla, we\u2019ve been working to ensure our repositories hosted on GitHub are protected from malicious modification. As the recent Gentoo incident demonstrated, such attacks are possible. Mozilla\u2019s original usage &hellip; <a class=\"go\" href=\"https:\/\/blog.mozilla.org\/security\/2018\/09\/11\/protecting-mozillas-github-repositories-from-malicious-modification\/\">Read more<\/a><\/p>\n","protected":false},"author":404,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[69],"tags":[],"coauthors":[320792],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v22.5 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Protecting Mozilla\u2019s GitHub Repositories from Malicious Modification - Mozilla Security Blog<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/blog.mozilla.org\/security\/2018\/09\/11\/protecting-mozillas-github-repositories-from-malicious-modification\/\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Hal Wine\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"4 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/blog.mozilla.org\/security\/2018\/09\/11\/protecting-mozillas-github-repositories-from-malicious-modification\/\",\"url\":\"https:\/\/blog.mozilla.org\/security\/2018\/09\/11\/protecting-mozillas-github-repositories-from-malicious-modification\/\",\"name\":\"Protecting Mozilla\u2019s GitHub Repositories from Malicious Modification - Mozilla Security Blog\",\"isPartOf\":{\"@id\":\"https:\/\/blog.mozilla.org\/security\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/blog.mozilla.org\/security\/2018\/09\/11\/protecting-mozillas-github-repositories-from-malicious-modification\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/blog.mozilla.org\/security\/2018\/09\/11\/protecting-mozillas-github-repositories-from-malicious-modification\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/blog.mozilla.org\/security\/files\/2018\/09\/image1-252x158.png\",\"datePublished\":\"2018-09-11T15:29:17+00:00\",\"dateModified\":\"2018-09-12T15:56:59+00:00\",\"author\":{\"@id\":\"https:\/\/blog.mozilla.org\/security\/#\/schema\/person\/d1ead927c9af863a6b6c19abcfdb1c4c\"},\"breadcrumb\":{\"@id\":\"https:\/\/blog.mozilla.org\/security\/2018\/09\/11\/protecting-mozillas-github-repositories-from-malicious-modification\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/blog.mozilla.org\/security\/2018\/09\/11\/protecting-mozillas-github-repositories-from-malicious-modification\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/blog.mozilla.org\/security\/2018\/09\/11\/protecting-mozillas-github-repositories-from-malicious-modification\/#primaryimage\",\"url\":\"https:\/\/blog.mozilla.org\/security\/files\/2018\/09\/image1.png\",\"contentUrl\":\"https:\/\/blog.mozilla.org\/security\/files\/2018\/09\/image1.png\",\"width\":953,\"height\":596,\"caption\":\"Image of README contents\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/blog.mozilla.org\/security\/2018\/09\/11\/protecting-mozillas-github-repositories-from-malicious-modification\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/blog.mozilla.org\/security\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Protecting Mozilla\u2019s GitHub Repositories from Malicious Modification\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/blog.mozilla.org\/security\/#website\",\"url\":\"https:\/\/blog.mozilla.org\/security\/\",\"name\":\"Mozilla Security Blog\",\"description\":\"\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/blog.mozilla.org\/security\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/blog.mozilla.org\/security\/#\/schema\/person\/d1ead927c9af863a6b6c19abcfdb1c4c\",\"name\":\"Hal Wine\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/blog.mozilla.org\/security\/#\/schema\/person\/image\/f9f8e5b90bdd0561a811c9e64561b7ca\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/eb8ae5dcc06a10bf5c9d7661dababa6d?s=96&d=identicon&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/eb8ae5dcc06a10bf5c9d7661dababa6d?s=96&d=identicon&r=g\",\"caption\":\"Hal Wine\"}}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Protecting Mozilla\u2019s GitHub Repositories from Malicious Modification - Mozilla Security Blog","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/blog.mozilla.org\/security\/2018\/09\/11\/protecting-mozillas-github-repositories-from-malicious-modification\/","twitter_misc":{"Written by":"Hal Wine","Est. reading time":"4 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/blog.mozilla.org\/security\/2018\/09\/11\/protecting-mozillas-github-repositories-from-malicious-modification\/","url":"https:\/\/blog.mozilla.org\/security\/2018\/09\/11\/protecting-mozillas-github-repositories-from-malicious-modification\/","name":"Protecting Mozilla\u2019s GitHub Repositories from Malicious Modification - Mozilla Security Blog","isPartOf":{"@id":"https:\/\/blog.mozilla.org\/security\/#website"},"primaryImageOfPage":{"@id":"https:\/\/blog.mozilla.org\/security\/2018\/09\/11\/protecting-mozillas-github-repositories-from-malicious-modification\/#primaryimage"},"image":{"@id":"https:\/\/blog.mozilla.org\/security\/2018\/09\/11\/protecting-mozillas-github-repositories-from-malicious-modification\/#primaryimage"},"thumbnailUrl":"https:\/\/blog.mozilla.org\/security\/files\/2018\/09\/image1-252x158.png","datePublished":"2018-09-11T15:29:17+00:00","dateModified":"2018-09-12T15:56:59+00:00","author":{"@id":"https:\/\/blog.mozilla.org\/security\/#\/schema\/person\/d1ead927c9af863a6b6c19abcfdb1c4c"},"breadcrumb":{"@id":"https:\/\/blog.mozilla.org\/security\/2018\/09\/11\/protecting-mozillas-github-repositories-from-malicious-modification\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/blog.mozilla.org\/security\/2018\/09\/11\/protecting-mozillas-github-repositories-from-malicious-modification\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/blog.mozilla.org\/security\/2018\/09\/11\/protecting-mozillas-github-repositories-from-malicious-modification\/#primaryimage","url":"https:\/\/blog.mozilla.org\/security\/files\/2018\/09\/image1.png","contentUrl":"https:\/\/blog.mozilla.org\/security\/files\/2018\/09\/image1.png","width":953,"height":596,"caption":"Image of README contents"},{"@type":"BreadcrumbList","@id":"https:\/\/blog.mozilla.org\/security\/2018\/09\/11\/protecting-mozillas-github-repositories-from-malicious-modification\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/blog.mozilla.org\/security\/"},{"@type":"ListItem","position":2,"name":"Protecting Mozilla\u2019s GitHub Repositories from Malicious Modification"}]},{"@type":"WebSite","@id":"https:\/\/blog.mozilla.org\/security\/#website","url":"https:\/\/blog.mozilla.org\/security\/","name":"Mozilla Security Blog","description":"","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/blog.mozilla.org\/security\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/blog.mozilla.org\/security\/#\/schema\/person\/d1ead927c9af863a6b6c19abcfdb1c4c","name":"Hal Wine","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/blog.mozilla.org\/security\/#\/schema\/person\/image\/f9f8e5b90bdd0561a811c9e64561b7ca","url":"https:\/\/secure.gravatar.com\/avatar\/eb8ae5dcc06a10bf5c9d7661dababa6d?s=96&d=identicon&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/eb8ae5dcc06a10bf5c9d7661dababa6d?s=96&d=identicon&r=g","caption":"Hal Wine"}}]}},"_links":{"self":[{"href":"https:\/\/blog.mozilla.org\/security\/wp-json\/wp\/v2\/posts\/2374"}],"collection":[{"href":"https:\/\/blog.mozilla.org\/security\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.mozilla.org\/security\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.mozilla.org\/security\/wp-json\/wp\/v2\/users\/404"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.mozilla.org\/security\/wp-json\/wp\/v2\/comments?post=2374"}],"version-history":[{"count":0,"href":"https:\/\/blog.mozilla.org\/security\/wp-json\/wp\/v2\/posts\/2374\/revisions"}],"wp:attachment":[{"href":"https:\/\/blog.mozilla.org\/security\/wp-json\/wp\/v2\/media?parent=2374"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.mozilla.org\/security\/wp-json\/wp\/v2\/categories?post=2374"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.mozilla.org\/security\/wp-json\/wp\/v2\/tags?post=2374"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/blog.mozilla.org\/security\/wp-json\/wp\/v2\/coauthors?post=2374"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}