{"id":11282,"date":"2018-03-08T07:39:32","date_gmt":"2018-03-08T15:39:32","guid":{"rendered":"https:\/\/blog.mozilla.org\/?p=11282"},"modified":"2018-03-08T14:48:05","modified_gmt":"2018-03-08T22:48:05","slug":"gender-bias-code-reviews","status":"publish","type":"post","link":"https:\/\/blog.mozilla.org\/blogarchive\/blog\/2018\/03\/08\/gender-bias-code-reviews\/","title":{"rendered":"Mozilla experiment aims to reduce bias in code reviews"},"content":{"rendered":"

Mozilla is kicking off a new experiment for International Women\u2019s Day<\/a>, looking at ways to make open source software projects friendlier to women and racial minorities. Its first target? The code review process.<\/p>\n

The experiment has two parts: there\u2019s an effort to build an extension for Firefox that gives programmers a way to anonymize pull requests, so reviewers will see the code itself, but not necessarily the identity of the person who wrote it. The second part is gathering data about how sites like Bugzilla and GitHub work, to see how \u201cblind reviews\u201d might fit into established workflows.<\/p>\n

The idea behind the experiment is a simple one: If the identity of a coder is shielded, there\u2019s less opportunity for unconscious gender or racial bias to creep into decision-making processes. It\u2019s similar to an experiment that began the 1970s, when U.S. symphonies began using blind auditions<\/a> to hire musicians. Instead of hand-picking known proteges, juries listened to candidates playing behind a screen. That change gave women an edge: They were 50 percent more likely to make it past the first audition if their gender wasn\u2019t known. Over the decades, women gained ground, going from 10% representation in orchestras to 35 percent in the 1990s.<\/p>\n

Mozilla is hoping to use a similar mechanism \u2013 anonymity \u2013 to make the code review process more egalitarian, especially in open source projects that rely on volunteers. Female programmers are underrepresented in the tech industry overall, and much less likely to participate in open source projects. Women account for 22 percent of computer programmers working in the U.S<\/a>, but only 11 percent of them contribute to open source projects. A 2016 study of more than 50 GitHub repositories<\/a> revealed that, in fact, women\u2019s pull requests were approved more often than their male counterparts \u2013 nearly 3% more often. However, if their gender was known, female coders were .8% less likely to have their code accepted.<\/p>\n

What\u2019s going on? There are two possible answers. One is that people have an unconscious bias against women who write code. If that\u2019s the case, there\u2019s a test you can take<\/a> to find out: Do I have trouble associating women with scientific and technical roles?<\/p>\n

Then there is a darker interpretation: that men are acting deliberately to keep computer programming a boy\u2019s club, rather than accepting high-quality input from women, racial minorities, transgender individuals, and economically underprivileged folks.<\/p>\n

A Commitment to Diversity<\/b><\/h3>\n

What does it mean to be inclusive and welcoming to female software engineers? It means, first of all, taking stock of what kind of people we think will do the best job creating software.<\/p>\n

\u201cWhen we talk about diversity and inclusion, it helps to understand the \u201cdefault persona\u201d that we\u2019re dealing with,\u201d said Emma Humphries, an engineering program manager and bugmaster at Mozilla. \u201cWe think of a typical software programmer as a white male with a college education and full-time job that affords him the opportunity to do open source work, either as a hobby or as part of a job that directly supports open source projects.\u201d<\/p>\n

This default group comes with a lot of assumptions, Humphries said. They have access to high-bandwidth internet and computers that can run a compiler and development tools, as opposed to a smartphone or a Chromebook. \u201cWhen we talk about including people outside of this idealized group, we get pushback based on those assumptions,\u201d she said.<\/p>\n

For decades, white men have dominated the ranks of software developers in the U.S. But that\u2019s starting to change. The question is, how can we deal with biases that have been years in the making?<\/p>\n

Inventing a Solution<\/b><\/h3>\n

Mozilla\u2019s Don Marti<\/a>, a strategist for Mozilla\u2019s Open Innovation group, decided to take on the challenge. Marti\u2019s hypothesis was: If I don\u2019t know who requested the code review, then I won\u2019t have any preconceived notions about how good or bad the code might be. Marti recruited Tomislav Jovanovic<\/a>, a ten-year veteran of Mozilla\u2019s open source projects, to create a blinding mechanism for code repositories like GitHub. That way, reviewers can\u2019t see the gender, location, user name, icon, or avatar associated with a particular code submission.<\/p>\n

Jovanovic was eager to contribute. \u201cI have been following tech industry diversity efforts for a long time, so the idea of using a browser extension to help with that seemed intriguing,\u201d he said. \u201cEven if we are explicitly trying to be fair, most of us still have some unconscious bias that may influence our reviews based on the author’s perceived gender, race, and\/or authority.\u201d<\/p>\n

Bias goes the other way as well, in that reviewers might be less critical of work by their peers and colleagues. \u201cOur mind often tricks us into skimming code submitted by known and trusted contributors,\u201d Jovanovic said. \u201cSo hiding their identities can lead to more thorough reviews, and ultimately better code overall.\u201d<\/p>\n

Test and Measure<\/b><\/h3>\n

An early prototype of a Firefox Quantum add-on can redact the identity of a review requestor on Bugzilla and the Pull Request author on GitHub. It also provides the ability to uncover that identity, if you prefer to get a first look at code without author info, then greet a new contributor or refer to a familiar contributor by name in your comments. Early users can also flag the final review as performed in \u201cblind mode\u201d, helping gather information about who is getting their code accepted and measuring how long the process takes.<\/p>\n

Jovanovic is also gathering user input about what types of reviews could be blind by default and how to use a browser extension to streamline common workflows in GitHub. It\u2019s still early days, but so far, feedback on the tests has been overwhelmingly positive.<\/p>\n

Having a tool that can protect coders, no matter who they are, is a great first step to building a meritocracy in a rough-and-tumble programmer culture. In recent years, there have been a number of high-profile cases of harassment at companies like Google<\/a>, GitHub<\/a>, Facebook<\/a>, and others. An even better step would be if companies, projects, and code repositories would adopt blind reviews as a mandatory part of their code review processes.<\/p>\n

For folks who are committed to open source software development, the GitHub study was something of a downer. \u201cI thought open source was this great democratizing force in the world,\u201d said Larissa Shapiro, Head of Global Diversity and Inclusion at Mozilla. \u201cBut it does seem that there is a pervasive pattern of gender bias in tech, and it\u2019s even worse in the open source culture.\u201d<\/p>\n

Small Bias, Big Impact<\/b><\/h3>\n

Bias in any context adds up to a whole lot more than hurt feelings. There are far-reaching consequences to having gender and racial bias in peer reviews of code. For the programmers, completing software projects \u2013 including review and approval of their code \u2013 is the way to be productive and therefore valued. If a woman is not able to merge her code into a project for whatever reason, it imperils her job.<\/p>\n

\u201cIn the software world, code review is a primary tool that we use to communicate, to assign value to our work, and to establish the pecking order at work in our industry,\u201d Shapiro said.<\/p>\n

Ironically, demand for programming talent is high and expected to go higher. Businesses need programmers to help them build new applications, create and deliver quality content, and offer novel ways to communicate and share experiences online. According to the group Women Who Code<\/a>, companies could see a massive shortfall of technical talent just two years from now<\/a>, with as many as a million jobs going unfilled. At 59% of the U.S. workforce, women could help with that shortfall. However, they make up just 30% of workers in the tech industry today, and are leaving it faster than any other sector. So we\u2019re not really heading in the right direction, in terms of encouraging women and other underrepresented groups to take on technical roles.<\/p>\n

Maybe a clever bit of browser code can start to turn the tide. At the very least, we should all be invested in making open source more open<\/a> to all, and accept high-quality contributions, no matter who or where they come from. The upside is there: Eliminate bias. Build better communities. Cultivate talent. Get better code, and complete projects faster. What\u2019s not to like about that?<\/p>\n

You can sign up for an email alert<\/a> when the final version of the Blind Reviews Experiment browser extension becomes available later this year, and we’ll ask for your feedback on how to make the extensions as efficient and effective as possible.<\/p>\n

 <\/p>\n","protected":false},"excerpt":{"rendered":"

Mozilla is kicking off a new experiment for International Women\u2019s Day, looking at ways to make open source software projects friendlier to women and racial minorities. Its first target? The code review process. Read more<\/a><\/p>\n","protected":false},"author":1462,"featured_media":11283,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[119,27801,1,67,66,27945],"tags":[317826,317830,317828,317827,317829],"coauthors":[290417],"yoast_head":"\nMozilla experiment aims to reduce bias in code reviews<\/title>\n<meta name=\"description\" content=\"Mozilla is kicking off a new experiment for International Women\u2019s Day, looking at ways to make open source software projects friendlier to women and racial minorities. Its first target? The code review process.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/blog.mozilla.org\/blogarchive\/blog\/2018\/03\/08\/gender-bias-code-reviews\/\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Judy McConnell\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"7 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/blog.mozilla.org\/blogarchive\/blog\/2018\/03\/08\/gender-bias-code-reviews\/\",\"url\":\"https:\/\/blog.mozilla.org\/blogarchive\/blog\/2018\/03\/08\/gender-bias-code-reviews\/\",\"name\":\"Mozilla experiment aims to reduce bias in code reviews\",\"isPartOf\":{\"@id\":\"https:\/\/blog.mozilla.org\/blogarchive\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/blog.mozilla.org\/blogarchive\/blog\/2018\/03\/08\/gender-bias-code-reviews\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/blog.mozilla.org\/blogarchive\/blog\/2018\/03\/08\/gender-bias-code-reviews\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/blog.mozilla.org\/blogarchive\/files\/2018\/03\/blind-review-hero.png\",\"datePublished\":\"2018-03-08T15:39:32+00:00\",\"dateModified\":\"2018-03-08T22:48:05+00:00\",\"author\":{\"@id\":\"https:\/\/blog.mozilla.org\/blogarchive\/#\/schema\/person\/e78adae84886c609040b3a8ebd49b9a3\"},\"description\":\"Mozilla is kicking off a new experiment for International Women\u2019s Day, looking at ways to make open source software projects friendlier to women and racial minorities. Its first target? The code review process.\",\"breadcrumb\":{\"@id\":\"https:\/\/blog.mozilla.org\/blogarchive\/blog\/2018\/03\/08\/gender-bias-code-reviews\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/blog.mozilla.org\/blogarchive\/blog\/2018\/03\/08\/gender-bias-code-reviews\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/blog.mozilla.org\/blogarchive\/blog\/2018\/03\/08\/gender-bias-code-reviews\/#primaryimage\",\"url\":\"https:\/\/blog.mozilla.org\/blogarchive\/files\/2018\/03\/blind-review-hero.png\",\"contentUrl\":\"https:\/\/blog.mozilla.org\/blogarchive\/files\/2018\/03\/blind-review-hero.png\",\"width\":1400,\"height\":700},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/blog.mozilla.org\/blogarchive\/blog\/2018\/03\/08\/gender-bias-code-reviews\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/blog.mozilla.org\/blogarchive\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Mozilla experiment aims to reduce bias in code reviews\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/blog.mozilla.org\/blogarchive\/#website\",\"url\":\"https:\/\/blog.mozilla.org\/blogarchive\/\",\"name\":\"The Mozilla Blog (Archived)\",\"description\":\"Dispatches from the Internet frontier.\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/blog.mozilla.org\/blogarchive\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/blog.mozilla.org\/blogarchive\/#\/schema\/person\/e78adae84886c609040b3a8ebd49b9a3\",\"name\":\"Judy McConnell\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/blog.mozilla.org\/blogarchive\/#\/schema\/person\/image\/bbd3cca6bf7d13cdb4506cc327abdcc9\",\"url\":\"https:\/\/hacks.mozilla.org\/files\/2017\/12\/Judy-DeMocker_avatar_1512521150-96x96.jpg\",\"contentUrl\":\"https:\/\/hacks.mozilla.org\/files\/2017\/12\/Judy-DeMocker_avatar_1512521150-96x96.jpg\",\"caption\":\"Judy McConnell\"},\"description\":\"Judy is a Technical Writer working with Mozilla. She has written about open source software for many years and now focuses on the open web platform.\",\"sameAs\":[\"https:\/\/x.com\/judydemocker\"],\"url\":\"https:\/\/blog.mozilla.org\/blogarchive\/blog\/author\/jdemockermozilla-com\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Mozilla experiment aims to reduce bias in code reviews","description":"Mozilla is kicking off a new experiment for International Women\u2019s Day, looking at ways to make open source software projects friendlier to women and racial minorities. Its first target? The code review process.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/blog.mozilla.org\/blogarchive\/blog\/2018\/03\/08\/gender-bias-code-reviews\/","twitter_misc":{"Written by":"Judy McConnell","Est. reading time":"7 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/blog.mozilla.org\/blogarchive\/blog\/2018\/03\/08\/gender-bias-code-reviews\/","url":"https:\/\/blog.mozilla.org\/blogarchive\/blog\/2018\/03\/08\/gender-bias-code-reviews\/","name":"Mozilla experiment aims to reduce bias in code reviews","isPartOf":{"@id":"https:\/\/blog.mozilla.org\/blogarchive\/#website"},"primaryImageOfPage":{"@id":"https:\/\/blog.mozilla.org\/blogarchive\/blog\/2018\/03\/08\/gender-bias-code-reviews\/#primaryimage"},"image":{"@id":"https:\/\/blog.mozilla.org\/blogarchive\/blog\/2018\/03\/08\/gender-bias-code-reviews\/#primaryimage"},"thumbnailUrl":"https:\/\/blog.mozilla.org\/blogarchive\/files\/2018\/03\/blind-review-hero.png","datePublished":"2018-03-08T15:39:32+00:00","dateModified":"2018-03-08T22:48:05+00:00","author":{"@id":"https:\/\/blog.mozilla.org\/blogarchive\/#\/schema\/person\/e78adae84886c609040b3a8ebd49b9a3"},"description":"Mozilla is kicking off a new experiment for International Women\u2019s Day, looking at ways to make open source software projects friendlier to women and racial minorities. Its first target? The code review process.","breadcrumb":{"@id":"https:\/\/blog.mozilla.org\/blogarchive\/blog\/2018\/03\/08\/gender-bias-code-reviews\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/blog.mozilla.org\/blogarchive\/blog\/2018\/03\/08\/gender-bias-code-reviews\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/blog.mozilla.org\/blogarchive\/blog\/2018\/03\/08\/gender-bias-code-reviews\/#primaryimage","url":"https:\/\/blog.mozilla.org\/blogarchive\/files\/2018\/03\/blind-review-hero.png","contentUrl":"https:\/\/blog.mozilla.org\/blogarchive\/files\/2018\/03\/blind-review-hero.png","width":1400,"height":700},{"@type":"BreadcrumbList","@id":"https:\/\/blog.mozilla.org\/blogarchive\/blog\/2018\/03\/08\/gender-bias-code-reviews\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/blog.mozilla.org\/blogarchive\/"},{"@type":"ListItem","position":2,"name":"Mozilla experiment aims to reduce bias in code reviews"}]},{"@type":"WebSite","@id":"https:\/\/blog.mozilla.org\/blogarchive\/#website","url":"https:\/\/blog.mozilla.org\/blogarchive\/","name":"The Mozilla Blog (Archived)","description":"Dispatches from the Internet frontier.","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/blog.mozilla.org\/blogarchive\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/blog.mozilla.org\/blogarchive\/#\/schema\/person\/e78adae84886c609040b3a8ebd49b9a3","name":"Judy McConnell","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/blog.mozilla.org\/blogarchive\/#\/schema\/person\/image\/bbd3cca6bf7d13cdb4506cc327abdcc9","url":"https:\/\/hacks.mozilla.org\/files\/2017\/12\/Judy-DeMocker_avatar_1512521150-96x96.jpg","contentUrl":"https:\/\/hacks.mozilla.org\/files\/2017\/12\/Judy-DeMocker_avatar_1512521150-96x96.jpg","caption":"Judy McConnell"},"description":"Judy is a Technical Writer working with Mozilla. She has written about open source software for many years and now focuses on the open web platform.","sameAs":["https:\/\/x.com\/judydemocker"],"url":"https:\/\/blog.mozilla.org\/blogarchive\/blog\/author\/jdemockermozilla-com\/"}]}},"_links":{"self":[{"href":"https:\/\/blog.mozilla.org\/blogarchive\/wp-json\/wp\/v2\/posts\/11282"}],"collection":[{"href":"https:\/\/blog.mozilla.org\/blogarchive\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.mozilla.org\/blogarchive\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.mozilla.org\/blogarchive\/wp-json\/wp\/v2\/users\/1462"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.mozilla.org\/blogarchive\/wp-json\/wp\/v2\/comments?post=11282"}],"version-history":[{"count":0,"href":"https:\/\/blog.mozilla.org\/blogarchive\/wp-json\/wp\/v2\/posts\/11282\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/blog.mozilla.org\/blogarchive\/wp-json\/wp\/v2\/media\/11283"}],"wp:attachment":[{"href":"https:\/\/blog.mozilla.org\/blogarchive\/wp-json\/wp\/v2\/media?parent=11282"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.mozilla.org\/blogarchive\/wp-json\/wp\/v2\/categories?post=11282"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.mozilla.org\/blogarchive\/wp-json\/wp\/v2\/tags?post=11282"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/blog.mozilla.org\/blogarchive\/wp-json\/wp\/v2\/coauthors?post=11282"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}