{"id":1902,"date":"2015-04-20T12:45:53","date_gmt":"2015-04-20T19:45:53","guid":{"rendered":"http:\/\/mozscienceblog.wpengine.com\/?p=1902"},"modified":"2019-02-28T12:40:12","modified_gmt":"2019-02-28T20:40:12","slug":"effective-code-review-for-journals","status":"publish","type":"post","link":"https:\/\/blog.mozilla.org\/foundation-archive\/mozilla-science\/effective-code-review-for-journals\/","title":{"rendered":"Effective Code Review for Journals"},"content":{"rendered":"<p><em>Nature Biotechnology<\/em> <a href=\"http:\/\/www.nature.com\/news\/rule-rewrite-aims-to-clean-up-scientific-software-1.17323\">recently announced<\/a> that it would be requiring authors to &#8216;check the accessibility of code used in computational studies&#8217;, in an effort to mitigate retractions and errors resulting from bugs &amp; under-validated code. The article quoted the Science Lab&#8217;s director, <a href=\"https:\/\/twitter.com\/kaythaney\">Kaitlin Thaney<\/a>, in observing the Science Lab&#8217;s position that openness in research is not only a matter of releasing information, but making sure it is effectively reusable, too, in order to reproduce and confirm results and carry that work forward.<\/p>\n<p>But, technical challenges remain. As was discovered in the series of code review pilot studies from the Science Lab and<span class=\"author-p-21003\"> Marian Petre from Open University<\/span> in 2013 and 2014, third parties reviewing code they weren&#8217;t involved in writing leads to superficial reviews without much value; see reflections on these studies from <a href=\"https:\/\/mozillascience.org\/code-review-for-science-what-we-learned\/\">Thaney<\/a> as well as <a href=\"https:\/\/mozillascience.org\/stating-the-obvious\/\">Greg Wilson<\/a>, in addition to recent comments to the same effect from Wilson <a href=\"http:\/\/software-carpentry.org\/blog\/2015\/04\/quality-is-free-getting-there-isnt.html\">here<\/a>.<\/p>\n<p>However, journals like <em>Nature Biotech<\/em> can still compel some very valuable change by marshaling a system of code review for their submissions. As we discuss in <a href=\"http:\/\/mozillascience.github.io\/codeReview\/intro.html\">our teaching kit on code review<\/a> (and as was originally investigated in <a href=\"http:\/\/smartbear.com\/smartbear\/media\/pdfs\/wp-cc-11-best-practices-of-peer-code-review.pdf\">this study<\/a>), much value can be derived from setting expectations for code clarity and integrity. By demanding authors submit a high-coverage test suite for any original code used,\u00a0 journals can encourage researchers to use this fundamental technique for ensuring code quality; also, as discussed in depth in the study linked above, the act of requiring authors to describe and justify the changes made at each pull request results in measurably less bugs committed &#8211; <em>before code review has even begun<\/em>. Specifically, journals could require:<\/p>\n<ul>\n<li>a passing test suite with a minimum standard of coverage (&gt;90%)<\/li>\n<li>a commit log consisting of small pull requests (&lt;500 lines each), each with an accompanying description &amp; justification of the changes made and strategies taken.<\/li>\n<\/ul>\n<p>Neither of these require reviewers to read code in-depth, but both push authors to <em>seriously<\/em> <em>reflect on their code<\/em>, and thus improve its quality.<\/p>\n<p>For more strategies on how to implement a system of code review for scientific software, check out <a href=\"http:\/\/mozillascience.github.io\/codeReview\/intro.html\">our curriculum on code review<\/a>. The ideas and strategies presented there are crafted with busy scientists in mind, and explore how to get the most out of short, low-time-commitment reviews; feedback and contributions always welcome over at the <a href=\"https:\/\/github.com\/mozillascience\/codeReview\/tree\/gh-pages\">project repo<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Nature Biotechnology recently announced that it would be requiring authors to &#8216;check the accessibility of code used in computational studies&#8217;, in an effort to mitigate retractions and errors resulting from bugs &amp; under-validated code. The article quoted the Science Lab&#8217;s &hellip; <a class=\"go\" href=\"https:\/\/blog.mozilla.org\/foundation-archive\/mozilla-science\/effective-code-review-for-journals\/\">Continue reading<\/a><\/p>\n","protected":false},"author":144,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[290376],"tags":[],"_links":{"self":[{"href":"https:\/\/blog.mozilla.org\/foundation-archive\/wp-json\/wp\/v2\/posts\/1902"}],"collection":[{"href":"https:\/\/blog.mozilla.org\/foundation-archive\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.mozilla.org\/foundation-archive\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.mozilla.org\/foundation-archive\/wp-json\/wp\/v2\/users\/144"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.mozilla.org\/foundation-archive\/wp-json\/wp\/v2\/comments?post=1902"}],"version-history":[{"count":0,"href":"https:\/\/blog.mozilla.org\/foundation-archive\/wp-json\/wp\/v2\/posts\/1902\/revisions"}],"wp:attachment":[{"href":"https:\/\/blog.mozilla.org\/foundation-archive\/wp-json\/wp\/v2\/media?parent=1902"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.mozilla.org\/foundation-archive\/wp-json\/wp\/v2\/categories?post=1902"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.mozilla.org\/foundation-archive\/wp-json\/wp\/v2\/tags?post=1902"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}