{"id":5533,"date":"2015-12-01T09:35:30","date_gmt":"2015-12-01T16:35:30","guid":{"rendered":"http:\/\/blog.mozilla.org\/metrics\/?p=5533"},"modified":"2019-09-18T12:05:33","modified_gmt":"2019-09-18T19:05:33","slug":"measuring-code-quality","status":"publish","type":"post","link":"https:\/\/blog.mozilla.org\/metrics\/2015\/12\/01\/measuring-code-quality\/","title":{"rendered":"Measuring Firefox Code Quality"},"content":{"rendered":"

Unlike physical objects, it might not always be apparent what wear and tear looks like in software. Parts don\u2019t squeak or crack or moan in the same way, and to the chagrin of many a researcher, there are no fundamental laws in software like there are in nature. And so best practices<\/i> and good advice<\/i> are generally the camels on whose backs we travel, occasionally sneering at onlookers for lacking the requisite level of faith in our idealized theories of how things ought to be done.<\/p>\n

Yet the road to engineering from mere development, a characterization that I first read in Garlan and Shaw\u2019s introduction to software architecture, is filled with noble efforts like that of devising metrics and building models of quality to help track and manage the complexity that is inherent in software. Complexity that is essential, as Brooks might say.<\/p>\n

But quality is ambiguous. Nobody really knows what one thing it is meant to measure, which is why we define quality in concrete and local terms. \u201cBuilding for quality means building for maintainability,\u201d one may proclaim, to which another may respond, \u201cTo us, it\u2019s more about performance.\u201d Often though, it is a combination of attributes that span multiple categories.<\/p>\n

Maintainability is what I\u2019m interested in, for personal reasons, as I have witnessed the wrath of\u00a0code that rips through muscle and bone and condemns the souls of all who dare approach it to eternal damnation\u2014technical debt<\/i>\u00a0as it is sometimes called. The metaphor is a useful one, since\u00a0it posits that software development malpractices incur a debt that must be paid at some point in the future in the form of time, effort or defects.<\/p>\n

This accumulation of debt can take months to become noticeable and so detecting it is only possible when we observe a system over a period of time. It is by way of this sort of transparency that we see things like drops and rises in a system\u2019s level of maintainability, indicators that we can then use to inform decisions about the product, process and project.<\/p>\n

In order to provide such a lens for those interested in tracking the maintainability of the Firefox codebase, I have put together a dashboard<\/a> that tracks six measures of architectural complexity. They constitute a set of measures that studies have shown to be good predictors of quality. Many other measures exist too.<\/p>\n

Size<\/h3>\n
    \n
  1. Lines of code<\/i>: The number of executable lines of code, not counting comments and blank lines. Though lines of code is the simplest measure of a system\u2019s complexity, some practitioners argue that it remains one of the best predictors of quality.<\/li>\n<\/ol>\n
      \n
    1. Files<\/i>: The number of files, not counting filtered files<\/a> and unit tests. A file is analogous to a component in other engineered systems, seeing as it is not as atomic as, say, a screw or a bolt and not as large as a module.<\/li>\n<\/ol>\n

      Intra-component complexity<\/h3>\n
        \n
      1. Cyclomatic complexity<\/i>: The number of linearly independent paths<\/a> of execution within a system per line of code. These are paths that occur as a result of branching constructs like if-else statements. For the sake of readability, the measure is per 1,000 lines of code, so a cyclomatic complexity value of 200 means that there are around 200 independent paths in every 1,000 lines of code.<\/li>\n<\/ol>\n

        Inter-component complexity<\/h3>\n
          \n
        1. Dependencies<\/i>: The number of files that the average file can directly impact. A file depends on another if it includes, calls, sets, uses, casts, or refers to one or more items in that file. We can determine the average number of dependencies in a system by building an adjacency matrix<\/a> of its components.<\/li>\n<\/ol>\n
            \n
          1. Propagation<\/i>: The proportion of files in a system that are connected, either directly or indirectly. In practical terms, propagation gives a sense of the total reach of a change to a file. We calculate propagation through a process of matrix multiplication<\/a>.<\/li>\n<\/ol>\n
              \n
            1. Highly interconnected files<\/i>: Files that are interconnected via a chain of cyclic dependencies. These are pairs of files in a system that have a lot of dependencies between each other. Highly interconnected files may be correlated with propagation.<\/li>\n<\/ol>\n

              * \u00a0* \u00a0*<\/p>\n

              The analysis runs daily on revisions in Mozilla\u2019s central tree<\/a>, first on the entire codebase and then on a set of top-level directories, which are meant to constitute individual modules. In addition to the interface, the dashboard includes an endpoint that allows one to specify the path to a file in the Firefox codebase, for which it returns the set of inward and outward dependencies. This information can be useful for things like determining what subset of tests to run for a particular commit.<\/p>\n

              \"Code<\/a>

              Code Quality, metrics.mozilla.com\/code-quality<\/a><\/p><\/div>\n

              The entire code for the analyzer and the interface, most of which is Python and JavaScript, is available in a public repository<\/a>, as is the documentation for how to set things up and modify default behavior.<\/p>\n

              By following the documentation, you should be able to run the analysis on your own codebases. A previous project<\/a>, for instance, ran it on 23 releases of Chromium, which served as a useful benchmark. Other areas of focus may include identifying refactoring patterns and looking into possible relationships between frequency of change to LOC and files and regressions. Furthermore, comparing metrics across modules might be valuable in providing insight into various team dynamics.<\/p>\n

              Seeing as the\u00a0work is still at a fairly embryonic stage, if you\u2019re interested in this sort of thing, I invite you to get in touch or contribute<\/a> your thoughts on GitHub.<\/p>\n","protected":false},"excerpt":{"rendered":"

              Unlike physical objects, it might not always be apparent what wear and tear looks like in software. Parts don\u2019t squeak or crack or moan in the same way, and to the chagrin of many a researcher, there are no fundamental … Continue reading<\/a><\/p>\n","protected":false},"author":542,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3647,491],"tags":[3669,472],"_links":{"self":[{"href":"https:\/\/blog.mozilla.org\/metrics\/wp-json\/wp\/v2\/posts\/5533"}],"collection":[{"href":"https:\/\/blog.mozilla.org\/metrics\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.mozilla.org\/metrics\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.mozilla.org\/metrics\/wp-json\/wp\/v2\/users\/542"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.mozilla.org\/metrics\/wp-json\/wp\/v2\/comments?post=5533"}],"version-history":[{"count":0,"href":"https:\/\/blog.mozilla.org\/metrics\/wp-json\/wp\/v2\/posts\/5533\/revisions"}],"wp:attachment":[{"href":"https:\/\/blog.mozilla.org\/metrics\/wp-json\/wp\/v2\/media?parent=5533"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.mozilla.org\/metrics\/wp-json\/wp\/v2\/categories?post=5533"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.mozilla.org\/metrics\/wp-json\/wp\/v2\/tags?post=5533"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}