{"id":949,"date":"2013-01-22T16:55:28","date_gmt":"2013-01-23T00:55:28","guid":{"rendered":"http:\/\/blog.mozilla.org\/security\/?p=949"},"modified":"2013-01-22T16:55:28","modified_gmt":"2013-01-23T00:55:28","slug":"using-coverage-data-for-security","status":"publish","type":"post","link":"https:\/\/blog.mozilla.org\/security\/2013\/01\/22\/using-coverage-data-for-security\/","title":{"rendered":"Using Coverage Data for Security"},"content":{"rendered":"<p>We recently started measuring <em>C\/C++ code coverage<\/em> on mozilla-central again and documented the various efforts around it in a <a href=\"https:\/\/developer.mozilla.org\/en-US\/docs\/Measuring_Code_Coverage_on_Firefox\">new MDN article<\/a>.<!--more--> This article describes why coverage measurements are useful, how to create the necessary builds and also provides a link to the <a href=\"http:\/\/people.mozilla.org\/~choller\/firefox\/coverage\/\">most recent coverage measurements<\/a> of <a href=\"https:\/\/developer.mozilla.org\/en-US\/docs\/Mozilla_automated_testing\">Mozilla&#8217;s automated test suites<\/a>. This data can also be used to improve the security of Gecko based products, Firefox included. Below we list some approaches to that you may want to consider when testing Mozilla software with automated tools.<\/p>\n<h4>Combining Vulnerabilities and Coverage<\/h4>\n<p>Ideally all code should be well tested, but we all know that competing priorities in software development don&#8217;t always make this possible. From a security perspective, it makes sense to focus on uncovered code that recently had one or more security problems to take advantage of this coverage inequity. One hypothesis is that such code suffers from structural problems, as such bugs with similar root causes might still be present.<br \/>\nAs an example of this; we started by creating a list of files that were patched due to security problems and then manually checked their testing coverage. The most interesting files were picked and <a href=\"https:\/\/bugzilla.mozilla.org\/showdependencytree.cgi?id=790572&amp;hide_resolved=0\">bugs were filed to increase their test coverage<\/a>.<\/p>\n<p>We created some <a href=\"https:\/\/github.com\/mozilla\/security\/tree\/master\/client\/stats\">publicly available scripts<\/a> that are able to collect the data automatically. Of course since current security bug reports are not available to the public, you will only be able to use this on older time frames where bugs have been unhidden.<\/p>\n<h4>Check Coverage of your Fuzzer<\/h4>\n<p>If you are using your own tools to test Firefox, then it often makes sense to try a <a href=\"https:\/\/developer.mozilla.org\/en-US\/docs\/Measuring_Code_Coverage_on_Firefox#Creating_your_own_Coverage_Build\">coverage build<\/a> to see what your tool is actually hitting and how often. This not only gives you a binary result of code that is tested or not; it can also tell you if certain code is reached, but very rarely compared to the rest of the covered code. Since fuzzers can be complex software they can also contain bugs, and while not hitting code at all might still be somewhat observable without special tools, hitting code just very rarely is hard to see.<br \/>\nTo try this out, just compile Firefox as described in the  <a href=\"https:\/\/developer.mozilla.org\/en-US\/docs\/Measuring_Code_Coverage_on_Firefox\">MDN article<\/a>, run your tool for a while and take a look at the lcov results.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>We recently started measuring C\/C++ code coverage on mozilla-central again and documented the various efforts around it in a new MDN article.<\/p>\n","protected":false},"author":409,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[30,69,73],"tags":[],"coauthors":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v22.5 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Using Coverage Data for Security - Mozilla Security Blog<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/blog.mozilla.org\/security\/2013\/01\/22\/using-coverage-data-for-security\/\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Christian Holler\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/blog.mozilla.org\/security\/2013\/01\/22\/using-coverage-data-for-security\/\",\"url\":\"https:\/\/blog.mozilla.org\/security\/2013\/01\/22\/using-coverage-data-for-security\/\",\"name\":\"Using Coverage Data for Security - Mozilla Security Blog\",\"isPartOf\":{\"@id\":\"https:\/\/blog.mozilla.org\/security\/#website\"},\"datePublished\":\"2013-01-23T00:55:28+00:00\",\"dateModified\":\"2013-01-23T00:55:28+00:00\",\"author\":{\"@id\":\"https:\/\/blog.mozilla.org\/security\/#\/schema\/person\/2fc6bcd986b8681dc6987d718de18ff4\"},\"breadcrumb\":{\"@id\":\"https:\/\/blog.mozilla.org\/security\/2013\/01\/22\/using-coverage-data-for-security\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/blog.mozilla.org\/security\/2013\/01\/22\/using-coverage-data-for-security\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/blog.mozilla.org\/security\/2013\/01\/22\/using-coverage-data-for-security\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/blog.mozilla.org\/security\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Using Coverage Data for Security\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/blog.mozilla.org\/security\/#website\",\"url\":\"https:\/\/blog.mozilla.org\/security\/\",\"name\":\"Mozilla Security Blog\",\"description\":\"\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/blog.mozilla.org\/security\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/blog.mozilla.org\/security\/#\/schema\/person\/2fc6bcd986b8681dc6987d718de18ff4\",\"name\":\"Christian Holler\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/blog.mozilla.org\/security\/#\/schema\/person\/image\/9e1c519469f37f1713d7aeb2e4b3f6c2\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/b29b83d12aba827b8ff68296e473bec5?s=96&d=identicon&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/b29b83d12aba827b8ff68296e473bec5?s=96&d=identicon&r=g\",\"caption\":\"Christian Holler\"},\"description\":\"Christian is a Firefox Tech Lead and Senior Staff Security Engineer at Mozilla.\",\"sameAs\":[\"https:\/\/x.com\/mozdeco\"]}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Using Coverage Data for Security - Mozilla Security Blog","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/blog.mozilla.org\/security\/2013\/01\/22\/using-coverage-data-for-security\/","twitter_misc":{"Written by":"Christian Holler","Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/blog.mozilla.org\/security\/2013\/01\/22\/using-coverage-data-for-security\/","url":"https:\/\/blog.mozilla.org\/security\/2013\/01\/22\/using-coverage-data-for-security\/","name":"Using Coverage Data for Security - Mozilla Security Blog","isPartOf":{"@id":"https:\/\/blog.mozilla.org\/security\/#website"},"datePublished":"2013-01-23T00:55:28+00:00","dateModified":"2013-01-23T00:55:28+00:00","author":{"@id":"https:\/\/blog.mozilla.org\/security\/#\/schema\/person\/2fc6bcd986b8681dc6987d718de18ff4"},"breadcrumb":{"@id":"https:\/\/blog.mozilla.org\/security\/2013\/01\/22\/using-coverage-data-for-security\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/blog.mozilla.org\/security\/2013\/01\/22\/using-coverage-data-for-security\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/blog.mozilla.org\/security\/2013\/01\/22\/using-coverage-data-for-security\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/blog.mozilla.org\/security\/"},{"@type":"ListItem","position":2,"name":"Using Coverage Data for Security"}]},{"@type":"WebSite","@id":"https:\/\/blog.mozilla.org\/security\/#website","url":"https:\/\/blog.mozilla.org\/security\/","name":"Mozilla Security Blog","description":"","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/blog.mozilla.org\/security\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/blog.mozilla.org\/security\/#\/schema\/person\/2fc6bcd986b8681dc6987d718de18ff4","name":"Christian Holler","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/blog.mozilla.org\/security\/#\/schema\/person\/image\/9e1c519469f37f1713d7aeb2e4b3f6c2","url":"https:\/\/secure.gravatar.com\/avatar\/b29b83d12aba827b8ff68296e473bec5?s=96&d=identicon&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/b29b83d12aba827b8ff68296e473bec5?s=96&d=identicon&r=g","caption":"Christian Holler"},"description":"Christian is a Firefox Tech Lead and Senior Staff Security Engineer at Mozilla.","sameAs":["https:\/\/x.com\/mozdeco"]}]}},"_links":{"self":[{"href":"https:\/\/blog.mozilla.org\/security\/wp-json\/wp\/v2\/posts\/949"}],"collection":[{"href":"https:\/\/blog.mozilla.org\/security\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.mozilla.org\/security\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.mozilla.org\/security\/wp-json\/wp\/v2\/users\/409"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.mozilla.org\/security\/wp-json\/wp\/v2\/comments?post=949"}],"version-history":[{"count":0,"href":"https:\/\/blog.mozilla.org\/security\/wp-json\/wp\/v2\/posts\/949\/revisions"}],"wp:attachment":[{"href":"https:\/\/blog.mozilla.org\/security\/wp-json\/wp\/v2\/media?parent=949"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.mozilla.org\/security\/wp-json\/wp\/v2\/categories?post=949"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.mozilla.org\/security\/wp-json\/wp\/v2\/tags?post=949"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/blog.mozilla.org\/security\/wp-json\/wp\/v2\/coauthors?post=949"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}