{"id":129,"date":"2021-01-15T05:09:32","date_gmt":"2021-01-15T13:09:32","guid":{"rendered":"https:\/\/blog.mozilla.org\/performance\/?p=129"},"modified":"2021-03-01T07:22:39","modified_gmt":"2021-03-01T15:22:39","slug":"performance-sheriff-newsletter-december-2020","status":"publish","type":"post","link":"https:\/\/blog.mozilla.org\/performance\/2021\/01\/15\/performance-sheriff-newsletter-december-2020\/","title":{"rendered":"Performance Sheriff Newsletter (December 2020)"},"content":{"rendered":"<p>In December there were <strong>241 alerts<\/strong> generated, resulting in <strong>39 regression bugs<\/strong> being filed on average <strong>6.4 days<\/strong> after the regressing change landed.<\/p>\n<p>Welcome to the December 2020 edition of the performance sheriffing newsletter. Here you&#8217;ll find the usual summary of our sheriffing efficiency metrics, followed by a review of the year. If you&#8217;re interested (and if you have access) you can view the <a href=\"https:\/\/sql.telemetry.mozilla.org\/dashboard\/perf-sheriff-newsletter\">full dashboard<\/a>.<\/p>\n<h2><a href=\"http:\/\/blog.mozilla.org\/performance\/files\/2021\/01\/se-dec.png\"><img decoding=\"async\" loading=\"lazy\" class=\"alignright wp-image-133 size-medium\" src=\"http:\/\/blog.mozilla.org\/performance\/files\/2021\/01\/se-dec-300x174.png\" alt=\"Sheriffing Efficiency (Dec 2020)\" width=\"300\" height=\"174\" srcset=\"https:\/\/blog.mozilla.org\/performance\/files\/2021\/01\/se-dec-300x174.png 300w, https:\/\/blog.mozilla.org\/performance\/files\/2021\/01\/se-dec.png 468w\" sizes=\"(max-width: 300px) 100vw, 300px\" \/><\/a>Sheriffing efficiency<\/h2>\n<ul>\n<li>All alerts were triaged in an average of <strong>1.1 days<\/strong><\/li>\n<li><strong>95%<\/strong> of alerts were triaged within 3 days<\/li>\n<li>Valid regressions were associated with bugs in <strong>3.4 days<\/strong><\/li>\n<li><strong>71%<\/strong> of valid regressions were associated with bugs within 5 days<\/li>\n<\/ul>\n<h2>2020 in Review<\/h2>\n<p>As we welcome a new year, let&#8217;s look back at how 2020 looked for performance sheriffing:<\/p>\n<ul>\n<li>Over <strong>2,500<\/strong> alert summaries were generated (~7 per day)<\/li>\n<li>A total of <strong>371<\/strong> regression bugs were opened<\/li>\n<li><strong>41%<\/strong> of these regression bugs were fixed<\/li>\n<li>All alerts were triaged in an average of <strong>1.4 days<\/strong><\/li>\n<li><strong>87%<\/strong> of alerts were triaged within 3 days<\/li>\n<li>Valid regressions were associated with bugs in <strong>2.8 days<\/strong><\/li>\n<li><strong>86%<\/strong> of valid regressions were associated with bugs within 5 days<\/li>\n<li><strong>119<\/strong> performance improvements were identified<\/li>\n<\/ul>\n<h2>Summary of alerts<\/h2>\n<p>Each month I&#8217;ll highlight the regressions and improvements found.<\/p>\n<ul>\n<li>\ud83d\ude0d <a href=\"https:\/\/bugzilla.mozilla.org\/buglist.cgi?bug_id=1679382,1675240,1678636,1645528,1678740,1680773,1679700,1681721,1680769,1654192,1680175,1669841\">12 bugs were associated with improvements<\/a><\/li>\n<li>\ud83e\udd10\u00a0<a href=\"https:\/\/bugzilla.mozilla.org\/buglist.cgi?bug_id=1680450,1680500,1680765,1680451,1681728,1683323,1683647,1683649,1683841,1683647\">9 regressions were accepted<\/a><\/li>\n<li>\ud83e\udd29\u00a0<a href=\"https:\/\/bugzilla.mozilla.org\/buglist.cgi?bug_id=1677972,1645528,1680743,1680769,1681030,1681030,1679482,1681730,1682556,1683870,1683856,1683885,1683903,1685020,1684434,1681030\">14 regressions were fixed (or backed out)<\/a><\/li>\n<li>\ud83e\udd25 <a href=\"https:\/\/bugzilla.mozilla.org\/buglist.cgi?bug_id=1680773,1682562\">2 regressions were invalid<\/a><\/li>\n<li>\ud83e\udd17 <a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=1683897\">1 regression is assigned<\/a><\/li>\n<li>\ud83d\ude28\u00a0<a href=\"https:\/\/bugzilla.mozilla.org\/buglist.cgi?bug_id=1681047,1682312,1682557,1682768,1683301,1683819,1683893,15554917\">8 regressions are still open<\/a><\/li>\n<\/ul>\n<p>Note that whilst I usually allow one week to pass before generating the report, there are still alerts under investigation for the period covered in this article. This means that whilst I believe these metrics to be accurate at the time of writing, some of them may change over time.<\/p>\n<p>I would love to hear your feedback on this article, the queries, the dashboard, or anything else related to performance sheriffing or performance testing. You can comment here, or find the team on Matrix in <a href=\"https:\/\/chat.mozilla.org\/#\/room\/#perftest:mozilla.org\">#perftest<\/a> or <a href=\"https:\/\/chat.mozilla.org\/#\/room\/#perfsheriffs:mozilla.org\">#perfsheriffs<\/a>.<\/p>\n<p>The dashboard for December can be <a href=\"https:\/\/sql.telemetry.mozilla.org\/dashboard\/perf-sheriff-newsletter?p_range=2020-12-01--2020-12-31\">found here<\/a> (for those with access).<\/p>\n","protected":false},"excerpt":{"rendered":"<p>In December there were 241 alerts generated, resulting in 39 regression bugs being filed on average 6.4 days after the regressing change landed. Welcome to the December 2020 edition of &hellip; <a class=\"go\" href=\"https:\/\/blog.mozilla.org\/performance\/2021\/01\/15\/performance-sheriff-newsletter-december-2020\/\">Read more<\/a><\/p>\n","protected":false},"author":513,"featured_media":131,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[281231],"tags":[457005,117,812,281231],"_links":{"self":[{"href":"https:\/\/blog.mozilla.org\/performance\/wp-json\/wp\/v2\/posts\/129"}],"collection":[{"href":"https:\/\/blog.mozilla.org\/performance\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.mozilla.org\/performance\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.mozilla.org\/performance\/wp-json\/wp\/v2\/users\/513"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.mozilla.org\/performance\/wp-json\/wp\/v2\/comments?post=129"}],"version-history":[{"count":0,"href":"https:\/\/blog.mozilla.org\/performance\/wp-json\/wp\/v2\/posts\/129\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/blog.mozilla.org\/performance\/wp-json\/wp\/v2\/media\/131"}],"wp:attachment":[{"href":"https:\/\/blog.mozilla.org\/performance\/wp-json\/wp\/v2\/media?parent=129"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.mozilla.org\/performance\/wp-json\/wp\/v2\/categories?post=129"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.mozilla.org\/performance\/wp-json\/wp\/v2\/tags?post=129"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}