{"id":2214,"date":"2017-06-28T09:47:30","date_gmt":"2017-06-28T16:47:30","guid":{"rendered":"https:\/\/blog.mozilla.org\/security\/?p=2214"},"modified":"2017-06-28T10:49:15","modified_gmt":"2017-06-28T17:49:15","slug":"analysis-alexa-top-1m-sites","status":"publish","type":"post","link":"https:\/\/blog.mozilla.org\/security\/2017\/06\/28\/analysis-alexa-top-1m-sites\/","title":{"rendered":"Analysis of the Alexa Top 1M sites"},"content":{"rendered":"

Prior to the release of the <\/span>Mozilla Observatory<\/span><\/a> a year ago, I ran a scan of the Alexa Top 1M websites. Despite being available for years, <\/span>the usage rates<\/span><\/a> of modern defensive security technologies was frustratingly low. A lack of tooling combined with poor and scattered documentation had led to there being little awareness around countermeasures such as <\/span>Content Security Policy<\/span><\/a> (CSP), <\/span>HTTP Strict Transport Security<\/span><\/a> (HSTS), and <\/span>Subresource Integrity<\/span><\/a> (SRI).<\/span><\/p>\n

A few months after the Observatory’s release \u2014 and 1.5M Observatory scans later \u2014 I reassessed the Top 1M websites. The situation appeared as if it was <\/span>beginning to improve<\/span><\/a>, with the use of HSTS and CSP up by approximately 50%. But were those improvements simply low-hanging fruit, or has the situation continued to improve over the following months?<\/span><\/p>\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
Technology<\/b><\/td>\nApril 2016<\/b><\/td>\nOctober 2016<\/b><\/td>\nJune 2017<\/b><\/td>\n% Change<\/b><\/td>\n<\/tr>\n
Content Security Policy (CSP)<\/span><\/td>\n.005%<\/span>1<\/sup>
\n<\/span><\/a>.012%<\/span>2<\/sup><\/span><\/a><\/td>\n		.008%<\/span>1<\/sup>
\n<\/span><\/a>.021%<\/span>2<\/sup><\/span><\/a><\/td>\n		.018%<\/span>1<\/sup>
\n<\/span><\/a>.043%<\/span>2<\/sup><\/span><\/a><\/td>\n		+125%<\/span><\/td>\n<\/tr>\n
Cookies (Secure\/HttpOnly)<\/span>3<\/sup><\/span><\/a><\/td>\n3.76%<\/span><\/td>\n4.88%<\/span><\/td>\n6.50%<\/span><\/td>\n+33%<\/span><\/td>\n<\/tr>\n
Cross-origin Resource Sharing (CORS)<\/span>4<\/sup><\/span><\/a><\/td>\n93.78%<\/span><\/td>\n96.21%<\/span><\/td>\n96.55%<\/span><\/td>\n+.4%<\/span><\/td>\n<\/tr>\n
HTTPS<\/span><\/td>\n29.64%<\/span><\/td>\n33.57%<\/span><\/td>\n45.80%<\/span><\/td>\n+36%<\/span><\/td>\n<\/tr>\n
HTTP \u2192 HTTPS Redirection<\/span><\/td>\n5.06%<\/span>5<\/sup>
\n<\/span><\/a>8.91%<\/span>6<\/sup><\/span><\/a><\/td>\n		7.94%<\/span>5<\/sup>
\n<\/span><\/a>13.29%<\/span>6<\/sup><\/span><\/a><\/td>\n		14.38%<\/span>5<\/sup>
\n<\/span><\/a>22.88%<\/span>6<\/sup><\/span><\/a><\/td>\n		+57%<\/span><\/td>\n<\/tr>\n
Public Key Pinning (HPKP)<\/span><\/td>\n0.43%<\/span><\/td>\n0.50%<\/span><\/td>\n0.71%<\/span><\/td>\n+42%<\/span><\/td>\n<\/tr>\n
\u00a0\u2014 HPKP Preloaded<\/span>7<\/sup><\/span><\/a><\/td>\n0.41%<\/span><\/td>\n0.47%<\/span><\/td>\n0.43%<\/span><\/td>\n-9%<\/span><\/td>\n<\/tr>\n
Strict Transport Security (HSTS)<\/span>8<\/sup><\/span><\/a><\/td>\n1.75%<\/span><\/td>\n2.59%<\/span><\/td>\n4.37%<\/span><\/td>\n+69%<\/span><\/td>\n<\/tr>\n
\u00a0\u2014 HSTS Preloaded<\/span>7<\/sup><\/span><\/a><\/td>\n.158%<\/span><\/td>\n.231%<\/span><\/td>\n.337%<\/span><\/td>\n+46%<\/span><\/td>\n<\/tr>\n
Subresource Integrity (SRI)<\/span><\/td>\n0.015%<\/span>9<\/sup><\/span><\/a><\/td>\n0.052%<\/span>10<\/sup><\/span><\/a><\/td>\n0.113%<\/span>10<\/sup><\/span><\/a><\/td>\n+117%<\/span><\/td>\n<\/tr>\n
X-Content-Type-Options (XCTO)<\/span><\/td>\n6.19%<\/span><\/td>\n7.22%<\/span><\/td>\n9.41%<\/span><\/td>\n+30%<\/span><\/td>\n<\/tr>\n
X-Frame-Options (XFO)<\/span>11<\/sup><\/span><\/a><\/td>\n6.83%<\/span><\/td>\n8.78%<\/span><\/td>\n10.98%<\/span><\/td>\n+25%<\/span><\/td>\n<\/tr>\n
X-XSS-Protection (XXSSP)<\/span>12<\/sup><\/span><\/a><\/td>\n5.03%<\/span><\/td>\n6.33%<\/span><\/td>\n8.12%<\/span><\/td>\n+28%<\/span><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

The pace of improvement across the web appears to be continuing at an astounding rate. Although a 36% increase in the number of sites that support HTTPS might seem small, the absolute numbers are quite large \u2014 it represents over <\/span>119,000<\/b> websites.<\/span><\/p>\n

Not only that, but <\/span>93,000<\/b> of those websites have chosen to be HTTPS by default, with <\/span>18,000<\/b> of them forbidding any HTTP access at all through the use of HTTP Strict Transport Security.<\/span><\/p>\n

The sharp jump in the rate of Content Security Policy (CSP) usage is similarly surprising. It can be difficult to implement for a new website, and often requires extensive rearchitecting to retrofit to an existing site, as most of the Alexa Top 1M sites are. Between increasingly improving documentation, advances in CSP3 such as <\/span>‘strict-dynamic’<\/span><\/a>, and CSP policy generators such as the <\/span>Mozilla Laboratory<\/span><\/a>, it appears that we might be turning a corner on CSP usage around the web.<\/span><\/p>\n

Observatory Grading<\/b><\/h3>\n

Despite this progress, the vast majority of large websites around the web continue to not use Content Security Policy and Subresource Integrity. As these technologies \u2014 when properly used \u2014 can nearly eliminate huge classes of attacks against sites and their users, they are given a significant amount of weight in Observatory scans.<\/span><\/p>\n

As a result of their low usage rates amongst established websites, they typically receive failing grades from the Observatory. Nevertheless, I continue to see improvements across the board:<\/span><\/p>\n\n\n\n\n\n\n\n\n\n
Grade<\/b><\/td>\nApril 2016<\/b><\/td>\nOctober 2016<\/b><\/td>\nJune 2017<\/b><\/td>\n% Change<\/b><\/td>\n<\/tr>\n
\u00a0A+<\/span><\/td>\n.003%<\/span><\/td>\n.008%<\/span><\/td>\n.013%<\/span><\/td>\n+62%<\/span><\/td>\n<\/tr>\n
A<\/span><\/td>\n.006%<\/span><\/td>\n.012%<\/span><\/td>\n.029%<\/span><\/td>\n+142%<\/span><\/td>\n<\/tr>\n
B<\/span><\/td>\n.202%<\/span><\/td>\n.347%<\/span><\/td>\n.622%<\/span><\/td>\n+79%<\/span><\/td>\n<\/tr>\n
C<\/span><\/td>\n.321%<\/span><\/td>\n.727%<\/span><\/td>\n1.38%<\/span><\/td>\n+90%<\/span><\/td>\n<\/tr>\n
D<\/span><\/td>\n1.87%<\/span><\/td>\n2.82%<\/span><\/td>\n4.51%<\/span><\/td>\n+60%<\/span><\/td>\n<\/tr>\n
F<\/span><\/td>\n97.60%<\/span><\/td>\n96.09%<\/span><\/td>\n93.45%<\/span><\/td>\n-2.8%<\/span><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

As 969,924 scans were successfully completed in the last survey, a decrease in failing grades by 2.8% implies that over <\/span>27,000<\/b> of the largest sites in the world have improved from a failing grade in the last eight months alone.<\/span><\/p>\n

In fact, my research indicates that over <\/span>50,000<\/b> websites around the web have directly used the Mozilla Observatory to improve their grades, indicated by scanning their website, making an improvement, and then scanning their website again. Of these <\/span>50,000<\/b> websites, over <\/span>2,500<\/b> have improved all the way from a failing grade to an A or A+ grade.<\/span><\/p>\n

When I first built the Observatory a year ago at <\/span>Mozilla<\/span><\/a>, I had never imagined that it would see such widespread use. 3.8M scans across 1.55M unique domains later, it seems to have made a significant difference across the internet. I feel incredibly lucky to work at a company like Mozilla that has provided me with a unique opportunity to work on a tool designed solely to make internet a better place.<\/span><\/p>\n

Please share the <\/span>Mozilla Observatory<\/span><\/a> and the <\/span>Web Security Guidelines<\/span><\/a> so that the web can continue to see improvements over the years to come!<\/span><\/p>\n

 <\/p>\n

Footnotes:<\/b><\/p>\n

    \n
  1. Allows <\/span>'unsafe-inline'<\/tt><\/span> in neither <\/span>script-src<\/tt><\/span> nor <\/span>style-src<\/tt><\/span><\/li>\n
  2. Allows <\/span>'unsafe-inline'<\/tt><\/span> in <\/span>style-src<\/tt><\/span> only<\/span><\/li>\n
  3. Amongst sites that set cookies<\/span><\/li>\n
  4. Disallows foreign origins from reading the domain’s contents within user’s context<\/span><\/li>\n
  5. Redirects from HTTP to HTTPS on the same domain, which allows HSTS to be set<\/span><\/li>\n
  6. Redirects from HTTP to HTTPS, regardless of the final domain<\/span><\/li>\n
  7. As listed in the <\/span>Chromium preload list<\/span><\/a><\/li>\n
  8. max-age<\/tt><\/span> set to at least six months<\/span><\/li>\n
  9. Percentage is of sites that load scripts from a foreign origin<\/span><\/li>\n
  10. Percentage is of sites that load scripts<\/span><\/li>\n
  11. CSP <\/span>frame-ancestors<\/tt><\/span> directive is allowed in lieu of an XFO header<\/span><\/li>\n
  12. Strong CSP policy forbidding <\/span>'unsafe-inline'<\/tt><\/span> is allowed in lieu of an XXSSP header<\/span><\/li>\n<\/ol>\n","protected":false},"excerpt":{"rendered":"

    Prior to the release of the Mozilla Observatory a year ago, I ran a scan of the Alexa Top 1M websites. Despite being available for years, the usage rates of … Read more<\/a><\/p>\n","protected":false},"author":1227,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[69],"tags":[],"coauthors":[],"yoast_head":"\nAnalysis of the Alexa Top 1M sites - Mozilla Security Blog<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/blog.mozilla.org\/security\/2017\/06\/28\/analysis-alexa-top-1m-sites\/\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"April King\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"4 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/blog.mozilla.org\/security\/2017\/06\/28\/analysis-alexa-top-1m-sites\/\",\"url\":\"https:\/\/blog.mozilla.org\/security\/2017\/06\/28\/analysis-alexa-top-1m-sites\/\",\"name\":\"Analysis of the Alexa Top 1M sites - Mozilla Security Blog\",\"isPartOf\":{\"@id\":\"https:\/\/blog.mozilla.org\/security\/#website\"},\"datePublished\":\"2017-06-28T16:47:30+00:00\",\"dateModified\":\"2017-06-28T17:49:15+00:00\",\"author\":{\"@id\":\"https:\/\/blog.mozilla.org\/security\/#\/schema\/person\/cab208d1e0249bd2da73bc39c3d542b7\"},\"breadcrumb\":{\"@id\":\"https:\/\/blog.mozilla.org\/security\/2017\/06\/28\/analysis-alexa-top-1m-sites\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/blog.mozilla.org\/security\/2017\/06\/28\/analysis-alexa-top-1m-sites\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/blog.mozilla.org\/security\/2017\/06\/28\/analysis-alexa-top-1m-sites\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/blog.mozilla.org\/security\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Analysis of the Alexa Top 1M sites\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/blog.mozilla.org\/security\/#website\",\"url\":\"https:\/\/blog.mozilla.org\/security\/\",\"name\":\"Mozilla Security Blog\",\"description\":\"\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/blog.mozilla.org\/security\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/blog.mozilla.org\/security\/#\/schema\/person\/cab208d1e0249bd2da73bc39c3d542b7\",\"name\":\"April King\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/blog.mozilla.org\/security\/#\/schema\/person\/image\/dd59dfe8e9604d209d11d9a03e8ab3a6\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/b874e43ec7b59fdca0b4f9b8c3b7cc1b?s=96&d=identicon&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/b874e43ec7b59fdca0b4f9b8c3b7cc1b?s=96&d=identicon&r=g\",\"caption\":\"April King\"},\"description\":\"IRC: April\",\"sameAs\":[\"https:\/\/pokeinthe.io\/\",\"https:\/\/x.com\/aprilmpls\"]}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Analysis of the Alexa Top 1M sites - Mozilla Security Blog","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/blog.mozilla.org\/security\/2017\/06\/28\/analysis-alexa-top-1m-sites\/","twitter_misc":{"Written by":"April King","Est. reading time":"4 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/blog.mozilla.org\/security\/2017\/06\/28\/analysis-alexa-top-1m-sites\/","url":"https:\/\/blog.mozilla.org\/security\/2017\/06\/28\/analysis-alexa-top-1m-sites\/","name":"Analysis of the Alexa Top 1M sites - Mozilla Security Blog","isPartOf":{"@id":"https:\/\/blog.mozilla.org\/security\/#website"},"datePublished":"2017-06-28T16:47:30+00:00","dateModified":"2017-06-28T17:49:15+00:00","author":{"@id":"https:\/\/blog.mozilla.org\/security\/#\/schema\/person\/cab208d1e0249bd2da73bc39c3d542b7"},"breadcrumb":{"@id":"https:\/\/blog.mozilla.org\/security\/2017\/06\/28\/analysis-alexa-top-1m-sites\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/blog.mozilla.org\/security\/2017\/06\/28\/analysis-alexa-top-1m-sites\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/blog.mozilla.org\/security\/2017\/06\/28\/analysis-alexa-top-1m-sites\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/blog.mozilla.org\/security\/"},{"@type":"ListItem","position":2,"name":"Analysis of the Alexa Top 1M sites"}]},{"@type":"WebSite","@id":"https:\/\/blog.mozilla.org\/security\/#website","url":"https:\/\/blog.mozilla.org\/security\/","name":"Mozilla Security Blog","description":"","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/blog.mozilla.org\/security\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/blog.mozilla.org\/security\/#\/schema\/person\/cab208d1e0249bd2da73bc39c3d542b7","name":"April King","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/blog.mozilla.org\/security\/#\/schema\/person\/image\/dd59dfe8e9604d209d11d9a03e8ab3a6","url":"https:\/\/secure.gravatar.com\/avatar\/b874e43ec7b59fdca0b4f9b8c3b7cc1b?s=96&d=identicon&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/b874e43ec7b59fdca0b4f9b8c3b7cc1b?s=96&d=identicon&r=g","caption":"April King"},"description":"IRC: April","sameAs":["https:\/\/pokeinthe.io\/","https:\/\/x.com\/aprilmpls"]}]}},"_links":{"self":[{"href":"https:\/\/blog.mozilla.org\/security\/wp-json\/wp\/v2\/posts\/2214"}],"collection":[{"href":"https:\/\/blog.mozilla.org\/security\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.mozilla.org\/security\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.mozilla.org\/security\/wp-json\/wp\/v2\/users\/1227"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.mozilla.org\/security\/wp-json\/wp\/v2\/comments?post=2214"}],"version-history":[{"count":0,"href":"https:\/\/blog.mozilla.org\/security\/wp-json\/wp\/v2\/posts\/2214\/revisions"}],"wp:attachment":[{"href":"https:\/\/blog.mozilla.org\/security\/wp-json\/wp\/v2\/media?parent=2214"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.mozilla.org\/security\/wp-json\/wp\/v2\/categories?post=2214"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.mozilla.org\/security\/wp-json\/wp\/v2\/tags?post=2214"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/blog.mozilla.org\/security\/wp-json\/wp\/v2\/coauthors?post=2214"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}