{"id":148,"date":"2020-03-30T06:56:26","date_gmt":"2020-03-30T06:56:26","guid":{"rendered":"https:\/\/blog.mozilla.org\/data\/?p=148"},"modified":"2020-05-05T08:23:37","modified_gmt":"2020-05-05T08:23:37","slug":"opening-data-to-understand-social-distancing","status":"publish","type":"post","link":"https:\/\/blog.mozilla.org\/data\/2020\/03\/30\/opening-data-to-understand-social-distancing\/","title":{"rendered":"Opening data to understand social distancing"},"content":{"rendered":"

As researchers work to understand and hopefully control the covid-19 pandemic, collaboration and sharing of data is essential.\u00a0 Organizations like Johns Hopkins University Center for Systems Science and Engineering <\/a>and Italy\u2019s Dipartimento della Protezione Civile<\/a> are publishing data sets to help others make sense of the impact of this pandemic.\u00a0 At Mozilla, we\u2019ve noticed a recent increase in desktop usage of Firefox. Because this data may have some value to researchers investigating social distancing measures in the current pandemic, we are releasing a dataset to support this collaborative effort.<\/p>\n

To explain visually, below is a plot of daily active users (DAU) of Firefox Desktop in France.\u00a0 We include both the actual metric and the forecasted metric, which represents what we would expect to see if everything were normal.\u00a0 Following is a plot of the metric deviation from forecast.<\/p>\n

\"DAU<\/p>\n

\"Deviation<\/p>\n

We see there is some normal range of variation, but there is a clear increase in usage outside the normal range starting on March 16th.\u00a0 Given the timing and unprecedented nature of the increase in desktop usage we have recorded, we think it may be related to social distancing measures being taken worldwide, although the relationship is not certain and Mozilla has not investigated alternative explanations for this increase in usage.\u00a0 It is possible that this data may be useful to help evaluate adherence with these measures, or even, by examining how the rate of disease growth changes following changes in Firefox usage, the impact that these measures are having.<\/p>\n

As data scientists, we would caution other researchers that our user base may not be representative of the general population and that we are still working to understand how other factors that affect online life (like the proliferation of online education or the increased consumption of news or other covid-19 related material) affect our data, but we hope that this data will provide some value to the research community and other interested organizations. We also hope to set an example and encourage our industry colleagues to collaborate openly and share data that may be of value in ways that are consistent with respect for the privacy of their users.<\/p>\n

We are making a table available that identifies deviations in our metrics over time for a large set of geographical units at the country and city level.\u00a0 As an organization steeped in openness and respect for individuals\u2019 privacy, we are publishing this non-personal data in accordance with our commitment to an internet that catalyzes collaboration for the common good <\/a>and with data practices that preserve anonymity for our users, by ensuring that all released data are based on an aggregation of at least 5000 users.\u00a0 Mozilla is making this database available under the Creative Commons CC0 public domain dedication<\/a>. That means we have waived all copyrights to the extent we can under the law and the data is public and free to use.\u00a0 The data are currently available as a CSV download here<\/a> or as a JSON endpoint here<\/a>.\u00a0 This will be replaced with a public BigQuery table and a JSON endpoint soon (this post will be updated). We describe the data briefly below. Inquiries can be directed to publicdata@mozilla.com<\/a>.<\/p>\n

Details of the data<\/b><\/p>\n

Our metrics have normal variation over time for many reasons, so rather than sharing the raw metrics, we are providing the deviation from a forecast. The forecast models what would have happened to our metrics since January 30th (the date the WHO declared a global public-health emergency) if \u201ceverything was normal,\u201d so deviations from the forecast represent an anomaly. Not all anomalies will be related to social distancing measures, but by analyzing the patterns over time, it may be possible to develop confidence that a deviation is attributable to the measures.<\/p>\n

The code used to produce this data is available here<\/a>. We would like to acknowledge our use of Facebook\u2019s Prophet forecasting library<\/a>. Its robustness makes it possible to develop useful forecasts for thousands of different geographical units over multiple metrics in a very short time.\u00a0 The library is freely available under an MIT open source license and does not require sharing any data with Facebook.<\/p>\n

The data describe the deviation from forecast of various usage metrics. Currently we include:<\/p>\n