Whenever an internet connection is cut in a country or city, the safety and security of millions of people may be at stake. Documenting outages helps internet access defenders understand when and where they took place even when authorities or service providers may deny them.
When large numbers of Firefox users experience connection failures for any reason, this produces an anomaly in the recorded telemetry data. At the country or city level, this can provide a corroborative signal of whether an outage or intentional shutdown occurred.
Several large technology companies, including Google and Cloudflare, publicly share data about outages of consumer-facing products in different ways. But researchers and journalists can usually only hone in on the exact nature of an outage by combining data from multiple sources.
Data validation process
Mozilla’s aggregate data for detecting outages is not publicly shared although it contains no personally identifiable information. In order to first assess the reliability of a dataset designed for this purpose, we invited a group of external researchers to query a dataset of aggregated signals that could be indicative of outages (e.g. the time it takes to perform a TLS handshake).
Researchers from Internet Outage Detection and Analysis (IODA) of the Center for Applied Internet Data Analysis (CAIDA), Open Observatory of Network Interference (OONI), RIPE Network Coordination Center (RIPE NCC), Measurement Lab (M-Lab), Internews and Access Now joined a collaborative effort in 2020 to compare existing data on outages with Mozilla’s dataset. This research took place over several months, anchored by a series of video calls in which researchers took turns sharing their screens to display visual explorations of data.
The main case studies were shutdowns in Belarus, Uganda and Myanmar, each with different time spans and characteristics. Individual research methods varied as teams undertook comparisons of Mozilla’s data with the data they typically use for this purpose.
New report explores the dataset
Today, OONI and IODA/CAIDA are releasing an independent report of their own research into Mozilla’s dataset that highlights their assessment of its advantages and limitations.
It’s called “Investigating Internet shutdowns through Mozilla telemetry” and it dives into the different cases, describing in which ways the dataset confirms or augments insights, and how it could be improved to better serve data researchers, journalists and civil society.
The collaborative research project ended in June 2021 with individual interviews of all participants by Mozilla to gather insights. They voiced unanimous support for facilitating greater access to the dataset to serve digital rights advocates worldwide.
In their report, OONI and IODA/CAIDA also confirmed its potential value to the measurement community if Mozilla’s dataset were to be shared publicly.
“It would be a really great addition to the datasets civil society uses to investigate and confirm internet outages,” says Arturo Filastò, the Project Lead and Co-Founder of OONI.
“The fact that it’s from so many different clients in locations where people are trying to browse the internet means that it’s a good representation of what the internet user experience is like around the world. The breadth and granularity of the data is unlike any other dataset out there,” says Filastò, adding that including mobile telemetry to the dataset would be a great advantage since shutdowns often target mobile networks in areas where desktop connectivity is limited.
With thanks to everyone who took part in or supported the research and interviews:
Simone Basso, Jochai Ben-Avie, Rafael Bezerra Nunes, Georgia Bullen, Jon Camfield, Federico Ceratto, Sage Cheng, Alberto Dainotti, Marianne Díaz Hernández, Michael Droettboom, Arturo Filastò, Georg Fritzsche, Saptarshi Guha, Eric Jjemba, Emily Litka, Lai Yi Ohlsen Ramakrishna Padmanabhan, Melody Patry, Jan-Erik Rediger, Stephen D. Strowes, Berhan Taye, Hamilton Ulmer, Vasilis Ververis, Maria Xynou, Mingwei Zhang.
This post was co-authored by Solana Larsen, Alessio Placitelli, Udbhav Tiwari.