Quantifying the effects of Firefox’s Tracking Protection

A number of people at Mozilla are working on a wonderful privacy initiative called Polaris. This will include activities such as Mozilla hosting its own high-capacity Tor middle relays.

But the part of Polaris I’m most interested in is Tracking Protection, which is a Firefox feature that will make it trivial for users to avoid many forms of online tracking. This not only gives users better privacy; experiments have shown it also speeds up the loading of the median page by 20%! That’s an incredible combination.

An experiment

I decided to evaluate the effectiveness of Tracking Protection. To do this, I used Lightbeam, a Firefox extension designed specifically to record third-party tracking. On November 2nd, I used a trunk build of the mozilla-inbound repository and did the following steps.

  • Start Firefox with a new profile.
  • Install Lightbeam from addons.mozilla.org.
  • Visit the following sites, but don’t interact with them at all:
    1. google.com
    2. techcrunch.com
    3. dictionary.com (which redirected to dictionary.reference.com)
    4. nytimes.com
    5. cnn.com
  • Open Lightbeam in a tab, and go to the “List” view.

I then repeated these steps, but before visiting the sites I added the following step.

  • Open about:config and toggle privacy.trackingprotection.enabled to
    “true”.

Results with Tracking Protection turned off

The sites I visited directly are marked as “Visited”. All the third-party sites are marked as “Third Party”.

Connected with 86 sites

Type            Website                Sites Connected
----            -------                ---------------
Visited         google.com              3
Third Party     gstatic.com             5
Visited         techcrunch.com         25
Third Party     aolcdn.com              1
Third Party     wp.com                  1
Third Party     gravatar.com            1
Third Party     wordpress.com           1
Third Party     twitter.com             4
Third Party     google-analytics.com    3
Third Party     scorecardresearch.com   6
Third Party     aol.com                 1
Third Party     questionmarket.com      1
Third Party     grvcdn.com              1
Third Party     korrelate.net           1
Third Party     livefyre.com            1
Third Party     gravity.com             1
Third Party     facebook.net            1
Third Party     adsonar.com             1
Third Party     facebook.com            4
Third Party     atwola.com              4
Third Party     adtech.de               1
Third Party     goviral-content.com     7
Third Party     amgdgt.com              1
Third Party     srvntrk.com             2
Third Party     voicefive.com           1
Third Party     bluekai.com             1
Third Party     truste.com              2
Third Party     advertising.com         2
Third Party     youtube.com             1
Third Party     ytimg.com               1
Third Party     5min.com                1
Third Party     tacoda.net              1
Third Party     adadvisor.net           2
Third Party     dictionary.com          1
Visited         reference.com          32
Third Party     sfdict.com              1
Third Party     amazon-adsystem.com     1
Third Party     thesaurus.com           1
Third Party     quantserve.com          1
Third Party     googletagservices.com   1
Third Party     googleadservices.com    1
Third Party     googlesyndication.com   3
Third Party     imrworldwide.com        3
Third Party     doubleclick.net         5
Third Party     legolas-media.com       1
Third Party     googleusercontent.com   1
Third Party     exponential.com         1
Third Party     twimg.com               1
Third Party     tribalfusion.com        2
Third Party     technoratimedia.com     2
Third Party     chango.com              1
Third Party     adsrvr.org              1
Third Party     exelator.com            1
Third Party     adnxs.com               1
Third Party     securepaths.com         1
Third Party     casalemedia.com         2
Third Party     pubmatic.com            1
Third Party     contextweb.com          1
Third Party     yahoo.com               1
Third Party     openx.net               1
Third Party     rubiconproject.com      2
Third Party     adtechus.com            1
Third Party     load.s3.amazonaws.com   1
Third Party     fonts.googleapis.com    2
Visited         nytimes.com            21
Third Party     nyt.com                 2
Third Party     typekit.net             1
Third Party     newrelic.com            1
Third Party     moatads.com             2
Third Party     krxd.net                2
Third Party     dynamicyield.com        2
Third Party     bizographics.com        1
Third Party     rfihub.com              1
Third Party     ru4.com                 1
Third Party     chartbeat.com           1
Third Party     ixiaa.com               1
Third Party     revsci.net              1
Third Party     chartbeat.net           2
Third Party     agkn.com                1
Visited         cnn.com                14
Third Party     turner.com              1
Third Party     optimizely.com          1
Third Party     ugdturner.com           1
Third Party     akamaihd.net            1
Third Party     visualrevenue.com       1
Third Party     batpmturner.com         1

Results with Tracking Protection turned on

Connected with 33 sites

Visited         google.com              3
Third Party     google.com.au           0
Third Party     gstatic.com             1
Visited         techcrunch.com         12
Third Party     aolcdn.com              1
Third Party     wp.com                  1
Third Party     wordpress.com           1
Third Party     gravatar.com            1
Third Party     twitter.com             4
Third Party     grvcdn.com              1
Third Party     korrelate.net           1
Third Party     livefyre.com            1
Third Party     gravity.com             1
Third Party     facebook.net            1
Third Party     aol.com                 1
Third Party     facebook.com            3
Third Party     dictionary.com          1
Visited         reference.com           5
Third Party     sfdict.com              1
Third Party     thesaurus.com           1
Third Party     googleusercontent.com   1
Third Party     twimg.com               1
Visited         nytimes.com             3
Third Party     nyt.com                 2
Third Party     typekit.net             1
Third Party     dynamicyield.com        2
Visited         cnn.com                 7
Third Party     turner.com              1
Third Party     optimizely.com          1
Third Party     ugdturner.com           1
Third Party     akamaihd.net            1
Third Party     visualrevenue.com       1
Third Party     truste.com              1

 Discussion

86 site connections were reduced to 33. No wonder it’s a performance improvement as well as a privacy improvement. The only effect I could see on content was that some ads on some of the sites weren’t shown; all the primary site content was still present.

google.com was the only site that didn’t trigger Tracking Protection (i.e. the shield icon didn’t appear in the address bar).

The results are quite variable. When I repeated the experiment the number of third-party sites without Tracking Protection was sometimes as low as 55, and with Tracking Protection it was sometimes as low as 21. I’m not entirely sure what causes the variation.

If you want to try this experiment yourself, note that Lightbeam was broken by a recent change. If you are using mozilla-inbound, revision db8ff9116376 is the one immediate preceding the breakage. Hopefully this will be fixed soon. I also found Lightbeam’s graph view to be unreliable. And note that the privacy.trackingprotection.enabled preference was recently renamed browser.polaris.enabled. [Update: that is not quite right; Monica Chew has clarified the preferences situation in the comments below.]

Finally, Tracking Protection is under active development, and I’m not sure which version of Firefox it will ship in. In the meantime, if you want to try it out, get a copy of Nightly and follow these instructions.

13 Responses to Quantifying the effects of Firefox’s Tracking Protection

  1. Thanks for publishing these results, it’s great to start quantifying these effects. Just a note about pref names: privacy.trackingprotection.enabled is a separate pref from browser.polaris.enabled. Enabling the polaris pref will enable the tracking protection pref, turn on DNT, and expose the tracking protection pref in the privacy preferences UI. Disabling the polaris pref will disable tracking protection and hide the UI (and leave DNT alone).

  2. Is there a way – as a user – to enhance the blocking list? I’d like to have “0” by using “easylist” list for example.

    Thanks for the great feature

    • There’s not currently an easy way to modify the blocklist right now, although that’s a great idea for power users. The blocklist is served by the Safe Browsing protocol, which has the advantage of being able to serve up large amounts of data incrementally, and the disadvantage of being more complicated.

      The prefs that control the blocklist are browser.trackingprotection.updateURL, browser.trackingprotection.gethashURL, and urlclassifier.trackingTable. The naming for these was chosen to match existing preferences for Safe Browsing phishing and malware checks. Anyone can start a Safe Browsing server and point these preferences to it. Mozilla’s implementation is at github.com/mozilla-services/shavar.

      • So this needs the SafeBrowsing-feature enabled?

        Doesn’t it send all kinds of data to Google since a few versions ago?

        • Hi Max,

          Safe Browsing is just a protocol that was invented by Google. It doesn’t actually require talking to Google, but Google is the default Safe Browsing provider for phishing and malware detection. If you prefer not to talk to Google, you can turn off phishing and malware detection (not recommended, from a security point-of-view) or point the phishing and malware Safe Browsing preferences to another provider, e.g. http://api.yandex.com/safebrowsing/. Mozilla is running its own Safe Browsing server to serve tracking protection lists, these don’t come from Google.

          Phishing and malware detection are independently controlled from tracking protection, though they both use the same Safe Browsing protocol and are enforced by the same part of the code base. The preference that Nick mentions in his post (browser.polaris.enabled) is sufficient to enable tracking protection and will not interfere with phishing and malware detection preferences.

          • Why is Mozilla using Google for Safe Browsing. I have read a lot criticism about Firefox because of this Feature. Especially from ├╝rivacy sensitiv people.
            Why is Firefox not using a Safe Browsing server from Mozilla?
            If you say Google has a good list then why is’nt Mozilla loading the list from Google and the Firefox users From Mozilla.
            I think this is a big point of criticism under Privacy sensitiv people.

            I’m sorry for my bad english.

          • Nicholas Nethercote

            See Monica’s response below. Also the Safe Browsing v3 API used by Firefox hashes all URLs so that Google cannot see which sites people are visiting. See https://developers.google.com/safe-browsing/ for details, this part in particular: “Privacy: API users exchange data with the server using hashed URLs, so the server never knows the actual URLs queried by the clients.”

        • (Reply to Alex @ 12:56, not Max — for some reason the reply button is missing from Alex’s comment)

          In answer to the question, why doesn’t Mozilla run its own Safe Browsing server for phishing and malware: that’s a very expensive proposition that we don’t have the expertise or resources to support right now. Mozilla should concentrate on its core competencies (building a browser) rather than trying to duplicate a pipeline that is essentially subsidized by Google, who has much more discretionary income than Mozilla.

  3. The variance is likely a result of real time auctions being won or lost by the 3rd party advertisers, and which tracking pixels are being loaded based on cookie data you would have picked up from moving around the web.

    That said, I am excited for this tech to make it into the stable channel.

  4. How does this interact with the tracking protection lists available in AdblockPlus ?

    Do those lists+ ABP make this feature redundant?

    • Nicholas Nethercote

      I know that you can run AdBlock Plus and Tracking Protection in tandem. I don’t know anything about AdBlock Plus’s tracking protection lists.

    • ABP and similar generally use nsIContentPolicy.shouldLoad to stop resources from loading. These content policy checks happen before the network channel is created, and before tracking protection checks. If you run ABP and tracking protection, it just means that ABP will most likely prevent many resources from loading before tracking protection sees them.