{"id":1972,"date":"2018-04-13T10:00:12","date_gmt":"2018-04-13T09:00:12","guid":{"rendered":"http:\/\/blog.mozilla.org\/press-uk\/?p=1972"},"modified":"2018-04-13T10:00:12","modified_gmt":"2018-04-13T09:00:12","slug":"cliqz-graduation-report","status":"publish","type":"post","link":"https:\/\/blog.mozilla.org\/press-uk\/2018\/04\/13\/cliqz-graduation-report\/","title":{"rendered":"Cliqz Graduation Report"},"content":{"rendered":"<p><span style=\"font-weight: 400;\">In late 2016, Mozilla embarked on a series of product experiments with our strategic partner Cliqz GmbH, a privacy-centric technology company based in Munich, Germany. The full Cliqz experience includes search recommendations, anti-tracking, and anti-phishing protections, and is best seen in their<\/span><a href=\"http:\/\/addons.mozilla.org\/en-US\/firefox\/addon\/cliqz\/\"> <span style=\"font-weight: 400;\">AMO add-on<\/span><\/a><span style=\"font-weight: 400;\"> and standalone<\/span><a href=\"https:\/\/cliqz.com\/download\"> <span style=\"font-weight: 400;\">desktop and mobile browsers.<\/span><\/a><\/p>\n<p><span style=\"font-weight: 400;\">We chose to begin our Mozilla\/Cliqz collaboration in Test Pilot, our Firefox platform to test new features in front of eager tech-savvy users. Our main goal was to understand how Firefox users valued Cliqz recommendations in the address bar. We removed the other feature \u00a0components so that they wouldn\u2019t unduly influence our feedback from users. We launched the stripped-down Cliqz version 1 address bar in Test Pilot in January 2017.<\/span><\/p>\n<p><a href=\"https:\/\/blog.mozilla.org\/press-uk\/files\/2018\/04\/c1.png\"><img decoding=\"async\" loading=\"lazy\" class=\"aligncenter size-full wp-image-1973\" src=\"https:\/\/blog.mozilla.org\/press-uk\/files\/2018\/04\/c1.png\" alt=\"\" width=\"679\" height=\"335\" srcset=\"https:\/\/blog.mozilla.org\/press-uk\/files\/2018\/04\/c1.png 679w, https:\/\/blog.mozilla.org\/press-uk\/files\/2018\/04\/c1-300x148.png 300w, https:\/\/blog.mozilla.org\/press-uk\/files\/2018\/04\/c1-600x296.png 600w\" sizes=\"(max-width: 679px) 100vw, 679px\" \/><\/a><\/p>\n<p><i><span style=\"font-weight: 400;\">Cliqz version 1<\/span><\/i><\/p>\n<p><span style=\"font-weight: 400;\">Normal Firefox telemetry counts the number of searches users perform in Firefox, such as in the address bar or search bar. Cliqz recommendations take users directly to their destination and also answer some questions directly in the address bar. Therefore, a key Test Pilot hypothesis was that users would perform fewer search engine searches.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Our version 1 launch allowed us to start a telemetry baseline to understand usage data, such as search. With Cliqz, we saw approximately a 25% decline in Test Pilot users\u2019 search engine search vs. their prior activity. This result confirmed our hypothesis that users would see direct answers to a large portion of their queries when using Cliqz.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">The version 1 launch also gave us crucial initial survey feedback within our first week. The surveys highlighted some major recurring themes:<\/span><\/p>\n<ul>\n<li style=\"font-weight: 400;\"><span style=\"font-weight: 400;\">Functionality. Many users saw fewer bookmarks and history results than they normally see in Firefox<\/span><\/li>\n<li style=\"font-weight: 400;\"><span style=\"font-weight: 400;\">Design. Some users did not like the design of the bar or found it confusing<\/span><\/li>\n<li style=\"font-weight: 400;\"><span style=\"font-weight: 400;\">Trust. The UI seemed foreign, hence users trusted it less or weren\u2019t confident using it.<\/span><\/li>\n<li style=\"font-weight: 400;\"><span style=\"font-weight: 400;\">Search status. Cliqz does not change users\u2019 default search engine, but because this was unclear many users complained that they\u2019d lost their favorite search engine.<\/span><\/li>\n<\/ul>\n<p><span style=\"font-weight: 400;\">Our user experience leads pushed for a joint work week to focus on the issues solvable through design. We ignored the requests around bookmarks and history for the time being, since those were out of scope for an initial UX overhaul. We held a<\/span><a href=\"https:\/\/cliqz.com\/en\/magazine\/techblog-new-face-cliqz-quick-search\"> <span style=\"font-weight: 400;\">joint UX work week<\/span><\/a><span style=\"font-weight: 400;\"> with the Mozilla and Cliqz teams in Toronto. In just 4 weeks, the team created a dramatic improvement. We launched the Cliqz version 2 in Test Pilot in mid-March 2017.<\/span><a href=\"https:\/\/blog.mozilla.org\/press-uk\/files\/2018\/04\/c2.png\"><img decoding=\"async\" loading=\"lazy\" class=\"aligncenter size-full wp-image-1974\" src=\"https:\/\/blog.mozilla.org\/press-uk\/files\/2018\/04\/c2.png\" alt=\"\" width=\"974\" height=\"512\" srcset=\"https:\/\/blog.mozilla.org\/press-uk\/files\/2018\/04\/c2.png 974w, https:\/\/blog.mozilla.org\/press-uk\/files\/2018\/04\/c2-300x158.png 300w, https:\/\/blog.mozilla.org\/press-uk\/files\/2018\/04\/c2-768x404.png 768w, https:\/\/blog.mozilla.org\/press-uk\/files\/2018\/04\/c2-600x315.png 600w\" sizes=\"(max-width: 974px) 100vw, 974px\" \/><\/a><\/p>\n<p><span style=\"font-weight: 400;\">The blue autosuggest bar returned to the top of the results pane. This bar exists in Firefox and informs the user that hitting \u201center\u201d allows them to search. We restyled the \u201cSmartCliqz\u201d information cards, which includes weather, currency conversion, and sports scores. We added icon styling to indicate bookmarks and open tabs. Iconography, fonts and line-spacing more closely resembled the Firefox Awesome Bar standards. In short: we made Cliqz feel more like \u00a0Firefox.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">The results were dramatic. The version 2 improved the Cliqz experience across every expected measure and even an unexpected one.<\/span><\/p>\n<ul>\n<li style=\"font-weight: 400;\"><span style=\"font-weight: 400;\">User satisfaction increased vs. the version 1<\/span><\/li>\n<li style=\"font-weight: 400;\"><span style=\"font-weight: 400;\">There were no more design complaints<\/span><\/li>\n<li style=\"font-weight: 400;\"><span style=\"font-weight: 400;\">Cliqz recommendation click throughs increased<\/span><\/li>\n<li style=\"font-weight: 400;\"><span style=\"font-weight: 400;\">Survey mentions of users unable to find their default search were entirely eliminated<\/span><\/li>\n<li style=\"font-weight: 400;\"><span style=\"font-weight: 400;\">External search referrals increased 7%. While there was no numeric goal, this increase confirmed that users better understood how to use their favorite search engine<\/span><\/li>\n<li style=\"font-weight: 400;\"><span style=\"font-weight: 400;\">Surprisingly, keyboard navigation increased dramatically. This was not something we anticipated. However, it confirmed that keyboard-centric users were more comfortable and expected the version 2 to behave like Firefox<\/span><\/li>\n<\/ul>\n<p><span style=\"font-weight: 400;\">We\u2019ve since made more improvements to the Cliqz experience. We collaborated with the Cliqz UX team on user research, which lead to many contextual and functional changes. We went back to fix many of the things we noticed in Test Pilot but drew out of scope in the UX overhaul. We improved the frequency, speed, and presentation of history results. We added context to many recommendations, such as adding source and timestamp to currency conversion and weather cards.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Cliqz has taken their vast product improvements and moved them into their core AMO add-on and desktop browser. Our time in Test Pilot was incredibly valuable, but our experimentation here has come to an end. We\u2019ll soon be retiring Cliqz in Test Pilot, and interested users will see an opt-in offer to install the full<\/span><a href=\"http:\/\/addons.mozilla.org\/en-US\/firefox\/addon\/cliqz\/\"> <span style=\"font-weight: 400;\">AMO experience.<\/span><\/a><\/p>\n<p><span style=\"font-weight: 400;\">Thank you to all of our Firefox Test Pilots!<\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p>In late 2016, Mozilla embarked on a series of product experiments with our strategic partner Cliqz GmbH, a privacy-centric technology company based in Munich, Germany. The full Cliqz experience includes &hellip; <a class=\"go\" href=\"https:\/\/blog.mozilla.org\/press-uk\/2018\/04\/13\/cliqz-graduation-report\/\">Read more<\/a><\/p>\n","protected":false},"author":493,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[121],"tags":[],"_links":{"self":[{"href":"https:\/\/blog.mozilla.org\/press-uk\/wp-json\/wp\/v2\/posts\/1972"}],"collection":[{"href":"https:\/\/blog.mozilla.org\/press-uk\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.mozilla.org\/press-uk\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.mozilla.org\/press-uk\/wp-json\/wp\/v2\/users\/493"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.mozilla.org\/press-uk\/wp-json\/wp\/v2\/comments?post=1972"}],"version-history":[{"count":0,"href":"https:\/\/blog.mozilla.org\/press-uk\/wp-json\/wp\/v2\/posts\/1972\/revisions"}],"wp:attachment":[{"href":"https:\/\/blog.mozilla.org\/press-uk\/wp-json\/wp\/v2\/media?parent=1972"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.mozilla.org\/press-uk\/wp-json\/wp\/v2\/categories?post=1972"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.mozilla.org\/press-uk\/wp-json\/wp\/v2\/tags?post=1972"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}