{"id":626,"date":"2009-06-25T09:23:16","date_gmt":"2009-06-25T16:23:16","guid":{"rendered":"http:\/\/blog.mozilla.org\/metrics\/?p=626"},"modified":"2019-09-18T12:05:41","modified_gmt":"2019-09-18T19:05:41","slug":"improving-a-users-experience-with-firefox-support-part-ii","status":"publish","type":"post","link":"https:\/\/blog.mozilla.org\/metrics\/2009\/06\/25\/improving-a-users-experience-with-firefox-support-part-ii\/","title":{"rendered":"Improving a User’s Experience with Firefox Support (part II)"},"content":{"rendered":"

We recently discussed<\/a> an optimization test at the Firefox Support site (support.mozilla.com<\/a>).\u00a0 At the time, we mentioned that:<\/p>\n

Next up, the SUMO team is planning to soon roll-out a much more substantial A\/B test at support.mozilla.com.\u00a0 With a somewhat radically different page design, our expectations for this next test are different.\u00a0 In other words, the more variation that is tested generally leads to bigger differences in the results.\u00a0 Stay tuned!<\/p><\/blockquote>\n

We just ran this “test #2” over the past week or so and we now have the results to share.\u00a0 For reference, the full project plan is here<\/a> and the two page versions we tested are below.<\/p>\n

Version #1 (Existing SUMO home page):<\/strong><\/p>\n

\"ab_test2_page3_blog\"<\/a><\/p>\n

Version #2 (New design for SUMO home page):<\/strong><\/p>\n

\"ab_test2_page4_blog\"<\/a><\/p>\n

What did the results say?\u00a0 Out of nearly 1.5 million visitors, we saw the following behavior:<\/p>\n

\"test_2_final_results\"<\/a><\/p>\n

What’s the impact of the “new design” page?\u00a0 Decreasing the bounce rate (or exit rate) by 0.5% translates to an improved user experience for approximately 260,000 Firefox users<\/strong> (annually).\u00a0 These are users who would have otherwise immediately exited the SUMO site (some perhaps feeling unsatisfied or unhappy).\u00a0 The one other critical metric for the SUMO team is the rate at which users utilize the knowledge base search box.\u00a0 Again, you’ll see that by this measure, the “new design” wins.\u00a0 11.9% of visitors searched vs. 10.6% on our control (or existing) page design.<\/p>\n

Thus, the SUMO team will now be implementing the “new design” as the permanent new Firefox Support home page.<\/p>\n

Moreover, we dug into the data here a little further.\u00a0 We asked ourselves, “of people clicking through or taking some action on either version of the SUMO home page, did one version affect the chances that the user clicked on something beyond their initial action?”\u00a0 The answer appears to be “yes”.<\/p>\n

First, let’s identify what are the other common actions taken by users at our two page versions, and then we can talk through what happened to users after this initial action.<\/p>\n

\"ab_test2_clickthru_actions\"<\/p>\n

The two most frequently clicked links on the existing SUMO home page \u2013 “Browsing basics” and “Customizing Firefox” \u2013 seem to provide content on their respective pages that is rarely used.\u00a0 During our test, we saw that less than 10% of users who navigated to “Browsing basics” further clicked on its relevant content.\u00a0 “Customizing Firefox” performed even worse, as less than 5% of users navigated to any of its content.\u00a0 Conversely, the two most popular links on our “new design” page \u2013 “Browse them all” (Popular Support Articles) and “Cannot clear location bar history” \u2013 appear to be clear improvements compared to the existing page.\u00a0 “Cannot clear location bar history” experienced 25% of its users following a suggested link and “Browse them all” saw 40% of its visitors use one of the suggested “Top Articles”.<\/p>\n

On many dimensions, this optimization effort at support.mozilla.com looks like a huge win for Firefox users.\u00a0 Please click over to the SUMO blog<\/a> for more details regarding future plans around further user experience improvements.<\/p>\n","protected":false},"excerpt":{"rendered":"

We recently discussed an optimization test at the Firefox Support site (support.mozilla.com).\u00a0 At the time, we mentioned that: Next up, the SUMO team is planning to soon roll-out a much more substantial A\/B test at support.mozilla.com.\u00a0 With a somewhat radically … Continue reading<\/a><\/p>\n","protected":false},"author":60,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[101],"tags":[],"_links":{"self":[{"href":"https:\/\/blog.mozilla.org\/metrics\/wp-json\/wp\/v2\/posts\/626"}],"collection":[{"href":"https:\/\/blog.mozilla.org\/metrics\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.mozilla.org\/metrics\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.mozilla.org\/metrics\/wp-json\/wp\/v2\/users\/60"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.mozilla.org\/metrics\/wp-json\/wp\/v2\/comments?post=626"}],"version-history":[{"count":0,"href":"https:\/\/blog.mozilla.org\/metrics\/wp-json\/wp\/v2\/posts\/626\/revisions"}],"wp:attachment":[{"href":"https:\/\/blog.mozilla.org\/metrics\/wp-json\/wp\/v2\/media?parent=626"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.mozilla.org\/metrics\/wp-json\/wp\/v2\/categories?post=626"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.mozilla.org\/metrics\/wp-json\/wp\/v2\/tags?post=626"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}