In the last 10 days, we’ve been running our second A/B test on SUMO to try a slight redesign of the in-product start page (the page you get to if you select Help from the menu of Firefox itself). This test is part of a bigger goal to reduce the number of people that leave the Firefox Support website immediately after visiting the start page — the so-called “bounce rate” of the page.

There can be many different reasons why people leave a website without interacting with it. When it comes to a support website, one of those reasons can be that the website isn’t helpful enough, or doesn’t provide sufficient instructions on how it should be used. This is something we are trying to minimize on SUMO so the support platform becomes as easy to understand as possible.

In order to improve the current start page, our first step was to figure out how people are using it today and identify areas where we could improve it. chofmann dug up a lot of the initial research about common web design mistakes which our start page was suffering from, and proposed some ideas on how we could use those insights when redesigning the page. chofmann and I then sat down and brainstormed about how we could improve the page, after which I created a simple mockup of our ideas.

Before we could actually test our ideas, we needed to turn the mockup into a polished web page that we would feel comfortable showing to our users, so we turned to Mozilla’s master of design and creativity, John Slater, who connected us with web designer Naz Hamid. The result of our collaboration can be seen below:

The new start page. Click on the image to see a version of it with notes explaining the differences between the current start page.

The test turned out to be successful. With the new start page:

• More people used the search box (+1.3%), which is the best way to use SUMO to find the solution to your problem.
• Fewer people left the site immediately without interacting with it (-0.5%), which means that more people are able to get their problem solved.

For the full report and many interesting insights about how people interact with this new page versus the current page, read the original blog post based on the full analysis of the A/B test by Ken Kovash and Mozilla intern Eric Hergenrader: Improving a User’s Experience with Firefox Support (part II).

Our effort to improve the support experience for our users will of course not end with this test. It’s an ongoing process and a continued focus of the SUMO team to make our support platform as easy to understand and use for as many users as possible.

When looking at the results of our test, it should be noted that the bounce rate is still very high (86%). As I mentioned earlier, there could be many other reasons why people quickly leave the website. One reason, that I suspect plays an important role here, is the fact that you can reach Firefox Support simply by pressing F1 on your keyboard. My theory is that many people accidentally do this when typing on a web page, leading to many unwanted visits to Firefox Support.

The most common SUMO bookmark?

That is one of our next things to test on SUMO: among the people that visit Firefox Support by pressing F1 on the keyboard, how many people close the website right away? Are the people that visit the site by selecting the Help option in the menu more interactive?

We will have the answers to these questions soon.