Category Archives: Forums

Your feedback for the new SUMO support forum needed

Hi,

We need your help in designing the presentation of the support forum statuses. If you have been following the contributor or the platform meeting, this is probably old news for you, but for everyone else: Big improvements to the support forums are coming.

A few month ago we set out to evaluate the support forum. The insights were partially surprising and partially confirmed our suspicions. The good news is: We are helping a huge number of people with their Firefox questions. However, the report also showed that we have two big issues:

  1. In too many cases people don’t come back, so we don’t know whether we are really helping them
  2. In too many cases we leave people hanging when they come back, either because we forget to get back to them, we’re out of our depth, or the question is not completely about using Firefox.

This quarter we set out to address these issues one by one:

  • We are looking into ways to let people easily reply in the forums when they receive a message from us, ideally they won’t need to even log in. This should increase the number of forum threads where we do get feedback on the solution we provided.
  • We have enlisted the help of the SUMO helpdesk to take care of issues we can’t solve alone. We can now escalate questions and have them spend time with us and the users to find the solution.
  • The heldpesk will also cover questions that are not completely about using Firefox.
  • We have introduced a new model for questions, specifically “states”. A question either “needs attention”, is “responded” to or is “done”. This way we can very easily tell which threads we haven’t responded yet, needing our attention the most.

Now, the challenge is to find a way to present all of this information in a way that is useful for as many contributors as possible. That includes current and future contributors. We have created a first mock-up, that you can see here: http://note.io/182uu9X The discussion around that has already started and you can follow along here: https://bugzilla.mozilla.org/show_bug.cgi?id=928901

We urgently need feedback on this, because it will be a large scale change, that will affect every contributor using the support forums today. In particular we are interested in answering the following questions:

  • What in particular are you looking for when using the support forums? Open questions, questions with lot’s of replies? Questions where you have replied?
  • Are their certain combinations of filters that make sense to you and that should be the default?

Please leave your replies in the forum. We will use this thread to develop this mock-up further.

Thanks!
Kadir

Open Badges 1.0: Minimum Awesome Achievement

sumo-kb-2013sumo-forum-1sumo-armysumo-l10n

Just before the Mozilla Summit back on September 30, we launched the first phase of SUMO Open Badges. They are: Minimum Awesome Achievement Badges for 2010, 2011, 2012 and 2013:

  • The KB badge for our awesome knowledge base writers (2010, 2011, 2012, 2013).
  • The Forum badge for our splendid English language forum support contributors (2010, 2011, 2012, 2013).
  • The Army of Awesome badge for folks who have contributed to the Mozilla SUMO twitter awesomeness (2010, 2011, 2012, 2013).
  • The L10N badge for our fantastic folks who translated KB articles from English (2010, 2011, 2012, 2013).

By “Minimum Awesome Achievement” we were trying to recognize significant non-trivial contribution on a yearly basis.

Four badges awarded algorithmically for four years: 2010, 2011, 2012 and 2013. In total: 16 badges. And we will issue these four badges automatically for 2014 and future years.

What’s next? Badges for the awesome folks who helped us with SUMO Live Chat support and badges for the “OG”s (the original SUMO gurus who contributed from 2007-2009). And adding our SUMO badges to the Mozilla Open Badges Backpack.

Got some ideas for future SUMO badges? Please add them to the future SUMO badge etherpad.

Want to be help lead the future of SUMO badges? I am looking for somebody to be a “co-driver”. All you need to do is read up about Open Badges and be passionate about them and be willing to work with me for about five hours a week (usually far less than five hours!). If you are interested email rtanglao AT mozilla.com

UPDATE: A big thanks to all who made SUMO open badges happen: Yvan for the awesome badge designs; Ricky and Will of the superb SUMO-DEV team for the SUMO badge code; Les for the excellent Django Badger code; splendid SUMO colleagues: Ibai, Rosana, Madalina, Kadir and Michael for feedback and guidance; Carla, Emily and Sunny of the Mozilla Foundation badges team for great Open Badges mentoring and ongoing badge system design advice; David of the community team for building the community of open badges within Mozilla and most importantly all those who contribute to support at mozilla!

Gravatar for Avatar

For a long time we have had small avatars on SUMO, but with the shiny new profile pages they look out of place. So we are going to support bigger avatars soon. But there is a twist: We won’t just increase the size of the avatars, we are switching to Gravatar instead. Gravatar is used by large number of online services including mozillians.org. This means that you don’t have to manage your avatar on SUMO separately unless you really want to. You can just upload an image once on Gravatar and have it show up on all kinds of webservices you are using.

Now, what does this change mean for you?

  1. If you are a new user and have a Gravatar account already, we’ll automatically use that when you create a new account on SUMO
  2. If you are a new user and don’t have a Gravatar account yet, you’ll get the default avatar as always and you can change your Avatar by going to the Gravatar page (linked from your profile on SUMO)
  3. If you are an existing user with a Gravatar account, we’ll replace the avatar you have here with the one in your Gravatar account.
  4. If you are an existing user without a Gravatar account nothing is changing for you. But if you want to change your Avatar later on, you’ll be linked to Gravatar.

If you don’t want your standard Gravatar image to be shown on SUMO, you can change your email address on SUMO. You don’t have to actually use a different email account. Just append “+SUMO” to it. So if my mail address is atopal@example.com it would become atopal+SUMO@example.com. This way you still get your email to atopal@example.com, but Gravatar won’t show the image here that you have set for atopal@example.com

This is now live on our staging server: http://support.allizom.org Please give it a try and let me know what you think. You can also comment directly in the bug for this: https://bugzilla.mozilla.org/show_bug.cgi?id=918854

The all new SUMO

Today we are going to make one of the biggest changes yet to SUMO, the Mozilla Support site, and this blog post is about what changes we are making. The changes will effect you the most as a user, but there are a number of changes for contributors as well.

First, a little history, what’s the SUMO team been up to this year?

For the last 9 month the SUMO team has been working on a new way to let users access our site content. Until recently the only way to reach most of our articles was by search, or by following links in articles. This is how wikis traditionally work. Of course that way of accessing content only works for a part of our visitors, some people want to search and some people want to browse to the solution, drilling down with ever finer topics to reduce the number of article that are related to the issue.

To come up with a new information architecture that would let people drill down like that, we first researched the mental model of our site users, how they think about issues and in what categories they would look for them. Based on that we created a small number of base categories and assigned our articles to those categories.

The next step was figuring out how to make this information architecture visible. We started to lay out a number of alternatives on paper and tested with real people in a lab. This paper prototyping gave us a way to test a number of ways to lay out the information very quickly. After a number of iterations we settled on the final designs and workflows.

Now we had everything to start adapting our site, but since this would be a big redesign, and we’d soon switch to the new unified One Mozilla design anyway, the decision was made to use this opportunity to rebuild the site based on the new theme, and that’s why the changes today not only affect the KB, but every part of our site.

So, what is changing? What does it look like?

The main change is, that we now support several products from one start page and all articles can be accessed by browsing. Let’s start with the start page:

We have the main topics on top, they allow you to start browsing by selecting your issue first, and then the product you have issue with.

One step below you can see the hot topics. Those are actually articles, things that came up recently and affect a large number of people. By providing them upfront we save a large number of people the hassle of searching or browsing for their solution.

Below that we have the product picker, this is a way to navigate our content by choosing the product first and then narrowing down the topics.

No matter what way you select, topic first or product first, you’ll end up narrowing down the number of articles to a scanable few and proceed to read one of the articles.

The important thing for localizers to note is: all of this is automated, there is no need anymore to create navigation pages and all the confusion that brought with it.

So, how did the article view change? On the surface not much has changed, but because we keep track of topics, we can now offer you a way to move to related topics, which is particularly interesting for people landing on articles from external searches:

Much, much more has changed, but this is the gist for the KB part of the site.

So, what has changed for forum contributors?

While the new iA did not touch the support forum per se, we took the redesign as an opportunity to improve a number of factors in the listing of questions for our contributors.

The new design is more friendly and clean, but at the same time gives more information about the thread content at the same time. This is especially helpful when contributors scan the thread listing page deciding which thread to pick next.

We already started rolling the design out to our contributor base over the last week and will start rolling it out to 1% of the general audience today. If everything goes to plan we’ll make it available to the general audience on Monday. If you want to try it out now, just register an account, and if you have any feedback, please use the comment section below.

The new information architecture will open up our content to a whole new group of users and make it much more accessible, while our new design is more coherent, taking into account all of the features we added since our first release while also being consistent with the Mozilla sites in general. All of this makes us very excited and hopeful that we’ll get that much closer to our number one goal: Happy users!

The new magic SUMO tool for frequent questions, and an interview with its developer, Tobias Markus

For a few weeks now we have been replying to almost every single question that is asked in the Mozilla support forums. That is up from about 50% last year and it shows the dedication of our community. Also over the last few month we have made a number of changes to SUMO to reduce the flow of simple questions that have answers in the KB already. That has been quite succesful, not only are we getting really hard questions now, but they are solved at a higher rate than before too.

The challenge now is that we have to ask for more information before we can solve an issue. Repeating those questions is quite tiring, so we decided to build a tool to help out our community and gather all the important questions to ask in one place.

We had actually planned to work on it later this year, but one of our great community members, Tobbi, saw an opportunity and offered to try his hand on it.

The result went online a few days ago, and is amazing. Sign into SUMO, and check it out for yourself:

  1. Chose a question
  2. Click on the response tool (the magic hat symbol)
  3. Select an answer or a request for specific information
  4. Customize as necessary and send!

Over time this tool will hopefully gather all the shared wisdom of our community and help us solve issues in as few steps as possible.

Interview with the developer, Tobias Markus.

This is not the first time a community member has been helpful in the development of Kitsune, the software that powers SUMO, but it’s the first major feature. We’d love to see much more of this, and I asked Tobbi for an interview to ask him how he got involved for anyone who wants to follow in his footsteps.

Tobbi, you have been a SUMO community member for quite a long time. Where did you start and how was your SUMO journey so far?

It was late 2008 when I started contributing to SUMO. My portal to the SUMO world was the Firefox LiveChat, which I joined as I really liked the personal touch that the chat had. Later on, I found out that there was a German localization group for SUMO articles, so I started translating articles into German and helped reaching the goal of then 100% localization coverage. I even gave forum support, although I preferred LiveChat because it was more personal than the forums.

How did you get the idea to help with the development of SUMO?

The idea for development started when I was working on a Firefox add-on together with Propeng. We implemented various features for the old forums to make them more contributor-friendly and add some tools that aimed at making life for contributors easier. However, due to the amount of unfinished code and the switch to Kitsune, the project was stalled. That’s when we decided to split the big project into smaller chunks to make them compatible again with the new forums. One of these smaller projects was the Knowledge Base article autocomplete that was landed a few months ago. I think it was Rosana who suggested integrating the Knowledge Base autocomplete into the Firefox support website directly. That’s when I started contributing to the development of SUMO.

Did you get any help with your project?

The SUMO team and other contributors really helped me a lot. At first I was posting an early draft to the contributors forum to gather suggestions from other contributors and the team. The feedback I got was huge. I spent the next couple of weeks reacting on it and making a lot of improvements to the code to enable things like instant preview, edit mode, and even flags that can limit certain responses to certain groups of contributors. The SUMOdev team also answered some implementation questions.

Once this was done, the next thing was to make the experience perfect for contributors. I brainstormed a lot with Bram [the SUMO UX designer] about best practices in order to get a good user experience design and spent some more weeks making the necessary changes.

The fourth step was a final code review by Ricky Rosario. That went smoothly as well, the necessary changes I had to do were minimal and most of them were about simple code style fixes. Once these were fixed, the code was ready to land.

The final step was to create the canned responses. I asked the community again to get back to me with some frequently used responses and the amount of responses I got was overwhelming. I chose around 10 of the most frequently used ones (based on my own judgement) and rewrote them a little. Kadir then helped me put each response in its own article and create a main article for the category structure that has links to all the individual responses.

What is your advise for people who want to get involved with SUMO development?

Drop by in #sumodev on irc.mozilla.org and say hello. The SUMOdev people answered any questions I had so that canned responses could become a success. Also, there is good documentation available online about setting up a local copy of SUMO. So, if you have a great feature in mind for the support website and want to start developing, give it a go!

Tobi, thank you for this interview, and in the name of the SUMO community: thanks for making everyone’s live so much easier!

Firefox support questions: on our way to 100%

Late last year, we set a goal for 2012 to achieve a 100% response rate in our community-powered support forum. In other words, every user who asked a question in our forum should get a response. It felt like an ambitious goal when we set it, because only six months ago, we averaged around just a 50% response rate.

Forum response rate within 72 hours

Average response rate has increased from ~50% to ~90% in just a few months.

A lot of things have changed since we set the goal, and they all contribute to this dramatic improvement in response rate:

  • Our community has grown bigger and is a lot more active. This makes answering questions more fun than before.
  •  We have started to host bi-weekly SUMO days focusing specifically on answering forum questions. Initially pioneered by Michelle, nowadays they are run by Madalina.
  •  We have improved the start page by introducing basic browseability that we’ve previously lacked. This has reduced the number of new questions in the forum and is part of the massive project to revamp the information architecture of SUMO to significantly increase the number of users who can help themselves on our site.

The increased response rate has a number of other side-effects as well, such as an increased chance of the user getting back to us to let us know if the answer solved their problem. The solved-rate has more than doubled since November last year.

This is just the beginning. We have yet to fully roll out the revamped site and information architecture mentioned above. We also just introduced some new cool features to the forum that makes it easier to see how many questions are still unanswered:

SUMO regular Tobbi is working on a Canned Response feature that will make it much easier to quickly respond to questions. You can watch a screencast of the work in progress. It’s already totally awesome!

Big thanks to everyone in the SUMO community who work hard to respond to our users’ questions! Without you, this wouldn’t have been possible.

Speaking of the forum… what are you waiting for? It’s SUMO Day today, so head on over and answer some questions! :)

The 8th SUMO sprint

Last Tuesday we finished the 8th SUMO sprint of this year. I’s been a short sprint, because for half of it Mozilla web developers met for a work week in California. Nevertheless we were able to land some great improvements to SUMO.

  1. One of our top priorities right now is improving our search results. To that end we started an initiative to rate KB and forum results as equals, based solely on the content instead of showing KB articles on top and forum posts at the bottom. We hope that this step will result in better search results by higher click through rates. This is a project that will take more than one sprint to be implemented. We completed the first of 3 phases in this sprint and are planning to implement the rest over the next sprints. You can read Will’s bug comment about it, if you want to know more about the gory details of the unification work.
  2. We were also able to implement a feature in the forums that gives our community an indicator for how we are doing and shows everyones impact on the forum: We call it the forum feedback indicator. It shows you how many questions were asked in the last 3 days and how many have a reply. It also links to the questions that haven’t been answered yet. The progress bar is orange, for any values below 100% and turns green when all questions are taken care of.

As with every sprint we also fixed a number of annoying bugs, and you can always follow along on the excellent scrumbugs. Here is the list for the 8th sprint.

However I’d like to point out one specific issue, that was bugging us in the forums for quite a while, creeping back when we thought we had squashed it: In some cases, a question displayed “No replies”, even though it already had replies. That bug is now hopefully fixed for good.

SUMO Thread analysis: Better Tools

At SUMO we always want to help our contributors help our users. To further this goal, we have begun analyzing SUMO threads, to see how we can help contributors improve their responses, and hopefully help more users. To help this goal, I and some other members of the SUMO team have analyzed a week’s worth of threads to try to find holes in our current process.

The Process

We analyzed all threads created on SUMO from April 1 to April 7. There were 365 total threads for this time period. We then arranged these threads into categories, 41 to be precise. These categories range from “Website Looks / Acts Wrong” to “Firefox Crashes” and everything in between.

Now, out of these categories, 23 of them (or 56%) have less than 5 threads per category, which was too small for analysis. Another was because of the Java blocklisting, which while it may be interesting for another analysis, for this case we just discarded those threads.

Then, we chose those categories that had the most useful information for our purposes. How many threads had responses, how many were useful responses, how many were just general replies (like “Try in Safe Mode”), etc. These threads turned out to be the following:
Firefox Crashes (19 threads)
Website Looks /Acts wrong (41 Threads)
Firefox is Slow (10 Threads)
Problems Caused by Plugins (10 Threads)

The Results

Out of all these threads, it became apparent that over half (50-60%) of threads have just very basic, general troubleshooting answers. These threads were also the ones that had very low (~20%) reply rate from the original poster. It seems from this that users want to reply when a contributor addresses their issue more directly than when there is just a general answer.

The answers that we found were not General Answers or Solid Answers were as follows:
Needs better Troubleshooting (roughly 12% of questions)
Too technical (Roughly 7%)

The rest of the questions that did not fall into these 3 categories were those with actual solid answers. While they may not have had a solution marked, they did have answers that, from how well they were written and how they addressed issues relevant to the original poster, seemed that they would have a high chance of fixing the problem. If we can increase the rate of users coming back to SUMO to update their questions, that will help the number of “Solid Answer” threads go up.

One good thing we found in our analysis was out of the 4 main groups of questions, only one question did not have an answer! This means we are doing really well on making sure we reply to 100% of threads.

Solutions

From the analysis, it seems that if we can help contributors provide more useful answers, we should begin to notice a higher percentage of solved and solid answers. To help with this goal, we have come up with a few different suggestions that we can begin to implement immediately:

Contributor guidelines:
Provide documentation on SUMO for contributors. This can range from how to begin diagnosing different issues (crash IDs, extensions, websites, etc.) to just general help for interacting with different users. We can give common issues, how to reply to them, tools to suggest to users, tools not to suggest, etc.

Contributor Workshops:
Beginning Class: Once every X weeks we have a class to teach people who want to contribute or have recently started contributing the basic ways to respond to threads, troubleshoot, act professionally, etc. These don’t even have to be ran by Mozilla Staff, experience and trusted contributors could be asked to help run these.

Special Guest Class: Developers, SUMO Staff, etc. can come and have a webinar to explain new features in firefox, how they work, what some common issues may be or are, and types of feedback they are looking for. Example, a Firefox dev in charge of the pdf.js feature could have a session about what it is, a basic overview of how it works, some known issues, how to fix them, and asking the community to keep an eye out and give feedback on X Y and Z. Then have a Q & A Time.

Specialty Webinars: Every so many months, or as needed based on feedback, the SUMO Staff gives sessions on diagnosing Hangs, how to read a Crash ID, website troubleshooting, etc.

Help Wanted!

Now, all of these are just ideas, for now. Obviously the sooner we get better tools to the community, the sooner we can improve the service we give to End-Users. We would love to get feedback from the community on ways they think that we can improve the currently available tools. Nobody knows ways we can help the community better than the community, so the more input we can get from you, the better! You can ping me on IRC (:Tyler), send me a message on SUMO (tylerdowner) or leave a comment. I’d love to hear from you and hear all your ideas! Specifically we would love feedback on these areas:
1. How useful is this current analysis, do you want to see more information from it, should we repeat it, if so how often, etc.
2. Do you feel these tools will help the community (you!), do you have suggestions, or even totally new ideas?

Thursday, March 29th is SUMO Day!

It’s time for a new SUMO day! This will happen on Thursday, March 29. We’ll be answering questions in the support forum and helping each other in #sumo on IRC from 9am to 5pm PST (UTC -7).

Join us, create an account and then take some time on Thursday to help with unanswered questions. Additional tips for getting started are on the etherpad. Our goal is to respond to every new question posted Thursday, so please try to answer as many questions as you can throughout the day.

Let’s make this an awesome day!

Madalina

SUMO Day – Mar. 15, 2012

This Thursday, March 15th is the next SUMO Day. We’ll be answering questions in the support forum and helping each other in #sumo on IRC from 9am to 5pm PST (UTC -7).

Just create an account and then take some time on Thursday to help with unanswered questions. Additional tips for getting started are on the etherpad. Our goal is to respond to every new question posted Thursday, so please commit to answering 10 questions throughout the day.

Grab your cape and join us! See you there.