Category Archives: General

SUMO development in 2012

We are more than halfway through 2012 and this seems like a good time to stop and take a look back at the year so far. In particular I’d like to provide insights into how we used the available development time and what we were able to achieve for the community this year.

Search and Browsing on SUMO

Late in 2011 we had found out that our site needed some serious improvement in it’s search and information architecture. We tackled our search issues right away, and with the help of our UX designers and Susan, our information architecture expert, we spent the first 3 month of this year to come up with a site structure that would not only speak to our current users, but would also be flexible enough for future products. And while we implemented some of the resulting suggestions right away, we then took another quarter to test the new information architecture rigorously in the real world, before we wrote the first line of code. I’ll have more to say about that in a future blog post, but I wanted to take the time today to look at our development efforts for this year.

In very broad strokes, we spend the biggest part of our development time on moving to a new search engine and tuning that engine to return much better results than before. We also spent a large chunk of our time on measuring and displaying our key performance indicators (KPI), the metrics we take to measure our success as SUMO. More recently we have focused the majority of our effort on implementing the proposed new information architecture and a new “One Mozilla” design that will bring us visually closer to the existing Mozilla ecosystem.

Where did all the time go?

For the first time this year, we also tracked what user roles we served with the development time we had. The following table charts the number of points we spent, each point stands roughly for one day of development. As you can see the total number of points varies throughout the year as the constellation of the development team changes. We separated the user roles into “contributor”, “user”, “sumo-team”, and “dev”, which is used for infrastructure work. That is not to say that the classification was always clear cut. For example a lot of the development time  spend on contributors was also beneficial for the sumo team and the other way around. You can see all individual sprints on Scrumbugs, if you want to have a closer look.

 

Legend:

  • Contributor: Code changes that affected contributor of the site the most
  • User: Code changes that affected visitors of the site the most
  • SUMO Team: Code changes that affected SUMO staff the most
  • Development: Infrastructure work that didn’t benefit any one role in particular.

Community Focus

As is clear from the data we put our focus on the user experience this year. However we also invested heavily into the experience for the most important aspect of SUMO: contributors. I don’t want to list the dozens of individual bugs that were fixed and smaller feature that were added as you can quite easily see each of our sprints on Srumbugs. but I wanted to take the time to mention some of them as a reminder of what we worked on for individual contributors this year.

Forum contributors

  • Forum contributors can now add links to KB articles easily by searching for them in the “add link” overlay, which removes the need to open a new window, doing the search and linking manually.
  • We are now hiding questions that are older than 90 days and don’t have any answers from most parts of the site, as well as hiding all threads that are older than 180 days, so that we can focus on the threads in the forum that need our attention the most
  • We added a feedback indicator to the forum, that now tells us how far we are away from our goal to answer all incoming questions within 72 hours.
  • Forum contributors can now use the magic hat, that will offer them various small snippets that they can use to ask for more information, or answer recurring questions with 3 simple clicks.
  • Also, we are now only bumping questions to the top when there is a reply and ignoring other activity, to make sure we can focus on the most important questions.

KB contributors:

  • KB contributors can now send messages to all recent contributors of an article, even if they approve their own edit.
  • We added a remaining characters counter to the search summary. Google only shows the first 160 characters of a site as the preview, and after deciding to make that the search results summary we also added a counter to make sure we stay within the limits.

Localizers

  • The “approved” mail notification now mentions the approver and the changes in the body, so that people can save a round trip to SUMO.
  • In the same way, the “ready for localization” email now features a diff, so contributors can tell from their inbox how much effort a new update would take.
  • We are now showing messages above English articles, if the visitor is coming from a localized article, and we ask people to help us with the translation of that article into the language of the visitor.
  • Localizers can now see all navigation articles on their L10n dashboard, which saves us searching for it on various documentation pages.

Army of Awesome:

  • The Army of Awesome snippets can now be easily edited on SUMO, since they are KB articles now, instead of being in code.
  • The Army of Awesome has now statistics that more accurately display the effort of our community and our goals.
  • SUMO members can now stayed signed in on the Army of Awesome and answer tweets without having to log-in to Twitter each time.

All:

  • We have added a simple quoting feature for private messages, that now makes it possible to cite text in a reply and giving context to a reply.
  • We have adjusted several time frames on the KB dashboards to be able to react to changes more quickly
  • Most parts of SUMO now feature time stamps that consider the time zone of the user instead of being fixed to Pacific Time
  • Support forum and discussion forum posts of users are now displayed on the profile pages of all users for quick reference.

What’s next?

All in all I’m very proud of what we all have done so far and I can’t wait to see in production the many changes that are planned for this year, the most important of which is currently under development: The new information architecture and redesign. The new iA has been on our minds for most of this year already offers tremendous wins for users and contributors alike, and I’ll soon blog about the project and its expected effect on our site. For now, a big thank you goes out to all sumo developers, contributors and members. You are what makes SUMO so awesome!

Firefox for Android Sync screencast

Firefox for Android has been rebuilt with a native front-end for performance and usability. The new app includes an update to the killer Sync feature that brings your tabs, bookmarks and settings from your desktop and tablet to your smartphone.

This screencast provides a high-level overview of Sync setup and use. It also shows how easy it is to completely clear all of your private data, history, bookmarks and top sites from the app when you want to.

This screencast was created with androidscreencast.jnlp, so there is some jerk and gradients that don’t reflect the true beauty and feel of the new mobile browser. Enjoy it on your Android 2.2 smartphone when the Beta lands on Google Play later this month!

Firefox for Android demo screencast

Firefox for Android has a speedy new front-end that we’ve been documenting on SUMO in preparation for the Firefox 14 release of the nativeUI to Google Play sometime this month. In this screencast, I show how Firefox on Android 4.0 renders WebGL and HTML5 pages with a bunch of fun interactive games, videos and animations that I control by touching or tilting the screen of my Samsung galaxy nexus.

This screencast was created with androidscreencast.jnlp so the frame-rate adds some jerk and gradients. You can have the buttery smooth experience after you install the new beta on your Android 2.2 device later this month.

Many of the demonstrations included are from Mozilla Developer Demo Studio.

SUMO Thread analysis: Better Tools

At SUMO we always want to help our contributors help our users. To further this goal, we have begun analyzing SUMO threads, to see how we can help contributors improve their responses, and hopefully help more users. To help this goal, I and some other members of the SUMO team have analyzed a week’s worth of threads to try to find holes in our current process.

The Process

We analyzed all threads created on SUMO from April 1 to April 7. There were 365 total threads for this time period. We then arranged these threads into categories, 41 to be precise. These categories range from “Website Looks / Acts Wrong” to “Firefox Crashes” and everything in between.

Now, out of these categories, 23 of them (or 56%) have less than 5 threads per category, which was too small for analysis. Another was because of the Java blocklisting, which while it may be interesting for another analysis, for this case we just discarded those threads.

Then, we chose those categories that had the most useful information for our purposes. How many threads had responses, how many were useful responses, how many were just general replies (like “Try in Safe Mode”), etc. These threads turned out to be the following:
Firefox Crashes (19 threads)
Website Looks /Acts wrong (41 Threads)
Firefox is Slow (10 Threads)
Problems Caused by Plugins (10 Threads)

The Results

Out of all these threads, it became apparent that over half (50-60%) of threads have just very basic, general troubleshooting answers. These threads were also the ones that had very low (~20%) reply rate from the original poster. It seems from this that users want to reply when a contributor addresses their issue more directly than when there is just a general answer.

The answers that we found were not General Answers or Solid Answers were as follows:
Needs better Troubleshooting (roughly 12% of questions)
Too technical (Roughly 7%)

The rest of the questions that did not fall into these 3 categories were those with actual solid answers. While they may not have had a solution marked, they did have answers that, from how well they were written and how they addressed issues relevant to the original poster, seemed that they would have a high chance of fixing the problem. If we can increase the rate of users coming back to SUMO to update their questions, that will help the number of “Solid Answer” threads go up.

One good thing we found in our analysis was out of the 4 main groups of questions, only one question did not have an answer! This means we are doing really well on making sure we reply to 100% of threads.

Solutions

From the analysis, it seems that if we can help contributors provide more useful answers, we should begin to notice a higher percentage of solved and solid answers. To help with this goal, we have come up with a few different suggestions that we can begin to implement immediately:

Contributor guidelines:
Provide documentation on SUMO for contributors. This can range from how to begin diagnosing different issues (crash IDs, extensions, websites, etc.) to just general help for interacting with different users. We can give common issues, how to reply to them, tools to suggest to users, tools not to suggest, etc.

Contributor Workshops:
Beginning Class: Once every X weeks we have a class to teach people who want to contribute or have recently started contributing the basic ways to respond to threads, troubleshoot, act professionally, etc. These don’t even have to be ran by Mozilla Staff, experience and trusted contributors could be asked to help run these.

Special Guest Class: Developers, SUMO Staff, etc. can come and have a webinar to explain new features in firefox, how they work, what some common issues may be or are, and types of feedback they are looking for. Example, a Firefox dev in charge of the pdf.js feature could have a session about what it is, a basic overview of how it works, some known issues, how to fix them, and asking the community to keep an eye out and give feedback on X Y and Z. Then have a Q & A Time.

Specialty Webinars: Every so many months, or as needed based on feedback, the SUMO Staff gives sessions on diagnosing Hangs, how to read a Crash ID, website troubleshooting, etc.

Help Wanted!

Now, all of these are just ideas, for now. Obviously the sooner we get better tools to the community, the sooner we can improve the service we give to End-Users. We would love to get feedback from the community on ways they think that we can improve the currently available tools. Nobody knows ways we can help the community better than the community, so the more input we can get from you, the better! You can ping me on IRC (:Tyler), send me a message on SUMO (tylerdowner) or leave a comment. I’d love to hear from you and hear all your ideas! Specifically we would love feedback on these areas:
1. How useful is this current analysis, do you want to see more information from it, should we repeat it, if so how often, etc.
2. Do you feel these tools will help the community (you!), do you have suggestions, or even totally new ideas?

Mobile Michelle: Up Up and Away

I joined Mozilla last month to serve as the SUMO Firefox Mobile Support Coordinator and it has been so exciting I wanted to invite you to become a Superhero with me.

Superpowers don’t come easy, so I’ve been participating in lots of exercises to increase my strength.

I attended a brainstorming session about use cases for mobile Firefox during my first week. We discussed everything from educational tasks, such as learning the recipes for a variety of mixed drinks, to traditional business communication needs. It was a passionate dialogue about mobile users, their characteristics, and how those characteristics drive needs on the mobile web.

There were a few takeaways from this think tank that I think we should consider when designing mobile support:

  • Every piece of data is useful and precious
  • Sometimes a data plan is too expensive and free wifi is not readily available
  • The internet can cause fatigue

I keep these in mind while planning the project to achieve SUMO’s mobile goal for this quarter, to make Android a top tier platform on FIrefox for Mobile Help. The plan proposes development of new knowledge base articles to describe Troubleshooting, Privacy & Security, and Advanced topics along with documenting new features. I also plan on making some changes to existing content to optimize for small screens, improve visibility of support resources and grow our community of experts.

Do join SUMO and participate in developing this content and in answering user questions with me. It has been incredibly rewarding to help new Firefox for Android users and I think you’ll enjoy it too. Just register and add some feedback on the Discussion tab for any article that you think needs some work, every extra voice will improve the final articles a great deal.

Firefox for Android is really new, so if you want to influence the direction of this technology, you should join in the conversation on the #mobile channel, I’m michelleluna, so say hello! All of the development and test engineers along with user experience designers, marketing and product managers and new folks like me hang out there.

If you’re in San Francisco, join our next meetup where we talk about Firefox for Android and enjoy brewery delights.

Help us test private messaging and group dashboards today

Hey everyone,

This is a quick reminder about our test day for private messaging and group dashboards today, Friday, July 8th, from 3pm to 12pm UTC. We want to test the recently finished implementation of private messaging and the groups dashboards on SUMO, and will meet on IRC, in #testday (please note, it’s not #sumo).

Especially for private messaging it’s important to have many people on IRC at the same time. So if you can manage it, please join us between 3pm and 7pm UTC , but of course we will be around all day. Also, we have a test plan ready, so you can systematically check if things are working as they should.

If you are a localizer, you can also request the group dashboard feature. This will add another tab to your dashboard and those of your team members. That tab will hold your localization dashboard and a message on top that you as the locale leader can change to message your team. Also, your localization group will get a profile page listing the locale leader and every member of the team. You can see here what the German localization group profile looks like.

For the groups dashboard I wrote a short tutorial that you can see here.

Please report any issues that you encounter on this etherpad so we can fix them quickly.

All info about the testday:

Date: Friday, July 8th
Time: 3pm to 12pm UTC
Place: IRC #testday (please note, it’s not #sumo)
Testplan: https://wiki.mozilla.org/QA/Execution/Web_Testing/SUMO/2.8
Feedback: http://etherpad.mozilla.com:9000/kitsune-groups-and-messages

Thanks, and hope to see you later today!

SUMO is adding new features: group dashboards and private messaging

Hello everyone,

Starting today we will be beta testing the new groups and private messaging features of Kitsune. We want to start with a small number of people for now, so we are going to cap this at about ten. We will have a test day for everyone on July 8th. If you are interested in testing private messaging and giving feedback on it, please let me know in this thread on the support community forum and I’ll activate it for you.

If you are a localizer, you can also request the group dashboard feature. This will add another tab to your dashboard and those of your team members. That tab will hold your localization dashboard and a message on top that you as the locale leader can change to message your team. Also, your localization group will get a profile page listing the locale leader and every member of the team.

You can see here what the German localization group profile looks like.

For the groups dashboard I wrote a short tutorial that you can see here.

Private messaging should be fairly intuitive. Once the feature is activated, just click on “Inbox” on the upper right corner of the page. Make sure that you only message people who are listed in the forum thread for now.

Please report any issues that you encounter on this etherpad so we can fix them quickly.

And of course, please let me know if you have any questions.

Thanks!
Kadir

Localization for Firefox 5 and beyond on SUMO

If you are a SUMO localizer now is a good time to prepare for the upcoming Firefox 5 release. Firefox 5 will be released on June 21, which is less than 2 weeks away and our articles are now ready to be localized. There are 7 updated articles for Firefox 5, and many of the updates are rather small, so you should have no problem updating your articles if you had them up to date for Firefox 4 already. As we have discussed before, in the future there will be 4 weeks of localization time beginning from the third week of the Beta period for every new Firefox release. But for Firefox 5 the time for localization is a bit shorter since the whole Firefox 5 development cycle is kept shorter than normal.

Michael Verdi has create a special page that let’s you easily keep track of all the articles that we have changed between Firefox 4 and Firefox 5. If you are a localizer you should bookmark that page so you can see at a glance the upcoming changes for future versions of Firefox.

As always your Localization Dashboard is the most important metric to see what to localize or update next. If you keep the top 20 articles localized at all times you cover over 50% of all visits to the KB already. But with upcoming new Firefox versions we can sometimes predict that an article is going to be very popular after the release, even if it doesn’t show up on the Localization Dashboard yet. So we’ll make sure to inform everyone about those cases.

If you have any questions about the process, an update is unclear or you want to help localize support articles into your own language, please don’t hesitate to contact us in the community forums.

SUMO in Ljubljana

Mozilla Balkans meetup group photo
The beautiful city of Ljubljana, shortened to Lublana by its natives (I wonder why), the capital of Slovenia, was the host to the second Balkans Meeting this year. I had the chance to represent the SUMO team this time, and ended up locked up in a cell in Ljubljana’s military prison. Read on, if you want to know why.

Last weekend was the second installment of the Balkans meeting, and this time it was much more hands-on oriented. We had the full day on Saturday to cover a whole range of subjects from SUMO, product L10n, QA to Add-on development. For SUMO this year was a year of changes in many aspects, and during my talk I used the chance to explain why we did what we did, and how we thought it would bring us closer to support each and every one of our 400 Million users worldwide.

The second half of the day was reserved for sprints and hands-on action. Since we had just released a brand new KB, this was an excellent opportunity to see the KB used in real live, and learn from the feedback. The participants worked mostly on the top 20 articles in the KB, which serve almost 50% of all visitors to the KB and have a really high benefit-cost ratio.

Particularly interesting was the feedback, most of it about the actual localization experience in the editor. Generally it was geared towards making the editor more user friendly (smaller fonts for more text, resizing of the editor window etc) and the localization experience smoother. In the latter case we are looking into a few ways to do that already, for example by providing a Google translated text as the base for the initial translation of the English text.

After a pretty solid 9 hours of work we headed for dinner in a traditional Slovenian Restaurant, with horse meat as a specialty. I didn’t try that, but my meal was delicious. There we also met our surprise guest for the evening, the new Mozilla CEO Gary Kovacs, who was adopted as a Balkan member for his Hungarian heritage ;)

Working hard during the SUMO sprint

After the hands-on the day before, Sunday was used for the goal setting process, which involved a review of the goals from the last time and their usefulness. The goal setting itself was pretty great, we had an etherpad where we collected the goals, the document was projected to the big screen then and live edited by a dozen people, which made for interesting discussions.

I’m happy to say that the Balkan communities decided to name the localization of the Top 20 SUMO articles into all Balkan languages as one of those goals, and I think the new KB had quite some influence here. There will be follow up calls to talk about the details of each goal, but all in all they look pretty good, and will surely give everyone enough to work on until the next Balkans meeting (I heard rumors about Athens, but who knows ;)

My cell in Ljubljana

So, how did I end up in a cell? Well, that was thanks to Matjaz, our wonderful host from Slovenia. He organize a hostel for the participants, but not just any hostel, it was a former military prison, taken over by university students and turned into a hostel, with the cells left intact and designed by different architects. It’s considered one of the best hostels worldwide, and I can really recommend spending a night behind bars, it’s unusual to say the least ;)

Big William is watching us

I’d like to thank everyone attending this years Balkan meeting, it was a pleasure to meet and work with you all, and special thanks to our host Mathjaz and the organizers, William and Milos, it was a productive and intense meeting, but with lots of opportunities for informal talks and get-togethers. I really enjoyed it, thanks for putting it all together!

The new SUMO is here, let’s put it to the test

We started working on our new SUMO Knowledge Base back in March and now, 9 months later, it’s finally here! We released our new KB to the public last night and it’s working great. Go check it out: http://support.mozilla.com. And come back for a localization sprint tomorrow (more about that below).

We are really excited, exhausted and extremely happy — this was a huge undertaking. After the months of planning and development, we spent the last few weeks testing the migration of thousands of articles to our new system. The good news is that all of that testing payed off. Thanks to everyone who contributed during the QA day, and other times, the migration went smooth, with no catastrophic problems whatsoever. Alas, no matter how much you test, you never catch all the issues. So, we are dedicating the next few days to finding and solving the little bugs we’ve found. Please let us know about any issues in #sumo or #sumodev on IRC or — better yet — file a bug so we can take care of it.

Localization Sprint

A new system always poses a lot of questions, especially for those who need to work with it. So we decided to have a Localization sprint to check our top articles and update or translate articles for Firefox 4. If you are a localizer, this gives you the chance to check your top 20 articles and report any issue to us right away. This way, we can try to solve it on the spot. It will also give you hands-on experience with the new KB for updating or translating articles for Firefox 4. And we’ll be ready on IRC to answer any questions about the syntax, templates (formerly, content blocks) or anything else. And we need your help, even if you’re not a localizer: All those newy imported articles need a search summary to show up on the search results page. Help us add those summaries.

Join us on Thursday, December 2nd. The sprint will start at 6am PST and last until 2pm PST. That’s 3pm to 11pm in Central Europe, or 2pm to 10pm UTC/GMT – yeah, we’re a global project ;)

Looking forward to seeing you all and putting the new KB to the test!