The Mozilla Support Blog

Rocking the helpful web since 2007.
Categories: about:sumo Contributor News Updates

2023 in a nutshell

Hey SUMO nation,

As we’re inching closer towards 2024, I’d like to take a step back to reflect on what we’ve accomplished in 2023. It’s a lot, so let’s dive in! 

  • Overall pageviews

From Jan 1st to the end of November, we’ve got a total of 255+ million pageviews on SUMO. We’ve been in a consistent pageview number drop since 2018, and this time around, we’re down 7% from last year. This is far from bad, though, as this is our lowest yearly drop since 2018.

  • Forum

In the forum, we’ve seen an average of 2.8k questions per month this year. This is a 6.67% down turn from last year. We also see a downturn in our answer rate within 72 hours, 71% compared to 75% last year. We also see a drop in our solved rate, 10% this year compared to 14% last year. On a typical month, our average contributors on the forum excluding OP is around 200 (compared to 240 last year).

*See Support glossary
  • KB

We see an increase over different metrics on KB contribution this year, though. In total, we’ve got a total of 1990 revisions (14% increase from last year) from 136 non staff members. Our review rate this year is 80%, while our approval rate is 96%, compared to 73% and 95% in 2022). In total, we’ve got 29 non-staff reviewers this year.

  • Localization

On the localization side, the number is overall pretty normal. Total revision is around 13K (same as last year) from 400 non-staff members, with 93% review rate and 99% approval rate (compared to 90% and 99% last year) from a total of 118 non-staff reviewers.

  • Social Support

From year to date, the Social Support contributors have sent a total of 850 responses (compared to 908 last year) and interacted with 1645 conversations. Our resolved rate has dropped to 40.74%, compared to 70% last year. We have made major improvements on other metrics, though. For example, this year, our contributors were responsible for more replies from our total responses (75% in total compared to 39.6% last year). Our conversion rate is also improving from 20% in 2022 to 52% this year. It means, our contributors have taken more role in answering the overall inbounds and have replied more consistently than last year.

  • Mobile Store Support

On the Mobile Store Support side, our contributors this year have contributed to 1260 replies and interacted with 3149 conversations in total. That makes our conversion rate at 36% this year, compared to 46% last year. And those are mostly contributions to non-English reviews.

In addition to the regular contribution, here are some of the community highlights from 2023:

  • We did some internal assessment and external benchmarking in Q1, which informed our experiments in Q2. Learn the results of those experiments from this call.
  • We also updated our contributor guidelines, including article review guidelines and created a new policy around the use of generative AI.
  • By the end of the year, the Spanish community has done something really amazing. They have managed to translate and update 70% of in-product desktop articles (as opposed to 11% when we started the call for help.

We’d also like to take this opportunity to highlight some Customer Experience team’s projects that we’ve tackled this year (some with close involvement and help from the community).

We split this one into two concurrent projects:

  • Phase 1 Navigation Improvements — initial phase aims to:
    • Surface the community forums in a clearer way
    • Streamline the Ask a Question user flow
    • Improve link text and calls-to-action to better match what users might expect when navigating on the site
    • Updates to the main navigation and small changes to additional site UI (like sidebar menus, page headers, etc.) can be expected
  • Cross-system content structure and hierarchy — the goal of this project is to:
    • Improve our ability to gather data metrics across functional areas of SUMO (KB, ticketing, and forums)
    • Improve recommended “next steps” by linking related content across KB and Forums
    • Create opportunities for grouping and presenting content on SUMO by alternate categories and not just by product

Project Background:

    • This research was conducted between August 2023 and November 2023. The goal of this project is to provide actionable insights on how to improve the customer experience of SUMO.
    • Research approach:
      • Stakeholder engagement process
      • Surveyed 786 Mozilla Support users
      • Conducted three rounds of interviews recruited from survey respondents:
        • Sprint 1: Evaluated content and article structure
        • Sprint 2: Evaluated the overall SUMO customer experience
        • Sprint 3: Co-design of an improved SUMO experience
      • This research was conducted by PH1 Research, who have conducted similar research for Mozilla in 2022.
  • Please consider: Participants for this study were recruited via a banner ad in SUMO. As a result, these findings only reflect the experiences and needs of users who actively use SUMO. It does not reflect users who may not be aware of SUMO or have decided not to use it. 

Executive Summary:

  • Users consider SUMO a trustworthy and content-rich resource. SUMO offers resources that can appropriately help users of different technical levels. The most common user flow is via Google search. Very few are logging in to SUMO directly.
  • The goal of SUMO should be to assist Mozilla users to improve their product experience. Content should be consolidated and optimized to show fewer, high quality results on Google search and SUMO search. The article experience should aim to boost relevance and task success. The SUMO website should aid users to diagnose systems, understand problems, find solutions, and discover additional resources when needed.

Recommendations:

  • Our recommendation is that SUMO’s strategy should be to provide a self-service experience that makes users feel that Mozilla cares about their problems and offers a range of solutions appealing to various persona types (technical/non-technical).
  • The pillars for making SUMO valuable to users should be:
    • Confidence: As a user, I need to be confident that the resource provided will resolve my problem.
    • Guidance: As a user, I need to feel guided through the experience of finding a solution, even when I don’t understand the problem or solutions available.
    • Trust: As a user, I need to trust that the resources have been provided by a trustworthy authority on the subject (SUMO scores well here because of Mozilla).
      • Modernizing our CMS can provide significant benefits in terms of user experience, performance, security, flexibility, collaboration, and analytics.
      • This resulted in a decision to move forward with the plan to migrate our CMS to Wagtail — a modern, open-source content management system focused on flexibility and user experience.
      • We are currently in the process of planning the next phases for implementation.
    • Pocket migration to SUMO
      • We successfully migrated and published 100% of previously identified Pocket help center content from HelpScout’s CMS to SUMO’s CMS, with proper redirects in place to ensure a seamless transition for the user.
      • The localization community began efforts to help us localize the content, which had previously only been available in en-US.
    • Firefox account to Mozilla account rebrand in early November.
    • Officially supporting account users and login less support flow (read more about that here).
    • This was a very challenging project, not only because we had to migrate our large codebase and very large data set from MySQL, but also because of the challenge of performing the actual data migration within a reasonable period of time, on the order of a few hours at most, so that we could minimize the disruption to users and contributors. In the end, it was a multi-month project comprising coordinated research, planning and effort between our engineering team and our SRE (Site Reliability Engineering) team. We’re now on a much better database foundation for the future, because:
      • Postgres is better suited for enterprise-level applications like ours, with very large datasets, frequent write operations and complex queries.
      • We can also take advantage of connection pooling via PgBouncer, which will improve our resilience under huge and often malicious traffic spikes (which have been occurring much more frequently during the past year).
      • Last but not least, our database now supports the full unicode character set, which means it can fully handle all characters, including emoji’s , in all languages. Our MySQL database had only limited unicode support, due to its initial configuration, and rather than invest in resolving that, which would have meant a significant chunk of work, we decided to invest instead in Postgres.

This year, you all continue to impress us with the persistence and dedication that you show to Mozilla by contributing to our platform, despite the current state of our world right now. To every single one of you who contributed in one way or another to SUMO, I’d like to express my sincere gratitude because without you all, our platform is just an empty shell. To celebrate this, we’ve prepared this simple dashboard with contribution data that you can filter based on username so you can see how much you’ve accomplished this year (we talked about this in our last community call this year).

Let’s be proud of what we’ve accomplished to keep the internet as a global & public resource for everybody, and let’s keep on rocking the helpful web through 2024 and beyond!

If you’re a looker and interested in contributing to Mozilla Support, please head over to our Contribute page to learn more about our programs!