Last fall we did a report on the SUMO support forum to assess the current situation, so we could make informed decisions about what to work on next. In particular, we wanted to know what was keeping us from solving all issues that were brought up in the forums. After analyzing several thousand support forum threads, we came up with a number of suggestions and decided to implement the low hanging fruits right away. Among those low hanging fruits were:
- We were dropping the ball on users, not getting back to them.
- We were telling users that we didn’t know how to solve their problem.
- There were off-topic issues that we didn’t know what to do with.
- People often didn’t tell us when a solution didn’t help, because it was hard to do so.
We knew that we had to address these issues, if we wanted to increase the “solution” rate. So, we came up with a number of programs and platform changes late last year to eliminate those issues. Specifically, we had the following planned:
- Make it easy for contributors to tell what needs their attention, so we can drop the rate of cases where we leave them hanging to zero.
- Extend the reach of the helpdesk to cover those cases where our community doesn’t know how to move forward, and by marking such threads as “hard” allow for specially motivated contributors to become focused on hard issues.
- Mark things that are off-topic as such, so we can hide them from the general audience and can take special care of them.
- Users can mark an issues as solved from their email, but telling us that the solution didn’t work required them to log-in. A big an unnecessary hurdle. People should not need to log-in. They should just click on the link in the email and start telling us it didn’t work.
I’m happy to say that we were able to follow through with our plans and those solutions are active on our production systems already. We are now seeing the first results of those changes and I’d like to share them with you. However, I need to preface this with a warning: Unfortunately the timing for the roll-out of those changes was as bad as it gets. First we had the release of a Firefox version that coincided with the launch, then we had the Christmas holidays and then new year. Each of these events leads to highly irregular usage patterns on SUMO, so we need to take the data with a pinch of salt. However, without any further ado, here are two charts, that show the first results:
Questions states on a higher level
Questions states on a higher level as a stacked area chart
You can see very clearly that we have significantly reduced the number of cases where we left people hanging, the cases that needed our attention. That was one of the primary goals of our efforts and it seems like we’ve achieved that part with great success. Unfortunately this didn’t translate directly into an increase in the “solution” rate yet, but as I mentioned before, the previous few weeks were highly irregular in terms of usage patterns, so we’ll need to wait a little longer, before we can say reliably how our changes affected the overall goal of increasing the “solution” rate.
In the mean time, we are coming up with ideas on how to challenge the bigger issues, now that we have taken care of the low-hanging fruits. I’m incredibly excited about the opportunities in front of us and the new ways we’ll try to address them with. If you are interested in taking part in that discussion, head over to the community discussion forums, and join us in our quest to solve all the issues.