I wanted to share a good example of how Firefox Support wasn’t just launched to fill the support need of our 270+ million Firefox users, but also to allow us to quickly discover new or emerging issues and escalate them so they reach the attention of the right people.
The background of this story is the problem many Firefox users experienced with anti-virus program BitDefender a few weeks ago, where BitDefender would quarantine one of Firefox’s program files — nssutil3.dll — treating it as a malicious trojan.
In September last year, Asa Dotzler initiated an effort to better monitor our marvelous world wide web for emerging during new Firefox releases to detect possible new problems with Firefox quickly. This is now known as the Release Rapid Response Team, or RRRT for short — and SUMO is an important part of it.
On the morning of April 17, our SUMO Live Chat administrator and general genius Matthew was anchoring the Friday morning Live Chat shift. In this shift, many Live Chat helpers were getting support requests from users with BitDefender. Matthew then filed a bug, alerted the RRRT, and set up a canned message so helpers could quickly respond.
A quick search in the support forum showed quite a few threads, so Cheng Wang, chemist, IRC hacker, and administrator of the support forum, set up “sumobot” in irc.mozilla.org channel #sumo to alert us whenever forum threads were posted with “nssutil3” or “bitdefender” in them, so we could quickly respond to those threads.
The first things we were telling users were that it was not actually a trojan and that we were working with BitDefender to figure out the false positive. SUMO contributors Noah and Quarantine deserve lots of props for making sure every thread about this issue got a quick response!
We had constant contact with Tomcat from QA, and Kev Needham, who hunted down BitDefender to get a first hand update about the situation. We were all in constant communication over IRC, so as soon as BitDefender released an update, we knew about it.
Once we got confirmation that the newest BitDefender fixed everything, Cheng posted to the bug and we were set. The whole thing was solved in less than 2 hours, which is amazing!
Many thanks to everyone on Live Chat for handing this on that end — these people really were pulling those chats out of the queue fast: noah, tmz, codylg13, dat, mzz, starpluck, leo, SliderMan, collin1000, and CoMpAnY.
A couple of weeks after this happened, I had the fortune to sit down in a conference room with Ken Kovash in Mountain View to discuss SUMO metrics. In this discussion, Ken helped me with setting up so we can see trended search terms on the website — search terms that are most rapidly changing in frequency in the last few days. So, rather than just looking at the most popular search terms on support.mozilla.com (which is almost always “bookmarks” and “clear history”), we can now also see trended search terms.
Ken and I looked back on the stats for April 17 and guess what? nssutil3.dll was at the top of the list! So now we have an even stronger zoom on our periscope lens in the ongoing hunt for problems on the web.
Expect to see this added to the Weekly Metrics shortly.
TMZ
wrote on