A long standing problem with AMO have been the add-on review queues and waiting times.
For those unaware, we have a review system that all new add-ons and add-on updates have to go through before they are available to the general public. The system is a combination of automatic code analysis and manual code and feature testing. The manual side is handled by a dedicated team of volunteers, known as AMO Editors. We’re a small group that has had to handle a massive amount of work, specially since the release of Firefox 3.5. We had spent the last year catching up with the numerous new submissions and updates, and the numbers will only grow once 3.6 is out.
Just 2 months ago, the new add-on review queue had over 600 pending add-ons, and the update queue had more than 250. Both had several months of waiting times, and add-on authors were increasingly unsatisfied, with good reason. Many trivial or important updates have had to wait a long time before seeing the public, causing users to complain and give add-ons bad reviews for the slow reaction of their authors. That’s just unfair to add-on authors, and we needed a solution.
I joined Mozilla recently with a very clear objective: to reduce the review queues back to a manageable size. I have helped implement several solutions that would improve the state of affairs, like posting detailed weekly queue reports in the AMO forums, and temporarily hiring professional add-on developers to help with the burden. We estimate that our current volunteer team is large enough to keep up with the inflow, so now we just need to reduce queue size down to our ideal waiting expectations (2 weeks for nominations and 1 week for updates). After that I hope we will regain stability.
I’m happy to announce that our efforts have paid off. The latest queue report indicates that we’re down to 301 nominations, and 120 updates. That’s half of what we had 2 months ago! Just last month we reviewed over a thousand files, and our pace is increasing every week. About half of our pending reviews are within our ideal waiting time frames, and many of the ones that have been waiting for longer require an admin review for various reasons. Things are looking great.
There’s still lots to do in order to improve add-on author experience, and that is an ongoing effort. I’ve received quite a few suggestions on how to improve the reports, and I hope to implement those soon. If you’re a developer and you have anything else you’d like help with, don’t hesitate to contact me 🙂