Mozilla Add-ons Community Blog
Categories: developers policy releases

Add-on Review Process Redesign

12 responses

Hello.

My name is Jorge Villalobos, and I’m the new (first, really) Add-ons Developer Relations Lead at Mozilla. I’ll be working on bringing the add-on developer community and Mozilla closer together. I have been an add-on developer for over 2 years, working on around a dozen add-ons during that time. I’ve worked on a few independent projects as well, Fire.fm being the most successful one, and the one I’m most proud of.

My initial focus in this role at Mozilla is to reduce the add-on review waiting times to a point where authors can have some certainty that their add-ons will be reviewed within a reasonable time frame. The current state of the queues is far from ideal, with the recent release of Firefox 3.5 being a big contributor to the rising tide of submissions. The queues are long, and add-on authors are not happy. I actually have a somewhat important update for Fire.fm waiting in the update queue, and I can’t help but feel a bit impatient about it.

To solve the queue situation, we are working on several solutions. We’re constantly looking for and introducing new editors to our team. We are working more closely with them to understand how they work and what their concerns are, and also to focus their efforts in the areas that have the greatest needs. We are attacking the queue problem from several different angles, some which will help us in the short term, and some which are more forward-looking, such as the one I’m introducing here.

We want to change how we handle add-on reviews, specially for updates. Our current system doesn’t handle well the fact that there are add-on authors that no longer need to have the constant scrutiny of the editor team, and don’t need to have their updates reviewed every single time. We think we need to introduce a trust factor into the process, that allows us to give more freedom of publication to authors that have proven themselves trustworthy. There are plenty of those, and I bet they are the most active authors on AMO. Reducing the amount of update reviews we give to trusted authors will give more time to our editors to focus on new add-on nominations and other updates, significantly reducing waiting times and making everybody happy.

I also cover some ideas for reviewing add-ons that are not extensions, which usually have longer waiting times when in reality they should be the easiest to check.

If you’re interested in the details, please read the proposal on Google Docs: Review Process Redesign Proposal. It’s very short, so it shouldn’t take more than a couple of minutes to read. You can take part in the discussion of the proposal in the mozilla.dev.amo newsgroup, or post a comment here. I’ll try to respond to all as time permits.

12 comments on “Add-on Review Process Redesign”

  1. avih wrote on

    Great to hear of this new, much needed IMHO, function (Add-ons Developer Relations Lead). On the face of things, you seem to fit the job well. Good luck with it.

    I’ve read your review process redesign proposal yesterday and will read it again soon. I also have few suggestions of my own which I’ll post on the new forum soon enough.

    While we’re at the new forum, may I suggest to move the AMO development discussions to that board as well? It can replace the function of mozilla.dev.amo, and a forum is usually easier to navigate, have distinct sections, etc.

    After all, extension developers are among your users, and our users are also your users. At least theoretically, we share the same goals, although from slightly different perspectives.

    Anyway, good luck again.

  2. RandomEngy wrote on

    Great to hear. It’s such a pain to wait months for your updates to go through with no indication of how soon it will happen.

  3. Matt wrote on

    Why not show us where we are in the queue? It would alleviate a lot of the stress with not knowing. I submitted an update 2 weeks ago, I think there were 255 updates in the queue, If I am now sitting at 212/400, I’d rather know that then know nothing. 🙂

  4. city_zen wrote on

    Maybe this is not the right place to post this, but I couldn’t find a more suitable one.
    In the last couple of days I’ve noticed that experimental add-ons are no longer listed when a search is performed at AMO. Is this intentional? Can someone else confirm this behavior? (maybe I have some settings wrong, I don’t know).

  5. fcp wrote on

    Glad to hear some ideas to reduce the queue time but, honestly speaking, I am confused. What is new in your proposal? There have already been “trusted add-ons” whose updates become immediately available on the public listing without editor reviews.
    Do you propose to mark more add-ons as trusted? If so, I think that it is reasonable.

  6. zamaan wrote on

    welcome to the seat Jorge.
    my themes updates currently reviewed in 1 week-10 days time.
    i would be much mush happy if i can get the review result in max 3 days.

  7. Ookong wrote on

    In general, how long does it take to review an addon update?

    I submit a major update for Ookong on Aug 27, it is in the queue now. https://addons.mozilla.org/en-US/firefox/addon/9308

    I agree with Matt’s suggestion to show where my addon is in the queue.

  8. Jorge wrote on

    @Matt, @Ookong: we’re posting reports every Friday at the forum, which should give you a decent idea of how long you’ll need to wait. Keep in mind that our review order isn’t quite firs-in-first-out
    https://forums.addons.mozilla.org/viewforum.php?f=21&sid=e49f6118aa1803607867f64e01b55489

    @fcp: the idea is to have an in-between solution. Trusted add-ons are never reviewed, ‘reliable’ add-ons would be reviewed sporadically.

  9. fcp wrote on

    Thanks for the clarification. It might be a little complicated, but I wish that your idea works well.

  10. Itamar Borochov wrote on

    Welcome Jorge,

    Thanks for the importent information !

  11. Mike wrote on

    The trust factor you speak of has been around for a while, but what I don’t understand about it is this: I have 3 add-ons, 1 of them is trusted and the other 2 aren’t. Surely I, as the developer, am trusted and not the add-on?

  12. pcarbonn wrote on

    I have read the proposal, but not all the comments it generated. Here are some thoughts, that may have already been raised.

    – if each submitters reviewed another extension, it would significantly reduce the queue. Mozilla could design, propose and administer a kind of contract that would encourage such behavior. For example, the extension from a submitter who has done a review of another extension would go up in the review queue. Making 2 reviews would even bring it up further.

    – functional and security reviews require different skills. While anybody could do a functional review, a security review is better done by a specialist. Some role specialisation could help here. (on the other hand, a security review often requires an understanding of the functional domain).