Thanks for the compatibility reports!

10 responses

A few weeks ago, we launched the Add-on Compatibility Reporter, a way for Firefox users to let us know if their incompatible add-ons were working properly or having issues. As of tonight, we’ve received 25,000 reports on 2,000 add-ons for Firefox 3.6 betas alone!

We just dispatched emails to the developers of those add-ons as follows:

  • 105 add-ons that we think are compatible. These add-ons had more than 80% of reports saying the add-on worked fine. For these add-ons, we suggest that the developer test the add-on out themselves and then bump compatibility to Firefox 3.6.*.
  • 24 add-ons with compatibility issues. These add-ons had more than 80% of reports saying that the add-on was not functioning properly. We ask the developers of these add-ons to review the submitted reports and look into the issues mentioned before bumping compatibility.
  • 321 add-ons with mixed reports. These add-ons have received a number of reports, but we couldn’t make a guess as to the status. We encourage these developers to view the submitted reports and then test their add-ons before bumping compatibility.
  • 830 add-ons with at least one report, but not enough to determine anything. We’ve asked these developers to test their add-ons with Firefox 3.6 and bump compatibility if everything checks out.

The response to the Compatibility Reporter has been amazing, with almost 20,000 active users helping us out. Your reports will make a big difference in being able to ship Firefox 3.6 without add-on compatibility issues, so keep it up!

10 responses

  1. Alfred Kayser wrote on :

    There is a problem with the reporter itself.
    Any ‘problems’ reported are lack any information about submitter, the problem description itself, and which version of the addon (extension or theme), so these reports are completely useless to me, as they don’t indicate what (if any) problem there is.

    Secondly, the labeling of the Compatability dropdown is unclear, and thirdly it should not directly generate a ‘report’ but popup a simple dialog with a entry box for the problem description and a textbox to optionally include the submitters email address for contact.

  2. Wim wrote on :

    I agree with Alfred, there are no comments so it’s not clear what went wrong. Also I see for “Woordenboek Nederlands” version 0.0, which doesn’t exist, very unclear.
    Some compatible and working add-ons can only be marked as “This does not work” and not as This works”.
    The filter option on the right doesn’t seem to work properly.

    Seems lik users ar just clicking their way through their add-ons (without updating) and mark things as not working.

    This add-on needs some clarification I guess.

  3. Neel Desai wrote on :

    Nice…. Really nice….

  4. Justin Scott (fligtar) wrote on :

    @Alfred – We added the extension version this weekend, so that will appear in future reports. For problem reports, we offer the user the option to enter comments, but we don’t require it as people would put useless comments.

    @Wim – The report viewer displays 0.0 for reports submitted before we started requiring the version. We should display nothing instead of 0.0, which is a bug that will be fixed in our next release.

    Add-ons can only be marked as “works properly” if they are shown as incompatible with your version of the application. For example, in Firefox 3.6, if the add-on is already compatible with Firefox 3.6, you can’t report that the add-on works properly because we assume that it does work properly since it was marked as compatible by the developer.

  5. Mime Cuvalo wrote on :

    Just a heads up: my extension was one that got mixed reports. The extension does work with 3.6, I just haven’t updated the maxVersion number yet.

    I suspect the users marking as broken are doing so b/c they see the add-on has become disabled (several reports said ‘does not appear anymore’ which means it was disabled).

    The reporter should ignore the case where the extension is simply disabled.

    Otherwise, good tool. Thanks!

  6. Nils Maier wrote on :

    We got:
    469 reports of add-on working properly in Firefox 3.6 Beta (69%)
    212 reports reports of add-on issues in Firefox 3.6 Beta (31%)

    From what I saw after glancing over the reports many, if not most, of the “issue” reports can be attributed to the addon being disabled because of maxVersion.
    Reports like:
    “worked with 3.6b2 but deactivated in 3.6b3! Activate button grey and unclickable”, “Needs to be updated.”, “not yet updated”
    Not helping at all and giving wrong results.

    Unfortunately most of the other reports are simply useless to us, because they just say “issues” but give no details at all. Or users often restate they OS (“Windows Vista”) which is already clear from the other report information.

    In conclusion I do think the compatibility reporter is a nice idea, but the quality of problem reports need to be improved dramatically to make it really useful.
    I have not used the reporter myself yet, but the results give me the impression that the documentation and in-product help (wizards, descriptions and such) need to be improved. First of all users should not post problem reports if “maxVersion” is the issue.

  7. Sam Hasler wrote on :

    Any chance the number of reports for each add-on could be made available? It would be interesting to compare it to download and usage stats for add-ons to see if there is a difference between installation numbers and which add-ons users care about. It could be that the number of compatibility reports for an add-on is a useful metric for ranking on AMO or deciding whether to feature an add-on.

  8. Pete wrote on :

    As Alfred says, while this is a handy tool, there really needs to be a way of asking for further information from the problem reporter.

    It’s a really good start though – thanks!

  9. mozdiav wrote on :

    please, enable seamonkey support to add-on_compatibility_reporter extension.
    I try it, and it work for me. You can see my reports in database.
    All changes than neeed:

    diff -rp -U 8 add-on_compatibility_reporter-0.4-fx+tb/chrome.manifest add-on_compatibility_reporter-0.4-fx+tb+sm/chrome.manifest
    — add-on_compatibility_reporter-0.4-fx+tb/chrome.manifest 2009-11-19 10:36:42.000000000 +0200
    +++ add-on_compatibility_reporter-0.4-fx+tb+sm/chrome.manifest 2009-12-21 02:36:04.163099200 +0200
    @@ -5,9 +5,10 @@ locale acr cs jar:chrome
    locale acr ja jar:chrome/acr.jar!/locale/ja/
    locale acr nl jar:chrome/acr.jar!/locale/nl/
    locale acr pl jar:chrome/acr.jar!/locale/pl/
    locale acr sk jar:chrome/acr.jar!/locale/sk/
    locale acr uk jar:chrome/acr.jar!/locale/uk/
    locale acr vi jar:chrome/acr.jar!/locale/vi/

    overlay chrome://browser/content/browser.xul chrome://acr/content/view/mainOverlay.xul
    +overlay chrome://navigator/content/navigator.xul chrome://acr/content/view/mainOverlay.xul
    overlay chrome://mozapps/content/extensions/extensions.xul chrome://acr/content/view/extensionsOverlay.xul
    diff -rp -U 8 add-on_compatibility_reporter-0.4-fx+tb/install.rdf add-on_compatibility_reporter-0.4-fx+tb+sm/install.rdf
    — add-on_compatibility_reporter-0.4-fx+tb/install.rdf 2009-11-19 10:36:54.000000000 +0200
    +++ add-on_compatibility_reporter-0.4-fx+tb+sm/install.rdf 2009-12-21 02:19:23.564308800 +0200
    @@ -28,11 +28,21 @@

    {3550f703-e582-4d05-9a08-453d09bdfdc6}
    3.0a1
    3.1a1pre

    +
    +
    +
    +     {92650c4d-4b8e-4d2a-b7eb-24ecf4f6b63a}
    + 2.0a1
    + 2.1a1pre
    +
    +
    +
    +

  10. radu wrote on :

    mozila 3.6 has a very big problem with the game tribal wars
    its very hard to connect to a world