Changes to storage.sync in Firefox 79

Caitlin Neiman

25 responses

Firefox 79, which will be released on July 28, includes changes to the storage.sync area. Items that extensions store in this area are automatically synced to all devices signed in to the same Firefox Account, similar to how Firefox Sync handles bookmarks and passwords. The storage.sync area has been ported to a new Rust-based implementation, allowing extension storage to share the same infrastructure and backend used by Firefox Sync.

Extension data that had been stored locally in existing profiles will automatically migrate the first time an installed extension tries to access storage.sync data in Firefox 79. After the migration, the data will be stored locally in a new storage-sync2.sqlite file in the profile directory.

If you are the developer of an extension that syncs extension storage, you should be aware that the new implementation now enforces client-side quota limits. This means that:

  • You can make a call using storage.sync.GetBytesInUse to estimate how much data your extension is storing locally over the limit.
  • If your extension previously stored data above quota limits, all that data will be migrated and available to your extension, and will be synced. However, attempting to add new data will fail.
  • If your extension tries to store data above quota limits, the storage.sync API call will raise an error. However, the extension should still successfully retrieve existing data.

We encourage you to use the Firefox Beta channel to test all extension features that use the storage.sync API to see how they behave if the client-side storage quota is exceeded before Firefox 79 is released. If you notice any regressions, please check your about:config preferences to ensure that webextensions.storage.sync.kinto is set to false and then file a bug. We do not recommend flipping this preference to true as doing so may result in data loss.

If your users report that their extension data does not sync after they upgrade to Firefox 79, please also file a bug. This is likely related to the storage.sync data migration.

Please let us know if there are any questions on our developer community forum.

25 responses

  1. rugk wrote on :

    I think you miss one bit of important information here. You do not mention the limit it actually enforces.
    So, actually, what *is* the limit, in numbers, I mean?

    1. Caitlin Neiman wrote on :

      You’re right — we’ll update the post. 🙂 These are the enforced limits: https://searchfox.org/mozilla-central/rev/8d55e18875b89cdf2a22a7cba60dc40999c18356/toolkit/components/extensions/schemas/storage.json#314-325 (the max amount of data, the max amount of data per item, and the maximum number of items).

    2. Mark Hammond wrote on :

      They are documented at https://developer.chrome.com/apps/storage#property-sync – we should get the same docs on MDN.

      1. Juraj M. wrote on :

        I’ve been testing the quotas and I can see I’m getting still the same error “QuotaExceededError: storage.sync API call exceeded its quota limitations.” for each type.
        Will there be a specific error message for each quota exceed type like in Chrome? I want to inform user what went wrong.

  2. tsalta wrote on :

    Please delete storage-sync-v2.sqlite-shm, storage-sync-v2.sqlite-wal like cookies.sqlite-wal, cookies.sqlite-shm, favicons.sqlite-shm, favicons.sqlite-wal when I exit Firefox.

    CCleaner > Custom Clean > Applications > Firefox > “Internet Cache” option deletes all *.sqlite-shm, *.sqlite-wal, so CCleaner deletes these two files too. storage-sync-v2.sqlite-wal store addon settings that uses the sync API, so I lose the addon settings changed that is not yet applyed to storage-sync-v2.sqlite.

    1. Mark Hammond wrote on :

      > CCleaner > Custom Clean > Applications > Firefox > “Internet Cache” option deletes all *.sqlite-shm, *.sqlite-wal,

      That’s a bug in ccleaner – it most certainly should not do that. These files are created by SQLite’s “WAL” mode and are necessary for operation – they hold actual data and aren’t simply left-over garbage. The “places” database uses the same mode and has those same files – if ccleaner is deleting them too, you have far bigger problems than extension storage being lost.

      1. tsalta wrote on :

        Already I report it to ccleaner forum. but they don’t treat it as bug.

        1. Mark Hammond wrote on :

          That’s a shame, because they are wrong. As mentioned, your places database will have those files too. Maybe point them at https://www.sqlite.org/wal.html#ckpt, which explains the wal file is only removed after “checkpointing” and explains that is done based on the size of the WAL file and has nothing to do with the database closing.

    2. Juraj M. wrote on :

      When was the `browser.storage.sync.getBytesInUse` API added?
      I see the bug is still opened:
      https://bugzilla.mozilla.org/show_bug.cgi?id=1385832
      And the compatibility table is also not updated:
      https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/API/storage/StorageArea/getBytesInUse#Browser_compatibility

      Also it may be worth mentioning that the new sync. is now synchronizes data more often:
      https://bugzilla.mozilla.org/show_bug.cgi?id=1649525

      1. Mark Hammond wrote on :

        > When was the `browser.storage.sync.getBytesInUse` API added?

        It was added to browser.storage.sync as part of the changes being described in this topic, but I don’t think it exists in browser.storage.local yet. I added a note to that bug.

        > Also it may be worth mentioning that the new sync. is now synchronizes data more often:

        That’s only *appears* to be true – the old sync mechanism synced just as often, it’s just that it caused such load on the servers that most of those attempts to sync failed. The new system is far more reliable but happens at the exact same frequency as before.

      2. Caitlin Neiman wrote on :

        Thanks Juraj — our documentation updates can run a little bit behind changes but we do have this flagged. 🙂

  3. Shane wrote on :

    So what am I supposed to use instead of storage.sync? I have been working on a fork of https://github.com/wildskyf/tab-notes
    If someone could explain what API I should be using instead that would be hugely helpful. I don’t really care about syncing to multiple profiles but I don’t know what else I could be using.

    1. Caitlin Neiman wrote on :

      You can continue to use storage.sync. 🙂 Just be aware that you need to keep your data storage within the browser’s limits (which you can find at https://searchfox.org/mozilla-central/rev/8d55e18875b89cdf2a22a7cba60dc40999c18356/toolkit/components/extensions/schemas/storage.json#314-325) otherwise you might experience some problems.

  4. John wrote on :

    Does the new implementation still relies on a explicit extension ID in the browser_specific_settings manifest.json key ?

    1. Caitlin Neiman wrote on :

      Yes! You still need the extension ID for the browser_specific_settings manifest.json key.

  5. Robin wrote on :

    I can understand a total byte maximum, but whats the point of the “QUOTA_BYTES_PER_ITEM” and “MAX_ITEMS” ? My extension uses nested JSON properties which in total doesn’t nearly exceed the total byte maximum, but it now breaks because it reaches the QUOTA_BYTES_PER_ITEM. (When storing an array with a lot of items for example)

    1. Caitlin Neiman wrote on :

      We didn’t want to provide unlimited storage, so we decided on the current settings to keep us in line with what Chrome was doing.

  6. Petri Kivimäki wrote on :

    What should I do when my friend needs to register account to Firefox trough my mobile phone

    1. Caitlin Neiman wrote on :

      Hi Petri, you can share these instructions with your friend: https://support.mozilla.org/kb/how-do-i-set-sync-my-computer#w_create-a-firefox-account

  7. V S Rawat wrote on :

    yesterday my ff 64 bit updated to v79.0 (w10 64 bit)

    And today I noticed that years of Noscript permissions/ refusal that I had saved are all gone.

    Sync is on but didn’t help.

    I checked my profile folders, but didn’t even find where Noscript files and data are stored in all those subfolders and files.

    it is very inconvenient. Why it happened? How to prevent it from recurring? And is there any way to get that data back?

    I don’t think I had ever exported noscript settings.

    At Noscript forum, members said that this could be due to “bugsy new firefox sync of 79.0”. I don’t know myself, but thought to tell you so that you can check if that is causing it.

    Thanks.

    1. Caitlin Neiman wrote on :

      Hi there, do you happen to have the app CCleaner installed by any chance? We’ve seen reports that users with CCleaner installed have recently lost their saved settings of NoScript (https://community.ccleaner.com/topic/58613-problem-with-firefox-79/?tab=comments#comment-322641).

  8. Nick Rishel wrote on :

    How does this impact (the absence of) storage sync on Firefox Mobile?

  9. Sebastian Weitzel wrote on :

    Thanks for the detailed information. I have a app and received some user complaints about stopped functionality due to the mentioned migration and quota enforcment. The app reportedly is not able to remove data and of course not store any new data (>100% usage).

    So I am wondering, is there a way to inspect the storage.sync area with the Firefox Storage Inspector? I was not able to get this information to display, and also didn’t find any other extension for this. Do you have any hints? I would like to offer users to manage the data with such “Storage Inspector” functionality, including the option to reset the data.

    Thank you!

  10. Mark Hammond wrote on :

    Hi Sebastian, sorry you are having trouble.

    This storage isn’t supported directly by devtools, but you can call the storage functions from the devtools console in the addon debugging window, which will show all the data. Your addon (and that console) should also be able call `.clear()` to remove all the data, which will obviously bring you back under quota – so you could imagine, say, fetching all the data (which works even if over quota), then calling `.clear()`, then putting some of the previously read data back.

    If this doesn’t work, please look for error messages in the addon or browser console.and open a bug, and we will have a look.

  11. Eric Jung wrote on :

    These new limits are affecting lots of FoxyProxy users now. Here are two examples:

    https://github.com/foxyproxy/firefox-extension/issues/95
    https://github.com/foxyproxy/firefox-extension/issues/94

    Are users able to manually increase the limits?