mwu's blog
A tale of a fearless coder
Extensions now installed packed
September 10th, 2010 by mwu
Bug 533038 has landed. Extensions are no longer unpacked by default when installed. If an extension with an id such as {d10d0bf8-f5b5-c8b4-a8b2-2b9879e08c5d} is installed, it will show up in the profile extensions directory as a file named {d10d0bf8-f5b5-c8b4-a8b2-2b9879e08c5d}.xpi instead of a directory named {d10d0bf8-f5b5-c8b4-a8b2-2b9879e08c5d}. This was done to reduce the startup impact of having too many files from extensions. It will also make it possible to fix startup cache/fastload invalidation by stat’ing the extensions directory.
Extension developers should note that they can opt back into unpacked installation by adding <em:unpack>true</em:unpack> to their install.rdf. Extensions with dictionaries, binary components, or window icons are most likely to be incompatible with packed installation.
Due to the entire extension being packed, developers no longer need to pack their chrome files into jars for better startup performance. If a developer chooses to keep packing the chrome files into jars, any jars inside the xpi should not be compressed since compressed jars inside the xpi use more memory to access. The files within the jar within the xpi, however, can continue to be compressed without much impact.
[...] This post was mentioned on Twitter by Planet Mozilla, Planet Repeater. Planet Repeater said: Michael Wu: Extensions now installed packed http://dlvr.it/535ZM [...]
For themes and extensions that want compatibility across Firefox versions, either use ‘unpack:true’ (resulting in ‘poorer’ performance in FF4), or don’t compress the inner jar (resulting in a bigger file on transport and on disk (for FF4). So this is not much of an improvement?
Not compressing the jar is actually marginal, because the jar inside the xpi can continue to compress its entries. It’s just the metadata (zip headers and central directory) that shouldn’t compressed. I’ve found that many extensions are in fact compatible without forcing unpacking so with proper packing, extensions can be compatible across old and new firefox versions without impact to optimal performance on each version.
So what’s the upgrade path for existing installed extensions? Do their folders get automatically deleted replaced with .xpi files on next upgrade, or do they need to be uninstalled and re-installed?
Yup. When upgrading/installing an extension, both possible locations of the old extension are deleted.
What does ‘window icon’ means? Images loaded in the extensions xul files like toolbar buttons, panel icons, example image in help guides?
It means the little icon in the left side of title bar that also shows up in your taskbar. https://developer.mozilla.org/en/Window_icons
Care to explain this further? Why a dictionary would be incompatible and why that’s just “likely” to happen?
Also: If I understand this change, this involves only new installations, not add-ons already installed (and unpacked). Am I right?
Thank you for the explanation. Didn’t expect that you can set them as this is not possible as a user action.
I do hope these now unpacked XPIs are uncompressed from their original form, otherwise it’s just stupidly decompressing something on every load when before you didn’t need to. Please tell me this didn’t just kill startup performance by default and explain to me what’s going on with this.
In other words, {d10d0bf8-f5b5-c8b4-a8b2-2b9879e08c5d}.xpi has no compression, right? (I hope) It isn’t just the installer xpi renamed?
The dictionary reading code assumes files so it can’t read from entries inside a zip.
This only involves new installations and upgrades.
It is in fact the installer xpi renamed. From a theoretical point of view, compressed xpis are faster for cold startup but slower for warm startup.
I did measure startup times on both warm and cold startup. I found the difference in warm start negligible and an improvement in cold start. The noisiness of the numbers with packed xpis was also lower. That was measuring with compressed xpis. Decompression is quite cheap relative to things like disk seeking.
This situation isn’t too different from the options extension authors had before with compressing or not compressing entries in their chrome jars, except now there is the option of choosing between compressed and uncompressed for all files in the extension. Most of the time though, it doesn’t matter a whole lot compared to the improvement made by minimizing fragmentation.
Can this be documented on https://developer.mozilla.org/en/Install_Manifests (and probably elsewhere as well)?
The bug has been marked dev-doc-needed so that page (and probably many others) should be updated when the documentation gets updated.
[...] that omnijar wasn’t awesome enough, Michael Wu went ahead and omnijared extensions. Most extensions will no longer need to be unpacked from xpi files. This also means that extension [...]
[...] are now installed packed to help Firefox startup time, great work by Michael Wu. This is transparent to users, 99.5% of whom [...]