{"id":50,"date":"2012-01-27T17:46:09","date_gmt":"2012-01-27T17:46:09","guid":{"rendered":"http:\/\/blog.mozilla.org\/nfroyd\/?p=50"},"modified":"2012-01-27T17:46:09","modified_gmt":"2012-01-27T17:46:09","slug":"datashrink-work","status":"publish","type":"post","link":"https:\/\/blog.mozilla.org\/nfroyd\/2012\/01\/27\/datashrink-work\/","title":{"rendered":"DataShrink work"},"content":{"rendered":"<p>Over the last month, I&#8217;ve been working on some patches to address <a title=\"invasive changes for relocation reduction\" href=\"http:\/\/blog.mozilla.org\/nfroyd\/2011\/12\/13\/invasive-changes-for-relocation-reduction\/\">the relocation issues<\/a> I&#8217;ve blogged about and just general space wastage in <tt>libxul<\/tt>.\u00a0 The upshot is that Firefox 13 will have shaved almost 100K of data and relocations for smaller binaries and slightly faster load times:<\/p>\n<ul>\n<li><a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=717311\">Bug 717311<\/a> was the big one; we were wasting 70K+ of space in some Unicode tables due to structure padding.\u00a0 Fixing this was just a matter of using the right bitfield datatype so the compiler would pack things correctly.<\/li>\n<li><a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=717501\">Bug 717501<\/a> was much the same: use smaller datatypes when the data you have fits in them.<\/li>\n<li><a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=711563\">Bug 711563<\/a> was more in line with the invasive relocation changes discussed earlier.\u00a0 The patches there rearranged the storage for the stub names to avoid relocations and generally packed things more efficiently.<\/li>\n<\/ul>\n<p>Not going in this cycle, but worthy of mention is <a href=\"https:\/\/bugzilla.mozilla.org\/show_bug.cgi?id=704848\">bug 704848<\/a> for rearranging the effective TLD name table to avoid relocations; that bug will save 40-50K of space in data and relocations.\u00a0 Jason Duell and I talked on IRC yesterday and while he&#8217;s in favor of the idea, he&#8217;d like to see if <a href=\"http:\/\/www.gnu.org\/s\/gperf\/\">gperf <\/a>makes things any better.\u00a0 No sense in constructing a table at runtime if you can construct it at compile time!<\/p>\n<p>Are there more instances of things that could be compressed like this?\u00a0 Yes, but the savings from them are likely to be much smaller on a case-by-case basis:<\/p>\n<ul>\n<li>Anything that uses <a href=\"http:\/\/mxr.mozilla.org\/mozilla-central\/source\/xpcom\/ds\/nsStaticNameTable.h#46\"><tt>nsStaticCaseInsensitiveNameTable<\/tt><\/a> could be tweaked to use less space and relocations by providing the entry data in a different format.<\/li>\n<li><a href=\"http:\/\/mxr.mozilla.org\/mozilla-central\/source\/parser\/htmlparser\/src\/nsElementTable.cpp\">nsElementTable.cpp<\/a>:<tt>gHTMLElements<\/tt> could probably be rearranged for the same sort of savings.<\/li>\n<li><a href=\"http:\/\/mxr.mozilla.org\/mozilla-central\/source\/xpcom\/ds\/nsAtomTable.cpp#540\">Initialization of atoms<\/a> could stand some rearrangement for relocation reduction, at the cost of some ugliness elsewhere.<\/li>\n<li>YARR, used by the JS engine, has <a href=\"http:\/\/mxr.mozilla.org\/mozilla-central\/source\/js\/src\/yarr\/RegExpJitTables.h\">some fabulous bit vectors represented as character arrays<\/a>; squashing those would shave 100K of data, but would likely be tricky.<\/li>\n<li>Eliminating null entries from the tables in table-driven QueryInterface methods would help, possibly at the cost of some extra parameter traffic to <a href=\"http:\/\/mxr.mozilla.org\/mozilla-central\/source\/xpcom\/glue\/nsISupportsImpl.cpp#41\"><tt>NS_TableDrivenQI<\/tt><\/a>.<\/li>\n<li>Actually, rewriting IIDs for XPCOM interfaces to fit in 32 bits could save a number of relocations in the tables for the aforementioned methods.<\/li>\n<li>The tables required by <a href=\"http:\/\/mxr.mozilla.org\/mozilla-central\/source\/nsprpub\/pr\/src\/misc\/prerrortable.c#200\"><tt>PR_ErrorInstallTable<\/tt><\/a> are breeding grounds for relocations; changing them is unlikely to happen anytime soon.\u00a0 (Those tables do account for a significant chunk of the relocations in <tt>libnss<\/tt> and <tt>libnspr<\/tt>, though.)<\/li>\n<li>Any number of places where we use pointers in constant data structures; there are quite a few of them, but converting each one saves 30-50 bytes each.\u00a0 Best to save these for when you are really bored.<\/li>\n<\/ul>\n<p>All of the above might amount to 200K of data+relocation savings.<\/p>\n<p>Really, though, there&#8217;s not that much to trim (or perhaps what is left to trim is decidedly non-trivial to trim).\u00a0 If you do something like:<\/p>\n<pre>readelf -sW dist\/bin\/libxul.so | grep OBJECT | awk '$3 &gt; 1000 { print }' | c++filt | less<\/pre>\n<p>in your objdir on a Linux-y system, you&#8217;ll see that quite a lot of the largest objects come from tables for character conversion or character detection.\u00a0 However, the authors of said code were already conscious of the space required by these tables and they have tended to use the smallest datatypes necessary.\u00a0 vtables make a number of appearances (hard to get rid of at the moment).\u00a0 There are also some tables for media codecs, which presumably are difficult to trim down.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Over the last month, I&#8217;ve been working on some patches to address the relocation issues I&#8217;ve blogged about and just general space wastage in libxul.\u00a0 The upshot is that Firefox 13 will have shaved almost 100K of data and relocations for smaller binaries and slightly faster load times: Bug 717311 was the big one; we [&hellip;]<\/p>\n","protected":false},"author":320,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"_links":{"self":[{"href":"https:\/\/blog.mozilla.org\/nfroyd\/wp-json\/wp\/v2\/posts\/50"}],"collection":[{"href":"https:\/\/blog.mozilla.org\/nfroyd\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.mozilla.org\/nfroyd\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.mozilla.org\/nfroyd\/wp-json\/wp\/v2\/users\/320"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.mozilla.org\/nfroyd\/wp-json\/wp\/v2\/comments?post=50"}],"version-history":[{"count":0,"href":"https:\/\/blog.mozilla.org\/nfroyd\/wp-json\/wp\/v2\/posts\/50\/revisions"}],"wp:attachment":[{"href":"https:\/\/blog.mozilla.org\/nfroyd\/wp-json\/wp\/v2\/media?parent=50"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.mozilla.org\/nfroyd\/wp-json\/wp\/v2\/categories?post=50"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.mozilla.org\/nfroyd\/wp-json\/wp\/v2\/tags?post=50"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}