Category: MemShrink

Categories
Firefox Memory consumption MemShrink

MemShrink progress, week 103–104

I’m back from vacation.  Many thanks to Andrew McCreight for writing two MemShrink reports (here and here) while I was away, and to Justin Lebar for hosting them on his blog.

Fixes

areweslimyet.com proved its value again, identifying a 40 MiB regression relating to layer animations.  Joe Drew fixed the problem quickly.  Thanks, Joe!

Ben Turner changed the GC heuristics for web workers on B2G so that less garbage is allowed to accumulate.  This was a MemShrink:P1.

Bas Schouten fixed a gfx top-crasher that was caused by inappropriate handling of an out-of-memory condition.  This was also a MemShrink:P1.

Randell Jesup fixed a leak in WebRTC code.

Changes to these progress reports

I’ve written over sixty MemShrink progress reports in the past two years.  When I started writing them, I had two major goals.  First, to counter the notion that nobody at Mozilla cares about memory consumption — this was a common comment on tech blogs two years ago.  Second, to keep interested people informed of MemShrink progress.

At this point, I think the first goal has been comprehensively addressed.  (More due to the  fact that we actually have improved Firefox’s memory consumption than because I wrote a bunch of blog posts!)  As a result, I feel like the value of these posts has diminished.  I’m also tired of writing them;  long-time readers may have noticed that they are much terser than they used to be.

Therefore, going forward, I’m only going to post one report every four weeks.  (The MemShrink team will still meet every two weeks, however.)  I’ll also be more selective about what I write about — I’ll focus on the big improvements, and I won’t list every little leak that has been fixed, though I might post links to Bugzilla searches that list those fixes. Finally, I won’t bother recording the MemShrink bug counts.  (In fact, I’ve started that today.)  At this point it’s pretty clear that the P1 list hovers between 15 and 20 most of the time, and the P2 and P3 lists grow endlessly.

Apologies to anyone disappointed by this, but rest easy that it will give me more time to work on actual improvements to Firefox 🙂

Categories
about:memory Firefox MemShrink

Recent about:memory improvements

The landing of patches from two recent bugs has substantially changed the look of about:memory.  When you load the page all see now is the following.

about:memory screenshot

Measurements aren’t taken away;  you have to click on the “Measure” button for that to happen.  Also, adding ?verbose to the URL no longer has an effect.  If you want verbose output, you need to click on the “verbose” checkbox.

The motivation for this change was that about:memory’s functionality has expanded greatly over the past two years, and cramming more functionality into the existing UI was a bad idea.  There are numerous advantages to the new UI.

  • No need to control behaviour via the URL, which is admittedly an odd way to do it.
  • You can switch between verbose and non-verbose modes without having to reload the page.
  • All the buttons are at the top of the page instead of the bottom, so you don’t have to scroll down.
  • You can trigger a GC/CC/minimize memory usage event without having do a memory measurement.
  • When you save reports to file, it’s clearer that a new measurement will be taken, rather than saving any measurements that are currently displayed on the screen.
  • The buttons are grouped by function, which makes them easier to understand.

There’s also the “Load and diff…” button, which lets you easily find the difference between two saved memory report files.  Here’s some example output taken after closing a tab containing a Google search result.

about:memory diff output

You can see that all the measurements are negative.  The [-] markers on leaf nodes in the tree indicate that they are present in the first file but not in the second file.  Corresponding [+] markers are used for measurements present in the second file but not the first.

Categories
about:memory B2G Memory consumption MemShrink

MemShrink progress, week 95–96

B2G Fixes

Kyle Huey made image.src='' discard the image immediately even if the image is not in the document.  This provides a way for image memory to be discarded immediately, which is important for some B2G apps such as the Gallery app.  This was a MemShrink:P1 bug.

Justin Lebar fixed a bad leak in AudioChannelAgent.

Mike Habicher fixed an image-related leak that was manifesting on B2G.

Other Fixes

I exposed the existing JSON memory report dumping functionality in about:memory.  As a result, gzipped JSON is now the preferred format for attaching memory report data to bugs.  This was a MemShrink:P1 bug.

Honza Bambas overhauled the DOM storage code.  Apparently this might reduce memory consumption, but I fully admit to not knowing the details.

Nicolas Silva fixed a leak in layers code relating to OMTC.

I removed the MEMORY_EXPLICIT telemetry measurement.  I didn’t want to do this, but it’s been causing hangs for some users, and those hangs are hard to avoid due to the complexity of reporting memory consumption from web workers.  Furthermore, in practice the measurement was sufficiently noisy that we’ve never gotten anything useful from it.

Help Needed

Mozilla code uses a mixture of fallible and infallible allocations.  In theory, any allocation that could allocate a large amount of memory (e.g. a few hundred KiB or more) should use a fallible allocator.  We’ve seen recently some cases where large allocations were being made infallibly, which led to OOM crashes.  Bug 862592 proposes adding an assertion to the infallible allocators that the request size isn’t too large.  This is an easy bug to get started with, if anyone is interested.  Details are in the bug.

Bug Counts

Here are the current bug counts.

  • P1: 15 (-4/+4)
  • P2: 145 (-0/+7)
  • P3: 131 (-2/+4)
  • Unprioritized: 5 (-1/+5)

Interregnum

I will be on vacation during May.  As a result, there will be no MemShrink reports for the next three fortnights.  (Although I’ll be away for just over four weeks, that period covers three MemShrink meetings.)  I’ll be back with the next report on June 11.  See you then!

Categories
about:memory MemShrink

Gzipped JSON is now the preferred format for attaching memory report data to bugs

Old: Text

about:memory is our main memory profiling tool.  You can copy and paste the memory reports shown by about:memory and they will reproduce nicely — so long as the displaying text is a fixed-width font — as the following example shows.

Main Process

Explicit Allocations
536.22 MB (100.0%) -- explicit
├──217.45 MB (40.55%) -- window-objects
│  ├───91.55 MB (17.07%) -- top(https://mail.google.com/mail/u/0/#label/A-moz%2Fbugz/13d41da77907a707, id=16)/active
│  │   ├──61.18 MB (11.41%) -- window(https://mail.google.com/_/mail-static/_/js/main/m_i,t/rt=h/ver=sobKNxkH_hk.en./sv=1/am=!rsy2VLKefbT1RPF-0_JbQoI4nDfwRxdk-CbEiLu_TGxDGzw05OfUOUODZR_O2flp0CcdPg/d=1)
│  │   │  ├──56.29 MB (10.50%) -- js
│  │   │  │  ├──56.11 MB (10.46%) -- compartment(https://mail.google.com/_/mail-static/_/js/main/m_i,t/rt=h/ver=sobKNxkH_hk.en./sv=1/am=!rsy2VLKefbT1RPF-0_JbQoI4nDfwRxdk-CbEiLu_TGxDGzw05OfUOUODZR_O2flp0CcdPg/d=1)
│  │   │  │  │  ├──35.23 MB (06.57%) -- gc-heap
│  │   │  │  │  │  ├──18.17 MB (03.39%) -- objects
│  │   │  │  │  │  │  ├───9.88 MB (01.84%) ── dense-array
│  │   │  │  │  │  │  ├───6.08 MB (01.13%) ── ordinary
│  │   │  │  │  │  │  └───2.21 MB (00.41%) ++ (2 tiny)
│  │   │  │  │  │  ├──10.71 MB (02.00%) ++ (6 tiny)
│  │   │  │  │  │  └───6.35 MB (01.18%) ── unused-gc-things
│  │   │  │  │  ├──12.01 MB (02.24%) ++ (7 tiny)
│  │   │  │  │  └───8.86 MB (01.65%) ++ objects-extra
│  │   │  │  └───0.19 MB (00.04%) ++ compartment(https://mail.google.com/_/mail-static/_/js/main/m_i,t/rt=h/ver=sobKNxkH_hk.en./sv=1/am=!rsy2VLKefbT1RPF-0_JbQoI4nDfwRxdk-CbEiLu_TGxDGzw05OfUOUODZR_O2flp0CcdPg/d=1, about:blank)
│  │   │  └───4.89 MB (00.91%) ++ (4 tiny)

Accordingly, until now the preferred way of reporting about:memory’s data in bug reports has been to include a copy of the entire contents of about:memory?verbose.

This worked well when about:memory contained a few dozen lines.  But the level of detail in the underlying memory reporters has increased greatly since then.  This is a good thing — more data is helpful — but it means that a full about:memory?verbose is now typically thousands of lines.  When viewing the data directly in about:memory?verbose this isn’t a problem, because you can easily collapse and expand sub-trees to focus on the interesting parts.  However, when reading pasted data as text, it’s overwhelming, even for experts.

New: Gzipped JSON

With the landing of bug 848560, about:memory can now import and export memory reports as gzipped JSON.  Use the “Write reports to a file” and “Read reports from a file” buttons at the bottom of about:memory.  (Note: the UI will likely change soon, but not substantively.)

(The JSON schema is documented in a comment near the top of xpcom/base/nsIMemoryInfoDumper.idl, for those who are interested.)

This is now the preferred format for attaching memory report data to bug reports.  It has two important advantages over text.

  • When you load one of these files into about:memory, you can do the aforementioned collapsing and expanding of sub-trees.
  • If you have two of these files, you can diff them using toolkit/components/aboutmemory/tools/diff-memory-reports.js.  (For the moment you’ll have to ungzip both files first, which is stupid.  Hopefully this’ll be fixed soon.)

The ability to compute memory report diffs will be extremely useful for diagnosing cases where Firefox’s memory consumption increases unexpectedly.  (There are already two bugs that are waiting on this feature for a better diagnosis.)

There is a bug open to implement a “Load reports from URL” feature in about:memory, which will make the viewing memory reports attached to bugs even easier.  It’s currently unassigned;  please email me or comment in the bug if you are interested in implementing it.

Finally, although the gzipped JSON format is now the preferred format for transferring the full memory report data, copy and paste is still useful for communicating snippets of about:memory’s contents.  Support for it won’t be disappearing.

Categories
about:memory Fennec Firefox Memory consumption MemShrink

MemShrink progress, week 93–94

After lots of activity in the previous month, the past two weeks have been fairly quiet for MemShrink.

AWSY

areweslimyet.com has been proving its worth.

Jonathan Kew reduced the amount of memory taken by fonts on Fennec at start-up, which was detected by AWSY/mobile.  Jonathan also reverted a change that AWSY detected as increasing Fennec memory consumption, and filed a follow-up to investigate further.

Joe Drew fixed a bad regression on AWSY relating to image decoding.  It’s not clear to me if this was a genuine regression that users would have seen, or if it was an artifact of the way AWSY does its measurements.  Either way, it’s good that it was fixed, and props to Joe for doing it so quickly.

Finally, we closed bug 833518, which was for an AWSY regression caused by the new DOM bindings.  This was previously improved by an Aurora-only hack, but enough cases have been translated to the new bindings that we’re naturally down almost to where we were.

Miscellaneous

The mobile team abandoned their goal of making Fennec work on phones with only 256 MiB of memory.  The rationale is that Android phones with only 256 MiB of RAM are uncommon, whereas low-end phones that meet the current minimum of 384 MiB are much more common.  The mobile team will of course continue to look for ways to improve memory consumption in order to make life for users with 384 MiB phones.

I modified the JS engine so that it doesn’t emit bytecode for asm.js functions in the normal case.  This reduced the memory consumption of the Unreal 3 demo used at GDC by about 100 MiB.  I also added a memory reporter for array buffers used by asm.js, which are often quite large and weren’t being measured on 64-bit platforms.

Alexandre Poirot fixed a leak relating to dev tools.

Randell Jesup fixed a small leak in WebRTC.

Help Needed

I’m working on adding a button to about:memory trigger the dumping of memory reporter data to file.  I have a patch awaiting review, but I’m getting a test failure on Windows.  The test saves gzipped memory reports to file, and then immediately loads that saved file (and uncompresses it) and checks the data looks as expected.  This works fine on Mac and Linux, but on Windows I’m sometimes getting incomplete data in the load step.  The file is quite short (just 253 bytes compressed, and 620 bytes uncompressed) and the truncation point varies between runs;  in the most severe occurrence only 9 bytes of uncompressed data were loaded, though the cut-off point seems to vary randomly.

I suspect there’s a file synchronization problem between the save and the load, even though gzclose() is called on the save file before the loading occurs.  If anyone has ideas about what the problem might be, I’d love to hear them.

Update: Nils Maier and an anonymous commenter pointed out the problem — I was using “r” instead of “rb” for the file mode.  On Windows, this causes mangling of EOL chars.

Also, we’re seeing some strange behaviour on Mac OS X where memory managed by jemalloc doesn’t appear to be released back to the OS as it should.  This is both alarming and hard to understand, which is not a good combination.

Good First Bugs

I have two easy bugs assigned to me that I probably won’t get around to for some time.  Both of them would be good first (or second, or third…) bugs.

  • Bug 857382 is about making about:memory handle memory report diffs more elegantly.
  • Bug 798914 is just a minor code clean-up.  Nothing too exciting, but first bugs often aren’t!

Please email or comment in one of the bugs if you are interested in helping.

Bug Counts

Here are the current bug counts.

  • P1: 15 (-0/+2)
  • P2: 138 (-4/+8)
  • P3: 129 (-2/+7)
  • Unprioritized: 1 (-3/+0)
Categories
B2G Firefox Memory consumption MemShrink

MemShrink progress, week 91–92

Bill McCloskey landed zones, a.k.a. compartment groups.  This mitigates the overhead of each compartment by allowing all compartments within a zone to share arenas and strings.  Roughly speaking, each tab gets its own zone, and there’s a system zone for Firefox’s internal use.  This was a MemShrink:P1 bug.

The following graph from areweslimyet.com shows the impact of zones — about 5/6 of the way along the graph there’s a distinct drop, most noticeable in the dark blue line.  The light green line (settled start-up) showed a ~6 MiB drop, which is ~10%.  Note that the fraction of JS memory in areweslimyet.com is less than that in typical sites, so the drop in the higher lines is smaller than the improvements normal users should see.

areweslimyet.com graph showing improvements due to zones

Avi Halachmi fixed a problem where badly managed gradients could cause spikes of 10s of MiBs when tab animations occurred.  This was a MemShrink:P1 bug.  The fix has been backported to Aurora.

Jed Parsons fixed excessive memory consumption by Persona after visiting the B2G marketplace.  At least, I think that’s what happened;  I won’t pretend to genuinely understand what went on in that bug.  This was a MemShrink:P1 bug.

Fabrice Desré fixed a bad B2G leak relating to error messages.  This bug was fixed before it was triaged in a MemShrink meeting, but it probably would have been a MemShrink:P1 because it could cause the phone to OOM after a few hours.

I removed all uses of nsFixedSizeAllocator.  This was only a small memory consumption win (a few 10s of KiBs) but it cut several hundred lines of code, and removed another low-quality custom allocator (and attractive nuisance) from the codebase.

I added a “js-main-runtime-temporary-peak” memory reporter, which measures the peak size of the main JSRuntime’s “temporary” pool, which mostly holds parse nodes.  These are short-lived but they can be quite large — 10s of MiBs in normal browsing, and we’ve seen it exceed 1.5 GiB on large asm.js examples.  Relatedly, I reduced the size of parse nodes by 12.5% on 64-bit platforms, and 16.7% on 32-bit platforms.

Interesting Open Bugs

Sometimes, especially on B2G, we have excessive memory consumption due to JS strings.  It would be useful to be able to dump the contents of all the strings in memory, to look for ones that shouldn’t be there.  Bug 852010 has been filed for this.  It shouldn’t be too hard, as it would fit in with the existing JSON memory report dumping infrastructure.  This is currently blocking bug 850893, a B2G MemShrink:P1 bug.  Please comment in the bug or contact me if you want to get involved.

Bug 846616 and bug 850545 both are about high “heap-unclassified” values.  Reducing “heap-unclassified” is getting very difficult, because the memory is often allocated in ways we can’t measure by third-party code such as Cairo and graphics drivers.  I suppose in the Cairo case we could put effort into adding some memory profiling infrastructure and try to get it upstreamed, but the driver situation seems pretty hopeless.

Bug Status

Here are the current bug counts.

  • P1: 13 (-3/+4)
  • P2: 134 (-2/+8)
  • P3: 124 (-4/+6)
  • Unprioritized: 4 (-6/+4)
Categories
Firefox Memory consumption MemShrink

MemShrink progress, week 89–90

IMAGES

The big news this week is that Timothy Nikkel finished bug 689623, which causes us to (for the most part) only decode images that are visible or nearly visible.  This goes a long way towards fixing our problems with decoded images which happens to be the #1 item on the MemShrink big ticket items list.  The fixing of this bug also happened to fix the following bugs in passing:  661304, 682230.

To give you an idea of what changed, consider an image-heavy page like this one.  Prior to Timothy’s change, shortly after loading that page on my machine Firefox’s memory consumption would reach 3 GiB, with 2.7 GiB of that being decoded (uncompressed) images.  If I switched to another tab then 2.45 GiB of those decoded images would be immediately discarded, then after 10–20 seconds the remaining 250 MiB would be.  If I switched back, all the images would immediately be decoded again and memory consumption would climb back to 3 GiB.

After Timothy’s change, Firefox’s memory consumption still peaks around 2.8 GiB just after loading the page, but then quickly drops back to 600 MiB as many of those decoded images are discarded.  After that, even when scrolling quickly through the page I couldn’t get memory consumption to exceed 700 MiB.

This change will make such pages much smoother on desktop machines that don’t have lots of RAM, because there won’t be any paging, which slows things down drastically.   Andrew McCreight wrote a comment that I can’t find now, but basically he said that on one of his machines, that page was extremely slow and painful to scroll through before the change, and after the change it was smooth and pleasant.

The peak that occurs when an image-heavy page first loads is still undesirable, and bug 542158 is open for fixing that, and it will build on the infrastructure that Timothy has developed.  When that’s fixed, it’ll also greatly reduce the likelihood that an image-heavy page will cause an out-of-memory (OOM) crash, which is great for both desktop (esp. 32-bit Windows) and mobile/B2G.

Miscellaneous

Nicolas Pierron fixed a leak in IonMonkey that was causing runaway memory consumption for Shumway demos on Android.  It has been backported to the Aurora and Beta channels.

I greatly reduced the size of the property cache.  This is a data structure the JS engine uses to speed up property accesses.  It used to be important, but is now barely necessary thanks to type inference and the JITs.  This reduced memory consumption by 75 KiB per JSRuntime on 32-bit, and by 150 KiB per JSRuntime on 64-bit.  There is one JSRuntime for each process, plus one for each worker, so for B2G it saves around 525 KiB at start-up.

We declared victory in the bug tracking the memory consumption of not-yet-loaded tabs.  When the bug was opened it was over 1 MiB per tab (64-bit builds), it’s now around 200 KiB.  Furthermore, if someone has many (e.g. hundreds) of not-yet-restored tabs they’re probably on a high-end machine, so the motivation for further reducing the memory consumption is low.  There is a bug open about some scripts run for each of these tabs, however;  avoiding these scripts may make start-up faster for heavy tab users.

Marco Bonardo reduced the maximum size of the cache used by each SQLite connection (excluding places.sqlite) from 4 MiB to 2 MiB.  Update: Marco corrected me in the comments — there are four places.sqlite connections, but only one of them (the one for the awesome bar) is excluded from the new 2 MiB limit.

Somebody fixed a leak in the Roboform add-on.  The fix is present in v7.8.7.5.

Olli Pettay improved our memory reporting by adding extra information to some system compartments that previously were undistinguished.  Our experience has been that as soon as somebody adds a new memory measurement, it finds a problem, and this was no exception, leading to the filing of bug 844661 .

Jonathan Kew has been busy.

I wrote a script that can take a diff of two JSON memory report dumps.  This will be very useful when comparing the memory consumption before and after doing something, for example.  If you didn’t know that JSON memory report dumps existed, you’re probably not alone — it’s an undocumented feature that’s currently only available on Linux and Android, and is triggered by sending signal 34 to the browser(!)  These dumps can be loaded by about:memory (see the buttons at the bottom) but there’s currently no easy way to trigger them, which is why today I filed a bug to make it easier.

I removed the pool used for recycling nsNodeInfo objects, which was causing them to never be released back to the OS.  Never-shrinking structures like this can become a problem in longer-running browser sessions — for example, after running MemBench, which opens and closes 150 tabs, this pool was unnecessarily hoarding 4 MiB.

INteresting Open Bugs

In my MemShrink reports I mostly write about bugs that have been fixed.  While this is satisfying and demonstrates progress, it might also give the impression that the MemShrink team has everything under control, when really we could use some help.

(In fact, the term “the MemShrink team” is something of a misnomer since there isn’t any such entity, officially.  “The people who regularly attend MemShrink meetings” would be a more accurate term.  But I’ll use “MemShrink team” as a short-hand for that.)

The MemShrink team has expertise in some areas of Mozilla code, such as DOM, cycle collector, JS engine (partial), Fennec, B2G, and memory profiling, and we tend to make good progress in those areas — we can fix bugs in those areas, and we generally pay attention to how these areas affect memory consumption.

But we lack expertise in areas like graphics, image-handling, layout, text rendering, storage, Jetpack/add-ons, and front-end.  Graphics is a particular problem, because graphics issues, esp. those involving drivers, can cause huge memory consumption blow-ups.  Bug 837542 is an example from the MemShrink:P1 list where gradients are somehow causing memory consumption to spike by 10s or even 100s of MiBs just by opening and closing a few tabs!  We triage bugs like that as well as we can, but often we’re just guessing, and we’re mostly helpless to address such problems.

Therefore, moving forwards I’m going to start mentioning interesting open bugs that don’t have someone working on them.

One example is bug 846173, which I filed when I noticed that fully loading the front page of TechCrunch takes over 100 MiB!  And it’s mostly because of the many Facebook “like” buttons, Google “+1” buttons, and Twitter “tweet this” buttons — see the about:memory output for the full gory details.  It’s obvious that most of these window objects are basically identical, except for the URL.  Could we do something clever to avoid all this duplication?  Justin Lebar wondered about a copy-on-write scheme.  Are there other ways we could improve this case?

Another example is bug 842003, which is looking for an owner.  Some basic leak-checking infrastructure in the IPC code could potentially detect some big problems for B2G.

In bug 842979 we’re seeing 10s or 100s of MiBs of orphan DOM nodes in long-running Gmail sessions.  It’s not clear if this is a bug in Gmail, or caused by add-ons, or something else.  Please comment in the bug if you have any additional data.

Another one:  DMD, which is one of our most important tools, is pretty much useless on Fennec because it can’t get stack traces.  If anyone who knows about such things would like to investigate, that would be very helpful.  (In fact, I got a crash in the stack tracing code when I most recently tried to run DMD on Mac, which I haven’t had time to investigate.)

Bug Counts

Here are the current bug counts.

  • P1: 12 (-6/+1)
  • P2: 128 (-7/+7)
  • P3: 122 (-0/+5)
  • Unprioritized: 6 (-2/+6)

The closed P1s were 661304, 689623, 837187, 841976, 841993, 842756.  Three of them related to an IPC leak fix that I mentioned in my last report.

Categories
Firefox Memory consumption MemShrink

Improving “Reset Firefox”

I recently wrote about the wonderful “Reset Firefox” feature, which in one fell swoop can fix all sorts of bad behaviours.  What follows is a number of complaints that commenters had, and some ideas about how to address them.

“The name is misleading”

It’s true.  It sounds like it will erase all of your data and customizations.  The SUMO page says the following.

The Reset Firefox feature can fix many issues by restoring Firefox to its factory default state while saving your essential information.

“Factory default state” and “while saving your essential information” are two phrases that sit together awkwardly. But it may be too late to change the name now.

“But I have lots of configuration tweaks”

And they’ll all be lost.  You can look in about:support to see “important modified preferences”.  Or you can look at the “user set” entries in about:config.  Or you can look at both, and wonder the two lists are different, and what the definition of “important” is.

It would be very helpful if you could see who or what was responsible for each of those changes.  Was it the user, via the Preferences dialog?  The user via about:config?  An add-on?  Something auto-updated by core Firefox code?  (And why do I always have dozens of “user set” preferences relating to printing, even on my Linux machine that I never print from?)

Would an API tweak help?  We could change the “setPref()” function (or whatever it’s called) to take an additional, optional argument that indicates who set it.  It wouldn’t fix the problem immediately but would give a path forward.

Update: I found bug 834034, which is about preserving a user’s custom spell-check dictionary.

“But I have hundreds of tabs open”

And they’ll all be lost, too.  Apparently the Session Manager extension solves this problem.  But that’s not obvious, and still a pain to manage.  Maybe we could mention it in the documentation?

Update: Alexander Limi pointed me in the direction of bug 833943, which is about fixing this.

“But I have Dozens of add-ons installed”

And one of them might be causing your problems.  You can get the list of installed add-ons from about:addons, so this one isn’t too hard to fix up manually.

“I can’t see the ‘Reset Firefox’ button” / “I want to reset a non-default profile”

The “Reset Firefox” button only shows up in the default profile.  (And sometimes not even there — I don’t get it on my default profile on my dev machine, where I have half a dozen profiles present.)

There’s a bug open about this.  I could be wrong, but it doesn’t sound that hard to fix.

“How do I undo it if it breaks something?”

The SUMO page addresses this.

After the reset is finished, your old Firefox profile information will be placed on your desktop in a folder named “Old Firefox Data.” If the reset didn’t fix your problem you can restore some of the information not saved by copying files to the new profile that was created. See Copying files between profile folders for instructions.

The instructions on copying files between profile folders are clear, except for the fact that even an expert user will have little idea which files within a profile are important.  It’s an intimidating process.

“Why isn’t this automated?”

Well, because it loses some of your configuration.  You don’t want Firefox discarding all your extensions every six months.

“Why is this even needed in the first place?”

Ah.  An interesting question.  What exactly are the sources of the problems that Reset Firefox fixes?

  • Problematic extensions.  This is probably a common cause of problems, especially for people who have toolbars they don’t want and things like that.  And I’ll give a special mention to GreaseMonkey users who have badly written scripts that do horrible things.
  • Bad preference settings.  In my experience, the most likely settings to cause problems are those involving hardware acceleration (e.g. bug 711656). Are there any other classes of preferences that could have big effects?  I guess if the JavaScript JITs were turned off that would slow things down drastically.  I don’t know how people end up with a bad setting, whether it’s by diffling in about:config or extensions or something else.  (The was a dev-platform thread about this last year.)
  • Corrupt(?) state.  This one’s vague, but I suspect it’s a factor.  Maybe some of the SQLite files?  Or perhaps the state isn’t corrupt, but just fragmented in some fashion?  For example, in an oldish profile on my Mac my places.sqlite file size is 73 MiB, and urlclassifier3.sqlite is 42 MiB, which seems like a lot.
  • Do we know of any other causes?

Update: in bug 754933 Michael Verdi lists the following preferences as “troublesome”:

  • home page
  • all search engines (location bar, search bar, right-click)
  • Application settings
  • History settings
  • Password settings
  • Proxy settings
  • Firefox update settings
  • SSL settings
  • All toolbars and controls

I don’t subscribe to the hardline “Reset Firefox shouldn’t even be necessary” viewpoint — software is hard.  But I would like to understand the root causes better, in order to understand if at least some of them could be reduced or prevented.

It’s not all bad

I hope this post doesn’t sound too negative.  I understand that Reset Firefox’s goal was to provide something really simple that’ll fix many problems, and it does that admirably.

It’s just a shame that some power users who genuinely want to improve Firefox’s behaviour are unable to try it because it discards too much data.  I wonder if having additional options (in an obscure corner of the UI) the way safe mode does — “preserve extensions”, “preserve preferences”, “preserve open tabs” — would help those users.  Then those users could use a more incremental approach:  discard some data away, see if that helps, discard some more, etc.

Or maybe that’s not worthwhile;  the people who comment on my blog aren’t exactly typical Firefox users!  And it could complicate things for the SUMO folks — they’d have to ask what options someone used when they initiated a Reset Firefox.  I’m not sure.  I’d just love to extend the goodness of Reset Firefox as far and wide as possible, and it seems like a few small tweaks might help.

Categories
Memory consumption MemShrink

areweslimyet.com custom runs now available

areweslimyet.com (a.k.a. AWSY) hosts the memory benchmark that the MemShrink team uses to detect memory consumption changes, and it automatically runs on every time mozilla-inbound is updated.

John Shoenick implemented support for measuring try builds, which means you can test the impact on memory consumption of a change you’ve made.  The instructions are here, but note that you need access to Mozilla’s VPN in order to queue up a run.  Ask in the #memshrink channel on IRC if you run into any problems.

Use the following try syntax to generate a suitable desktop build without running any of the usual try test suites.

try: -b o -p linux64 -u none -t none

At the moment, the best way to run it is to build (via the try server) an unmodified browser as a control, and also a second, modified version with your changes.  You can then compare the two points easily.

It only works with Linux desktop builds.  AWSY has a mobile page but custom runs for mobile are not yet available.

Categories
Firefox Memory consumption MemShrink

“Reset Firefox” is wonderful

TL;DR: Please recommend the “Reset Firefox” feature to anyone who complains about odd, persistent performance problems and/or high memory consumption.

In Ars Technica’s article about the Firefox 19 release, carbon fibre made a comment complaining about Firefox’s performance.

Firefox still doesn’t fix memory issues and smoothness, at least on my experience. It’s constant random jerks and extreme mem usage, until restarted. I use chrome but I have been wanting to switch FF for awhile, but I can’t, as long as they fix these issues, long standing issues since ver. 4.

There were four notable responses.

korg250 (my emphasis):

They are losing market share by not fixing the memory issues. Even on my 8GB SSD i5 system I have to restart it when it reaches over 1 GB because it starts to slow down A LOT. And it get over 1 GB very fast for someone who constantly open/close tabs like me (even in safe mode).

I was an avid FF user until I saw myself installing an add-on to restart FF when it reaches X MB of memory usage. Today I use Opera for most of my browsing but I some sites does not work properly with Opera and some extensions only exist on FF.

Shame.

abhi_beckert:

Do you have extensions installed? Maybe one of them is causing it.

crislevin:

every benchmark I saw, firefox uses less memory than everybody else, I really don’t get why people complain about it?

nnethercote:

You might have problems with your profile. Visit about:support, and click on the “Reset Firefox” button in the top right corner. It’ll generated a new profile that preserves much of your browsing history, though it will disable any add-ons you have. (It’ll give you details about exactly what it preserves and discards before doing so, so you can cancel it if you change your mind.) This step fixes a lot of performance problems like the ones you describe.

To summarize the responses:

  • Me too!  It sucks.
  • Are extensions to blame?  (A reasonable question, though less likely since Firefox 15 fixed most extension-related leaks, and one that can be difficult to answer conclusively.)
  • Really?
  • Is a busted profile to blame?  Try “Reset Firefox” to determine if so.

carbon fibre didn’t respond, but korg250 (the “Me too!” commenter) took up the commenting baton…

korg250 (again):

In my case it is not about memory usage, but how slow it becomes when it reaches 1 GB of usage.

I will try the about:support tip listed before.

grumpy2:

Out of curiosity, which version of Firefox are you basing this on?

Because the thing about these “I used Firefox until X got too much for me, and I switched” kind of imply something quite significant, which is “I am no longer using Firefox, and can therefore not speak for the possible improvements that have happened since I switched”.

For the last couple of versions, Firefox has shown extremely good memory usage in my experience. And yes, they were absolutely horrible at this in earlier versions, and I’ve been *that* close to switching a number of times because of it. I’ve ranted a lot about how their memory usage was just so broken I doubted they’d ever be able to fix it. But in my experience, they have.

(And yes, this is 120% subjective, and I am certainly not arguing that if your experience differs then you are wrong. But I have noticed a vast improvement in recent versions. As I recall, the first of these fixes started trickling in around the FF13 timeframe, but I could be wrong on that.)

otomo1001:

I think you’ve some other computer issues. Here is my system, 2.4gig firefox process. Still running fine and dandy no slowdown, this is 18 mind you.

http://imgur.com/gNmZK84

korg250 (again):

I am always in the latest version. Was using 18.0.2 until today. Now I am on the 19 after resetting via about:support (still watching how it goes).

Maybe my installation had something wrong and even after the updates the memory issue remained – I don’t know. But I made the “switch” over the last week, when I compared speed between FF and Opera.

korg250 (again):

The reset via about:support worked for me. My FF is running smoothly for over 24 hours. Thanks guys!

Excellent!

Many people have used Firefox for years, and so the chance of mysterious profile problems occurring are quite high.  “Reset Firefox” cleans that dirty slate.  As the support page says, it will “easily fix most problems”.  It’s a simple, one-shot process with a high effectiveness rate that can be explained in a single sentence — perfect for mentioning in an online forum.  (Indeed, I should have linked to the support page in my comment, and then I could have written less explanation.)

Ideally it would also be tried by anyone who tries switching back to Firefox from another browser.  In fact, if Firefox is started up for the first time in a while (3 months? 6 months?) it would be great if it offered to automatically do this.