{"id":1027,"date":"2011-07-15T17:50:06","date_gmt":"2011-07-15T06:50:06","guid":{"rendered":"http:\/\/blog.mozilla.org\/nnethercote\/?p=1027"},"modified":"2011-07-15T17:50:06","modified_gmt":"2011-07-15T06:50:06","slug":"building-a-page-fault-benchmark","status":"publish","type":"post","link":"https:\/\/blog.mozilla.org\/nnethercote\/2011\/07\/15\/building-a-page-fault-benchmark\/","title":{"rendered":"Building a page fault benchmark"},"content":{"rendered":"

I wrote a while ago about the importance of avoiding page faults<\/a> for browser performance.\u00a0 Despite this, I’ve been focusing specifically on reducing Firefox’s memory usage.\u00a0 This is not a terrible thing;\u00a0 page fault rates and memory usage are obviously strongly linked.\u00a0 But they don’t have perfect correlation.\u00a0 Not all memory reductions will have equal effect on page faults, and you can easily imagine changes that reduce page fault rates — by changing memory layout and access patterns — without reducing memory consumption.<\/p>\n

A couple of days ago, Luke Wagner initiated an interesting email conversation with me about his desire for a page fault benchmark, and I want to write about some of the things we discussed.<\/p>\n

It’s not obvious how to design a page fault benchmark, and to understand why I need to first talk about more typical time-based benchmarks like SunSpider.\u00a0 SunSpider does the same amount of work every time it runs, and you want it to run as fast as possible.\u00a0 It might take 200ms to run on your beefy desktop machine, 900ms on your netbook, and 2000ms to run on your smartphone.\u00a0 In all cases, you have a useful baseline against which you can measure optimizations.\u00a0 Also, any optimization that reduces the time on one device has a good chance of reducing time on the other devices.\u00a0 The performance curve across devices is fairly flat.<\/p>\n

In contrast, if you’re measuring page faults, these things probably won’t be true on a benchmark that does a constant amount of work.\u00a0 If my desktop machine has 16GB of RAM, I’ll probably get close to zero page faults no matter what happens.\u00a0 But on a smartphone with 512MB of RAM, the same benchmark may lead to a page fault death spiral;\u00a0 the number will be enormous, assuming you even bother waiting for it to finish (or the OS doesn’t kill it).\u00a0 And the netbook will probably lie unhelpfully on one side or the other of the cliff in the performance curve.\u00a0 Such a benchmark will be of limited use.<\/p>\n

However, maybe we can instead concoct a benchmark that repeats a sequence of interesting operations until a certain number of page faults have occurred.\u00a0 The desktop machine might get 1000 operations, the netbook 400, the smartphone 100.\u00a0 The performance curve is fairly flat again.<\/p>\n

The operations should be representative of realistic browsing behaviour.\u00a0 Obviously, the memory consumption has to increase each time you finish a sequence, but you don’t want to just open new pages.\u00a0 A better sequence might look like “open foo.com in a new tab, follow some links, do some interaction, open three child pages, close two of them”.<\/p>\n

And it would be interesting to run this test on a range of physical memory sizes, to emulate different machines such as smartphones, netbooks, desktops.\u00a0 Fortunately, you can do this on Linux<\/a>;\u00a0 I’m not sure about other OSes.<\/p>\n

I think a benchmark (or several benchmarks) like this would be challenging but not impossible to create.\u00a0 It would be very valuable, because it measures a metric that directly affects users (page faults) rather than one that indirectly affects them (memory consumption).\u00a0 It would be great to use as the workload under Julian Seward’s VM simulator<\/a>, in order to find out which parts of the browser are causing page faults.\u00a0 It might make areweslimyet.com<\/a> catch managers’ eyes as much as arewefastyet.com<\/a> does.\u00a0 And finally, it would provide an interesting way to compare the memory “usage” (i.e. the stress put on the memory system) of different browsers, in contrast to comparisons of memory consumption which are difficult to interpret meaningfully.<\/p>\n

 <\/p>\n","protected":false},"excerpt":{"rendered":"

I wrote a while ago about the importance of avoiding page faults for browser performance.\u00a0 Despite this, I’ve been focusing specifically on reducing Firefox’s memory usage.\u00a0 This is not a terrible thing;\u00a0 page fault rates and memory usage are obviously strongly linked.\u00a0 But they don’t have perfect correlation.\u00a0 Not all memory reductions will have equal […]<\/p>\n","protected":false},"author":139,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[4544,4546,311],"tags":[],"_links":{"self":[{"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/posts\/1027"}],"collection":[{"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/users\/139"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/comments?post=1027"}],"version-history":[{"count":0,"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/posts\/1027\/revisions"}],"wp:attachment":[{"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/media?parent=1027"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/categories?post=1027"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.mozilla.org\/nnethercote\/wp-json\/wp\/v2\/tags?post=1027"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}