Main menu:

Site search

Categories

Archive

Browser Sunspider update

IEBlog recently showed some Sunspider results that show them currently ahead of us. I just want to correct that a little bit. :-)

The latest Firefox shown on their graph is Firefox 4.0 beta 6, which is well behind our nightly releases at this point, including in performance. Their graph shows the Chrome 8 nightly, but nightlies for no other browsers. So I decided to do a comparison test using Sunspider 0.9.1 on the latest version I could get for each of the big 5 browsers. Results:

So, not terribly different from their results, except that it shows we’re a hair faster as of now, at least on this machine. We are currently finishing up a last few perf projects before we turn to focus on getting it ready for release. I’m expecting us to get 3-10% faster by then, but that’s just a guess. I’m sure IE is still working on perf as well.

A final point is that the graph shows pretty clearly that Sunspider scores are converging: the days when one browser would be 2x or 6x as fast as another are gone. Now it’s more like 1.1x or 1.2x. I know there are a few more tricks we can play that are too complicated to do before Firefox 4.0, but they would make only a small dent in Sunspider scores. By next year, either we’ll all be fast and basically done with JS perf work, or else it will be time to update the benchmark to help drive the next round of perf improvments.

Update: sdwilsh pointed out to me that the IEBlog post shows a date of “10/25″ for Firefox, which isn’t the build date of beta 6. So it might actually be a nightly build from a week or two ago. Not clear.

Comments

Comment from Matt Parlane
Time: November 2, 2010, 2:10 pm

But AWFY shows TM as faster than V8 on SunSpider on both x86 and x64… or is that just a difference between Mac and Windows?

Pingback from Tweets that mention David Mandelin’s blog » Browser Sunspider update — Topsy.com
Time: November 2, 2010, 2:30 pm

[...] This post was mentioned on Twitter by kangax and Mats Bryntse, David. David said: I just did a short blog post on current nightly/beta Sunspider scores: http://blog.mozilla.org/dmandelin/?p=158 [...]

Comment from Randy Peterman
Time: November 2, 2010, 2:46 pm

I think that I’ve seen enough complaints about benchmarks that a new benchmark mechanism should be built. I think it should actually load up (in an iFrame or through some other mechanism) the top 100 websites and plow through them all start to finish.

Comment from Jim
Time: November 2, 2010, 3:23 pm

I think the 25th of October date is date they benchmarked the browsers. Certainly in benchmark articles I’ve seen in the past on the IE blog they’ve never been inclined to use Firefox nightlies or to change any settings from the default.

Pingback from YT Intro v5.5.6 Update! | Script ebooks
Time: November 2, 2010, 4:51 pm

[...] David Mandelin's blog » Browser Sunspider update [...]

Comment from Read more
Time: November 3, 2010, 11:09 am

I’ve always thought yours was a bit faster as well.

Trackback from === popurls.com === popular today…
Time: November 3, 2010, 11:20 am

=== popurls.com === popular today…

yeah! this story has entered the popular today section on popurls.com…

Comment from Fuse
Time: November 3, 2010, 2:14 pm

Was it a x86 or a x64 Windows? I believe Firefox’s JIT compiler used to run on x86 OSes only; whereas V8, for example, is able to perform native compilation on other platforms as well.

Consequently, the difference could be greater on 64 bits systems with 64 bits browsers.

Comment from matti
Time: November 3, 2010, 2:18 pm

Geckos JIT runs on X86 and X64 and on PPC (I’m not sure about PPC).

Comment from james
Time: November 3, 2010, 2:45 pm

Stable releases are more important than the beta or alpha builds…
http://philosophyparadise.com/blog/?p=222
Firefox clearly lags behind.

Comment from Hongli Lai
Time: November 3, 2010, 3:09 pm

You might know that a lot of people are upset about Firefox performance. Most of them are not upset about Javascript performance or other raw-CPU-time performance, but the sluggish feeling of the UI (things like startup time and the feeling that typing in the awesome bar is sluggish when it’s autocompleting). I think that might be more important than Javascript performance at this point.

Comment from berler
Time: November 3, 2010, 3:27 pm

Why provide a graph when it’s not even clear what is better? Is the graph showing load times/latency (in which case lower would be better) or is it showing some other speed metric such as pages loaded per minute (in which case higher is better).

Please label the Y axis with units, and have an indication of what is better.

Pingback from Browser Speed | Marketing Blog | Creative Development
Time: November 3, 2010, 3:40 pm

[...] Blog Browser SpeedPosted on 04. Nov, 2010 by Marketing in Web DesignEBlog recently showed the Mozilla blog some Sunspider results that show them currently ahead of Firefox. Below are the results that the [...]

Comment from dmandelin
Time: November 3, 2010, 3:40 pm

berler: I figured everyone who cared to read this post knows that SunSpider scores are in ms, and lower is better. Sorry if my lack of Y axis labels offended you.

Pingback from Interesting Reading #615 – Netflix detroying internet? Virgin birth, Strongest body armor, Awesome timelapse video and much more! – The Blogs at HowStuffWorks
Time: November 3, 2010, 7:07 pm

[...] Browser Sunspider update – “the days when one browser would be 2x or 6x as fast as another are gone. Now it’s more like 1.1x or 1.2x. I know there are a few more tricks we can play that are too complicated to do before Firefox 4.0, but they would make only a small dent in Sunspider scores. By next year, either we’ll all be fast and basically done with JS perf work, or else it will be time to update the benchmark to help drive the next round of perf improvments…” [...]

Pingback from Browser Parity | adPineapple
Time: November 3, 2010, 7:32 pm

[...] quote from Mozila’s blog: “The days when one browser would be 2x or 6x as fast as another are gone. Now it’s more [...]

Comment from Anonymous
Time: November 3, 2010, 8:53 pm

You should try for politics, this is a great attempt at spin. These individual tests are great for blogs but won’t mean much in the long run.

Comment from Jason Knight
Time: November 3, 2010, 9:33 pm

I agree with Hongli on this one: JS performance is not the most important thing for me. UI responsiveness is much more important.

Comment from sysprv
Time: November 3, 2010, 11:25 pm

d00d, JS performance is good enough now, and will be for a while until people think of even more amazing things to do with it. Please fix up other things with FF…

Comment from logos
Time: November 4, 2010, 10:52 am

yeah this is getting boring; especially when you consider that JS scores (on sunspider for instance) are now too close between browsers to generate performance gaps. But these gaps still exist, and they’re not javascript related anymore, and for the time being, in real world (ie browsing) conditions, FF4b8pre is almost as fast as Chrome dev (9.0.570), while IE9 is faster then both. It launches faster, and it browses faster. Don’t get me wrong, I don’t use IE on a regular basis at all and i’m certainly not advertising it, just been testing IE9 and it’s a damned rocket. I just don’t like the interface (UI mainly), but it will change…not enough to convert me I guess… but you got to be blind if you don’t admit that: the fastest browser atm is very obviously Internet Explorer 9 beta, period.

Comment from logos
Time: November 5, 2010, 1:24 am

just to slightly correct my statement about FF and Chrome. I’m not changing a word to what I said about IE9, it’s not that, it’s more about FF4 and Chrome.

I actually tested that a few days ago, even thinking then that FF might have become faster than Chrome. Then I deactivated the “adblock” extension in Chrome and saw that it was indeed slowing chrome like hell. Compared again with FF and Chrome seemed (a bit) faster again. This was a subjective impression…I just compared again yesterday after posting here and… I can definitely say that FF4b8pre is now faster than Chrome (dev) almost on any site, i.e. in terms of complete loading of web pages or page rendering, whatever you wanna call it >>> Firefox 4 is now definitely faster then Chrome.
So this is all just confirming what I said in my last post here: javascript performance is not the decisive parameter anymore. On sunspider (on my system), I get about 530ms with IE9beta, about 480ms with FF4, and 380 ms with Chrome dev 9.0. This order is exactly inverted in real world browsing, where IE9 comes first, followed by Firefox4 and then by Chrome…(there is one exception: when it comes to video streaming, FF4 comes in last position here).

ps: scrolling pages while loading or once loaded is also smoother in FF then in Chrome, but this is not new, it’s always been like that. What’s “new” is Firefox4 CPU usage in live stream videos, it’s a disaster, when it use to be just the contrary in FF 3.6; I mean it use to be the best browser in terms of resource usage when watching streaming (live or not) videos. Now it’s the worse…hope this will change until the final comes out early next year what what I heard. I won’t mention starting times, it has improved, but that’s not yet what it should be.

Comment from tony
Time: November 12, 2010, 6:25 am

hello friends ‘m love one you

Comment from arapça tercüme
Time: November 14, 2010, 9:13 am

Thanks for this nice post.

Comment from New York Mets Jerseys
Time: November 17, 2010, 9:33 pm

This is a cool adn nice read. The blog is written in such a way that it is so easy to read and understand. I AM a fan of your blog. Thanks for sharing this information.
This is a great blog posting and very useful. I really appreciate the research you put into it…

Comment from Professional SEO
Time: November 22, 2010, 9:49 pm

I love the speed battle! I’ve never once noticed any version of IE to be faster than Mozilla. My IE is up to date and clean while my Mozilla browser is full up plugins and mods and it still smokes IE. This is on a VISTA machine at my office. The only reason I still use IE is because my Outlook is set up to use IE for links in emails and I’m too lazy to change it. Keep up the good work!

Comment from umuts
Time: November 24, 2010, 8:16 am

thank you

Comment from hotel amenities
Time: November 25, 2010, 9:48 pm

ps: scrolling pages while loading or once loaded is also smoother in FF then in Chrome, but this is not new, it’s always been like that. What’s “new” is Firefox4 CPU usage in live stream videos, it’s a disaster, when it use to be just the contrary in FF 3.6

Comment from canada goose outlet
Time: November 25, 2010, 9:48 pm

just to slightly correct my statement about FF and Chrome. I’m not changing a word to what I said about IE9, it’s not that, it’s more about FF4 and Chrome.