BETA Browser Benchmarks

Back on July 5th, I benchmarked a few browsers, to see how the new mainstream versions performed. The results came back that popular open source browser, firefox, was kinda crap, and that Chrome did well, but not quite as well as Opera. This time around, I'm testing some old browsers, for comparison, but also the new browsers, AND the betas for the big 4.

Contents:

1. Recap
   
2. Versions tested
   
3. Peacekeeper Benchmark
   
4. Kraken Benchmark
   
5. Sunspider Benchmark
   
6. Acid3 test
   
7. Conclusions
   

RECAP

The tests are run with only the browser going, in the order Peacekeeper, Kraken, Sunspider, all in the same tab. On loading the new test's page, it's left for 30 seconds to allow for stabilization from the previous test, and any cache clearing, or other operations to take place. System Stats: HP Pavilion a6535c

• Intel Core2Quad Q6600 Cpu (2.4 Ghz quad core)
• 6Gb DDR2-6400 Ram
• nVidia GeForce 9500GS graphics card
• Windows Vista home Premium (64bit)

VERSIONS TESTED

(Old versions in red, current in blue, betas in green )

• Internet Explorer 7
• Internet Explorer 8
• Internet Explorer 9.0.7930.16406 (BETA)
• Opera 9.62
• Opera 10.53
• Opera 10.62
• Opera 10.70 snapshot build 9048 (BETA)
• Firefox 3.5.11
• Firefox 3.5.13
• Firefox 3.6.10
• Firefox 4 Beta 6 (BETA)
• Chrome 5.0.375.127
• Chrome 6.0.472.59
• Chrome 7.0.517.5 (BETA)

So a mixture of old, current, and beta browsers. The tests will be the Futuremark Peacekeeper tests, Mozillas new Kraken test, SunSpider, and for the Betas, the Acid3 test.

PEACEKEEPER

First the tests from last time.

Peacekeeper benchmarks, July 5th 2010 (bigger is better)

Several of these versions have been tested again, for reference. The new graph looks like this.

Peacekeeper benchmarks, September 16th 2010 (bigger is better)

As you can see, when the same versions are compared, there is a slight increase in scores this time. However, there is a clear winner in Chrome on the peacekeeper tests, running away with things. Both Firefox and Internet explorer have made big strides with their beta versions, advancing their scores significantly, but they are both still left behind by Opera and Chrome. Interestingly, Chrome's beta scored slightly worse than the stable version, and Opera's beta only gave a slight increase.

KRAKEN

This test uses the the new 'Kraken' benchmark by Mozilla. This is a varient of the Sunspider tests, that aims to be a more accurate benchmark system than the existing sunspider tests. The problem is, some older browsers have problems with it. Opera 9.64 and IE 7 both locked up running it, while IE8 constantly popped up a dialogue box warning that a script was slowing things down. There is, consequently, no results for those browsers. The raw data is as follows: (click data links for full details)

Browser	Kraken (ms)
Internet Explorer 7	N/A
Internet Explorer 8	N/A
Internet Explorer 9	61475.0
Opera 9.62	N/A
Opera 10.53	14500.1
Opera 10.62	14312.4
Opera 10.70	13933.6
Firefox 3.5.11	27696.6
Firefox 3.5.13	27573.7
Firefox 3.6.10	29374.2
Firefox 4 Beta 6 (BETA)	22578.5
Chrome 5.0.375.127	21508.8
Chrome 6.0.472.59	19633.1
Chrome 7.0.517.5 (BETA)	19154.8

And in graph form

Kraken Tests 9-16-10, Shorter is better
Click to Enlarge

As you can see, here Opera was much faster than Chrome, and the Firefox beta was closing in on it fairly nicely. It will be seen just how trusted this test will become.

SUNSPIDER

Last time's sunspider tests were a close run thing between Opera and Chrome

SunSpider Tests July 5 2010

This time, it was still close between them. (click data links for full results)

Browser	Sunspider (ms)
Internet Explorer 7	7232.2
Internet Explorer 8	6735.2
Internet Explorer 9	606.8
Opera 9.62	4733.0
Opera 10.53	538.4
Opera 10.62	503.6
Opera 10.70	502.4
Firefox 3.5.11	1622.6
Firefox 3.5.13	1496.2
Firefox 3.6.10	1342.4
Firefox 4 Beta 6 (BETA)	764.4
Chrome 5.0.375.127	472.2
Chrome 6.0.472.59	444.0
Chrome 7.0.517.5 (BETA)	408.6

and in Graph form

SunSpider Tests 16-9-2010
(click to enlarge)

Chrome just about squeaks it from opera, although the real shock is IE9, which gets fast enough to split the difference between Firefox and Opera/Chrome. an impressive performance, although with the Kraken performance taken into account, it may just be a case of benchmark optimization.

ACID 3

The acid3 test is a test of browser standards. it's not a new test, but one that firefox and IE have traditionally not done well at. How have these Beta's handled it? (click to enlarge)

CONCLUSIONS

Both firefox and Internet explorer have made massive strides over their previous versions, as befits major version changes. They're getting good, but they're still not quite there yet. Despite the major version name change (a result of Google's recently announced version policy) the Chrome beta is just a slight improvement, as is the Opera Beta. As before, it's a close thing between them, and the choice is down to personal preference.

One thing to notice though, that is obvious when you look at the acid3 screens, is that the design is becoming more homogeneous,  they're all going to a single main button, with a few basic browsing controls, in an aero window.