Wednesday, June 6, 2012

FireFox 13 Windows7 Shootout

Source Technovore
It’s that time, again, another browser benchmark showdown.

This time we’re starting with Windows7, and doing Vista later. Over the last few months, things have moved on a bit, Chrome’s moved on from v16 to v19, Firefox has also moved on 3 versions to 13, Opera’s gone from 11.61 to 11.64 (and an open 12 beta) and Safari’s gone from 5.1.2 to 5.1.7. Only IE has stayed untouched. So has there been much improvement over these new versions, or is it a new coat of paint over the same old tat?

Leading with Windows 7 for a change, we’re covering the following versions, both new and slightly older. In total ten browsers were tested, 3 each of Firefox and Opera, two chrome and one each of Safari and Internet Explorer.
Browser
Version
Release Date
Chrome
18.0.1025.168
2012-03-28
Chrome
19.0.1084.46
2012-05-15
Opera
11.61
2012-01-24
Opera
11.64
2012-05-10
OperaNext
12.0 beta build 1429
2012-05-24
Safari
5.1.7
2012-05-09
Internet Explorer
9.0.8112.16421
2011-03-14
Firefox
10.0.2
2012-02-16
Firefox
12.0
2012-04-24
Firefox
13.0
2012-06-05
The Windows 7 test rig is unchanged from previous pieces, and its specs are available here.

There are some more benchmarks added from previous instances, but the memory test is unchanged.

The tests are (links will launch the benchmarks in a new window)
  • Peacekeeper – Futuremark’s test has been a staple of the Pol&P2P browser benchmarks since day 1.
  • ClubCompy – a benchmark written in simple code that tests various aspects of the javascript engine.
  • Sunspider – another test that’s been in since day 1, and developed by the Webkit team
  • V8 – A Javascript suite created by Google
  • Kraken – A browser benchmark by the Mozilla team.
  • HTML5Test – a test of html5 capabilities
  • Memory test – a real-world test of memory usage using 8 common realworld sites.

Peacekeeper

The standard peacekeeper test covers a wide variety of tests, from data handling to 3d acceleration, video playback and text parsing. Here we have the overall scores

As you can see, the browsers have split into two fairly equivalent groups, one of Chrome and Opera, and one of Safari, Internet Explorer and Firefox. First, while Chrome just edges out Opera in the scores, most of that comes from, as always, its amazing data scores particularly in ‘array weighted’, where it performed 3x better than its nearest competitor. Elsewhere it didn’t do so well, performing worse than Opera overall. Chrome is, however the only one to run all the video formats. Also, Opera 12 does have 3d acceleration support, but it’s disabled by default and doesn’t work with Peacekeeper (although does with other sites)

The other three also keep a stable position, also fairly unchanged from last test. Again, each browser plays to its strengths, IE excels in rendering, Firefox in video  playback and canvas applications, and Safari in DOM operations, yet all three are still worse than Opera or Chrome.

ClubCompy

Clubcompy is a site aimed at teaching computer programming to kids. They include a simple set of benchmarks that test various aspects of a javascript engine, focused as follows.


The clear winner was Chrome, but a surprise second was Safari. Opera, IE and Firefox were not that far behind. The key test was test two, which did half-area fills, and test 3, which was based on Mandelbrot’s. Both focused on control flow and context switching.


Sunspider and Kraken

Two similar codebase tests, Sunspider and Kraken next. The first, Sunspider by the webkit (safari and chromium) developers, has traditionally been an IE win thanks to code optimisation. This time there’s been an upset as that’s been taken away in the latest upgrade of Chrome, who goes from last place to first. This now leaves Safari in last place (it was unusual that both webkit browsers were last in the webkit-designed tests). Firefox and Opera seem to be playing around in the same area as before.



Mozilla’s Kraken test has Firefox doing pretty well, but it’s Chrome that steals the show here again. Opera is about three times slower than Firefox, and Internet Explorer is about four times slower, with Safari splitting the difference.

V8

V8 is Google’s own benchmark test. It contains 8 tests at present, and Chrome leads the way on all but one. This includes a massive improvement on the NavierStokes test which involves double precision array manipulation. The only test where it fell short was Splay, dealing with automatic memory management subsystems, where both Firefox and Opera excelled. Firefox came a competent second in all the other tests. Safari and IE where pretty far down, often just trailing Opera, and they could do with picking their game up.


HTML5

Minor gains in HTML compatability have been made by Chrome and Firefox, and a rather larger one by Safari since the last test. The OperaNext beta of Opera12 has a major jump, that will at least bring it within striking distance of Chrome, and regain some headway over Firefox.

Memory test

As before, we took 8 fairly typical pages, that encompass a wide variety of web technologies, including dynamic content and video. They were loaded at the same time, so as to display the same content. As before they are

Again, Safari was by far the best for memory usage. Firefox isn’t that far behind, although no long term testing was done to check for the constant claims of memory leaks. Opera and IE performed about the same, and Chrome, as always did the worst, presumably due to the sandboxing of processes.


Raw Data


Chrome 18
Chrome 19
Opera 11.61
Opera 11.64
Opera 12 b1429
Safari 5.1.7
IE 9
Firefox 10
Firefox 12
Firefox 13
PEACEKEEPER
SCORE
2556
2397
2540
2533
2335
1525
1504
1468
1404
1419
RENDERING
77.8
74.38
86.76
86.77
41.14
56.88
64.66
33.92
34.55
32.7
RenderGrid01
164.21
164.66
199.51
199.24
70.77
144.36
194.11
86.98
123.11
109.91
RenderGrid02
153.26
147.63
196.09
194.8
77.13
125.91
154.11
65.2
59.44
47.3
RenderGrid03
22.33
19.24
27.94
27.5
14.38
13.9
9.42
6.18
5.62
5.99
renderPhysics
65.19
65.44
51.84
53.11
36.49
41.45
62.04
37.8
34.65
36.7
HTML5 Capabilities
7
7
5
5
5
3
3
6
6
6
webglSphere
67.92
58.36
no
no
no
no
no
41.6
45.56
47.52
videoPosterSupport
yes
yes
yes
yes
yes
yes
yes
yes
yes
yes
videoCodecH264
yes
yes
no
no
no
yes
yes
no
no
no
videoCodecTheora
yes
yes
yes
yes
yes
no
no
yes
yes
yes
videoCodecWebM
yes
yes
yes
yes
yes
no
no
yes
yes
yes
workerContrast01
2730.1
2302.63
2177.53
2247.56
3319.79
no
no
4651.1
5155.51
4598.79
WorkerContrast02
2963.09
2986.35
3054.97
3097.21
3394.91
no
no
5590.86
6229.82
6345.86
gamingSpitfire
50.42
50.26
49.33
49.85
53.5
50.96
49.51
59.12
54.5
51.16
HTML5 Canvas
13.38
12.3
31.22
32.01
36.92
8.26
12.16
15.04
14.85
14.89
exp ripple 01
20.8
19.0
48.0
49.5
55.7
12.9
19.5
23.4
22.8
23.0
exp ripple 02
8.6
8.0
20.3
20.7
24.5
5.3
7.6
9.7
9.7
9.7
Data
56,621.9
54,695.2
23,023.0
22,315.3
23,979.9
20,686.3
23,202.0
22,902.1
23,340.8
23,729.8
arrayCombined
9,071.5
8,689.0
4,326.5
4,374.0
4,745.5
6,173.0
4,360.5
5,269.0
6,186.0
6,486.0
arrayWeighted
353,418.5
344,293.0
122,514.0
113,848.5
121,175.0
69,322.0
123,456.8
99,546.0
88,026.0
86,818.0
DOM Operations
10,385.9
9,359.4
12,173.3
12,079.2
11,132.0
10,524.2
4,060.6
8,337.0
8,047.5
8,051.4
domdomGetElements
692,000.0
734,278.0
200,943.5
193,478.0
184,906.5
668,758.5
171,821.3
528,742.5
456,659.5
445,549.0
domDynamicCreationCreateElement
13,400.5
11,494.5
13,960.1
14,291.4
15,695.0
26,244.5
1,721.0
16,877.0
14,309.0
13,801.0
domDynamicCreationInnerHTML
13,486.5
12,182.0
34,779.0
34,398.0
31,062.5
22,075.5
4,775.5
29,611.0
28,412.5
29,140.0
domJQueryAttributeFilters
4,296.5
3,948.0
17,764.0
17,877.5
14,889.0
4,434.0
1,780.5
3,137.5
2,832.5
2,948.5
domJQueryBasicFilters
1,696.0
1,600.0
1,741.5
1,737.5
1,616.0
1,361.1
1,160.0
1,027.0
941.5
918.0
domJQueryBasics
4,382.0
4,175.5
1,998.5
1,976.0
1,971.5
2,781.5
1,104.0
2,426.5
2,115.0
2,186.5
domJQueryContentFilters
2,474.0
1,920.2
1,896.0
1,923.0
1,316.1
2,275.5
1,018.5
930.0
1,626.0
1,578.5
domJQueryContentFilters
6,168.5
5,015.0
21,955.0
22,151.0
20,787.0
5,566.5
4,707.5
4,027.5
3,639.5
3,479.0
domQueryselector
23,071.0
21,103.5
23,380.5
22,014.5
22,440.5
19,224.5
19,442.5
25,145.5
22,845.0
24,416.0
Text parsing
178,261.2
168,780.9
139,171.9
139,368.4
171,153.1
80,737.4
103,850.3
96,633.1
56,533.9
61,865.6
stringChat
62,458.5
61,382.5
49,547.5
49,156.5
52,886.5
22,442.5
53,705.7
58,052.0
54,377.0
54,644.0
stringDetectBrowser
399,811.0
350,217.5
717,336.0
757,051.5
1,081,630.5
278,747.0
322,580.7
308,852.5
214,522.0
239,941.0
stringFilter
38,376.5
37,766.5
21,030.0
20,661.0
24,751.0
36,917.5
29,365.5
3,053.0
2,424.5
2,749.5
stringValidateForm
90,058.5
839,362.5
1,217,862.5
1,236,628.0
1,571,619.0
340,423.0
239,808.2
393,489.0
302,938.5
349,013.0
stringWeighted
201,958.5
200,991.5
57,355.5
55,300.0
66,005.0
43,635.5
99,009.9
77,658.0
67,403.0
72,028.0
CLUBCOMPY
SCORE
12285
12062
9349
9287
8779
10338
8427
7488
7425
7790
Test 1 iterations
612
632
582
576
550
588
538
479
482
505
Test 2 iterations
87
93
37
38
34
50
27
26
26
27
Test 3 iterations
7470
7530
4950
4890
4800
5700
5920
4140
4080
4020
Test 4 iterations
146400
135360
108520
106440
98640
121880
83780
81600
84480
86920
Test 5 iterations
145960
135840
103720
105240
96560
121880
82580
81840
81360
87360
SUNSPIDER
Time
303.9
235.1
285.7
289.2
274.6
296.5
254.3
259.9
279.3
269.3
V8
Score
7195
9162
3614
3578
3666
2606
2234
5986
6002
5795
Richards
11042
6577
3107
3223
3139
3071
2252
7304
7904
7477
DeltaBlue
10850
14883
2807
2585
2810
2059
2288
8298
7326
6923
Crypto
13521
13672
4042
4041
4008
3235
3694
12497
12576
12332
RayTrace
11089
15730
4736
4708
4543
3464
1862
2821
3028
2564
EarleBoyer
19863
18588
4732
4684
4734
3084
3508
6637
6617
6528
RegExp
2135
2236
1449
1440
1568
1249
1818
1202
1061
1132
Splay
3294
3635
7391
7228
7888
2868
1555
7571
8695
8785
NavierStoles
2862
15606
3441
3476
3470
2719
1766
12783
12514
11961
KRAKEN
Time
3401.6
3307.2
12876.5
12864.8
12972.4
14849.6
15953.1
3913.9
4054.2
4136.5
HTML5TEST
Score
400
402
338
338
385
319
138
342
345
345
Bonus
13
13
9
9
9
9
5
9
9
9
Posted by K`Tetch at 6/06/2012 06:48:00 PM
Labels: , , , , , ,

No comments:

Post a Comment

Subscribe to: Post Comments (Atom)