 |
| Source Technovore |
We will be testing Firefox, Opera, Internet Explorer9, Chrome, and Safari. Compared with the results of last year, not long after Firefox5 came out, how have things changed?
First things first, I was somewhat goaded into this test. A Firefox fan challenged me, during a discussion in the UK Pirate Party IRC channel to look at Firefox again, when Firefox10 came out. Being an open-minded person, I agreed to look at it.
Vista-based tests were all performed Wednesday, Feb 1 2012 on the standard Vista test machine (specs here). The only hardware change was the addition of an external 1Tb USB HDD, which was disconnected for the tests. The machine was run on a user account set specifically for the testing, and the same order and method used on each browser.
Browser | Version | Release Date | Notes |
Chrome | 16.0.912.77 | January 23 2012 | |
Opera | 11.61 | January 24 2012 | |
Internet Explorer | 9.0.8112.16421 | March 14 2011 | 32-bit version |
Safari | 5.1.2 | November 29 2011 | |
Firefox | 3.6.26 | January 31 2012 | Not used in Memory Test |
Firefox | 5.0 | June 21 2011 | Not used in Memory Test |
Firefox | 10.0 | January 31 2012 |
The inclusion of Firefox 5 was to allow a comparison with earlier benchmark tests along with IE9. In addition, the Long Term Release version of Firefox (currently at 3.6.26) was tested at the benchmarks, but not in the memory test, as it is no longer possible to run two versions at the same time.
The results are as follows
Yet again, Chrome is the clear leader. Opera has dropped back a bit from Chrome – they used to be fairly close – meanwhile Firefox and Safari are still pretty close. Internet Explorer, because it’s now 11 months old, is dropping back. By contrast, three of the other four had versions less than a month old, while Safari is 3 months old, and that shows.
If we look at the score breakdown (not graphed this time) it’s clear to see why Chrome’s in the lead. In pretty much every test, it was the front runner, with Opera the only one able to beat it, on two tests – the HTML5 canvas ripple-test, where it was 2 frames per second faster; and DOM operations where not only Opera, but Safari took Google’s client to task. The huge lead for chrome, though, comes from its long-time strength, data parsing, where it was 3.5x faster than all the others. It’s also the only client to handle all the video playback codecs.
It’s a clear win for Chrome here.
Here we can see the results. Some definite progress has been made by Mozilla, and it’s certainly catching the IE browser quickly. Opera has actually slowed down some, back to the same sort of performance of 2 years ago. Perhaps it was a testing error, but we will have to wait and see.
Peacekeeper Test.
The peacekeeper test has changed since the last time I benchmarked, and the new version aims to test things in more stable way, and run on more devices. The Firefox 5, and IE9 results can be compared with the previous test’s results if a past comparison is desired.The results are as follows
Yet again, Chrome is the clear leader. Opera has dropped back a bit from Chrome – they used to be fairly close – meanwhile Firefox and Safari are still pretty close. Internet Explorer, because it’s now 11 months old, is dropping back. By contrast, three of the other four had versions less than a month old, while Safari is 3 months old, and that shows.
If we look at the score breakdown (not graphed this time) it’s clear to see why Chrome’s in the lead. In pretty much every test, it was the front runner, with Opera the only one able to beat it, on two tests – the HTML5 canvas ripple-test, where it was 2 frames per second faster; and DOM operations where not only Opera, but Safari took Google’s client to task. The huge lead for chrome, though, comes from its long-time strength, data parsing, where it was 3.5x faster than all the others. It’s also the only client to handle all the video playback codecs.
It’s a clear win for Chrome here.
Sunspider
As always, we check out the javascript engine with the sunspider 0.91 test. |
| Sunspider 0.9.1 (shorter are better) |
HTML5
Using the tests from html5tests.com, all browsers picked up some points, except IE9. Surprisingly, that also includes FireFox5, which gained 10 points from its last test back in the summer despite the browser being completely untouched. I can only attribute that to the test being altered to conform to the new html5 specs.
While only Chrome managed to break the 300 point mark last time, this time all except IE9 and the older Firefox’s made it. Opera is where Chrome was, while firefox has just nudged ahead for once. Chrome is way off in the distance though as we can see here.
The breakdown is as follows.
Surprisingly, Chrome wasn’t the worst one, as most people would assume. Internet Explorer was worse in every department, and Opera was also worse in all-but-one group. The surprise though has to be Safari. It was far and away the lightest on the system. However, if I had tried another 8 tabs (and when devising this test, myself and Illunatic did come up with about 20 in total, including some NSFW) we might see a difference.
Firefox, while it’s improving, still requires too much in the way of outside software to be made ‘full features’, and while it’s a criticism that can also be thrown at Chrome, it doesn’t have Chrome’s speed, while its security is starting to look a little ragged. The 3.6.x release is also showing slight improvement, perhaps as much as they can squeeze out of it without a radical code overhaul, and the point of continuing the 3.6 path is that it’s NOT an overhaul.
Safari is competent, and a choice if you’ve got memory limitations, yet it’s just ‘competent’ and not ‘Good’ or ‘great’ at anything in particular. As for Internet Explorer, what was decent 11 months ago is starting to look a little long-in-the-tooth now, even despite its Sunspider score. While it’s certainly better than previous versions, Microsoft needs to update things more often, at least every 6 months. However, with Internet Explorer 10 already in development, it’s unlikely to happen.
Windows7 Testcoming very soon DONE
While only Chrome managed to break the 300 point mark last time, this time all except IE9 and the older Firefox’s made it. Opera is where Chrome was, while firefox has just nudged ahead for once. Chrome is way off in the distance though as we can see here.
Memory
One of the more frequent criticisms of both Chrome and Firefox has been their memory usage. To test the real-world memory usage, an 8-tab assortment of pages was used, which are surprisingly system-heavy. They were:- Gmail, logged in but Gtalk turned off.
- Logged in http://facebook.com with sidebar and chat active
- www.hulu.com front page
- Translate.google.com, translating the main page of piratpartiet.se
- A 10 minute 720p youtube video (http://www.youtube.com/watch?v=FicK4gP7kNc) played for the first 10 seconds, and the rest left to buffer.
- Maps.bing.com showing Atlanta, with the traffic layer enabled
- Twitter, logged into my account (@ktetch) with ssl
- Finally, this site (ktetch.co.uk)
 |
| The memory test in progress |
This gave a balance of data-heavy pages, and active content pages. Browsers were run simultaneously, to ensure they all had the same content. Each browser had the first tab set up, then every browser had the second, with all actions being the same. The figures were collected from Chrome’s memory manager, but were checked (which is one reason this took a week to get out) using figures from Process Explorer.
The breakdown is as follows.
Surprisingly, Chrome wasn’t the worst one, as most people would assume. Internet Explorer was worse in every department, and Opera was also worse in all-but-one group. The surprise though has to be Safari. It was far and away the lightest on the system. However, if I had tried another 8 tabs (and when devising this test, myself and Illunatic did come up with about 20 in total, including some NSFW) we might see a difference.
Conclusion
Which is the best browser for Vista? It really is hard to beat Chrome. Not only for performance, but for adopting the new technologies, it’s way ahead of the rest. Doubtless it was the reason Mozilla has attempted to emulate Google with the rapid release schedule. Opera is also hanging in there too. If speed isn’t the top priority, but reliability and built-in features are, or if you’re just not happy with Chrome/Google’s privacy position, then it’s certainly the one to go for.Firefox, while it’s improving, still requires too much in the way of outside software to be made ‘full features’, and while it’s a criticism that can also be thrown at Chrome, it doesn’t have Chrome’s speed, while its security is starting to look a little ragged. The 3.6.x release is also showing slight improvement, perhaps as much as they can squeeze out of it without a radical code overhaul, and the point of continuing the 3.6 path is that it’s NOT an overhaul.
Safari is competent, and a choice if you’ve got memory limitations, yet it’s just ‘competent’ and not ‘Good’ or ‘great’ at anything in particular. As for Internet Explorer, what was decent 11 months ago is starting to look a little long-in-the-tooth now, even despite its Sunspider score. While it’s certainly better than previous versions, Microsoft needs to update things more often, at least every 6 months. However, with Internet Explorer 10 already in development, it’s unlikely to happen.
Windows7 Test
Raw Data
Browser | Chrome | Opera | Safari | IE | Firefox | Firefox | Firefox | |
Version | 16.0.912.77 | 11.61 | 5.1.2 | 9.0.8112.16421 | 10.0 | 5.0 | 3.6.26 | |
Position | 1 | 2 | 4 | 5 | 3 | - | 6 | |
PEACEKEEPER | ||||||||
Overall Score | 2629 | 111912 | 1410 | 1121 | 1484 | 1143 | 641 | |
Rendering | 65.10 | 68.52 | 58.17 | 36.99 | 48.49 | 36.65 | 28.61 | |
RenderGrid01 | fps | 140.17 | 160.46 | 149.19 | 66.76 | 177.00 | 174.59 | 94.39 |
renderGrid02 | fps | 118.83 | 133.28 | 127.71 | 66.73 | 116.31 | 113.58 | 58.26 |
renderGrid03 | fps | 18.14 | 24.23 | 13.02 | 7.78 | 5.54 | 4.76 | 5.82 |
renderphysics | fps | 59.43 | 42.55 | 46.14 | 53.99 | 48.46 | 19.12 | 20.90 |
HTML5 Capabilities | of 7 | 7 | 5 | 3 | 3 | 6 | 6 | 3 |
webglSphere | fps | 58.04 | NO | NO | NO | 60.79 | 61.22 | NO |
VideoPosterSupport | Yes | Yes | Yes | Yes | Yes | Yes | Yes | |
videoCodecH264 | Yes | NO | Yes | Yes | NO | NO | NO | |
videoCodecTheora | Yes | Yes | NO | NO | Yes | Yes | Yes | |
videoCodecWebM | Yes | YES | NO | NO | Yes | Yes | NO | |
workerContrast01 | ops | 2405.60 | 1738.97 | - | - | 3658.45 | 3750.75 | - |
Workercontrast02 | ops | 3783.73 | 2555.20 | - | - | 6486.82 | 3906.66 | - |
Gamingspitfire | fps | 50.12 | 51.69 | 51.74 | 53.28 | 55.60 | 55.17 | 53.28 |
HTML5 Canvas | 23.82 | 25.58 | 9.15 | 10.28 | 23.00 | 17.76 | 6.39 | |
experimentalRipple01 | fps | 37.46 | 39.51 | 14.34 | 16.58 | 36.32 | 27.86 | 10.30 |
experimentalRipple02 | fps | 15.14 | 16.57 | 5.84 | 6.37 | 14.56 | 11.32 | 3.96 |
Data | 56369.14 | 16677.73 | 17978.98 | 17325.7 | 17706.15 | 13937.04 | 9876.45 | |
arrayCombined | ops | 9543.0 | 3721.0 | 5669.0 | 4169.5 | 4587.0 | 3871.0 | 3887.0 |
arrayWeighted | ops | 332964.5 | 74750.5 | 57019.5 | 71994.24 | 68347.0 | 50178.5 | 25095.0 |
DOM operations | ops | 9077.41 | 9674.29 | 9239.96 | 3541.38 | 7034.93 | 5362.16 | 3503.7 |
domGetElements | ops | 512657.5 | 132719.0 | 345499.0 | 125628.14 | 304024.0 | 290994.5 | 104686.5 |
domDynamicCreationCreateElement | ops | 15289.0 | 12068.79 | 23581.5 | 1437.5 | 14572.0 | 5860.5 | 6319.0 |
domDynamicCreationInnerHTML | ops | 15600.5 | 26414.5 | 19405.0 | 3994.5 | 22368.5 | 7170.0 | 12180.0 |
domJQueryAttributeFilters | ops | 3976.5 | 10986.5 | 4313.5 | 1681.0 | 2977.0 | 2711.5 | 1511.5 |
domJQueryBasicFilters | ops | 1302.0 | 1561.5 | 1456.5 | 1118.5 | 1014.0 | 919.04 | 539.23 |
domJQueryBasics | ops | 3458.5 | 2253.5 | 2696.5 | 994.5 | 2210.5 | 1787.5 | 992.0 |
domJQueryContentFilters | ops | 2174.0 | 16522.0 | 2245.5 | 960.0 | 876.5 | 1096.5 | 748.5 |
domJQueryHierarchy | ops | 5137.5 | 16522.0 | 5236.5 | 4204.5 | 3840.5 | 3304.0 | 1809.0 |
domQueryselecto | ops | 17112.0 | 17302.0 | 15588.5 | 16091.5 | 18959.5 | 18573.0 | 9020.5 |
Text Parsing | 158377.91 | 90367.15 | 62998.29 | 75744.14 | 51880.51 | 40048.46 | 17119.27 | |
stringChat | ops | 55575.5 | 40492.0 | 21290.0 | 44892.0 | 46347.5 | 47124.0 | 22805.5 |
stringDetectBrowser | ops | 357355.5 | 340031.0 | 184300.0 | 195694.72 | 202913.5 | 141160.0 | 114453.0 |
Stringfilter | ops | 35279.5 | 19264.0 | 32562.5 | 25987.0 | 2652.0 | 3012.5 | 164.42 |
stringValidateForm | ops | 782809.5 | 439897.0 | 212463.5 | 171232.88 | 258577.5 | 84657.0 | 77276.0 |
stringWeighted | ops | 181681.5 | 51649.7 | 36554.5 | 63775.51 | 58280.0 | 60727.5 | 44337.0 |
SunSpider | ms | |||||||
HTML5 Test | ||||||||
Basic | /475 | 374 | 329 | 302 | 141 | 332 | 296 | 183 |
Bonus points | 13 | 9 | 9 | 5 | 9 | 9 | 4 | |
Memory Usage | ||||||||
Private | Kb | 471864 | 560704 | 100576 | 548724 | 278056 | ||
Shared | Kb | 17393 | 21072 | 28436 | 35144 | 30060 | ||
Total | Kb | 489257 | 581776 | 129012 | 583868 | 308116 | ||
Virtual Memory | ||||||||
Private | Kb | 494024 | 614784 | 111748 | 619404 | 266784 | ||
Mapped | KB | 184436 | 16520 | 17284 | 272128 | 118360 |
No comments:
Post a Comment