Thursday, April 14, 2011

IE9 v Firefox4 Round 2 - Windows7

A week or two ago, I looked at a number of browsers, and compared them all. While it was done with my standard method on windows Vista, Windows7 is now more dominant. It's also in many cases 'faster'. So, Could browsers be written so as to unlock potential in Windows7 that isn't there in Vista? That would clearly be the case with Internet Explorer 9, which isn't even offered for XP, because of it's reliance on newer versions of windows. So, let's see if the version of windows really does make a difference.

Background
As promised, I've run some tests using windows 7 as well. The test machine used here is a Toshiba Satelite C655 laptop run in 'high performance' power mode. The specs are:

  • Intel i3-380m Processor (2.53 Ghz, dual core with hyper-threading for 4 effective cores)
  • 4Gb DDR3-1066 RAM (PC3-8500)
  • Mobile Intel HD graphics
  • Windows 7 Home Premium SP1 (64-bit)

Note
This is a different system to the Vista testbed, so the results are not directly comparable. and are instead for relative comparison.

The test method was identical to the test method used for Vista. Tests were run on a clean user session, with no extensions running, just a plain vanilla configuration. The tested browsers were:

  • Internet Explorer 9.0.8112.16421 32-bit mode ('IE9-32')
  • Internet Explorer 9.0.8112.16421 64-bit mode ('IE9-64')
  • Firefox 4.0
  • Opera 11.01
  • Chrome 10.0.648.204


Results
Previously, I'd used 4 tests, Peacekeeper, Sunspider (0.91), Acid3 and html5test. Of these, the latter two focus entirely on the capabilities of the browser, with no regard to performance. Theoretically they should be identical then, unless part of the operating system adds some functionality. In order to be sure, the browsers were run through those tests, but the results were identical to the same versions already tested on Vista, and are not included here. Peacekeeper and Sunspider tests results are the main focus here.

The Peacekeeper test is the first one up, as always. IE9-64, is not included on the graph, as Peacekeeper only shows the best result for the browser version on their graphs. However, it was significantly slower in both tests. Here's the peacekeeper graph for the 32-bit browsers:

Peacekeeper tests, i3-380m/Win7
click to enlarge
Opera is a lot closer than expected to Chrome, while there's a marked drop down to IE9-32 and Firefox. The score for IE9-64 was lower still, almost as far behind Firefox4, as Firefox was behind IE9-32. However, while that tells one story, it's not quite the case that 'Chrome is a little bit faster than Opera' for instance. As Peacekeeper consists of several tests (explained in their FAQ), looking at them broken down by test might shed more light.

What this shows is that Opera actually beats Chrome on almost all of the tests. The only place it wins is in the data processing, where it blows the others away. In addition, the Complex Graphics score is not included in the final score (since at the time of the tests establishment, it wasn't common on all browsers, although it is now, at least for the major Windows ones). Thus it would seem the final score should be closer between the two.


So, now for Sunspider. The tests were a bit quicker, and fairly close, but again, IE9-64 was significantly slower.
Shorter is better
Conclusions
So, are there any major differences between Vista and Win7 performance? Well, let's compare their scores (remember, different hardware, so it's only a relative comparison, rather than direct) between the exact same versions, tested on the two.

As you can see, the biggest boost is to IE, obviously, breaking clear of Firefox. Opera also makes a big improvement as well, closing up on Chrome.

For Sunspider, we get this graph
Here the biggest gain is from Firefox, which bypasses Chrome, and is on a near-parity with Opera. By contrast, IE, makes the smallest improvement, but it is still the fastest.

IE9 and Opera both seem to prefer Windows7 over Vista, moreso than their competitors. Were it not for Chrome's extraordinary 'data' result in the peacekeeper test, it wouldn't even be an issue. Firefox4, much hearalded though it might be, is however being left far behind, kept afloat by an array of add-ons (many of which are based on features integrated into Opera). As I write this, Opera 11.10 has also been released, While Microsoft i s playing along with an early release of IE10. Since this is a comparison between Vista and Win7, I didn't test them. However,  It will be interesting to see how they fare.

Raw Data

Windows 7 data
Browser Sunspider (ms) Peacekeeper
Total Rendering Social Networking Complex Graphics Data DOM operations Text Parsing
IE9 – 64bit 1147.7 3692 4946 1918 7474 4257 2229 7426
IE9 – 32bit 244.9 5685 5484 2687 11629 12658 2790 11430
Firefox 4.0 273.2 4400 2333 3056 8560 8383 4083 6760
Opera 11.01 268.7 8454 7077 5986 15230 6985 8072 18082
Chrome 10.0.648.204 291.3 9102 4689 4474 14815 30258 6031 16325

Windows Vista data
Available HERE

2 comments:

  1. Great comparison. Thanks for taking the time to do that. I agree, the things that keep me with Firefox are the add-ons.

    I have had more issues with the latest versions of Firefox(not with FF4 yet). One is not properly closing. Sometimes I will close the browser and then try and reopen it and Windows tells me that I already have FF running. I either have to wait or restart my computer.

    Hopefully with the release of Aurora http://www.softwarecrew.com/2011/04/firefox-aurora-release-heralds-new-development-cycle-for-firefox/, this will allow less issues/bugs. It looks as if each release will have a 6 week cycle Hopefully this will help with stability.

    ReplyDelete
  2. Maybe. The way to deal with your Fx 'closing' problem is to go to the taskmanager, and look for the firefox task in the processes tab. select it, and hit 'end task'.

    Orphaned processes like this happen a lot when a program crashes. I see it when testing alpha/Beta µTorrent clients a fair bit.

    ReplyDelete