Forgot your password?
typodupeerror
Firefox Internet Explorer News

IE9, FF4 Beta In Real-World Use Face-Off 358

Posted by timothy
from the it's-a-walk-off dept.
An anonymous reader writes "Most browser benchmarks are isolated, artificial tests that can be gamed by browser vendors optimizing those specific cases. With only those benchmarks to go on, the folks at LucidChart were skeptical that the IE9 beta would actually outperform other modern browsers in real-world applications. To separate hype from reality, they built their first browser benchmarking tool, based in LucidChart itself. This benchmark is to SunSpider what a Left4Dead 2 benchmark is to 3Dmark Vantage. Product specs don't matter, only real-world performance on a real-world application. The results were surprising. IE9 held its own pretty well (with a few caveats), and the latest Firefox 4 beta came in dead last."
This discussion has been archived. No new comments can be posted.

IE9, FF4 Beta In Real-World Use Face-Off

Comments Filter:
  • by LostCluster (625375) * on Friday September 17, 2010 @11:40PM (#33617436)
    Browser makers design for tests, real world data shows exact opposite results to what you expect. We've got a crew working on the story overnight and will have a full update for you on the weekend edition of Wicked Early News, we start before normal people wake up.
    • by beelsebob (529313) on Saturday September 18, 2010 @01:45AM (#33617896)

      Actually, these results look very similar to the ones the WebKit guys (both chrome and safari teams) publish. They're almost always saying chrome and safari are similarly fast in the lead, firefox lags slightly, and IE8 is way slower... These are the same results as here. It just appears that IE9 is now added to the pile, and added at the top.

      So what have we learned
      1) Mozilla are good at lying about benchmarks (actually, we already knew that, they've been claiming the next big firefox release would be faster than everything for a while now)
      2) IE9 is quick

      The question is... is IE9 correct. I'll take works correctly but takes time over doin it rong quickly any day.

      • by GMC-jimmy (243376) on Saturday September 18, 2010 @05:02AM (#33618438) Homepage

        I'm not so quick to forget why I dropped IE in the first place. To get away from ActiveX and general apathetic browser security. Where's that represented in benchmark?

        • by h4rm0ny (722443) on Saturday September 18, 2010 @07:02AM (#33618808) Journal

          I'm not so quick to forget why I dropped IE in the first place. To get away from ActiveX and general apathetic browser security. Where's that represented in benchmark?

          Well, that's more specifically why you dropped IE 5.5 and IE6. By the time you get to IE8, these aren't really issues and Firefox has traded its early security-consciousness for usability. And it looks from those charts that IE9 runs pretty fast. That lines up with my own experiences of the beta. Though it's worth keeping in mind that the IE9 is a beta so it's not 100% fair to draw conclusions until it goes to release.

          The most interesting things in the article aren't in TFS, though. One is that javascript processing is now apparently so fast that it's dwarfed by the time it actually takes the browser to render an updated page. Another is that in order to get the results from Firefox that they did, they actually had to drop a number of anomalous results where it ran vastly more slowly for unknown reasons. I'm not surprised as Firefox has been getting fatter and fatter ever since 3. The third is that Chrome blows everything else out of the water. They used an old version of Opera which is a shame as I have the newer one and my anecdotal impression is that it's snappier than the previous one so it would have been worth using the latest Opera. But still, Chrome apparently renders far faster than both IE9 and FF. I'd love to know why. Is it just that it's a new project designed from scratch without the cruft that other browsers come with? Does it lack support for significant functionality? Has there been a lot of low-level optimization?

          I think in a years time, the real battle is going to be between IE and Chrome. Even today I mainly only use FF for web-development due there being some excellent development add-ons for it that IE can't compete with. Opera is a nice general browser and my default, but its cookie management is shit. Much of FF's funding comes from Google, who I presume would want to push Chrome and who I guess fund Firefox as a means of keeping Microsoft from re-establishing browser-dominance. But that may not be sufficient reason to really push it to be the best it can be, just to keep it "good enough".

          I don't wish to be disrespectful to the FF developers. I know how complicated a code-base that size is and they're to be commended on producing a browser that serves well. But it's become a big, unwieldy beast in comparison to the leanness of Chrome and the new IE.

          • by Bert64 (520050) <bert@slash d o t . f i renzee.com> on Saturday September 18, 2010 @07:17AM (#33618852) Homepage

            The prevalence of firefox has helped the security of IE considerably, for a number of reasons...
            Before they started losing market share, MS had no intention of improving IE at all.. Also now that no browser has over 90% market share they become far less attractive targets for malware authors, who instead now either target specific areas (eg ie6 is still huge in corporate settings) or other software which has a huge market share (adobe flash/pdf, msoffice).
            So long as there are a handful of browsers out there competing with each other.
            Were it not for firefox, ie7 would be a slightly warmed over ie6, and would still be getting attacked on a daily basis and users would have nowhere to go.

            Incidentally, those benchmarks show ie9 coming in 3rd from last, only firefox 4 beta (by a small margin) and ie8 (by a huge and laughable margin) are slower. Actually released browsers such as firefox 3.6, safari 5, chrome 6 and opera 10 are all ahead of both betas of ie9 and firefox 4.

            Now as for why the firefox 4 beta is slower than 3.6, it could be compiled in a debug mode (as betas often are), its new javascript engine has only just been integrated and needs tweaking etc...

            The fact that current beta versions of both ie9 and firefox 4 are way behind current non beta versions of other browsers is rather poor, and you would hope that both will be fixed by the time their final releases come around.

            As for Google, they make their money from the web and generally don't care which browser you use to access it, so long as that browser is fast enough to deliver a reasonable experience and standard enough so it doesn't force them to spend a lot of time kludging around. Older versions of IE were a threat to google as they made their web based apps appear slow, new versions appear better however IE is still controlled by google's biggest competitor who, given the chance, would use it against google in any way they can. When it comes to firefox/opera/chrome/safari i doubt google really cares which you use.

    • Fuck this shit (Score:5, Interesting)

      by Massacrifice (249974) on Saturday September 18, 2010 @02:19AM (#33618002)

      Really, fuck it. I've had it with corporate-sponsored dick-fighting contest about which browser is the fastest. I really, really couldn't care any less. Features, openness, security, standards compliance, yeah. But If I want a fast app, I'll go native, thank you. Maybe I'm too old, but I've always thought HTML sucked as a programming paradigm. As an information distribution mechanism, sure. But for interactivity? Please. It's about time somebody called bullshit on this. Hell, a goddamn Visual Basic app from fifteen years ago kicked the butt of most modern web sites in usability, performance and ease of maintenance. The only thing that makes the web so attractive is the barrier to entry : free, nothing to install, immediate access to the average brains of millions. Just like TV. No thanks for dumbing it down to this. And now you wanna make it faster? Piss off. Go write real code that does something, not just another abstraction layer.

      • Re:Fuck this shit (Score:5, Insightful)

        by Gamer_2k4 (1030634) on Saturday September 18, 2010 @02:36AM (#33618066)
        Meanwhile, in the real world, people actually want the source of all their information to be fast. Go figure.
      • Re: (Score:3, Interesting)

        by strikethree (811449)

        "The only thing that makes the web so attractive is the barrier to entry : free, nothing to install, immediate access to the average brains of millions."

        Ah. You forgot one other very important thing that make the web so attractive as a platform: Control. "They" have all the control over the app/service. They can changes terms of service, price, availability, etc. at any time they want and you have NO RECOURSE.

        Control baby. Control. It is good to be King.

        • Re:Fuck this shit (Score:4, Insightful)

          by CastrTroy (595695) on Saturday September 18, 2010 @08:51AM (#33619232) Homepage
          There are some good things about the web.

          Instant updates to all your users at exactly the same time. No more worrying about if your users are using old, insecure, incompatible versions.

          For times when users need to be connected to eachother, having everybody go through a standard HTTP server is the easiest way to getting rid of networking problems, not having to worry about firewalls, and not having to expose your users computers to the web.
        • Re: (Score:3, Insightful)

          by Oligonicella (659917)
          Actually, no they don't. A few houses worth of friends and mine all interconnect without the use of anyone's services. Now, if I want to read The Tokyo Daily, I have to use something that connects to it. Can't afford the 12,000 mile of cabling m'self. They have every right to charge me for the use of all the shit they had to do to get my browser linked up with Tokyo.
      • Re: (Score:3, Interesting)

        by Stick_Fig (740331)

        The only thing that makes the web so attractive is the barrier to entry : free, nothing to install, immediate access to the average brains of millions. Just like TV. No thanks for dumbing it down to this. And now you wanna make it faster? Piss off. Go write real code that does something, not just another abstraction layer.

        Fun fact: Programmers don't choose the platform. Users do.

  • by Tumbleweed (3706) * on Friday September 17, 2010 @11:44PM (#33617446)

    I'd like to see Chrome 7 results in there...

    • by icebike (68054) on Saturday September 18, 2010 @12:18AM (#33617598)

      Me too.

      Its odd how both the summary above and the linked article sort of over look the fact that Chrome just blew the doors off of every other browser and the compared the production version to the latest and greatest of the others.

      Chrome really deserves top billing, but the story is about the who is going to come in dead last.

      Yawn.

      • by moderatorrater (1095745) on Saturday September 18, 2010 @01:34AM (#33617850)
        Modern browsers are so fast that the difference is miniscule. If you're looking at using IE6 or using Chrome, then obviously Chrome needs to be praised. If you're comparing several browsers that are all fast enough that there's no strong difference between them in real world use, then mocking the loser is just more fun :D
        • by icebike (68054) on Saturday September 18, 2010 @01:43AM (#33617890)

          Fast enough unless you need to work a major portion of your day using one. Then you find out there are significant and meaningful differences in speed that affect your ability to get things done.

          Fast enough for someone who checks in once a day to look at email is easy.

          • by CAIMLAS (41445) on Saturday September 18, 2010 @02:18AM (#33617998) Homepage

            Absolutely. I find it humorous how I will occasionally hear coworkers cursing:

            1) The speed of their browsers. "Render, god damn it!" echos down the halls.
            2) The ability to quickly switch tasks/tabs within the browser (ie responsiveness vs. speed). "Fucking flash!"
            3) The stability of the browser. I don't really care so much if a single tab crashes; I'll just reload it. Someone with 40+ tabs in firefox, however, is stuck waiting a minute or so while whatever they were doing crawls back from the dead. (Users who don't have session management in their browser are even less fortunate.)

            Meanwhile, I sit there contentedly working away, not distracted by such things, due to using Chrome and a lightweight window manager on Linux. I only start noticing a slow down when I'm being inefficient, anyway - IE, doing too much at once, getting distracted, and not getting anything done.

            Of course, the slow users don't complain all that much, either. Seems they can't quite keep up with much of anything. :P

            A little speed in the right places makes a huge difference.

    • by TeXMaster (593524) on Saturday September 18, 2010 @03:13AM (#33618178)
      The versions used for tests were quite debatable. For IE and FF they put up both the final version and the current beta, while for the other browsers they only show the results for the current release ... mostly: the latest released Opera version is 10.60. A fair comparison should have included Chrome 7 and Opera 10.70 and replaced Opera 10.53 with 10.60.

      I guess the actual selection of versions shows how the point of the article was more about bashing FF4 compared to IE9 (in which it also failed, given the very small difference between them) rather than doing a honest comparison of all of the browsers.

  • Real test? (Score:5, Funny)

    by haystor (102186) on Friday September 17, 2010 @11:45PM (#33617450)

    Unless the test contains porn and all the accompanying popups, it's not a real world test.

  • I have been thinking about using Chrome for some time, it seems faster and already has a respectable community around it. But I also would like to avoid google stuff and I'm already used to Firefox, not sure if it is worth the trouble.... Any opinions?

    • by FutureDomain (1073116) on Friday September 17, 2010 @11:56PM (#33617506)
      Try one of the Chrome forks, such as ChromePlus, SRWare Iron, Comodo Dragon, or a pure Chromium build. They're just like Chrome, but without the questionable Client-ID and RLZ modules that Google put in Chrome. I typically use ChromePlus since it has several features that I like builtin, but I've been trying the IE 9 beta and I like that as well. It's faster than Firefox in my opinion and I absolutely love the UI layout.
      • Re: (Score:3, Informative)

        by oddfox (685475)

        SRWare Iron was created for the sole purpose of earning the "creator" some money on ad revenue. To borrow from my previous post on the subject:

        Everyone mentioning SRWare Iron should know about this little tidbit: The story of Iron [neugierig.org]. The article and the linked IRC log [neugierig.org] tell a very interesting story about a guy less concerned with having a good reason to fork and more concerned with making money off of adsense and publicity for creating a "privacy-respecting" Chrome which is basically a perpetually outdated Ch

    • Re: (Score:2, Informative)

      by ChipMonk (711367)
      If you're concerned about Google eavesdropping on your browsing habits, you might try the SRWare Iron [srware.net] browser instead. It's Chrome minus the snooping built-ins [wikipedia.org].
    • by rmcd (53236) * on Saturday September 18, 2010 @12:42AM (#33617696)

      I use Ubuntu 10.04. I have mostly switched to chrome (not completely; there are still sites that don't work properly with it). My problem with firefox was memory usage. I tend to have *lots* of tabs open and I often don't reboot for weeks. Firefox memory usage creeps up over time and my laptop slows. I keep reading that this is no longer supposed to happen, but it happens to me. Chrome with a comparable number of open tabs does not slow everything else down.

      If Firefox were better behaved I would stick with Firefox.

      • by ADRA (37398)

        Browser open several days fully loaded with things? It takes all of 5 seconds to reload a browser. If its really that large of a burden what's the problem? I'm amazed at the behavior changes that I've adapted too now that browsers save open tabs on close/crash. Bookmarks have seen substantial decline, and I don't leave a browser open unless I'm in core development time or looking something up online. With a 5 second start time, there's really no bother for me. Maybe if you have dozens of media rich tabs co

    • by CAIMLAS (41445)

      I've been using Chrome for about two years now, I think (mostly Chromium, actually, but the performance improvement does not match the stability/memory use right now, so thinking of going back to Chrome). I gave it a try in version 4, and was underwhelmed. I switched outright when it was first released as available for Linux - in the early alpha stages of 5 - and haven't looked back once.

      I was never a 'heavy extension user' in Firefox (at least not since 3.x), but I was almost always "cutting edge" (alpha,

    • by ADRA (37398)

      As a guy who doesn't specifically care what to use besides IE, I can say that I always end up going back to FF from chrome. I try it for a few days, but I can't explain it but there's always something that I need in FF. The browser just seems slightly less likely to work with sites (not necessarily the browser's fault mind you), and I get crashes in it. Despite what I hear from others, I rarely get FF crashing on me. Now that there's plugin isolation, I imagine its less crash friendly. The browser is still

  • by gandhi_2 (1108023) on Friday September 17, 2010 @11:49PM (#33617476) Homepage

    No native Gopher support?

    From my cold, dead hands!

    • by Tumbleweed (3706) *

      No native Gopher support?

      I don't care if it's native Gopher or imported Gopher, as long as it's here legally.

  • Frames per Second? (Score:2, Insightful)

    by Anonymous Coward

    Frames per second seems like pretty much the opposite of "real-world" for how 99% of users use their browsers.

  • "Real World"? (Score:5, Insightful)

    by farnsworth (558449) on Friday September 17, 2010 @11:58PM (#33617516)
    I'm not sure what is "real world" about spinning a UML box around another UML box in a giant (presumably) canvas-based javascript app.

    For me, "real-world" means: is gmail fast enough? is opening a new tab fast? is image rendering fast enough? is html video fast enough? is the occasional embellished html5 animation fast enough? is typing into the address bar fast enough?

    I'm sure their diagramming app is cool and everything, but I don't think I've ever seen anyone use anything like it, so I'm not sure what is "real world" about using it for a benchmark.

    They even said that they altered the test in the middle to fix IE's performance problem. Come on.
    • Re: (Score:2, Insightful)

      by Danieljury3 (1809634)

      For me, "real-world" means: is gmail fast enough? is opening a new tab fast? is image rendering fast enough? is html video fast enough? is the occasional embellished html5 animation fast enough? is typing into the address bar fast enough?.

      I use gmail in basic HTML mode. It takes away some of the things I never use and, in my opinion looks much cooler (none of that modern looking nonsense). I also barely ever type into the address bar.

  • by Anonymous Coward

    Firefox 4 beta 6 doesn't have the new JavaScript engine in it. Beta 7 will have it. But there's no particular need to wait for beta 7 as they could benchmark a nightly now. They also don't mention what kind of video card they've got in that laptop. IE9 and Firefox 4 can take better advantage of a good video card on Windows 7 than the other browsers tested and that may significantly influence a charting benchmark like this one.

  • Figure of Speech (Score:2, Insightful)

    by Anonymous Coward

    I love how a difference of a few milliseconds (looks to be 5ms) means a browser "tanks" and its position, when compared to other browsers, can be described as "dead last." Oh no, we're not painting a bias picture here.

  • The results were surprising. IE9 held its own pretty well (with a few caveats), and the latest Firefox 4 beta came in dead last.

    No. Not surprising at all... but I still prefer Firefox for everyday use.

  • jaegermonkey (Score:3, Informative)

    by yakumo.unr (833476) on Saturday September 18, 2010 @12:13AM (#33617576) Homepage

    I'd be much more interested to see it being done with the builds of FF 4 that have jaegermonkey enabled. Though that should be merged into the main branch fairly soon with any luck.

    http://www.conceivablytech.com/2673/products/first-look-firefox-4-jaegermonkey/ [conceivablytech.com]

    • by BZ (40346)

      It has been, already. Trunk builds of Firefox have it, as will Firefox 4 beta 7.

  • by BZ (40346) on Saturday September 18, 2010 @12:16AM (#33617588)

    The way this benchmark measures "intra-frame time" is broken. In particular, it uses a setInterval with a 1ms delay. No browser actually respects that 1ms. Chrome clamps it to 5ms; others clamp it to 10ms, all to avoid the website thrashing the CPU pointlessly.

    The upshot is that Chrome's interframe delay in the graph is about 5ms and Firefox 3.6's interframe delay is aboug 10ms. Which this particular benchmark can't tell apart from "no delay at all", given its methodology.

    Firefox 4 beta, IE9 beta, Safari, and Opera seem to have delays greater than 10ms, so they're clearly doing some work they can't finish in 10ms.... or have slightly buggy timer implementations. Or both.

    Of course in practice frame rates above 60fps or so are pointless since the screen doesn't redraw that often. ;)

    On the other hand, on Mac, on modern hardware, I get 4.5fps in Chrome 7 dev on a random trial document I just tried, with JS render tiems on the order of 7ms (with a 7ms standard deviation) and "intra-frame time" of 224ms with a 900ms standard deviation (yes, those numbers are nuts). Firefox 4 beta comes in at about 11s for the JS (with 3ms stddev) and 125ms for the "intra-frame time" (with a claimed stddev of 0, which looks really suspicious).

    It'd be nice if there were non-obfuscated source for this benchmark so its number-crunching could be evaluated; that 0 stddev is ... highly improbable.

  • by phantomcircuit (938963) on Saturday September 18, 2010 @12:21AM (#33617616) Homepage

    This benchmark can be run by anyone in LucidChart. First, sign up for a free account here.

    Nuff said

  • by whoever57 (658626) on Saturday September 18, 2010 @12:27AM (#33617644) Journal
    TFA (yes, I actually read it) says: "Firefox 4.0 Beta 6 came in behind all other browsers except for IE8". That's quite different from "dead last".
    • Re: (Score:3, Funny)

      by gmuslera (3436)
      IE8 counted as "braindead last"... so the next place was that one for FF4
  • Now, will IE9 scale with a faster GPU? I'd like to see some benchmarks with different video cards.

  • Who cares? (Score:5, Interesting)

    by Goonie (8651) <robert,merkel&benambra,org> on Saturday September 18, 2010 @01:15AM (#33617792) Homepage

    I use Firefox and IE regularly, have played with Chrome, and occasionally use Safari on the Macs at work.

    I honestly can't notice any difference between any of them in rendering speed.

    99.99% of the time, web browsing performance is network-limited anyway.

    Surely standards support and browser stability are more important features, at least on platforms with more grunt than an iphone?

  • The story doesn't mention the GPU in use, but it does mention it's an i7 processor. So I assume it's using an i965-class GPU. These aren't exactly known for speed or stability on linux. I believe FF4 uses Cairo, which in turn uses XRender, and my experience with integrated Intel GPUs and XRender is that pure software ( ie X on FBDev ) is faster. I would have liked to have seen a system used which could actually accelerate the drawing operations.

  • The results were surprising. IE9 held its own pretty well (with a few caveats), and the latest Firefox 4 beta came in dead last."

    Why is this surprising? Firefox has been the slowest of the current crop of browsers (IE8 excepted) for quite a while now.

  • And once again I see no mention of lag or latency behind everyday controls, one of the real factors which affect a user's perception of speed and responsiveness. The kind which gets them to say, "It feels faster, but I don't know how". I'm talking about switching between tabs, closing tabs, clicking the browser's back or forward button, and general UI navigation. You want hundredths of a second or less for these kind of actions.

  • by macshit (157376) <miles.gnu@org> on Saturday September 18, 2010 @02:03AM (#33617952) Homepage

    I'm not sure how they get off calling this a "real world benchmark", as it seems to bear almost no resemblance to what people normally use web browsers for: "The benchmark works by simply dragging a part of the diagram around the page for five seconds." WTF?

    It certainly doesn't seem to be any more useful than the other browser benchmark being touted these days, and arguably it's much less useful, because it measures a single very narrow aspect of browser operation, one which has little connection with typical browser usage.

    Moreover, the slashdot summary seems to go to great lengths to emphasize how "badly" FF4 did on this (useless, remember) benchmark, and to pump up IE9: "The results were surprising. IE9 held its own pretty well (with a few caveats), and the latest Firefox 4 beta came in dead last" -- but if you actually look at the results that emphasis is misplaced: almost all the browsers were quite close to each other, with a few outliers, but in no cases was FF4 an outlier, and indeed was pretty much identical to IE9 (on this test).

    The only clear result I can see is: When doing a certain very specific type of javascript rendering, most modern browsers have pretty much identical performance, though chrome's particularly fast, and IE8 particularly slow.

    Of course, that isn't very interesting to anybody except LucidChart users, of course, nor very likely to generate any controversy...

  • Chrome is faster because it massively favors speed over customization and features. FF is slower because it favors customization, and assumes, correctly, that no one actually actually gives a flying fark if it needs slightly more than a 1/50th of a second to render a page that Chrome can do in 1/100th of a second. This isn't a problem, nor is it news. Now of course, you may do the occasional task where those milliseconds actually matter because your browser is processing something enormous, but then just in

  • Stability and web standards implementation is what matters most.

To understand a program you must become both the machine and the program.

Working...