Mozilla Unleashes the Kraken 363
An anonymous reader writes "Mozilla has released the first version a new browser benchmark called Kraken. Mozilla's Robert Sayre writes on his blog, 'More than Sunspider, V8, and Dromaeo, Kraken focuses on realistic workloads and forward-looking applications. We believe that the benchmarks used in Kraken are better in terms of reflecting realistic workloads for pushing the edge of browser performance forward. These are the things that people are saying are too slow to do with open web technologies today, and we want to have benchmarks that reflect progress against making these near-future apps universally available.' On my somewhat elderly x86_64 Linux system Google Chrome 6.0.472.55 beta completes the Kraken benchmark in 28638.1 milliseconds, Opera 10.62 completes it in 23612.4 milliseconds, and the current Firefox 4 nightly build completes it in 19897.5 milliseconds."
Obvious... (Score:3, Interesting)
How about IE performance? Too bad to even mention?
Re:Obvious... (Score:5, Funny)
Re: (Score:3, Funny)
It's still running.
You mean it hasn't crashed yet?
Re: (Score:2, Informative)
Intel T7500, 2.2GHz, 2GB Ram, Win 7 x64
Crome 7.0.517.5 dev - 19849.5ms
Re: (Score:2)
Re: (Score:2)
they run a 2.67 stock
the i7 955 extreme runs at 3.3.. so mr AC... how you keeping that processor cool?
are you actually using a Mac or a hackintosh sorta dude?
what bios(for hackintish) settings and what EFI (for Mac)settings do you use to clock it to that?
Re: (Score:2)
Re: (Score:3, Informative)
What's odd (to me) is that Chrome 5.0.375.29 beta (I'm still running a beta of 5? Odd) on Ubuntu on a Core2Duo in my T61 Thinkpad just ran the benchmark in 21486.1ms +/- 0.6%
Why wouldn't the Corei7 processor in his OSX with newer Chrome perform better?
Re:Obvious... (Score:5, Interesting)
It's still running.
Laugh all you want but I have had it running on IE 8 (Windows 7 64 bit) for the past 5 minutes and it is still stuck at the first stage. So I think we have a legitimate reason why Internet Explorer was not included...
Also got a warning that "A script on this page is causing your web browser to run slowly."...
Re: (Score:2)
IE7 (on WinXP 32 bit) stops at the first test because of a script failure. Are you sure it still ran?
Re: (Score:3, Informative)
On my somewhat elderly x86_64 Linux system
Re: (Score:3, Funny)
If you're suitably masochistic:
http://www.tatanka.com.br/ies4linux/page/Main_Page [tatanka.com.br]
Re: (Score:2)
How about IE performance? Too bad to even mention?
I started it on the latest IE 9 Preview, but it seems like it's taking at least around 3-4 times as long time to finish as Chrome or Firefox, so I aborted it. :-(
Javascript (Score:5, Insightful)
Shame it only benchmarks one small part of the browser - Javascript.
Re:Javascript (Score:5, Insightful)
The scary bit is that the world is quickly moving in a direction where serious desktop applications will be written in... Javascript.
So much for Java, .NET ; as soon as its possible to earn money through the Google App store for your Web app there will be a torrent of these applications being release to the world.
The web browser is the new platform.
It feels like going back 20 years in time.
Re: (Score:2)
Re: (Score:3, Interesting)
On the server side, fast is not enough. I don't want to be reimplementing functionality I know I can get elsewhere very very cheaply.
Re: (Score:2, Insightful)
Except that you now need what was then a supercomputer just to run Hello, world.
Re: (Score:2)
Well, there is a difference this time. Last time (1995-2000) they tried to make javascript run on the server side. This is nonsense.
Today, it's just the client getting thicker. Which is fine in my view, and JavaScript fit the bill.
Of course, if you regard javascript as the thing you look up examples for in Google in order to program it, you're screwed. JavaScript is a great and powerful language. JQuery is there to iron out the main differences in most browsers. Short of that and with the help of CSS3, HTML
Re: (Score:3, Insightful)
Many people have told me that... so far I'm unimpressed by their arguments. Yes, there are a lot of people who abuse js -- but how would that change if we gave them another language? Yes, working with the DOM in a cross-browser way is a pain in the ass -- but how would that change if you did that via another language? Yes, some js engines have bugs and performance issues -- but how do we know the engines for the other languages would be better (remember that we need an engine
Re: (Score:2)
I touched on it many times. What's horrible about it is that if something goes wrong - NOTHING happens. When you're trying to debug something and end up filling it full of popups to see when its failing...
Try GWT if you've got naught to do - it converts java to javascript code.
Re: (Score:3, Insightful)
Sorry, but this goes to the unimpressing-arguments pile... Debugging javascript is not more difficult than debugging anything else that runs in an environment like the browser and the tools that are available are fairly good. "Debugging by popups" was a choice you made, not something related to javascript the language.
Re: (Score:2)
...
Really? Show me something which provides me a break capability, with the ability to inspect variables, and single step thru the code.
Re: (Score:2)
Firebug does all those things, AFAIK. Although I certainly agree that it's not as convenient a debugging environment as Eclipse and other full blown IDEs.
Re: (Score:2)
Firebug makes it damn convenient. You set a breakpoint and refresh your page. It stores the breakpoints between refreshes and stops the code. It allows you to set watches on variables and step through or into function calls... just like any other language.
I bemoan developers I work with that don't use Firebug/Chrome to debug and use alert boxes to debug.
Re: (Score:3, Informative)
I figured I didn't need to as a AC had already mentioned that in a reply to you, but I was referring to Firebug. It has been able to do basic debugging for at least four years, and before that Venkman had those features (from 2002 or so)...
Current Firebug of course does loads more than just plain js debugging and should definitely be a tool in every serious web developers toolbox.
Somewhat similar functionality is found in at least Chrome developer tools, IE developer tools, Opera Firefly and Safari Drosera.
Re: (Score:2)
eh, sorry -- I thought you were the same guy as the original poster. Please do 's/you/Haedrian/' in your head when you read that.
Re: (Score:2)
There you go. [getfirebug.com]
Re: (Score:2)
The chrome and safari tools are actually better, at least in one crucial bit: If I set a break point inside a function that's wrapped in a closure, I can't see the closure variables in firebug, but can in chrome.
Re: (Score:2)
Or you could use Firebug, IE developer tools, Chrome Developer Tools etc
Re: (Score:2)
Re:Javascript (Score:5, Funny)
Re: (Score:2)
The language is horrible for all it lacks. I have read the Crockford book because I am writing larger and larger pieces of code in Javascript and found myself drifting.
I want to code things in a structured fashion and to do this I have a constant need to hack my own solutions.
OO in Javascript is a terrible hack. No ability to check if an object decents from a specific class. (yes, you can hack your own)
No native ability to include other source files (yes, you can fake it -- but it forces me to build my lo
Re: (Score:3, Insightful)
Sure, but the whole point of many Javascript-enabled applications is manipulating the DOM. Which is often really slow. Something as simple as putting a couple of invisible divs on the page and measuring their height is measured in milliseconds, not microseconds.
So while some applications obviously aren't possible without a fast Javascript engine, I think if you really want to make the web faster for people, you need to include a DOM benchmark. Something like inserting text, inserting elements, moving elemen
I hope that Firefox isn't playing Microsoft's game (Score:5, Insightful)
Re: (Score:2, Insightful)
If you hadn't noticed, every synthetic benchmark released from a browser vendor favoured their engine, at time of release. At least Google had balls to call it v8bench.
While I believe all benchmarks (and non-comprehensive ACID tests) to be 3dmark-style pissing contests where they encourage developers to fast-path specific used functions, I have more confidence in Mozilla producing another (Dromaeo also tried to have a more realistic workflow).
Re: (Score:2)
While I believe all benchmarks (and non-comprehensive ACID tests) to be 3dmark-style pissing contests
I want a benchmark that shows a large Slashdot article loading over 3G (at -1, no Javascript). For some reason it still completely freezes Firefox.
Sadly, it's the only browser out there with Noscript and Adblock.
Re: (Score:2)
That's a great start. If you can tell me how I could get the functioanlity of Vimperator, Web Developer, and Video Download helper, then I might be looking at a new browser.
Re:I hope that Firefox isn't playing Microsoft's g (Score:4, Insightful)
Its not like only MS and Mozilla as browser vendor released their own benchmark in with their product is doing good.
Besides, whats so bad about it? Ain't it obvious they are gonna include in their benchmarks stuff that they feel is important and as a consequence - made it good during browser development?
It just shows that other browsers than FF lack in some areas, with might - or might not - be important.
Re:I hope that Firefox isn't playing Microsoft's g (Score:5, Insightful)
The major browser Javascript engines (with the apparent exception of IE) are all now within the ballpark of each other. And they all make slightly different tradeoffs and are optimized for slightly different conditions, and have all released benchmarks that illustrate the strong points of their browsers.
If you look at v8bench (Google's Javascript benchmark), sunspider (the Webkit Javascript benchmark), and now Kraken (Mozilla's Javascript benchmark), you'll see that the latest browser versions are basically within 5-30% of each other on identical hardware. Which one comes out ahead depends on whose set of optimization parameters you think is most important.
Attacking Mozilla for doing the same thing every other major browser maker does (not that your post was, but other posters have) is silly.
Re:I hope that Firefox isn't playing Microsoft's g (Score:5, Informative)
One of Mozilla's longstanding issues with some of the other benchmarks is that they test toy problems that take longer to set up than to run. Yes, that favors browsers with JS engines that set up for execution quickly, and that portion of the engine is important. It doesn't show the real speedups for intensive applications in the browser, though. Optimizing the slow parts is the priority of most people right now, and getting the application set up a little faster at the beginning isn't as big a deal unless you have a lot of small scripts in one page.
An earlier blog post by Sayre [mozilla.com] and some of the comments to it display some of the issues.
Re:I hope that Firefox isn't playing Microsoft's g (Score:4, Interesting)
Why don't they just grab the (say) 200 most visited sites on the internet, copy the JavaScript and use that to benchmark instead?
Simples.
Re:I hope that Firefox isn't playing Microsoft's g (Score:4, Interesting)
Because the amount of time it takes to run the javascript on the top sites is pretty small (which is what the IE team was talking about around IE8's release). Performance on those sites mostly doesn't depend on whether your JS engine is the one in Chrome dev or the one in IE7. I only say "mostly" because I wouldn't be surprised if gmail is in the top 200. ;)
If you're going to worry specifically about JS performance (which is an assumption; the IE team is still saying that this focus is a mistake and to some extent they're right), you want to be benchmarking things that are gated on JS performance. That means identifying t the things that are slow with current JS engines and that people would like to be doing but can't because of said slowness, whatever those things are, and benchmarking those.
Re: (Score:2)
Lies, damned lies and benchmarks.
Re:I hope that Firefox isn't playing Microsoft's g (Score:5, Insightful)
Have you considered that it may well be the other way around?
If Mozilla, Google, MS, Apple or whoever truly believe that those particular aspects of a browser are the most important, doesn't it make sense that they would optimize their browsers for those aspects? I think it makes sense that they would write tests for the exact same aspects that they have been optimizing their browsers for, -because- they believe these are the key aspects.
Lacking an objective measure, all you can do right now is decide with whom you agree the most and probably use their browser or another browser that ranks well on their test - if these benchmarks are a critical decision factor for you.
Re: (Score:2)
Would be funny is someone releases a benchmark which shows that their software is less capable than others.
Re: (Score:2)
I use Firefox exclusively on my laptop. Speed is not its strong point. The main reason that I use FF is for a few add-ons that I now find necessary (mostly Vimperator to get vi keybindings). Speed is a reasonable trade off for me to have such a customizable browser, and I always assumed that the speed loss may have been due to the XUL stuff.
I think that FF being slow is common knowledge these days. I think that Mozilla is trying to show that the reason their browser is slower on some (most) things is becaus
Re: (Score:2)
other benchmarks were released by Apple (sunspider) and Google (v8) (and the last one is actually from Mozilla)
I believe Mozilla is actually always striving for the best of the community unlike the 2 big giants, they've always proved to do so. (afaik!)
Sunspider shows Safari as fastest, V8 Chrome as fastest (at least last I checked), and the Mozilla bench was a more mixed result.
but anyway back to the point, why would you believe Google and Apple and not Mozilla?
Re: (Score:2)
Re:I hope that Firefox isn't playing Microsoft's g (Score:2)
The charitable way to look at it is that each vendor writes a benchmark for the aspects of the browser which they think are most important, and those aspects are also obviously what they've put the most development emphasis on since they're regarded as the most important.
If you write a benchmark against your top development priorities and a rival eats your lunch despite having a different set of development priorities, you're probably not a top-tier vendor.
Am I the only one? (Score:4, Insightful)
Re:Am I the only one? (Score:5, Insightful)
Well... Mozilla _has_ concentrated on low RAM usage in the past. The actual memory usage of Gecko is significantly lower than its competitors if you load some pages and measure it.
At this point, they're actually trading off space for performance (e.g. making some core objects slightly bigger to improve certain performance characteristics).
Re: (Score:2)
Well... Mozilla _has_ concentrated on low RAM usage in the past. The actual memory usage of Gecko is significantly lower than its competitors if you load some pages and measure it.
Not that I want to complain but comparing memory usage for Firefox and Opera on my work laptop (running Windows Vista Business 64-bit, C2D) and Firefox and Safari 5 on my home system (fully patched OS X, Core i7) I have to say that Firefox is disappointing when it comes to memory usage. Compared to Opera it's a resource hog on my work laptop, if I start Opera and Firefox at the same time and use Opera over the day with only some minor Firefox usage it will still use 50% more RAM than Opera at the end of the
Re: (Score:2)
Interesting. One question. Is this a Firefox profile with extensions installed? If so, which ones? Whenever I've looked over here (Linux and Mac), recent Firefox has used less memory than recent Opera....
Re: (Score:2)
Well, the memory usage definitely gets worse if I have Web Developer, Firebug, Screengrab and YSlow installed. But even without these add-ons it still feels like memory usage could be lowered, if performance reflected the amount of RAM being used then I wouldn't mind it most of the time but when the browser as a whole still feels sluggish compared to Safari, Opera and Chrome it kind of annoys me.
Re: (Score:2)
Re: (Score:2)
It is mostly a general feeling of sluggishness compared to Safari (on OS X) and high memory use compared to Opera (on Windows). As an example, I had to restart it earlier today after it topped 1 GiB of RAM, since then I've had 3-4 tabs open, all containing internal web service function listings and current memory usage for firefox.exe is supposedly just short of 300 MiB (though this is with Web Developer, Firebug, YSlow, Adblock+ and Screengrab installed). As a comparison, my current Opera session has been
Re:Am I the only one? (Score:5, Informative)
For this particular Slashdot page right now, with both browsers opened fresh for it, Firefox 4.0 beta 6 uses 23 megabytes less resident memory than Chrome 5.0.375.125 does. It also uses about 1800 megabytes less virtual mapped memory, not that that matters nearly as much, but it's a big number in difference.
Epiphany 2.30.2 uses 11 megabytes less residential still, but about as much virtual as Chrome.
Galeon 2.0.7 uses about the same residential memory as Firefox and about twice as much virtual.
Midori 0.2.6 uses 5 megabytes less residential than Firefox, and about 1850 megabytes more virtual.
Arora 0.10.2 uses about twice as much residential memory as Firefox, and about twice as much virtual.
Dillo only needs 11 megabytes to render the page, but that doesn't have JavaScript and only shows a handful of comments without being able to get more.
Fennec 1.0 uses about the same memory footprint as Firefox 4.0 Beta 6, despite being the small-device Mozilla browser.
What is your exact complaint about Firefox's memory use? Are you still experiencing the huge memory leakage and growth from the 2.0 series?
Re: (Score:2)
What is your exact complaint about Firefox's memory use? Are you still experiencing the huge memory leakage and growth from the 2.0 series?
This.
Re: (Score:2)
Re: (Score:2)
Re: (Score:3)
Well, not to be too snarky, but my browsing profile currently matches what you say, with the added bonus of my constantly browsing the comments on Slashdot with Full/All-Comments on all the time. I've been browsing for about 30 minutes, I have 4 tabs open, and the following Addons: Adobe Contribute, ComEd's Real-Time Prices Toolbar, LastPass, FlashGot, Vuze Remote, and AdBlock.
I also have a Gmail session open, with Voice/Video Chat Enabled. My firefox is currently using 220 MB of Real Memory.
If I close Gmai
Re: (Score:2, Insightful)
What I want is low RAM usage
Ok, so you want a lighter, leaner browser...
and more features/plugins.
And at the same time more feature-rich.
Wait, what?
Re: (Score:2, Insightful)
Re: (Score:3)
It's gotta be one of your extensions. I run Firefox non-stop for a week at work and rarely close it during that week. I have noticed no leaks.
This is testing, debugging and running various web pages at work developed by some less than stellar programmers (and some very competent ones too.) I frequent Slashdot and a few other places around the web for reference material as well. I couldn't tell you how many tabs I open and close on a weekly basis.
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
I've got 8Gb, why does Firefox still constantly swap to disk?
If I open a bunch of pages RAM usage says (eg.) "about 500Mb" for firefox.exe but the pagefile-delta column constantly says "I'm thrashing!" and the browser is almost unusable. Wait ten seconds for each mouse click to do something.
Re: (Score:2)
Re: (Score:2)
Kraken (Score:2)
As made popular in the Pirates of the Caribbean films? Fortunately theres plenty of prior art on this legendary sea monster, so they can't sue you.
Re: (Score:2)
Or the original Clash of the Titans years before Johnny Depp was on 21 Jumpstreet? Or in 1870 when Verne mentioned it? Or in 1830 when Tennyson wrote of it?
It's also an actual myth, many of which are feature in the Titan films and in Pirates of the Carribean films, too. I don't think they need to worry about a trademark over an old Norse fishing legend.
The circle is now complete! (Score:5, Insightful)
Google wins in their test [googlecode.com]! (that curiously heavily exploit recursion and other good parts of the V8 engine)
Microsoft wins in their tests [microsoft.com]! (that curiously heavily test only DirectX acceleration)
Re:The circle is now complete! (Score:5, Informative)
You forgot the part about Apple winning in their test (sunspider), and the curious cache of the values of the sin() function in their JS engine that just happens to be the right size for that test.
Fundamentally, browser makers optimize their engine for what they consider important. They also put the things they consider important into benchmarks. The result is somewhat predictable.
Now Peacekeeper is an interesting mention, except I've actually looked at its code. This is a benchmark that measures things like 10,000 calls each of which removes 20 elements from an array that starts with 100,000 elements. It has (failed, interestingly) attempts to browser-sniff and run different code in different browsers. I wouldn't take its numbers to mean much of anything, in general, without some careful study of the exact tests you're looking at. Of course it's also measuring a lot more than just JavaScript; in that sense it's better than most of the benchmarks out there, if you think it manages to correctly measure the things it claims it's measuring.
One other thing, by the way: I fully expect that on Kraken shipping Chrome and Opera are faster than Firefox 3.6.
Re: (Score:2)
One other thing, by the way: I fully expect that on Kraken shipping Chrome and Opera are faster than Firefox 3.6.
Firefox 3.6 is pretty sluggish compared to the latest 4.0 betas, and I beleive that b6 will have a lot more JS optimisation enabled.
So yeah, I'd expect FF 3.6 to be the slowest of the bunch. Excluding IE of course, which is in a whole different league of slow.
Actually, there are probably several empty leagues between FF/Chrome/Safari/Opera and IE
Re: (Score:2)
> So yeah, I'd expect FF 3.6 to be the slowest of the bunch.
Right. Which is consistent with other JS tests, so no mysteries or benchmark-skewing by Mozilla needed there... ;)
> Actually, there are probably several empty leagues between FF/Chrome/Safari/Opera and IE
That may or may not be the case with the IE9 betas, depending on which benchmarks you use. Though there's some weirdness; see http://blog.mozilla.com/rob-sayre/2010/09/09/js-benchmarks-closing-in/ [mozilla.com] the paragraph starting "One last issue that
Re: (Score:3, Informative)
Actually FF4 nightlies beat IE9 in many of the IE9 testdrive tests. FF4 has the same Direct2D, Direct Write, and DirectX 9 hardware acceleration that IE9 does but FF4's javascript engine is better which gives it better FPS in those tests. FF4's javascript engine is a lot faster and there's still lots of room for improvement. FF4 Beta 7 will have the new javascript engine but it's already been merged to the nightly trunk branch so you can try it now if you want. Firefox had hardware acceleration first (in ni
Unfair method? (Score:3, Insightful)
Biased? (Score:3, Insightful)
We believe that the benchmarks used in Kraken are better in terms of reflecting realistic workloads
Or just better in terms of reflecting where their product is strongest?
Isn't it just a little bit suspicious when the browser people release a benchmark that scores their own browser as the fastest? Intel's benchmark in 2002 [pcworld.com] was known to have emphasized performance traits specific to Intel chips.
One thing I don't understand. (Score:2, Insightful)
Who cares (Score:5, Insightful)
My browser's performance has always been "good enough". Can we talk about ergonomy, reliability, compatibility, please ?
7892.1ms with minefield 4b6pre (Score:2)
compare http://bit.ly/bFQ17C [bit.ly]
Mozilla releases a Windows botnet (Score:3, Interesting)
Real world performance (Score:2)
Meanwhile, when I actually use my web browser in the real world, Chrome seems much faster than Firefox. So what do I need any kind of synthetic benchmark to tell me what I already know? I'll find out who's faster when they release a non-beta. If performance matters that much to my experience, I may switch, but whoever I use better support the types of plug-ins that I want. Chrome is just about there. It's nice to see rapid innovation and competition in the browser again.
Great! (Score:2)
Tried in in firefox (Score:2, Informative)
Re: (Score:2)
Here are my results.
Latest Mozilla Central nightly on Mac OS X 32 bit: 9339.5ms [mozilla.com]
Latest Webkit nightly. 64 bit: 15736.5ms [mozilla.com]
I'll run a Mozilla Central 64 bit in a moment.
Re: (Score:2)
Latest Mozilla Central 64 bit: 9409.6ms [mozilla.com]
Latest Chromium 32 bit 18766.2ms [mozilla.com]
Re: (Score:2)
Latest Opera Snapshot: 15603.7ms [mozilla.com]
Re: (Score:2)
FF 3.6.9 on MacBook Pro '08: 24930.0ms +/- 0.4% [mozilla.com]
(not testing under ideal conditions - other tabs open, other programs running, etc, in an attempt to reflect "real conditions")
Re: (Score:2)
28.6s is less accurate than 28638.1ms.
Re: (Score:2)
28.6s is less accurate than 28638.1ms.
28.6s is less precise than 28638.1ms but they could be equally accurate depending on the test conditions.
Re: (Score:2)
Re: (Score:2)
Not an SI unit.
Re: (Score:2)
Re: (Score:2)
Apologies, a deci is an SI unit (why did I post that?), but I have never heard anyone use it. You never, ever measure a deci-metre or deci-gramme.
dB is 100% not a SI unit.
Re: (Score:2)
Actually, you do hear about decimeters... 1/10th of a meter. Coincidentally, 1 cubic decimeter is exactly 1 litre.
Re: (Score:2)
Are you comparing your 11180.7 number to the numbers in the article submission to conclude that "Chrome is still king"? Or did you run other browsers on your hardware and OS? The hardware and OS _do_ make a difference, you know... especially that hardware bit.
Re: (Score:2)
Are you seriously trying to compare a single test on your system to a test run on completely different hardware? You can't say anything about Opera 10.70 vs FF nightly with your single result.
Re: (Score:2)
Took wel over 40 seconds on my 3.6.9 Linux Firefox, freezing my other tabs most of the time. But it did finish.