Theora Ahead of H.264 In Objective PSNR Quality 313
bigmammoth writes "Xiph hackers have been hard at work improving the Theora codec over the past year, with the latest versions gaining on and passing h.264 in objective PSNR quality measurements. From the update: 'Amusingly, it also shows test versions of Thusnelda pulling ahead of h.264 in terms of objective quality as bitrate increases. It's important to note that PSNR is an objective measure that does not exactly represent perceived quality, and PSNR measurements have always been especially kind to Theora. This is also data from a single clip. That said, it's clear that the gap in the fundamental infrastructure has closed substantially before the task of detailed subjective tuning has begun in earnest.'
Momentum is building with a major Open Video Conference in June, the impending launch of Firefox 3.5 and excitement about wider adoption in a top-4 web site. It's looking like free video codecs may pose a serious threat to the h.264 bait-and-switch plan to start charging millions for internet streaming of h.264 in 2010."
What? (Score:5, Informative)
H.264 is a specification, not a codec.
There are various codec implementations of it.
x.264 being the most popular.
Main Concept being the best overall.
Nero being one of the first to market and as usual being slow and bloated and buggy.
DivX as usual being late to market but driving the push for playback in embedded devices, while being at the top in terms of quality and decoding speed.
Re:bullcrap (Score:5, Informative)
What software agreement? I think that they are licensing patents. They have merely said that you don't have to pay to use the patents before 2010, but if you use the patents after that, you may need to pay (depending on volume). Yes, products that have shipped will be safe, but most companies want to continue shipping products, which will be affected by the royalty demands.
Re:Problems..... (Score:1, Informative)
"Unfortunately I can't really play any of those formats save for on my computer"
Wrong [sourceforge.net]
Oh, and don't forget to rockbox [rockbox.org] your iPod.
Re:bullcrap (Score:3, Informative)
"...even if there was a clause in there stating "we can change this at anytime"..."
That's *exactly* what's in the MPEG licenses. And software vendors don't get indefinite licenses for distributing MPEG implementations, they have to reup on a regular basis.
Repost of TFA in case of Slashdot/Streisand affect (Score:2, Informative)
Since the last update and alpha release, work has centered on two basic tasks: correcting the substantial energy leakage in Theora's forward DCT and optimization of the quantization matricies (and matrix selection). Here's an early example of Thusnelda with some early quant matrix tuning, along with the new forward DCT versus Theora 1.0 discussed below (same encoder parameters, equal bitrates):
Greg Maxwell has been doing automated regression and comparison testing of the ongoing Thusnelda work against previous versions of Theora, and because there's so much anecdotal FUD flying around about Thusnelda and (especially) h264, he threw h264 (the x264 encoder) into the testing mix too. The following PSNR chart is data collected against the 'Akiyo' QCIF test clip:
X axis is kbps, Y axis is PSNR in dB
The important thing to note is that objective error steadily decreases from Theora, to the SVN version of Thusnelda, to the early experimental Thusnelda work that includes some matrix optimization (but not yet adaptive quantization). Also worth noting is that something is very very wrong with Theora support in older versions of ffmpeg, which for some reason, outside reviewers insist on using to compare Theora against other codecs. The bug is not actually in ffmpeg2theora; the same ffmpeg2theora version linked against a recent ffmpeg does not exhibit the same problem.
Amusingly, it also shows test versions of Thusnelda pulling *ahead* of x264 in terms of objective quality as bitrate increases. It's important to note that PSNR is an objective measure that does not exactly represent perceived quality, and PSNR measurements have always been especially kind to Theora. This is also data from a single clip. That said, it's clear that the gap in the fundamental infrastructure has closed substantianlly before the task of detailed subjective tuning has begun in earnest.
Forward DCT
The original VP3 was designed with a forward/inverse DCT pair without perfect reconstruction that exhibits substantial and highly nonuniform energy leakage. It appears that the only real consideration in the design and implementation of the original transform pair was speed on a single platform [a classic case of premature optimization].
Original transform error
The peak and mean square error charts (values arranged by position in the 8x8 output matrix) make clear just how poor the original forward DCT is. (This is an excerpt from the full test and is representative of the results across all input conditions)):
IEEE1180-1990 test results (VP3):
Input range: [-256,255]
Sign: 1
Iterations: 10000
Peak absolute values of errors:
3 3 2 2 2 2 2 2
2 2 2 2 2 2 2 2
2 3 2 2 2 2 2 2
2 2 2 2 2 2 2 2
2 2 2 2 2 2 2 2
2 2 2 2 2 2 2 2
2 3 2 2 2 2 2 2
2 2 2 2 2 2 2 2
Worst peak error = 3 (FAILS spec limit 1)
Mean square errors:
2.1289 0.9616 0.6611 0.3385 0.3458 0.6426 0.5268 0.3499
0.4746 0.6312 0.6130 0.4239 0.4310 0.6287 0.6312 0.4315
0.4706 0.6238 0.6300 0.4228 0.4159 0.6278 0.6357 0.4191
0.3642 0.5461 0.5286 0.3527 0.3467 0.5368 0.5413 0.3405
0.3483 0.5285 0.5463 0.3531 0.3499 0.5389 0.5294 0.3421
0.4331 0.6090 0.6244 0.4272 0.4218 0.6296 0.6172 0.4209
0.4164 0.6225 0.6191 0.4248 0.4285 0.6206 0.6331 0.4269
0.3419 0.5315 0.5428 0.3586 0.3560 0.5299 0.5390 0.3482
Worst pmse = 2.128900 (FAILS spec limit 0.06)
Overall mse = 0.523162 (FAILS spec limit 0.02)
Improved transform
A
Re:bullcrap (Score:3, Informative)
How is that any different than a company selling a physical product deeply discounted or below cost for an initial period of time in order to gain market share?
Correct! (Score:4, Informative)
And following Thusnelda will be Snuppflog. They're just internal project names.
Intel chooses boring internal codenames like towns, we choose silly things that our incredulous detractors dare us to use. But only if we like them.
Re:What? (Score:5, Informative)
Main Concept being the best overall.
Oh? this [doom9.org] (and this [doom9.org] follow up post) seem to indicate that it's not so clearcut. Looks like x264 beat MainConcept in most tests, and the major tests it lost in were rather unrealistic.
But in the interest of full disclosure, Dark Shikari is one of the main developers on x264, so he's got an obvious bias. Doesn't necessarily make him wrong though.
No, confuzzled hacker (Score:5, Informative)
You have to measure the PSNR of each codec with the same tool, silly (and avoid doing colorspace conversions which are lossy in the interchange. Keep the output in YCb'Cr' format). If you're using the x264 encoder's reported PSNR *cough*ahem* it's known to be wrong. It always reports way higher than other tools, like it's forgetting chroma is subsampled or its log-space algebra is just wrong or something.
Let me check myself with the clip linked in the article....mmmm lessee.... yep! that's what you're doing. So, BZZZT, no gold star, try again.
Royalties payable to Adobe (Score:3, Informative)
The host can use any codec they want and it is transparent to the client. By doing this, the client never notices, and they don't pay royalties.
The manufacturer of the playback device (if not a PC) pays royalties to Adobe for Flash Player and passes these on to the client.
It's more likely than you think.
What is "centipedes in my vagina"? Oh wait, this isn't Jeopardy!.
Dumbest, most meaningless post story ever. (Score:2, Informative)
http://mirror05.x264.nl/Dark/x264vsElecard/ [x264.nl]
Of worth noting is that in these screenshots, Elecard has a higher PSNR than x264.
Re:Use your peepers. (Score:3, Informative)
Yeaaah. First of all, let me just say that I'm not claiming Theora is better than H.264, or even on quite equal footing (as gmaxwell said, that isn't really even the point). So there, that's out of the way.
In any case, your suggestion to eyeball these comparisons that are just insanely old considering the improvements Theora has gone through is pretty clueless, more so with you even admitting to their datedness. (Sure, x264 has improved as well, but Theora has had the *cough* benefit of rather much more low-hanging fruit due to the not very high quality of the original encoder inherited from VP3.)
So I'm gonna ask _you_ to use your eyeballs and follow the link in this very article, 'cause there are before/after shots there of old and new Theora encoder output. Then come back saying that these ancient comparisons are representative of the performance of the current code. That is, after all, what this article is about.
Re:No, confuzzled hacker (Score:4, Informative)
Well, one tester (and Greg's graph was generated to rebut his graph) was converting each output frame to PNG and then feeding them into one of a number of PSNR tools one by one to get a PSNR result. The conversion from YCb'Cr' to RGB is lossy, but apparently this particular author didn't know that.
He was also using multiple PSNR tools because some were mysteriously not working with some video inputs. Given that there's no one standardized way to calculate PSNR, that led to additional data lulz.
And for x264, he apparently didn't generate his own numbers, he just used someone else's published numbers.
Anyhow, He reported that x264 was 30-ishdB (!) better than Theora. Wha? If every Theora frame was black, that still wouldn't account for that much difference. YUV12 is only 40-45dB deep!
In other words, the whole point of the graph was originally to illustrate and rebut these errors, and it turned out to be a nice regression test too.
Also, for the record, the x264 curve is not perfectly smooth, but that it's as smooth as it is attests to the fact that it's a nicely tuned codec. That Theora is lumpier is one indication it still has more tuning to be done.
Also, Greg's response below is way more levelheaded than mine. He actually collected the data himself (so has more detailed, accurate and first-hand knowledge) and he also probably hasn't been drinking whiskey all night.
Re:I contradict myself. (Score:1, Informative)
The example frame grab is from the new star trek promo, as a slashdot user you should recognize it. It's a very high action clip.
The particular frame was selected by a a reddit user advocating H.264 (vs old theora) in a thread a couple of weeks back.
The PSNR test is against Akiyo which is a boring talking head clip, but a standard in codec testing.
I'm not sure where your claim of contradiction comes from: The submitter to Slashdot exaggerated, not the Theora developers. The point here is that Theora is improving and that it's at least in the same general ballpark for some clips.
Re:Why the idiotic naming again? (Score:5, Informative)
"Ogg" is actually a term from an early internet game.
http://en.wikipedia.org/wiki/Ogging
Theora is named after Theora Jones, a secondary heroine character from the movie 'Max Headroom: 20 Minutes into the Future' about a dystopian future where video media is overwhelming, centralized, oppressive, dangerous, and an off switch on a television is illegal:
http://en.wikipedia.org/wiki/Theora_Jones#Theora_Jones
"Xiph" is actually from the Greek ξÎÏÎÏ (sword) by way of 'Xiphophorus' (sword-bearing, pseudolatin?) from the genus name of a fish (Xiphophorus helleri). Which is where I picked it up in middle school. I'd been using it for my software projects since I was 14 or so and by the time Xiph.Org was a real thing [many many years later] I wanted to change the name to something less silly and my co-founders voted me down. They liked Xiph. It became the precedent-setting silly name.
Vorbis is from Terry Pratchett's _Small Gods_ and I dearly hope Mr. Pratchett considers it a compliment. It was meant as tribute to my favorite fictional villain, Archdeacon Vorbis. "A mind like a steel marble"
Re:Performance matters BIG TIME (Score:2, Informative)
The catch is that industry involvement alone gives H.264 an edge. IC companies are putting effort into making low-power chips that can decode H.264 in hardware, in realtime. So the CPU-efficiency of Vorbis goes out the window, and everyone sticks with the existing standard.
Re:You know (Score:2, Informative)
The point is not the surround support in MP3, the point is when you're looking to upgrade from MP3, a few years down the line. At which point your obvious options are AC3 or later AAC (lots of surround support, along with hardware decoders), WMA (took a while, but good surround support now), maybe RealAudio (who cares, not I), or ogg (rather poor surround support). At which point you're going to go with AAC.
Reading comprehension, it's useful.
Re:You know (Score:3, Informative)
PSNR metrics were calculated wrong - x264 Theora (Score:5, Informative)
Turns out there was an error in the methadology used in the original comparison, which hit x264 for more than 4 dB of difference.
http://www.reddit.com/r/programming/comments/8iphn/theora_encoder_improvments_comparable_to_h264/c09eyvc [reddit.com]
Edit: HAHAHA! We figured out what was wrong--thanks a ton, gmaxwell, for coming on IRC and resolving this! Turns out his testing methodology was flawed... but not in the way I thought!
Turns he out he did everything correctly... but he used ffmpeg for outputting the raw y4m file to have its quality measured by dump_psnr (but not for theora). Apparently, ffmpeg flags the output chroma as "420mpeg2" instead of "420", which results in over 4db of PSNR being slashed off of x264's results unfairly.
Oops. We already have a patch submitted to ffmpeg for the problem and a retraction of the Theora comparison results is in the works. Thanks to gmaxwell for taking the initiative and David Conrad (Yuvi) for finding the bug!
The Doom9 thread on the same topic:
http://forum.doom9.org/showthread.php?t=146893 [doom9.org]
Anyway, given H.264 is a more recent codec that is highly optimized for PSNR and has had many years of refinement in a number of implementations, it's hard to conceive of how Theora could even approach it in compression efficiency, let alone beat it.
Well... (Score:2, Informative)
You certainly can't link against it without being GPL/MIT yourself but I doubt that stops you using calling a program that does the encoding of a file with x264. As an example LAME is GPL and gets used all over the place. The ffmpeg stuff is also quite popular.
I guess "popular" would need to be defined. Are we counting programs or are we counting videos produced with a particular encoder? I'd guess that whatever Adobe ships would be the most popular for the later and some open source thing if you were counting the former...