Slashdot Log In
Theora Ahead of H.264 In Objective PSNR Quality
Posted by
timothy
on Thu May 07, 2009 06:55 PM
from the why-ayn-never dept.
from the why-ayn-never dept.
bigmammoth writes "Xiph hackers have been hard at work improving the Theora codec over the past year, with the latest versions gaining on and passing h.264 in objective PSNR quality measurements. From the update: 'Amusingly, it also shows test versions of Thusnelda pulling ahead of h.264 in terms of objective quality as bitrate increases. It's important to note that PSNR is an objective measure that does not exactly represent perceived quality, and PSNR measurements have always been especially kind to Theora. This is also data from a single clip. That said, it's clear that the gap in the fundamental infrastructure has closed substantially before the task of detailed subjective tuning has begun in earnest.'
Momentum is building with a major Open Video Conference in June, the impending launch of Firefox 3.5 and excitement about wider adoption in a top-4 web site. It's looking like free video codecs may pose a serious threat to the h.264 bait-and-switch plan to start charging millions for internet streaming of h.264 in 2010."
Related Stories
[+]
Technology: YouTube, HTML5, and Comparing H.264 With Theora 361 comments
David Gerard writes "Google Chrome includes Ogg support for the <video> element. It also includes support for the hideously encumbered H.264 format. Nice as an extra, but ... they're also testing HTML5 YouTube only for H.264 — meaning the largest video provider on the Net will make H.264 the primary codec and relegate the equally good open format Ogg/Theora firmly to the sidelines. Mike Shaver from Mozilla has fairly unambiguously asked Chris DiBona from Google what the heck Google thinks it's doing."
DiBona responded with concerns that switching to Theora while maintaining quality would take up an incredible amount of bandwidth for a site like YouTube, though he made clear his support for the continued improvement of the project. Greg Maxwell jumped into the debate by comparing the quality of Ogg/Theora+Vorbis with the current YouTube implementations using H.263+MP3 and H.264+AAC. At the lower bitrate, Theora seems to have the clear edge, while the higher bitrate may slightly favor H.264. He concludes that YouTube's adoption of "an open unencumbered format in addition to or instead of their current offerings would not cause problems on the basis of quality or bitrate."
[+]
Technology: Concrete Comparisons of Theora Vs. Mpeg-4 325 comments
icknay writes "With the upcoming Firefox 3.5 and HTML5 video, there's natural interest in Theora vs. Mpeg-4, but without much evidence either way. Here's clips encoded at various rates to provide concrete comparison between Theora and Mpeg-4. Theora performs decently, but requires more bandwidth than Mpeg-4 (although this is a 1.1alpha release of Theora and Theora has a much better license than Mpeg-4). The quality comparisons are very subjective, but you can try the clips yourself and see how it breaks down. There was an earlier discussion about this, but it lacked much concrete evidence. (Disclosure: it's my page.)"
[+]
Technology: Google Acquiring VP3 Developer On2 Technologies 133 comments
R.Mo_Robert writes "BetaNews is reporting that Google is acquiring On2, the video codec company and original developers of the VP3 codec from which Theora is derived. The article suggests that this may mean Google is backing Ogg Theora as the HTML5 video standard, but this is likely not the case--with Theora already being open-source and On2 having disclaimed all rights and patents, there is no reason Google should have needed to do this to push Theora. You may recall from some time back that HTML5 no longer specifies which video codec(s) a browser should support due to there being, unfortunately, no suitable codec at this time. But Google (known for supporting H.264) practically owns Web video with YouTube in most people's minds, so their influence could really swing the future of HTML5 video either way. It remains to be seen whether Google's acquisition of On2 has any bearing on their plans for video on the Web."
This discussion has been archived.
No new comments can be posted.
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
Full
Abbreviated
Hidden
Loading... please wait.
Free codecs are not a major threat (Score:3, Insightful)
Unless some major device manufacturers or youtube like heavyweights get behind it, it's gonna be pretty much limited to the geek community.
Re:Free codecs are not a major threat (Score:5, Funny)
This might not pose that much of a threat to H264, sounds like another OGG or FLAC.
Theora sounds like another OGG, huh? Imagine that.
Parent
Re:Free codecs are not a major threat (Score:4, Insightful)
Parent
Still doesn't mean much (Score:3, Insightful)
First off, most people don't care about lossless compression. It's a niche market. After all, even on extremely good sound gear, you are hard pressed to pick out 256k MP3 from uncompressed in blind tests. Also, popular though it might be, it wasn't popular enough for the big boys to pick up. Both Apple and Microsoft did their own lossless formats. Windows Media Audio has a lossless mode, and Apple uses ALAC. Now while Windows Media Player will happily play FLAC if you install a DirectShow codec (don't know
Re: (Score:3, Interesting)
First off, most people don't care about lossless compression. It's a niche market. After all, even on extremely good sound gear, you are hard pressed to pick out 256k MP3 from uncompressed in blind tests.
And this matters because...? high definition video is also a niche market, as Blu-Ray vs DVD sales analysis would show. Yet obviously we're talking about popularity within its scope, otherwise not even the iPod would be popular, if we were to consider the entire human race.
Also, popular though it might be, it wasn't popular enough for the big boys to pick up. Both Apple and Microsoft did their own lossless formats.
Remember WMA? and AAC? no, the "big boys" ignored FLAC not because it wasn't popular enough, it was because both have *very* strong NIH sentiments against it, as they did with MP3.
Re:Still doesn't mean much (Score:4, Funny)
You know what else is a niche market, PCs. Most people in the world don't own one, so it is a niche market. And you know what isn't a niche market, stupid fucking wankers talking shit.
Parent
RIAA statistical methods.... (Score:5, Funny)
Talk to average users, and ask them "what is a flac file?", and "what is a wav file?", then ask them "which one would you use to record audio?". 99.999% would say "wav".
Actually, that percentage of your 'average users' would just *blink* with glazed over eyes...and not have a clue what you are talking about.
I say this after having worked tech support for Creative Labs, dealing with mp3 players and your 'average users.
Now I will agree that more 'average users' will recognise a *.wav file as a sound file compared to recognising a *.flac file as a sound file...if we leave 'lossless' and other qualifiers out of the equation.
But 99.999%????...'average users'???
Hah! I would not touch that statistic with a bleach-soaked 10 foot pole, because I know where you pulled it from, and it's drawing flies already, because it stinks so bad!
Parent
Re: (Score:3, Insightful)
Except that Theora is pretty much inferior in all qualities except being free.
"But I'm Richard Stallman." "Oh, Okay then" (Score:5, Funny)
To be fair, the whole thing is part of 'the Ogg Project'. Saying 'Theora is Ogg' is not actually incorrect, and it might get you laid at parties.
No need to be such a stickler, here have a beer.
Parent
Problems..... (Score:3, Interesting)
Re:Problems..... (Score:4, Interesting)
Parent
Re:Problems..... (Score:5, Interesting)
Parent
Re: (Score:3, Interesting)
Re: (Score:3, Interesting)
As much as I hate Apple's domination of the hardware music player market, that's the reality.
I agree 100%. It is to the point now that for mp3 players, everyone is trying to 'play catch-up' with Apple, for better or worse. It's just the fact of reality for now.
It could change, and probably will.
I however will not even try to attempt to predict what the changes may be...there are too many variables in too many related/significant areas IMHO.
It would be easy at this stage to expect some undefined hardware/software combo to take off in directions most people could not predict accurately: cell/smart ph
What? (Score:5, Informative)
H.264 is a specification, not a codec.
There are various codec implementations of it.
x.264 being the most popular.
Main Concept being the best overall.
Nero being one of the first to market and as usual being slow and bloated and buggy.
DivX as usual being late to market but driving the push for playback in embedded devices, while being at the top in terms of quality and decoding speed.
Re:What? (Score:5, Informative)
Main Concept being the best overall.
Oh? this [doom9.org] (and this [doom9.org] follow up post) seem to indicate that it's not so clearcut. Looks like x264 beat MainConcept in most tests, and the major tests it lost in were rather unrealistic.
But in the interest of full disclosure, Dark Shikari is one of the main developers on x264, so he's got an obvious bias. Doesn't necessarily make him wrong though.
Parent
Re: (Score:3, Insightful)
I'm sorry, what? (Score:5, Funny)
Please tell me that's not an actual product name.
Correct! (Score:4, Informative)
And following Thusnelda will be Snuppflog. They're just internal project names.
Intel chooses boring internal codenames like towns, we choose silly things that our incredulous detractors dare us to use. But only if we like them.
Parent
What are the settings? (Score:5, Interesting)
I don't mean to belittle Theora, I've really been rooting for them over the years. And this recent test does look fantastic.
But I can't help wonder what settings they are testing x264 with. x264 has recently been shown [doom9.org] to be highly sensitive to clips like the Akiyo one tested here -- it also lost to some other H.264 encoders that it usually beats fairly consistently. The version and settings used to encode this one make a WORLD of difference.
You know (Score:5, Insightful)
I'm less worried about benchmarks, more worried about, you know, seeing an actual production, ready for end-user codec released. This only finally happened end of 2008 to all of no fanfare (I didn't see it on /. or anywhere). That is a loooong time they've been messing with it (2001 was when VP3 with open).
The problem is, if you take forever to make it "perfect" you miss the boat. The reason MP3 got so popular is not because it was the first compressed music standard capable of near CD quality. It was also not because it is the best lossy compression standard. It is because it was good enough, at the right time. It's compression level was small enough that people found it usable (as opposed to things like ADPCM which do knock the size down, but not enough) on the technology of the day, and it did it while giving quality good enough people liked it.
So in my opinion it really is to late, they needed to release a couple years ago. As it stands, I think they've missed the boat. Blu-ray is done and uses VC-1, MPEG-4, and MPEG-2, ATSC is done, uses MPEG-2, Flash Video uses H.263 and VP6 (and also H.264), mobile stuff uses MPEG-4 (part 2 and 10). They have just missed the boat. So they release a codec in a year or two or five that's maybe a little better than MPEG-4 part 10... Ok so what? Nobody will really care. Net connections only get faster, harddrives get larger, so even if you offer 20% better compression it doesn't matter, people will stick with the standard.
Vorbis had more of a chance since it actually did get released around the time that there was interest in upgrading from MP3 to something better for some things. However they largely lost out (it does have some use, in game engines for example) in part because of their silly naming and in part because of their poor surround support. However Theora is too little too late as far as I can tell. The world is already settling in to their HD codecs and once the standards get entrenched, they'll stay there until there's a compelling reason to switch.
Timing is important. If your product isn't ready when it is needed, it isn't going to get used no matter how awesome it is in the end.
Re: (Score:3, Interesting)
It is because it was good enough, at the right time.
This is such an awesome, succinct way of explaining the sometimes-inexplicable success of so many things. I will be sure to use it again!
I actually don't think it's too late for Theora to have an impact though.
The big thing is the tag that is being considered for HTML5. If Firefox and Opera and Chrome all bust out solid support in release builds soon, we'll be converting our video library to support it (catalogue of video game trailers on ausgamers.com - I realise one site isn't enough, but if others feel
Re: (Score:3, Informative)
I contradict myself. (Score:5, Insightful)
The benchmark that looks good in the lab.
YMMV.
The "objective" benchmark that has been "especially kind to Theora."
What the hell am I to make of that?
It's one clip -
apparently of a geek dead on his feet after pulling one too many all-nighters.
You can drown in techno-babble.
I want to see video.
Richly detailed backgrounds.
Textures. Wood and fur and cloth and grass. Subtle rendering of flesh tones.
Give me a real taste of how well your codec handles action. Take your camera outdoors. In the rain. Out on a boat. Take it on stage.
Re: (Score:3, Insightful)
Richly detailed backgrounds.
Textures. Wood and fur and cloth and grass. Subtle rendering of flesh tones.
Give me a real taste of how well your codec handles action. Take your camera outdoors. In the rain. Out on a boat. Take it on stage.
Show me *any* geek who has ever done that. Or even seen any of that. We have only one action. With one skin tone. And you don't wanna see any part of it. Believe me. ^^
Performance matters BIG TIME (Score:3, Interesting)
Image quality vs bitrate means very little without mentioning CPU/memory usage. H.264's greatest weakness is the heavy CPU load on playback, it's just not friendly to low-cost and/or mobile devices. If Theora can get within the ballpark in terms of quality, but beat H.264 in speed, that could be the edge it needs to hit the mainstream.
Right now it's little more than an academic experiment. Floating point everything can give you fantastic quality, but it will crawl so slowly that people will choose a lesser-quality alternative that runs faster.
PSNR metrics were calculated wrong - x264 Theora (Score:5, Informative)
Turns out there was an error in the methadology used in the original comparison, which hit x264 for more than 4 dB of difference.
http://www.reddit.com/r/programming/comments/8iphn/theora_encoder_improvments_comparable_to_h264/c09eyvc [reddit.com]
Edit: HAHAHA! We figured out what was wrong--thanks a ton, gmaxwell, for coming on IRC and resolving this! Turns out his testing methodology was flawed... but not in the way I thought!
Turns he out he did everything correctly... but he used ffmpeg for outputting the raw y4m file to have its quality measured by dump_psnr (but not for theora). Apparently, ffmpeg flags the output chroma as "420mpeg2" instead of "420", which results in over 4db of PSNR being slashed off of x264's results unfairly.
Oops. We already have a patch submitted to ffmpeg for the problem and a retraction of the Theora comparison results is in the works. Thanks to gmaxwell for taking the initiative and David Conrad (Yuvi) for finding the bug!
The Doom9 thread on the same topic:
http://forum.doom9.org/showthread.php?t=146893 [doom9.org]
Anyway, given H.264 is a more recent codec that is highly optimized for PSNR and has had many years of refinement in a number of implementations, it's hard to conceive of how Theora could even approach it in compression efficiency, let alone beat it.
Re:bullcrap (Score:5, Insightful)
Parent
Re: (Score:3, Informative)
How is that any different than a company selling a physical product deeply discounted or below cost for an initial period of time in order to gain market share?
For one, it's usually illegal (Score:5, Insightful)
How is that any different than a company selling a physical product deeply discounted or below cost for an initial period of time in order to gain market share?
That practice is called 'dumping' and is illegal for most goods and services, at least in the United States.
Parent
Re: (Score:3, Interesting)
Then why haven't video game companies been hammered? Just about all video game consoles for the last decade or so have been sold at a loss for market share.
Re:For one, it's usually illegal (Score:4, Insightful)
Nah more like if they sold PS3's for $50 until their competitors withdrew from the market then jacked the price to $1500, but dumping isn't that big of an issue with luxury goods like game consoles anyway, because people will just stop buying them if the price is too high.
The robber barons were famous for doing that kind of thing to crush anyone who didn't bend to their will, but with more important goods (steel, oil). And now for some reason Carnegie and Rockefeller are names most people respect. So much for karma eh?
Parent
Re:bullcrap (Score:5, Informative)
What software agreement? I think that they are licensing patents. They have merely said that you don't have to pay to use the patents before 2010, but if you use the patents after that, you may need to pay (depending on volume). Yes, products that have shipped will be safe, but most companies want to continue shipping products, which will be affected by the royalty demands.
Parent
Re: (Score:3, Informative)
"...even if there was a clause in there stating "we can change this at anytime"..."
That's *exactly* what's in the MPEG licenses. And software vendors don't get indefinite licenses for distributing MPEG implementations, they have to reup on a regular basis.
Re:bullcrap (Score:4, Insightful)
Software patents should all be invalid.
There are numerous and completely independent ways for people to construct software that does the same thing. Software and data compatibility is far more important that limiting what programmers can write independently without also being required to research whether or not their work is already covered under a patent somewhere.
And to be clear, what software patents do most often is PREVENT people from being paid for their original work or at the very least allow some otherwise uninvolved party to come in and tax your ability to market your work if not block it entirely.
Software protected by copyright? I'm not entirely down with that but it makes a lot more sense than patenting software.
Parent
Re: (Score:3, Insightful)
do you really hate paying people for their work that much?
It depends how much...
Im happy to pay people how much i think their work is worth to me, but only a victim would pay what a capitalist says their work is worth.
Re: (Score:3, Insightful)
No one person can be objective, a fair value can only be reached when their is competition.
Patents and copyrights are government granted monopolies, therefore anti-competition, and thus will always be unfairly priced.
Re: (Score:3, Interesting)
if your laying out cash on infrastructure i'd say it serves you right for not doing your homework first.
Re: (Score:3, Insightful)
once it more widely adopted and all your infrastructure is organized around using it you have to start paying in 2010. Which is not exactly heavily publicized.
How does being in the license agreement itself count as "not heavily publicized?" C'mon, people... anyone who signs a legal agreement like a patent license without having a lawyer look over it is a moron.
It's not bait and switch if they tell you about the switch up front.
Re: (Score:3, Interesting)
Are you forgetting the previous bait and switches (Score:5, Insightful)
No royalties were levied on mp3 implementations until MPEG changed their minds in 1998, ironically not long after the format really took off, and delivered Cease-and-Desists to every free encoder project and a bunch of companies too.
"Thanks, boys, for promoting our format for us. We thought it was only good for hold music over ISDN! Since you did such a fabulous job, we're gonna have to ask you to hand everything over right fucking now or we sue you into oblivion. Don't let the door hit your ass on the way out."
Don't you remember that was the whole reason Ogg and Vorbis got started? We just had Unisys/GIF threaten to sue everyone, then we had MPEG threatening to sue everyone and someone finally had the guts to say no fucking more. MPEG can't even keep its own members from suing each other, and you plan to trust them for the basis of your own smaller business?
But one thing is funny, MPEG has mostly (mostly) behaved since then. Maybe MPEG is only playing fair now *because* of Ogg? Ogg is pretty much the only viable non-MPEG codec effort left.
Parent
modern PC hardware can handle Dirac (Score:4, Insightful)
It's free, from the BBC. It's never blocky because it uses wavelets.
Parent
Re: (Score:3, Funny)
Re:Fighting the money machine never works!! (Score:4, Funny)
Nope, and as a developer for Internet Explorer, I thought I'd heard of every image format already!
Parent
Re: (Score:3, Interesting)
Yes, I was being sarcastic. I was actually showing that free solutions CAN catch on and dominate.
No, confuzzled hacker (Score:5, Informative)
You have to measure the PSNR of each codec with the same tool, silly (and avoid doing colorspace conversions which are lossy in the interchange. Keep the output in YCb'Cr' format). If you're using the x264 encoder's reported PSNR *cough*ahem* it's known to be wrong. It always reports way higher than other tools, like it's forgetting chroma is subsampled or its log-space algebra is just wrong or something.
Let me check myself with the clip linked in the article....mmmm lessee.... yep! that's what you're doing. So, BZZZT, no gold star, try again.
Parent
Re:No, confuzzled hacker (Score:4, Informative)
Well, one tester (and Greg's graph was generated to rebut his graph) was converting each output frame to PNG and then feeding them into one of a number of PSNR tools one by one to get a PSNR result. The conversion from YCb'Cr' to RGB is lossy, but apparently this particular author didn't know that.
He was also using multiple PSNR tools because some were mysteriously not working with some video inputs. Given that there's no one standardized way to calculate PSNR, that led to additional data lulz.
And for x264, he apparently didn't generate his own numbers, he just used someone else's published numbers.
Anyhow, He reported that x264 was 30-ishdB (!) better than Theora. Wha? If every Theora frame was black, that still wouldn't account for that much difference. YUV12 is only 40-45dB deep!
In other words, the whole point of the graph was originally to illustrate and rebut these errors, and it turned out to be a nice regression test too.
Also, for the record, the x264 curve is not perfectly smooth, but that it's as smooth as it is attests to the fact that it's a nicely tuned codec. That Theora is lumpier is one indication it still has more tuning to be done.
Also, Greg's response below is way more levelheaded than mine. He actually collected the data himself (so has more detailed, accurate and first-hand knowledge) and he also probably hasn't been drinking whiskey all night.
Parent
Royalties payable to Adobe (Score:3, Informative)
The host can use any codec they want and it is transparent to the client. By doing this, the client never notices, and they don't pay royalties.
The manufacturer of the playback device (if not a PC) pays royalties to Adobe for Flash Player and passes these on to the client.
It's more likely than you think.
What is "centipedes in my vagina"? Oh wait, this isn't Jeopardy!.
Re: (Score:3, Informative)
Yeaaah. First of all, let me just say that I'm not claiming Theora is better than H.264, or even on quite equal footing (as gmaxwell said, that isn't really even the point). So there, that's out of the way.
In any case, your suggestion to eyeball these comparisons that are just insanely old considering the improvements Theora has gone through is pretty clueless, more so with you even admitting to their datedness. (Sure, x264 has improved as well, but Theora has had the *cough* benefit of rather much more low
Re:Why the idiotic naming again? (Score:5, Informative)
"Ogg" is actually a term from an early internet game.
http://en.wikipedia.org/wiki/Ogging
Theora is named after Theora Jones, a secondary heroine character from the movie 'Max Headroom: 20 Minutes into the Future' about a dystopian future where video media is overwhelming, centralized, oppressive, dangerous, and an off switch on a television is illegal:
http://en.wikipedia.org/wiki/Theora_Jones#Theora_Jones
"Xiph" is actually from the Greek ξÎÏÎÏ (sword) by way of 'Xiphophorus' (sword-bearing, pseudolatin?) from the genus name of a fish (Xiphophorus helleri). Which is where I picked it up in middle school. I'd been using it for my software projects since I was 14 or so and by the time Xiph.Org was a real thing [many many years later] I wanted to change the name to something less silly and my co-founders voted me down. They liked Xiph. It became the precedent-setting silly name.
Vorbis is from Terry Pratchett's _Small Gods_ and I dearly hope Mr. Pratchett considers it a compliment. It was meant as tribute to my favorite fictional villain, Archdeacon Vorbis. "A mind like a steel marble"
Parent