1440073
story
Comments: 316 +- IT: 2nd Multi-Format 128kbps Public Listening Test on Thursday May 13 2004, @04:50PM
background: url(//a.fsdn.com/sd/topics/topicmusic.gif); width:60px; height:49px;
music
background: url(//a.fsdn.com/sd/topics/topicmediaall.gif); width:73px; height:59px;
media
background: url(//a.fsdn.com/sd/topics/topicencryption.gif); width:80px; height:63px;
encryption
background: url(//a.fsdn.com/sd/topics/topicsecurity.gif); width:59px; height:78px;
security
technology is sexy writes "Roberto Amorim has launched his latest public listening test evaluating the performance of different audio codecs at 128kbps, among them Apple's AAC implementation (used in iTunes), LAME, Ogg Vorbis fork auTuV, WMA, Musepack and even Sony's Atrac3 format, which is soon to be used in their own music store. Read more on Hydrogenaudio and check out the results of prior tests. As opposed to most evaluations of audio codecs, this is a scientific test adhering to ITU-R BS.1116-1 as much as possible while still allowing everybody to participate."
Related Stories
He had occasional flashes of silence that made his conversation perfectly
delightful.
-- Sydney Smith
All trademarks and copyrights on this page are owned by their respective owners. Comments are owned by the Poster. The Rest © 1997-2009 Geeknet, Inc.
Listening test? (Score:5, Funny)
Ogg! (Score:4, Funny)
Now that that's out of the way, let the insightful comments begin.
Re:Ogg! (Score:5, Informative)
Parent
Re:Ogg! (Score:4, Funny)
Nope, it just doesn't have the same ring to it.
Plus, vorbisty just doesn't work.
Parent
Re:Ogg! (Score:4, Funny)
Now that has a ring to it!
Parent
Re:Ogg! (Score:4, Funny)
Vorb-Vorb Vorbbity Vorb Vorb.
Bissy Bis... ba bis bis bis.
Vorbbity Vorbbity va va vorb. bissity bis.
Parent
How about: (Score:3, Informative)
Of course, if that turns out to be inferior to any of the other formats, it would prove that something's wrong with the tests.
Re:How about: (Score:5, Funny)
Aflac? What does a silly duck have to do with sound compression?
Parent
Re:How about: (Score:3, Funny)
Re:Ogg! (Score:5, Insightful)
Parent
Re:Ogg! (Score:5, Funny)
I, for one, would welcome our new filter overlords.
Parent
Objective audio analysis (Score:2, Informative)
So uh, why is this necessary, exactly?
Re:Objective audio analysis (Score:5, Interesting)
Parent
Re:Objective audio analysis (Score:5, Interesting)
Because "human auditory capacity" is not fully understood. Sure we can give standard frequency response graph, but most of these codecs take advantage of psycho-accoustic hearing models -- where certain frequencies mask other frequencies in our perception. Since this is a developing field, objective listening tests could really help determine what's working and what's not.
From my understanding of MP3 compression and others, the compression protocols take advantage of this frequency masking, so if humans can't hear it, it removes it. It also obviously takes into account frequency ranges of hearing. As a side note, I think it might be neat to be able to compress 30-50% better based on your personal hearing characteristics, but it'd stink if you got old and had to not only wear a hearing aid, but also start collecting MP3's all over again.
Parent
Re:Objective audio analysis (Score:4, Interesting)
Apparently he doesn't realize that this is a double-blind test - meaning neither the listener nor the tester knows what codec is being presented at any given time.
I'm taking the test now (well, not right now, taking a break) and it's about as scientific as I think you could make a public test taken in the home. Yes, the samples get compressed and then put in easily accessible folders with proper file name extensions, but you never know what you're actually listening to when you're running the testing program. All you have is a source file for comparison, then two buttons marked "1" and "2", one of which is the source again, the other a randomized codec. You never know which of the two buttons is the uncompressed source and you also never know which codec you're hearing. The results are also encrypted, so it's not as if you can just go into the results files and look at what codecs you favor.
I suppose someone who's truly got the Ear of the Gods could listen to the samples outside of the testing program, pick various identifiable traits out of each, then listen for those traits in the testing program and vote up or down whatever codecs he or she chose, but that would be exceedingly difficult and more than a little time-consuming. I can't see how it would be worth it, especially as no single test result is going to skew the overall results to any significant degree.
This is the first time I've ever taken a test like this and I am honestly pretty shocked at how good all of these codecs sound. I am having a really hard time even deciding which is the compressed track most of the time, and I consider myself something of an audiophile. I'm even listening in a fairly controlled environment with a good pair of headphones, at a volume loud enough to hear any background noise clearly but below any clipping whatsoever. I will be surprised if any codec really does significantly better than the others consistently when we see the final test results.
Parent
Re:Objective audio analysis (Score:5, Insightful)
For example, conventional wisdom says that the human ear cannot detect sounds above roughly 20kHz, yet there is at least some anecdotal evidence that higher order harmonics shape what we hear.
If "normal" human auditory capacity was a completely decoded topic, there wouldn't be nearly as much a need for different approaches to music compression (it would be a much simpler problem with fewer possible solutions)
Parent
Re:Objective audio analysis (Score:4, Insightful)
Well I could be wrong, and forgive me if I've misinterpreted your post...but
Don't all of these compression algorithms rely on psychacoustic modeling to remove 'extraneous' information from the bitstream?
If that is correct, and the algorithms are implemented correctly, then really what we are looking for is the best perceived result.
Just because the output meets the algorithm input->output specs, justn't mean it's the best output as perceived by humans.
Maybe think of it as optimizing sort routines? Yep, bubble-sort or b-tree still output a sorted list, but the perceived value is that the b-tree is better because it performs it's function more quickly.
This isn't an exercise in getting the frequencies algorithmically correct - the end result has to be listenable.
Humans are analog devices...
Parent
Re:Objective audio analysis (Score:5, Informative)
Also, a frequency plot tells us nothing about the phase or frequency distribution at certain times in the signal. I can make a sine sweep that would match exactly the spectrum of a pop song, but obviously would sound nothing like it.
There are ways of objectively measuring the performance of perceptual encoders, but frequency analysis isn't really one of them.
Parent
Re:Objective audio analysis (Score:4, Informative)
Parent
Re:Objective audio analysis (Score:3, Informative)
Also, all frequencies aren't of equal importance to a our ears. Our hearing is best in the middle range (near where the important elements of speech are), and taper off above
No matter *what* (Score:2, Insightful)
Re:No matter *what* (Score:4, Insightful)
Uh, if the sample is the same length, and the but rate is the same, won't the file size be the same as well? A 10 second sample at 128 Kb Per Second should be 1280Kb regardless of the format, no?
And, just FYI, MOST people, something like 95% of listeners cannot tell the difference between 128kbps sample and the original. I generally can't, even with decent headphones on.
I think that all you compression elitist snobs work for HD manufacturers, trying to get me to buy a 250GB drive to store the same amount of music as my 60GB will hold!
Parent
Uh, file size *is* bitrate... (Score:5, Insightful)
BTW, I think the difference between MP3 and Vorbis at 128 kb/s is perfectly noticeable. MP3 sounds rather bad, vorbis sounds pretty good. And the point is precisely to tell which format sounds best, so you don't want to do 512 kb/s bitrate where all formats sound close to CD quality.
Parent
Re:Uh, file size *is* bitrate... (Score:3, Insightful)
When you go above 128kbps, most formats become indistiguishable from the uncompressed sample. I mean hell, most people CAN NOT hear the
Re:Uh, file size *is* bitrate... (Score:3, Interesting)
Re:No matter *what* (Score:5, Insightful)
Not even if it's about average quality speakers?
Not even if it's about some rather cheap speakers?
I can't say I hear much of a difference with modern codecs, and I own some average speakers. Maybe 128 kbps mp3 can sound bad (although that depends a lot on the kind of music), but that's an aging codec anyway. I think encoded files in the 192 - 256 kbps range is the best, and 128 kbps ogg's often acceptable, especially with the DFX plugin (or similar) for Winamp to compensate for shortcomings in compressed formats.
I'd definitely not call 128 kbps in modern codecs "disgusting". In ogg's I've found it to be roughly as 160-192 kbps mp3's and that's perfectfly fine for my ears.
Parent
Re:No matter *what* (Score:4, Informative)
Parent
Re:No matter *what* (Score:5, Insightful)
The fact is that for a lot of people, knowing the best codec at 128kbps is worth knowing because:
1) They are using portable devices where they are space constrained
2) They are using portable devices that may not have the perfect fidelity of a high-end sound system, but can go anywhere with them.
3) They are using their portable device in a somewhat noisy environment that overshadows any sound quality issues caused by a lower bitrate.
Parent
Re:No matter *what* (Score:4, Informative)
Try doing the test, you might be surprised, or conversely if you're not surprised, you might contribute valuable information to the project.
Parent
Re:No matter *what* (Score:3, Informative)
While the sine wave's frequency is known exactly (within the resolution of your sampling frequency) the amplitude is not- you always have loss due to quantization noise. You may be thinking of the fact that the fourier transform will have only one harmonic and thus the quantization noise doesn't come into play.
Consider the signal to quantization noise rate (SQNR):
SQNR (dB) = 20log(Vsignal/Vquantization_noise)
With linear quantization, your quantization is evenly spaced and the noi
Speakers (Score:3, Insightful)
Performance is only one more factor (Score:5, Insightful)
But in the real world other factors may be more important to chose a coded, like for example general acceptance, freely available code and specs, and a large content base available.
You see: performance will increase allways in all codecs with time... so this kind of testing is only a minute factor amongst others.
What's the point of 128kbps? (Score:4, Insightful)
Re:What's the point of 128kbps? (Score:3, Interesting)
Same deal for MPEG-2 encoders, they all look great at 7 Mbit+/sec but the real test is 3-4 Mbit/sec.
VBR? (Score:3, Informative)
Vorbis does variable bit rate and you set the quality you want. That way you don't waste lots of bits where they are not needed. My 4MB ogg file sounds as good or better than my little brother's 6MB mp3. The difference is more songs on my 256MB compact flash card. Yes, it's easy to play that music on my Zaurus, which cost about as much or less than DR
Re:What's the point of 128kbps? (Score:3, Informative)
I use `lame --preset standard`, which ends up being VBR in with a max of 110-290, hovering mostly around 190-210 range. It's one of the reasons I don't use OGG, it doesn't have any preset's so I'm supposed to just decide on a good level myself. I'd rather use something th
Sound quality is in the speakers (Score:5, Insightful)
Too bad they didn't challenge Apple (Score:3, Insightful)
If they'd included both versions of iTunes/QuickTime in this test, perhaps they could have helped shame Apple into fixing what they broke.
Re:Honesty of responders (Score:5, Insightful)
Not possible. All you will get is a bunch of WAV-files, you have no way to tell which file belong to which codec.
That said, I don't care which codec wins the test because Vorbis is still the only one free from patents and the margins are so incredibly small.
Vorbis will win for me even in the unlikely scenario that it comes out last.
Parent
Re:Honesty of responders (Score:3, Interesting)
Of course, proving the patent-freeness of Vorbis requires searching every single patent with a fine-toothed comb, further indicating how messed-up the whole patent system is at this point.
I just have to wonder how many companies are waiting to pounce on the first major commercial user of Vorbis with a patent suit
Re:Honesty of responders (Score:4, Interesting)
Check the contents of the sampleXX.zip files; you actually get an mp3, an
Parent
Re:Okay... (Score:5, Informative)
The sad thing is that somebody went to the trouble of putting together a perfectly reasonable, logical post just to throw in a porn link. *sigh*
Parent
Re:The best 128kbps audio format (Score:3, Informative)
And this is for monaural sound. If you want stereo, cut the sampling rate in half -- this might cut it for voice, but it won't work for anything else.
you have no clue ... (Score:3, Informative)
The bit rate of
Re:you have no clue ... (Score:3, Funny)
>All you have to do is limit the length of the song to
good lord. There's TWO of them.
Re:What ever happened to r3mix.net? Any replacemen (Score:5, Insightful)
The best replacement for r3mix.net in my opinion is HydrogenAudio [hydrogenaudio.org] . The forums are frequented by a lot of professionals, as well as developers of LAME, FLAC, Nero AAC, Musepack, Wavpack, and other codecs.
Parent
Re:What ever happened to r3mix.net? Any replacemen (Score:5, Informative)
Parent
Re:A nice idea (Score:3, Insightful)
Premise of test is somwhat flawed (Score:4, Insightful)
After a while, once you have weeded out bad ways, one is going to reach the following situation. Each algorithm will perform very well for a large set of music and poorly for some small set of music. Barring pathologies, The poor set will be assymtotically fixable by increacing the bit rate. By the way this is not just my opinion. Theres theorems that say this is true of any compression scheme when applied to all problems.
what does this mean? it means that the end user is never going to work at the truly low end of the bit rate specrrum because they want something that virtually always works. Plus they want a wee bit more just in case they have to transcode it. So if the recommended rate is 128 people will encode at 160.
So these comparisons need to be done not at the bitter edge where music flaws are easy to spot because NO ONE WILL ACTUALLY MAKE THAT THE OPERATING POINT THEY USE. That is to say everyone knows vorbis sounds so-so at 64KB while MP3 sound much worse. But no one wants So-So they want darn good. So they are going to recors their Mp3 at 160 and at 160 Ogg and Mp3 sound so close that the size of the test you'd have to do to pick up the difference is silly.
the proper way to do this is the following. Pick the gold standard format, say MP3 and its standard excellent operating point, say 160. now test all the others at lower bit rates than 160, and see which one has the lowest bit rate that scores as good as the Mp3 at 160.
comparing all methods at a constant bit rate, esepciall a low one, is stupid
Parent
Re:There is no satisfying audiophiles (Score:3, Insightful)
I've found most of the people on Hydrogenaudio to be incredibly pragmatic. Perfection isn't the only parameter of importance. If it were, they'd not be wasting time testing codecs at 128kbps, except to demonstrate their unsuitability compaired to losless formats. They'd not be wasting time letting phillistines with their waxy untrimmed ears particpate in listening tests with their