Catch up on stories from the past week (and beyond) at the Slashdot story archive

 


Forgot your password?
Close
typodupeerror
×
Music Media

Listening Comparisons For Audio Codecs At 64kbps 331

Posted by simoniker from the passing-the-audio-taste-test dept.
waaka! writes "Hydrogenaudio has just wrapped up a listening test of various audio codecs at 64kbps. Check out the results, where Ogg Vorbis performed quite well, scoring significantly better than WMA, RealAudio and QuickTime AAC, and kept pace with MP3Pro and HE-AAC (AAC with the SBR extensions that MP3Pro uses). Clearly, though, no codec can honestly claim 128 kbps MP3 quality at 64 kbps. The charts at the end show entries for 128kbps LAME MP3 and 64kbps FhG MP3, but these are used as high and low anchors for reference, as MP3 is really out of its league at bitrates such as these."
This discussion has been archived. No new comments can be posted.

Listening Comparisons For Audio Codecs At 64kbps More

Listening Comparisons For Audio Codecs At 64kbps

Comments Filter:

  • Before anybody says.. (Score:5, Insightful)

    by Anonvmous Coward ( 589068 ) on Monday September 22, 2003 @04:29PM (#7028343)
    ... "Why not just encode it at 384 and be done with it?", consider that PDAs have 64 megs of RAM, cell phones get no better than 56k, and that not everybody has broadband.

    • Re:Before anybody says.. (Score:5, Funny)

      by Capt'n Hector ( 650760 ) on Monday September 22, 2003 @04:30PM (#7028353)
      That's what the 40 gig iPod is for.

      • obligatory audiophile style rant (Score:5, Insightful)

        by plip ( 630579 ) on Monday September 22, 2003 @10:47PM (#7030960)

        I want to give full respect to the people who put all the research into creating these new audio formats. The results are truly phenomenal for 64kbps codecs. It's a fabulous academic demonstration.

        However, each is what it is. A 64kbps codec.

        I have about $6000 invested in my 2-channel +subwoofer setup here at home, and I consider that moderate compared to what you can truly achieve. I love listening to music, and it is completely remarkable when it is reproduced as realisticly as possible. So I go to painstaking methods to make sure the AC power is clean, the wiring is right, the distortion is low as possible. The signal to noise ratio is far between, with a good amp, and great speakers... I am especially pleased when the recording I am playing on my wonderful system is in the best production quality that it can possibly be.

        As amazing as they are, these 64kbit formats are useless on a person like me. I crave LOSSLESS not LOSSY. I might as well be listening to music on a $60 AIWA boombox, since it would sound relatively similar either way. All the subtle beauty and realism of the music is completely wasted with destructive compression.

        And for those of you that say it's for portable devices, It's not too unreasonable to get a portable player that plays high streaming VBR mp3s with some nice ~$100-$150 headphones. The small little investment to hear your music from 20hz-20khz flat response with low distortion is worth every single penny.

        I simply do not understand the need to take our ever improving technology and lower the quality of the music. If anything, it should be increasing... higher resolutions. 24bit/192khz technologies, and wonderful DSP equalizers, large portable storage devices... they are all realities now, but nobody seems to care but the fanatics like me. I would think that techno geeks would care more about the music they love, but that does not seem to be the case. The only logic that I can fathom to explain why is that perhaps they don't even know what they're missing. I know I didn't, until I actually experienced how good sound quality can be on the right system.
    • If you want to use a PDA for any amount of MP3s, the 64 of RAM won't cut it. You'd need something like a IBM microdrive for that kind of storage.
      • Excuse me, but what are you talking about? Perhaps you're just exaggerating? But most mp3 files are no bigger than 5MB. How does that not fit in 64MB?
        • Excuse me, but what are you talking about? Perhaps you're just exaggerating? But most mp3 files are no bigger than 5MB. How does that not fit in 64MB?

          "For any amount of MP3s", as the grand-parent poster put it, would equate to at least more than a CD.

          64MB, with everything else, won't store more than a CD.

          Understand?

    • To paraphrase Gates (Score:5, Funny)

      by commodoresloat ( 172735 ) on Monday September 22, 2003 @05:20PM (#7028724)
      Nobody will ever need less than 64kbps of audio.
    • Actually, my cell phone gets around something like 12kb/s actual throughput. In other words, I've actually downloaded a file using my cell phone for Internet access, and I got around 12kB/s doing it. (Shh, my plan says I'm not allowed to...) Mind you, it has really lousy ping times, on the order of a few seconds. This makes browsing the Internet on the phone seem very slow and makes remote terminal connections very interesting - be sure to type it right, the first time.

      So while using a cell phone for

  • But how... (Score:5, Funny)

    by Atario ( 673917 ) on Monday September 22, 2003 @04:30PM (#7028344) Homepage
    ...to compensate for everyone's crappy $1.99 computer speakers?
    • 1. Take user to goodwill or other cheep 2nd hand store and show them the plethra of cheep vintage amps amps and speakers, many of which really whip the lama's ass.

      2. Hook up this vintage amp to computer and watch user go "wow that sounds so much better".

      3. Profit (from the lack of snap crackle and pop)

      This is a touch off the topic, but i've observed that many 2nd hand stores are no longer stocking computer equipment and monitors. I understand the reason even, people go there and give them worthless c

  • Sometimes you don't need high audio quality (Score:4, Interesting)

    by Anonymous Coward on Monday September 22, 2003 @04:30PM (#7028352)

    Sometimes, simple audio clips [oldmencrying.com] don't require the highest quality. Function triumphs over high performance hot-rodding.

  • CD (Score:5, Interesting)

    by Leffe ( 686621 ) on Monday September 22, 2003 @04:31PM (#7028361)
    Clearly, though, no codec can honestly claim 128 kbps MP3 quality at 64 kbps.


    You used to compare against CD quality.

    Oh well, times change, I guess it's time to throw all my CDs away and instead store all music in this new exciting digital format.

    And seriously, does anyone listen to music encoded at 64 kbps? 128 is the bare minumum.
    • "Oh well, times change, I guess it's time to throw all my CDs away and instead store all music in this new exciting digital format. And seriously, does anyone listen to music encoded at 64 kbps? 128 is the bare minumum."

      Well....dunno about throwing them all away...but, I am in the process of ripping all my CD's to my newly built media computer. Using FLAC lossless format...

      Just checked for sound the other day, and was fantastic!! I guess the ogg/mp3 thing is ok for poor listening environments (car, port

      • only very few professional and gifted people can distinguish or even recognize a good encoded sound from lossless sound on 100 % top end HIFI systems.

        Check out the C't listening test (blind test!) done in 2002 or 2003, which showed that people producing classical music, people finetuning codecs and many others were not consistently able to tell the difference. The best tester was someone with a hearing damage on one ear. The psychoaccustics obviously did not work 100% for him.

        BTW: OGG won that test for ~1
    • If you intend on streaming your audio. Audio can stream decently at 128kbps, in fact, for modems it is a must. Internet radio, and even most stuff on mp3.com et al sounds decent for previewing purposes when played at lower bitrates.

    • Clearly, though, no codec can honestly claim 128 kbps MP3 quality at 64 kbps.

      You used to compare against CD quality.

      Oh well, times change, I guess it's time to throw all my CDs away and instead store all music in this new exciting digital format.

      And seriously, does anyone listen to music encoded at 64 kbps? 128 is the bare minumum.

      This isn't the "Archival/home music jukebox near-perfect CD quality on a high fidelity speaker set" comparison. At this bitrate, it's the "best space constrained portable dev

    • And seriously, does anyone listen to music encoded at 64 kbps? 128 is the bare minumum.

      Some people use 64 k... internet radio sometimes does, for instance... but that's not why they used it...

      I imagine that it was used because with today's lossy codecs, the bitrate has to be abysmal to have a measurably inferior sample. Bitrates of, say, 384 kbps, for these codecs are very very difficult to rate to the average listener.

      I can only assume that this test was done with the assumption that, for a given code
    • Bare minimum? I don't think so.

      I have a portable digital audio player with 64 megs. It is not my home system, where things are done at 128 (really, the default of oggenc...), it is what I carry around. I'd rather listen to (almost) twice as many songs at 64kbps then half as many at the same quality (since I am using cheap-o earphones.) Currently I am doing it at 96kbps, but I might go down to 64 after reading this article.
    • I'd consider the bare minimum for music to be about 70 or 80 kbps Vorbis, but I don't have great quality audio hardware.

    • Re:CD (Score:3, Informative)

      by harlows_monkeys ( 106428 )
      And seriously, does anyone listen to music encoded at 64 kbps? 128 is the bare minumum

      If you mean "does anyone rip things at 64 kpbs?", then I'd guess mostly not. However, if you really mean what you asked, then plenty of people do.

      Take a look at live365.com [live365.com]. A huge number of the streaming stations there are at 64 kbps or less.

      I listen to filk radio [filk.com] via live365 a lot, for example, and it is below 64kbps.

      64k and below can work fine for listenting to music. However, many people listen to the encodi

  • Streaming audio (Score:2, Interesting)

    by Brahmastra ( 685988 )
    I've also noticed sometimes that 28 kbps streaming audio in some sites is much better than 56 kbps audio on other sites

    • Re:Streaming audio (Score:2, Interesting)

      by Tyrdium ( 670229 )
      Yes, but bitrate isn't the only factor in streaming audio. If the latency from the 28 kbps site is relatively low compared to that of the 56 kbit site, it will sound better because it will actually be able to make use of the bandwidth. There was a site on this, but I lost the url. Sorry!
      • uuuhuuhhuhhhh... latency?

        I think the difference would more likely be due to stereo/mono, samplerate, and codec differences (tfa). latency's just going to affect how quickly you get the packets, not the sound quality.

    • Re:Streaming audio (Score:4, Insightful)

      by connect4 ( 209782 ) on Monday September 22, 2003 @04:48PM (#7028489)
      Sometimes people try to encode stereo into less than 64kbs. It's just crazy. I've listened to mono 56kbps mp3 that sounds a lot better than stereo 64kbs.

      I'd much rather listen to a favourite song in mono with reasonable reproduction quality, than in stereo that sounds like it's coming out of a tunnell.
    • The difference is usually down to mono v stereo encoding which makes a big difference at low bitrates.

  • MP3 is the standard. (Score:5, Insightful)

    by Schezar ( 249629 ) on Monday September 22, 2003 @04:38PM (#7028413) Homepage Journal
    MP3s are ubiquitous. My computer, DVD player, portable audio player, and car stereo all support it. The same can't be said for other formats.

    More to the point, why are all of these competitions at such low bitrates? The differences in quality between various types of audio compression become indistinguishable (and therefore irrelevant) as you raise the bitrate.

    I just use good old variable bitrate MP3 and forget about it. Simple and standard.

    • Re:MP3 is the standard. (Score:5, Insightful)

      by molarmass192 ( 608071 ) on Monday September 22, 2003 @04:50PM (#7028505) Homepage Journal
      You're thinking in terms of music only. The MP3 patent (#5,579,430) only expires on Jan 26, 2015. So if you're a software/game maker with slim margins to start with, you'd want a format that's not patent encumbered or that cost less. Sure a $1 royalty fee for each decoder doesn't seem like much until you ship 500k units and have to cut that check to Fraunhofer. If the quality is comparable and a lower/no cost option is available, you'd have a pretty strong case for not using MP3.

      • Re:MP3 is the standard. (Score:5, Informative)

        by StaticEngine ( 135635 ) on Monday September 22, 2003 @05:39PM (#7028916) Homepage
        The Game Industry has embraced OGG, although somewhat silently. With slim budgets, we're always looking for the cheap (and free) solution, and OGG is perfect when we want compressed audio at a good quality.

        The sole deciding factor in whether or not compressed audio really gets used in a game is available minspec bandwidth. If marketing is forcing us to target a 500MHz machine, and decompressing OGG audio kills our framerate, then audio compression goes. It the sad truth that the tech heads do not call the shots in this department.

      • Re:MP3 is the standard. (Score:4, Insightful)

        by repetty ( 260322 ) on Monday September 22, 2003 @05:49PM (#7029005) Homepage
        Well, it's all a matter of perspective, isn't it? After all, a lot of game makers would love to ship 500k units.

        And personally, I'd love the burden of paying $50,000 taxes every year, if you catch my drift.

  • Comment removed (Score:5, Insightful)

    by account_deleted ( 4530225 ) * on Monday September 22, 2003 @04:42PM (#7028451)
    Comment removed based on user account deletion

  • some good listening test material (Score:5, Informative)

    by MarcoAtWork ( 28889 ) on Monday September 22, 2003 @04:43PM (#7028457)
    I saved this thread quite a while ago and I agree with several of the recommendations (notably with the 'Tori Amos' 'Boys for Pele' CD, not that it's the type of music I usually listen to, but I have to admit the production values are outstanding), after all using hyper-compressed (re: other slashdot articles) crappy source material is not that helpful in terms of figuring out how good the various encoders really are...

    the thread on google [google.com]

    Personally I rip my own CDs with lame --alt-preset extreme (on said Tori Amos' CD it seems it hovers around 224kbps with -lots- of frames at 256 and 320), for fun I transcoded (I know, transcoding is bad, mmkay?) a few of them to vorbis 48kbps and it's amazing how good they sound at that low of a bitrate.

    • Re:some good listening test material (Score:4, Informative)

      by default luser ( 529332 ) on Monday September 22, 2003 @05:25PM (#7028773) Journal
      (I know, transcoding is bad, mmkay?)

      Actually, it's not as bad as you think, given the circumstances.

      The general problem with re-encoding audio is errors will become magnified versus a direct encode to the lower bitrate. If you take a 192k or 160k CBR mp3 encode and downsample it to some other format, it is going to sound like crap. But you have to remember that modes like LAME --alt-preset virtually eliminate errors in audio reproduction.

      Sure, the inaudable tones have been removed, but every bit of the audible spectrum has been accurately rendered, making it nearly as good as the original source for the purposes of transcoding.

      I rip all my albums using --alt-preset standard, and I transcode them to 128k ABR for my handheld mp3 player. I've never been able to hear any perceptible difference between this and a direct-from-CD 128k ABR encode.

  • Just Habit... (Score:2, Informative)

    by Ironix ( 165274 )
    I always encoded my MP3s 224 kbps and when iTunes came out I simply continued the tradition.

    In any case, I can certainly notice the improvement from MP3s encoded at the same rate.

    • Re:Just Habit... (Score:4, Informative)

      by hondo77 ( 324058 ) on Monday September 22, 2003 @05:10PM (#7028660) Homepage
      Try 128 AAC. I can hear that it's better than 192 MP3 even through my dinky little headphones. Better sounds and smaller files make me a happy guy.

      • Re:Just Habit... (Score:3, Insightful)

        by tho 1234 ( 709100 )
        Try a different encoder. There is so much variance between different MP3's that simply stating a bitrate doesn't say very much. An MP3 encoded at ~192 average bitrate using LAME alt preset extreme actually sounds slightly better better to my ears than AAC with any encoder at the same bitrate. On the other hand, if your 192k/s MP3 is ripped with xing or downloaded off Kazza, then chances are it will be much worse than any AAC file. Try some different encoders first, MP3 may be an outdated format, but mor

  • question (Score:5, Interesting)

    by zymano ( 581466 ) on Monday September 22, 2003 @04:44PM (#7028470)
    Are we at the LIMITS of compression technology ? Is there anything new being worked on by anyone ?

    • Re:question (Score:2, Funny)

      by pbox ( 146337 )
      I can compress anything to a single bit. The decompression algrithm, however, depends on what you have compressed earlier ;-)
    • My Limp Bizkit and Britney Spears collections are compressed down to zero bits. This won't work for everyone, though.

    • Re:question (Score:5, Interesting)

      by merlin_jim ( 302773 ) <James@McCracken.stratapult@com> on Monday September 22, 2003 @05:35PM (#7028878)
      Are we at the LIMITS of compression technology ? Is there anything new being worked on by anyone ?

      Well we are certainly near the limit of lossless compression. In that there is a well-studied field of computer science (information theory), which provides a framework to determine the theoretically maximum amount of lossless compression possible given a particular data sample, and the best lossless compression algorithms we can come up with are within a small percentage of that figure. FYI, a fundamental tenant of information science is that everything can be reduced to a certain atomic level of representation, and that this atomic piece is the "information" contained within "data"... and that one cannot convey "information" in less space than this atomic piece.

      For instance, I've heard that common every day american english conveys approximately 1.2 bits of information per word... meaning that the least redundant approximation of human speech would need that bit rate to represent it.

      As far as lossy compression, there might or might not be more work to be done. The problem is that the human ear and the human auditory nervious response are far from being fully characterized, though we do have a good start on it.

      The idea of a lossy compression algorithm is to remove pieces of information that the human ear and/or auditory nervous response is not sensitive too... therefore increasing the theoretically possible maximum compression without adversely affecting the signal representation. As we as a species come to characterize these human responses, we will certainly see better codecs coming out. I do however believe that we're rapidly approaching an asymptotic level of understanding where further levels of effort and research into codecs is not economical with regard to expected payoffs...
      • Well we are certainly near the limit of lossless compression

        For general-purpose compression, yes.

        Keep in mind, though, that Shannon's theory only applies to context-free compression, by which I mean something slightly different than the normal information-theoretical use of the word "context"...

        As a trivial example, consider a multiplicative congruential random number generator (the one most C libs use for "rand()"). If you take the output of that and try to compress it, you get very poor results.

        • Re:question (Score:3, Insightful)

          by dmaxwell ( 43234 )
          With a sufficiently complex model, we should have the ability to record an entire concert as little more than a MIDI-like file, containing the excitiation parameters for each instrument involved.

          Thats the sticky part though. A really good model of a musical instrument or human vocal tract will require significant memory and CPU resources. Compression has always entailed a tradeoff between filesize and resources to decode it. Your proposal represents one of the extremes. Even with today's tech, I don'
        • With a sufficiently complex model, we should have the ability to record an entire concert as little more than a MIDI-like file, containing the excitiation parameters for each instrument involved.

          While that is an interesting (and frankly exciting) idea, I was specifically addressing current state of the art, which is focused on faithful reproduction of a given waveform, not necessarily reducing that waveform to a set of parameters in some MIDI-like encoding theme. That would certainly be several orders of
      • For instance, I've heard that common every day american english conveys approximately 1.2 bits of information per word... meaning that the least redundant approximation of human speech would need that bit rate to represent it.

        No, the entropy of English is ~1 bit per character, not ~1 bit per word.

        Here's one reference [stanford.edu]

  • Why so low? (Score:4, Informative)

    by Izago909 ( 637084 ) <tauisgod@gmaiPOLLOCKl.com minus painter> on Monday September 22, 2003 @04:53PM (#7028526)
    Anything below 128k/s (in my opinion) is only good for streaming and embedding. Even 128 is the bare minimum for anything that sounds decent. Are there any comprehensive articles that deal with comparing high encoding rates (192+) of multiple formats?

    It should also be noted that it is not recommended using CBR encoding with OGG. It is a native VBR codec that is only forced into CBR for steaming. The quality of CBR is much lower than VBR. It would be very nice to see a comparison that uses VBR for all codecs that stick to the same bitrate range.
    • Right, streaming and embedding. Why is it so hard to imagine that this is, in fact, a large and rapidly growing market in which these codecs are competing? Besides, in the high-bitrate end, these next-gen codecs are practically indistinguishable, so why bother testing them? The new frontier is in the low-bitrate market...

      Anyway, I was pleasantly surprised at the quality of OGG at 64 kb/s. It's easily FM quality, and FAR better than MP3 at a similar rate, making it a superb codec for live audio streamin
    • Hmmmm, RTFA. Vorbis (along with the other codecs that support it) used VBR in the tests.
      • The article was about 64k recordings while my question was about 192k+ recordings. Seeing as how I read the article, and it did not answer my question about high encoding rates, I posted a question for the community assuming that some people would be helpful and/or informative.

        Thank You... Come Again.
    • Probably because the average listener can't even tell MP3 from the original CD at 128kbs, let alone the next gen formats.

      Yes I know there are people that can tell the difference, but there are a lot of people that cant.

      If you push the boundries then you can start to see them differentiate themselves a lot more clearly. It's really quite amazing how far you can push Vorbis and still have it sound acceptable, and even when it degrades, it does so in a much more pleasant way than MP3 - no horrible metallic

      • Re:Why so low? (Score:3, Informative)

        by Izago909 ( 637084 )
        Actually, even at 128 (at 48 kHz) you can tell a difference. With mp3, higher frequency sounds (ex. cymbal crashes) can artifact heavily. The more that's going on, the worse it gets. Higher range vocals also are affected. I have some bebop styled tracks that use a lot of the stand up bass and brass percussion. The vocals often sound very metallic, especially when she starts hitting the higher notes.

        For most of my archival I use OGG at a quality setting of 7 (~224k/s) and transcode it to mp3 @ 128-192 when

    • Re:Why so low? (Score:3, Informative)

      by proxima ( 165692 )
      Well, 64kbps is a good rate for streaming and low-capacity situations (like flash-based mp3 players). If ogg can manage to become more popular in hardware, it would make an excellent alternative to standard mp3 encoding.

      That said, I've fallen down the quality slope - with hard drives so large now I've decided just to encode all my music with FLAC [flac.org] and have absolutely no quality loss (lossless compression; flac is to ogg as PNG is to JIF). Granted, I don't know if I can tell the difference between ~256kbp

  • An interesting intepretation (Score:3, Insightful)

    by nsample ( 261457 ) <nsample AT stanford DOT edu> on Monday September 22, 2003 @05:01PM (#7028592) Homepage

    The poster offers an interesting interpretation of the results, but only his/her comments support Ogg Vorbis in this case. The numbers tell a completely different story.

    The analysis presented leads us to one conclusion: use Lame 128. It's strictly better than all other options. Do not use FhG MP3. Easy.

    If you're willing to slip to 4th best encoder, then consider Ogg Vorbis. 4TH BEST. That's hardly the rosey picture painted in the article.

    Also, don't be deceived by the "confidence intervals" shown in the graph. They're all drawn to the same widths for each set! At best, this is an approximation. At worst, the author is simply using a program that draws in some uniform (and meaningless) bars. Fear graphs.
    • The analysis presented leads us to one conclusion: use Lame 128. It's strictly better than all other options. Do not use FhG MP3. Easy. If you're willing to slip to 4th best encoder, then consider Ogg Vorbis. 4TH BEST. That's hardly the rosey picture painted in the article.

      Orrr..... you could use Ogg at 128kbps, which would be an apples-to-apples comparison, one in which Ogg (or AAC, for that matter) would surely come out on top.

    • It's only 4th best if you forget to notice that the #1 codec was 128KbPS MP3. All the other codecs were at 64, including Vorbis.
  • Why is it that no-one ever quotes their hearing test results when doing these subjective tests? It's just like when my co-worker tells me that watching DVD's on my 1600x1200 resolution screen is not as sharp as his German 100MHz standard television.
    The fact that his visual acuity has been compared to Mr Magoo never comes into the equation...
  • that Joe Schmo out there with the Windows machine will be pretty much sticking to WMA. Sure hardcore audiophiles can tell a difference between formats but the average computer idiot doesn't care.

    The saddest part of all is that WMA is a beast that is growing and will be hard to get rid of. Since MS has submitted this format for inspection for widespread adoption, they will continue to force their way into this becoming the de facto standard even though it sucks ass. More importantly, because of the draconia
  • Interesting to look at, and I'm not very suprised that the "we sound as goot at half the bitrate" claim wasn't true, but I do have two observations.

    First, I know and listen to some of those songs. It's nice to see band(s) I listen to, it makes the test seem much less... abstract. It seems like these tests usually use music I've never even heard. (For the curious, TMBG and John Linnell).

    Second, I would have liked to see the results presented as "quality relative to 128kb MP3", since that's the "gold standa

  • "We've done statistical operations on non-quantitative data."

    ...uh, I guess I could go on, but that fact alone kills this before it even starts. When are people gonna learn? When the hell are people gonna learn? It doesn't matter how you "encode" or "enumerate" it, quantitative operations done to non-quantitative data have NO MEANING. NONE.

    @!#%$%#@ it. I suffered through three semesters of Stats in college, and the acid-reflux flare-up I get reading this kind of "test result" is the burden I bear for

    • "quantitative operations done to non-quantitative data have NO MEANING. NONE."

      Thank you, but of course the people who need to get the message, won't. They *want* to be influenced.

      To me there are two levels of audio/video quality. Good Enough for the only copy of the Master, and Not Good Enough.

      If I can have 32-bit waves sampled at >=96kHz, that's what I want. (And I get very, very tired of hearing about why I don't need that much digital headroom.)

      I'm old enough that I'm not supposed to care about

  • Okay, maybe I'm nitpicking... (Score:5, Interesting)

    by shirai ( 42309 ) on Monday September 22, 2003 @05:17PM (#7028706) Homepage
    Just some relevant data that doesn't appear on the front page is that the test is blind and they do compare the audio to an uncompressed reference. You have to click through to "Return to Roberto's Listening Tests page" to find this information though. Just thought I'd mention this because my first thought was what they are comparing the sound to.

    Comparing without a reference reflects how much you like the encoding of the codec, not how accurate it is to the original. For example, if a codec boosts the bass or encodes slightly louder, you may interpret this as better sound. For example, when auditioning speakers, you must always balance the output of the speakers as most people will psychologically prefer the louder (most sensitive) speaker. This does not mean the speakers are accurate however.

    At any rate, here is the relevant quote on that page:


    One of the most acclaimed methods of comparing codec quality is by performing so-called "Double Blind Listening Tests". In this sort of test, the participant compares various encoded samples against each other and against an uncompressed reference sample. The blind part means that the participant doesn't know which sample was encoded by which encoder. That guarantees there'll be no psychological bias towards his/her favorite codec, or against the codec he/she dislikes.


    Note that the quote (and here's the nitpick) suggests that double-blind means that the participant doesn't know which encoder is used. Double-blind means that both the participant and the person running the test don't know. By the way, this is, indeed as accurate as double-blind (since, well, the computer might know but surely doesn't care to influence the results). And I realize he doesn't say "double-blind means" but seems to suggest the definition of double-blind. Anyways, that's just the nitpick. Please don't mod me down for it. It's just an observation and I'm trying to build some Karma!

  • Digital Music artist perspective (Score:5, Informative)

    by merlin_jim ( 302773 ) <James@McCracken.stratapult@com> on Monday September 22, 2003 @05:21PM (#7028734)
    As an artist that releases mainly online, I found these results very interesting, and thought I'd share my feelings with the slashdot community.

    While MP3Pro and Vorbis were good competitors overall, and have a fairly good footprint to boot, I'd have to say that if I'm forced to encode to 64MBit/s, I'd absolutely choose Ahead HE AAC, if I'm judging solely on this comparison (which I am at this point in time...)

    Why? Because there was no sample that Ahead HE AAC did POORLY at. MP3Pro and Vorbis (and all the other codecs) each had one or two samples that they just totally choked on, quality-wise. So if I was forced to use a 64 MBit/s codec, it would absolutely be Ahead HE AAC, because while it didn't score highest on every test, and the three codec were virtually tied across the whole competition, I would feel far safer trusting my best digital work to a codec that, according to this test, would have the least chance of representing it particularly poorly.

    I wonder how these results compare to higher encoding rates; I could easily imagine that most codecs have a sweet spot, where the encoding quality/bitrate maximizes... it would be interesting to do some research to find this sweet spot.

    Anyone want a quick way to slashdot a server? :D
  • >It's worth mentioning that, while Ahead HE AAC,
    >Vorbis, MP3pro and WMA were encoded in VBR mode,
    >Real Audio and QuickTime were encoded in CBR mode
    >since these codecs don't offer a VBR mode.
    >Lame MP3 was encoded at ABR mode because that's
    >how Lame performs better at this bitrate.

    It explains. The "64kbit/s" is only an average.

    In general adaptive sampling methods such as VBR should always outperform constant sampling methods like CBR.
  • While there is a need for 64kbps bit rates, this is, *at best* FM radio quality.

    This is not high fidelity and certainly not for critical listening.

  • Wrong settings for Quicktime? (Score:4, Interesting)

    by RalphBNumbers ( 655475 ) on Monday September 22, 2003 @05:39PM (#7028913)
    As I understand it, the "Best" mode, which they used to encode the QT AAC clips, was actually optimized for audio with sample rates well above CDs' 44.1khz. For audio that originated on CD, the "Better" setting would have been more appropriate. (this setting does seem really unintuitive, I would hope for better from apple)

    I wonder if/how this would have affected the scores.

    I was surprised to se QT AAC ranked so low after it recently won a similar test among AAC encoders, was that HE AAC encoder not included in the previous test?
  • ...RCA Orthophonic records were judged superior to Edison Blue Amberols--even at 160 RPM.

  • SBR (Score:4, Informative)

    by zurab ( 188064 ) on Monday September 22, 2003 @05:48PM (#7028999)
    It's worth pointing out that at least MP3Pro and HE-AAC from tested codecs use SBR. SBR is a method (mostly post-process) that allows transmission of lower half of audio spectrum, and have the decoder "guess" what the the other part of the spectrum would have been. While this allows for "cool-sounding" audio at low bitrates, the generated part of the spectrum is not actually an encoded original audio, but rather its "guessed" reconstruction. SBR is also patented.

    Search for more info on SBR if interested, like this one [codingtechnologies.com].

  • rjamorim conducted the test, not HA (Score:3, Informative)

    by Compact Dick ( 518888 ) on Monday September 22, 2003 @09:21PM (#7030436) Homepage
    While Hydrogen Audio did provide the resources to host this test, the real work was done by Roberto Amorim, who organised this monster.

    Credit where credit is due.

  • Check yer fly, your ignorance is showing... (Score:5, Insightful)

    by CaptainPhong ( 83963 ) on Monday September 22, 2003 @10:41PM (#7030920) Homepage
    Ok, first of all, while the test was DISCUSSED on HydrogenAudio, and most of the participants are HA regulars, Roberto Amorim did all the hard work of organizing the test and compiling the results, dealing with complaints, etc. To not give him credit is not very nice.

    Second, there was a 128kbps test a month or two ago (which for some reason got repeatedly rejected when submitted to slashdot). You can see it here [ciara.us]. Unfortunately, the results there aren't quite as interesting (it was mostly a big tie). Unfortunately, tests at higher bitrates are difficult because detecting problems at, say, 160kbps often requires well trained ears and good audio equipment.

    Third, it's a good idea when commenting on an article to actually read it and click around on a few links to actually have an idea what you are talking about. Many /.ers seem to be only half-literate (can write but not read). There is a hilarious number of denegrating comments here by people who know nothing about either statistics or psychoacoustic audio compression. ABC/HR type methodology is the standard for comparing the relative quality of audio sources. Also, a great deal of effort went in to assuring that the best settings for the best encoders for each codec were used for the test. A little reading of the pre-test discussions would reveal this. Further, HydrogenAudio is not a club of audiofools who spend zillions of dollars on fancy speaker cable without any science to back it up. It is an objectivist forum. Anyone who makes statements without backing them up (with something like ABC/HR or ABX results) gets flamed HARSHLY. Some of the regluars have PhDs on various audio topics. They know what the fuck they're doing.

    Fourth, just because you don't have a use for 64k audio, doesn't mean the results are meaningless. Lots of people have small-capacity players, and some codecs can tolerate that bitrate for very casual listening (such as in the car). Lots of streaming audio sources are at this bitrate or lower. Satellite radio is at 64k or lower. Also, it's not a good idea to try to extend these results to other bitrates. MPC for example, isn't even worth considering at 64kbps, but at bitrates over about 140kbps, it will beat the pants off of anything else.

    Finally, for those who want to know more, or want their audio collections to sound best, read the FAQs at HA. Many codecs have a preset where they are transparent for the vast majority of samples; usually a VBR setting that averages somewhere between 160 and 200kbps (such as lame --preset standard, mppenc --standard, oggenc around -q5 or -q6).

Slashdot Top Deals

If a 6600 used paper tape instead of core memory, it would use up tape at about 30 miles/second. -- Grishman, Assembly Language Programming

Close