Follow Slashdot stories on Twitter

 


Forgot your password?
Close
typodupeerror
Music Media

Audio Format Listening Tests Concluded 377

Posted by timothy from the good-time-to-bet-on-ogg dept.
Pointing to the conclusions of this listening study, nullity writes: "The results are interesting, and show a high variation in the performance of the various codecs on different musical styles. Ogg seems to work well on dance music, WMA8 on chamber music, etc."
This discussion has been archived. No new comments can be posted.

Audio Format Listening Tests Concluded More

Audio Format Listening Tests Concluded

Comments Filter:

  • WMA8 (Score:5, Funny)

    by af_robot ( 553885 ) on Tuesday July 30, 2002 @05:00AM (#3976809)
    Ogg seems to work well on dance music, WMA8 on chamber music, etc.

    Like requiem...

    • Re:WMA8 (Score:2, Funny)

      by Snork Asaurus ( 595692 )
      Ogg seems to work well on dance music, WMA8 on chamber music , etc.

      This might explain why I only think of WMA as being good enough for recording the sounds that I hear while sitting my "chamber" on my "throne".

      (no offence to those sounds intended).

  • While mp3s encoded at lower bit rates seem to have a tingly sound sometimes, almost every wma file I've gotten or encoded ALWAYS sounds like someone is blowing windchimes in my music. I can't stand it.

    In my opinion, 192kbps MP3 is the way to go, but then what do I know? I can't hear about 13khz.

    Chris
    • I have problems with some 192Kbps CBR MP3s, I find --r3mix usually sounds a lot better (and takes up less space), though sometimes I neeed to bump it up to ABR (usually around 220Kbps if it's a difficult piece).

  • Who cares about 64 kbps tests? (Score:5, Insightful)

    by splorf ( 569185 ) on Tuesday July 30, 2002 @05:03AM (#3976818)
    These tests are all at 64 kbps and most people use much higher bitrates for real music. I'd like to see comparisons at 128k bits minimum, and preferably 160k or 192k, which is what most quality mp3's are at, for direct comparison.

    • Re:Who cares about 64 kbps tests? (Score:5, Insightful)

      by keller ( 267973 ) on Tuesday July 30, 2002 @05:20AM (#3976859)
      People who wnat to stream audio of course! .K

    • Re:Who cares about 64 kbps tests? (Score:5, Interesting)

      by ProtoCat ( 452381 ) on Tuesday July 30, 2002 @05:22AM (#3976863) Homepage
      64Kbps is where the flaws of a codec are truely exposed. It's a great median between being too high to produce much results and too low where everything completely falls apart. You may not think any of this has any relevance to you as you're encoding above 128Kbps, but it actually does make a difference when you stress your encoder with a difficult piece of music.

      However, if the difference between sounding 'good' and sounding 'accurate' mean little to you, as someone who'd make an argument of 64Kbps tests being worthless would, then you really aren't the intended audiance of such tests. You can merrily use any of those encoders at 128-192Kbps without ever really noticing or caring much.

      I, personally, would like to see OGG1.0, MP3 Pro and WMA8 take on some real tough to beat codecs such as Dolby's AAC High-Complexity Mode (which no AAC freely available encoder supports, including QuickTime) and Sony's ATRAC3. But, that'd be kinda moot, because most people out there do not have access to those toys.

      For now, I'm content to just watch people hop around and proclaim whatever they want as king of audio formats while sticking to 256Kbps Fraunhoffer MP3 (archival purposes) and 192Kbps LAME HQ MP3 (general usage) as something both widely supported and pratically indistinguishable from the source. Even if AAC-HC and ATRAC3 were freely available, it'd take an awful large effort to wean people off of MP3 so far as support base and to migrate them to a new format. New P2P programs, new players/plug-ins (in some cases) and new hardware players. Not gonna happen for a while.

      • Re:Who cares about 64 kbps tests? (Score:5, Insightful)

        by nagora ( 177841 ) on Tuesday July 30, 2002 @05:39AM (#3976904)
        64Kbps is where the flaws of a codec are truely exposed.

        Running your car over a cliff is where the flaws in its safety system are truely exposed but I don't tend to drive over cliffs much.

        However, if the difference between sounding 'good' and sounding 'accurate' mean little to you, as someone who'd make an argument of 64Kbps tests being worthless would, then you really aren't the intended audiance of such tests.

        What do you mean by this? 64Kbits is worthless for listening to any music I own while 128 is good enough to not actually annoy me much of the time so why should I be interested in these tests? Are you saying that the intended audience for these tests are people that are not interested in the quality of the music they're listening to?

        TWW

        • Re:Who cares about 64 kbps tests? (Score:5, Insightful)

          by gleam ( 19528 ) on Tuesday July 30, 2002 @06:08AM (#3976961) Homepage
          You don't often intentionally hurl your car at 45 miles an hour into a steel box, either, but insurance companies do it all the time to see how well a particular car stands up to the abuse.

          Even if you don't knowingly take the insurance institute's results (or federal crash-test ratings) into account, the company selling you insurance does, and your premiums will be higher.

          To say "just because i'll never do something this way it has no merit" is silly. Performance in a 45-mile-per-hour offset crash will tell a car company how well it would stand to you accidentally bumping into the corner of your garage, or into the bumper of another car.

          Tests like this are important because they're indicative of performance at all bitrates. If you want to know WHICH codec will sound the best at 128kbit, you should look at which codec sounds the best at 64kbit--the two are likely to be the same.

          There are two intended audiences for this test: 1) people trying to decide which audio format to use for a stream (which are very often in the 32-64kbit range)

          and

          2) people who realize these tests can tell us much more than simply which codec performs best at 64kbit, and want to know how to maximize the quality-to-diskspace ratio on their own encodings.

          Hope this clears something up for you.

          -gleam
          • If you want to know WHICH codec will sound the best at 128kbit, you should look at which codec sounds the best at 64kbit--the two are likely to be the same.

            Any proof of that?

            TWW

            • Agreed. Even certain MP3 encoders very quite greatly, depending on the bitrates that are used. BladeENC really blows at less than 256 kbps, but shines at higher bitrates. Xing just blows all the time. LAME just rocks no matter what, and is especially nice with its VBR implementations. FIIS's officialy encoder is the best for lower bitrates.

              They aren't always the same.
        • What do you mean by this? 64Kbits is worthless for listening to any music I own while 128 is good enough to not actually annoy me much of the time so why should I be interested in these tests?

          I suppose he meant that since at 128Kbps all codecs perform quite well. Therefore personal preferences affect the results too much, i.e. some people like bass boosts and lots of treble, although this is not accurate in the sense that it differs from the original recorded signal.

          If 128kbps is 'good enough' for you then I too suppose you would fall into the sounding 'good' instead of sounding 'accurate' category.
          • If 128kbps is 'good enough' for you then I too suppose you would fall into the sounding 'good' instead of sounding 'accurate' category.

            If accurate is what you want you don't use any of these: just save raw samples! There is a balance in all compressed formats between good enough and accurate. Anyone reading this test is, by definition, looking for "good" rather than accurate.

            TWW

        • Running any car over a cliff will destroy it, regardless of safety systems. This is not a valid test of a car safety system.
    • 64kbps are more interesting when it comes to streaming music, obviously, so the test is valid for that purpose. I would love to get internet radio at something better than 64kbps 22khz, mono mp3s that doesn't eat bandwidth.

      But most importanatly, at which kbps does the codecs become equal? MP3s do sound a lot better at 192kbps, and surely will beat ogg at 64kps. The music won't be better with one format over the other, as long as the "I can't belive it is lossy" barrier has been reached. Which format will make the music sound as the original (all the others can be discarded now thank you), and of those, which does it at the lowest bitrate?
    • like to see comparisons at 128k bits minimum, and preferably 160k or 192k, which is what most quality mp3's are at

      Yes - mp3's. I thought both wma and ogg used lower bitrates because they were more efficient.

      A bitrate isn't a measure of quality - it's more like a measure of consumed bandwidth.
    • I'd like to know if there are and codecs that support constant quality but a variable bit rate?

      A codec with a target bitrate of 64k but maintains quality by channing between say 1k(for silence) and 100k through the streem would be nice.

    • The whole point of the test, and I say this without reading the article, was that with these codecs, you don't have to code at higher bitrates any more.
    • A better test would be to loop an encode. take a piece and encode/decode/encode/decode this will magnify the problems with the codec. granted encoding at a very low bitrate will do this also. but then some codecs (mp3) use different systems for different bitrates. and also the encoder has a really large part of it at least in the mp3 world. lame with the right settings is the absolute best mp3 encoder availabe short of the $25,000.00 hardware unit used by studios and radio station conglomerates. Yes it blows away every pay-for encoder on the market... hands down.

      so I do agree that the 64K test is a bit inaccurate, but it was for a reason and unfortunately the testers were not educated enough in what they were testing in order to perform an accurate and meaningful test.

      I would grade these results as valuable as that of an 8th grade physics student... there is some info there but nothing that makes a difference.
    • I actually haven't done many tests with Ogg 1.0 yet, due to a lack of time, but in the future, when portable devices are available that play Ogg, I really might be interested in ~64 kbps rates. Right now I have a Rio Volt and encode with lame 3.92 set at vbr -q 9. The files usually average a tiny bit under 128kbps. No, they aren't perfect when listened to with a decent sound setup, but they're not real bad.

      But in reality, 99% of the reason I bought the Rio Volt was to be able to take one or two CDs in my car and have a decent selection of music. My car has a pretty crappy sound system, and the whole thing is being played through a tape adapter which further mangles the sound, and to be honest, I really can't tell a difference when I'm driving with the noise of the road/wind/AC.

      If Ogg made it possible to have 64kbit files at about the same quality as the mp3s I have now, then I could fit twice as much music on one CD, which would put it at around 25+ albums. It'd be fantastic to be able to carry around my entire CD library in one of those 10 or 20 CD cases. Portable devices are where filesize reigns king.
    • Take a look around his site [ff123.net]. He's done other listening tests, including two [ff123.net] separate [ff123.net] tests at 128 kbps, with MPC [musepack.org] and AAC [aac-audio.com] highest in the first and MPC and Ogg Vorbis highest in the second.

      Note the tests at 128 kbps seem much more difficult to discern a clear winner without resorting to some statistical work.

  • Hmmmm (Score:3, Funny)

    by cca93014 ( 466820 ) on Tuesday July 30, 2002 @05:04AM (#3976821) Homepage
    Tests confirmed that attempting to encode "Aphex Twin" with any of these codecs caused the PC to tremble at a frequency that, when connected to a refracting laser stuck up Bill Gates' ass, had it spell out "we're all dead" on the nearest wall.

  • Hard disks get cheaper and cheaper, and 200 Gb hard drives will get common in a few months.

    Also, more and more people get a reasonnable bandwidth.

    So is there still any point in lossy compression codecs? FLAC's lossless compression doesn't compare with Ogg, but neither does the quality. FLAC preserves the original quality. Disco, chamber music, anything will sound great if the original was great.

    • I guess we still need lossy codecs, imagine how big those ipods become as soon as you have to carry around such a huge HD-monster. Besides, HDs use way too much energy for portable devices. And those Flash-Cards or MemorySticks are far from 200GB at the moment.
      • The iPods do use hard drives. Yet they still have 10 hour battery lives (according to Apple).
        • The iPods do use hard drives. Yet they still have 10 hour battery lives (according to Apple).

          That's because the good people at Apple who designed the iPod are very, very smart. They had Toshiba design a low-power, 2.5" hard disk that spins at 5400rpm, and only access the hard drive to buffer files into memory while listening. When the drive isn't in use, it's locked.

    • Whilst 200Gb hard drives will probably be common in desktop machines ina few months, it'll be a *long* time before one fits in my back pocket.

      Furthermore, hard drives have moving parts, moving parts mean increased power consumption. For low power devices we'll want static media, which is lots more expensive, and so we'll want to use less of it.

      Don't get me wrong - there will always be a place for lossless compression / uncompressed formats, but likewise, there is still a time and a place for decent lossy compression. And at the end of the day, when you are sitting on a crowded bus using headphones the size of smarties, I'm not sure many people can tell that their music isn't perfect quality...

    • Re:Do we still need lossy compression? (Score:5, Insightful)

      by rseuhs ( 322520 ) on Tuesday July 30, 2002 @06:24AM (#3976987)
      Have you done a blind test?

      Do the following:

      Get your great .wav, encode it to 96kbps .ogg and copy both files 5 times on to your xmms-playlist.

      Then move the playlist up with Alt+LMB so you only see the playlist controls. (Or configure your windowmanager in a way you can have windows on top of your active window) Of course you have to make sure the main window doesn't reveal any information either.

      Choose "Misc -> Sort -> Randomize"

      Use a pen + pencil to guess what piece of music is what.

      If you really can tell the difference (I can't) repeat with 128kbps.

      Why should anybody care about lossless compression if you can't hear the difference?

      • Why should anybody care about lossless compression if you can't hear the difference?
        Because it depends on a lot of factors; the audio you're encoding (some might be fine, others might show nasty artifacts), the audio equipment (I could do the test and think it's fine, but I'm using taped-together £10 cheapo headphones, and the guy with a £300 hi-fi setup and a machine that doesn't drown out half the frequency range could well notice a huge difference), and on the individual (not everybody hears the same frequency range, or even pays the same level of attention).

        What we really need is some sort of audio quality metric; take each encoding, compare with the uncompressed original, determine how much it differs.
      • well, if one comes to the conclusion that he can not tell the difference he can take the encoding that uses the least amount of hard drive space, and use that encoding from now on, to save on space. it may be only a small amount, but even "not being able to tell the difference" can help out with that.
        • well, if one comes to the conclusion that he can not tell the difference he can take the encoding that uses the least amount of hard drive space, and use that encoding from now on, to save on space.

          Great...except if you later beg, borrow or steal (er, or buy!) better headphones or speakers you may spend a signifgant portion of your life re-encoding your "couldn't tell the differnece" files. (I "had" to do this after buying a very nice set of headphones)

          Also if you (possabilly illegally...possiabbly legally) share any of this encoded with your friends, be ready for them to give you an endless amount of crap...even if their stereo is worse then yours :-)

      • Why should anybody care about lossless compression if you can't hear the difference?

        Maybe you don't, because you can't hear the difference.

        But a lot of people can. And it's more evident when you get better quality gear to output the audio.

        What, you can't tell the difference between an MP3 and CD audio on your computer? Gee, I don't suppose that could be influenced by the crappy soundcard (probably built into the MB or a Creative Labs card) or the crappy speakers (insert virtually any speaker system that's computer oriented here).

        Play it either on a real piece of electronics w/ a good decoder and amp stage or a good sound card plus a good amp through good quality speakers (which, again, pretty much excludes anything designed for computer use, as well as the crap you can buy at Best Buy, Circuit City, etc.) and virtually anyone can tell the difference.

        Yes, as you get higher bitrates the differences become fewer and fewer until only the "Golden Ears" can hear the difference. But you have to get to that point first, and 64 kbps or 128 kbps just doesn't approach that point with current compression methods.
        • So with special equipment you can hear the difference.

          What was your point again?

          ogg @ 64kbps is still much better than radio and most people are very happy with radio.

          Oh, and you do realize that what is on a CD is not some kind of sacred "original", it's just a digitalized version which is just an approximation to the real thing - just like ogg and mp3.

          Even if I could hear the difference (which I can't) I would have to ask wether it would really makes a difference. Is a song encoded in ogg really "worse" than the same song on CD? In some listening tests mp3s were getting better notes than .wavs (see my other post), so maybe I should only listen to encoded stuff because it's better than wav?

          This stuff just reminds me when CDs were introduced. The place was crawling with people who claimed to be able to hear the difference (actually I believe them. A CD lacks the cracks you sooner or later will have on vinyl), with good equipment, yadda, yadda, yadda.

          Now, the very same people (at least it looks like this) are defending the CD versus mp3.

          You will be assimilated :-)

  • This isn't terribly surprising, as it was already known that the different formats have different frequency responses. More specifically, the way they compress the music dictates what frequencies are cut out. MP3s, for example, are notorious for removing high and low frequencies from music - not a big deal for casual listeners, but those with high-end stereo systems will definitely notice the lack of high overtones, and the "flat" low end bass response. WMA sort of sounds like certain frequencies are cut from the raw audio, leaving the rest to fill in as sort of an approximation of the original full sound - it sounds hollow and "chime-y". Ogg has its defining sound characteristics as well. Thus, it isn't surprising that different styles of music sound better encoded in different formats, as different styles take advantage of different frequencies. Rock music has high frequency cymbals and low frequency bass drums and guitars, as well as a very full mid-range, so a well-rounded encoding system works well. Classical is somewhat more compressed, as a result of the physical limitations in terms of sound reproduction of the instruments, so to the undiscerning ear, a format with especially good mids will suffice. The examples go on and on, but the point is that different tools are needed for different jobs - if nothing else, this study shows that having having a number of encoding tools on hand is actually a good thing. When you look in your tool box, you've got more than a couple Phillips-head screwdrivers - you should have enough tools to deal adequately with any job. The same applies to music.

  • Should compare Ogg as a single entry (Score:3, Insightful)

    by Russell Coker ( 125579 ) <russell&coker,com,au> on Tuesday July 30, 2002 @05:18AM (#3976852) Homepage
    Let's assume that anyone who likes Ogg and is seriously into music will compress their music with both Ogg variants and use the best variant for each file.
    Therefore we should also consider taking the best of the two results and comparing it to mp3.
    From a quick look at the results it appears that Ogg will still be edged out by mp3 when analysed in such a fashion, but it's much closer.
    Also a test on several bit rates would be useful.
    • Let's assume that anyone who likes Ogg and is seriously into music will compress their music with both Ogg variants and use the best variant for each file.
      Even better, let's assume that anyone who is seriously into music will compress their music with everything and use the best variant for each file.

      Kinda defeats the purpose, eh?

      --
      Damn the Emperor!
    • Let's assume that anyone who likes Ogg and is seriously into music will compress their music with both Ogg variants and use the best variant for each file.
      Therefore we should also consider taking the best of the two results and comparing it to mp3.

      No we shouldn't.

      You would be chronically distorting the results by merging both Ogg variants. What you would end up with is MP3Pro vs. some super-Ogg thing that doesn't actually exist.

      They should be tested seperately. If people are going to do what you suggest then, yes, they will get the best. The problem is that you're making additional assumptions about usage which will serve no other purpose than to boost Ogg's results.

    • No, those of us who are seriously into music and audio quality will buy bigger disks and compress with a lossless encoder like flac.

  • I guess grip will have to use Genre info from CDDB to decide what to encode the the files as now. I wonder if you coudl set up something to optimize individual tracks. Like scan a wav and pick the best codec for the frequencies used in the audio.
  • I'd be interested to know how these codecs perform when streaming things like news or talk radio or foreign language lessons. Clarity at a low bandwidth would support a lot of simultaneous listeners from a low-end server. Clarity at medium bandwidth could provide the extra sound quality needed for something like language learning/practice, again from relatively modest servers.

    • Take a look at Speex [sourceforge.net], an open source codec project aimed at speech compression. It uses Ogg for its file format (but don't confuse it with Vorbis). The quality's pretty good already.

  • MP3PRO not MP3 (Score:4, Interesting)

    by Anonymous Coward on Tuesday July 30, 2002 @05:32AM (#3976891)
    I noticed a number of confused posters here... The tested codecs were AAC/MP3PRO/OGG/WMA, not MP3. Had mp3 been tested, it would have lost every round as all of the tested codecs are vastly superior to plain MP3 at this bitrate.

    It also should be noted that the only two samples that WMA beat OGG at (indeed the only ones that it didn't totally flop on) were two very simple samples that are demonstrations of two differnt weaknesses in the current revision of vorbis. Orignally the results page had some very interesting commentary from Monty on this, but it looks like it got pulled.

    With the exception of those two samples, OGG clearly won. Even including those, it was only beat out by MP3PRO by a small margin. When you factor in that MP3PRO isn't available at anything but such low bitrates and that it's substantially more propritary then MP3, it seems like pretty much a no-contest.

    • Yes, look at the SCORES (Score:3, Interesting)

      by tweakt ( 325224 )
      OVERALL RANKINGS (12 SAMPLES)

      mp3pro 49.00
      oggq0 44.00
      ogg64 40.00
      wm8 24.00
      aac 23.00

      The AC above me speaks the truth. mp3PRO has no hope of gaining enough market share to become a worthy competitor. It's a very proprietary extention to MP3. OGG being open source and free (as in beer) has clear advantages for hardware vendors (where it really counts). Lets hope the codec is easy to embed into portable products.

      I want my Portable OGG CD Player! I'll buy the first one that comes out. Could you imagine? Twice the capacity of normal players and it STILL sounds better (or same capacity truly indistinguishable from CD -- at only 128k). Right now I have to encode my mp3's at ~180-220kbit to get something acceptable. =/

      • Re:Yes, look at the SCORES (Score:4, Interesting)

        by micromoog ( 206608 ) on Tuesday July 30, 2002 @09:19AM (#3977444)
        mp3PRO has no hope of gaining enough market share to become a worthy competitor. It's a very proprietary extention to MP3.

        mp3PRO has one very specific advantage over all the other formats on the market-share front. It has the characters m, p, and 3 in it. Everyone has heard of mp3, and people who don't care about the open source cause (read: the vast majority of people) will buy an mp3PRO device way before considering an Ogg Vorbis device.

        As I've said before, name is really important when marketing comes into play. And Ogg Vorbis' name simply blows.

      • Twice the capacity of normal players and it STILL sounds better (or same capacity truly indistinguishable from CD -- at only 128k).

        Vorbis 1.0 does sound amazingly good at ca. 128 kbps (VBR -q 4). That's what I've been using lately for CDs that I rip. But it's not "indistinguishable from CD" in all cases. On at least one song ("Feed My Hungry Soul" by the Lords of Acid), I can differentiate Vorbis -q 4 from the original in ABX testing. And I'm not a trained listener, and not using high-end equipment.

        I urge everyone to encode for themselves, using their favorite music CDs, and decide what works best for them. Some people are very sensitive to the lossy stereo separation that Vorbis RC3 and 1.0 employ at low-to-mid-bitrate settings. I was able to hear this clearly on several of the samples in the 64kbps test, though I'd never noticed it at higher quality levels.
    • When you factor in that MP3PRO isn't available at anything but such low bitrates and that it's substantially more propritary then MP3, it seems like pretty much a no-contest.

      Except that to get the best quality with Ogg I have to encode in one of two ways depending on the type of music.

      With MP3Pro, there is only one.

      For the music purists and geeks, this would matter. But for everyone else on the planet they'd rather have one encoding that does well.

      It would appear that MP3Pro is pretty close to that.

  • Multi-Codec Codec anyone? (Score:4, Interesting)

    by BlackGriffen ( 521856 ) on Tuesday July 30, 2002 @06:34AM (#3977008)
    Considering that different codecs do better at different music w/ different frequency spreads, who else thinks that the next generation of audio codecs will be multi-modal; in effect, be several codecs in one. Then have each codec specialize on certain types of music. Perhaps even have them run in an advanced mode where they do a frequency analysis of whole songs, rather than just using genre, to automatically select the best codec for the job. Perhaps even use different codecs for different sections of the song. That would definitely help songs like Bohemian Rhapsody and orchestas with movements, etc.

    Would this be too time consuming to implement or what?

    BlackGriffen
    • Using different codecs for different parts of a single song would be technically possible, but I don't think it would be desireable.

      Different codecs have different sound qualities, because of the different ways in which they discard information in order to achieve their lossy compression. So I think that using different codecs within the same song would result in a pretty wide variation of the way music sounded within a single compressed song...

  • What about VBR? (Score:2, Insightful)

    by xA40D ( 180522 )
    Okay, so everybody has pointed out that a 64k bitrate is useless for their needs. Some claim 128K is the best, others 192K. Personally I prefer a variable bitrate - indeed OGG encodes VBR by default.

    But is their some fundamental reason why nobody else insists on VBR?

  • Outiers skewing the results? (Score:4, Interesting)

    by Outland Traveller ( 12138 ) on Tuesday July 30, 2002 @07:18AM (#3977089)
    Looking at the data, it looks the two samples where Ogg performed poorly ended up being encoded at a significantly smaller average bitrate than any of the other encoders.

    The table at the end lists LiszBMinor with an average ogg bitrate of 45 and BachS1007 with an average bitrate of 47. Since the other codecs encoded those samples at a bitrate 64 or higher, this may explain the results.

    The results may point to a flaw in Ogg's VBR login rather than in the lossy compression scheme.

    • Re:Outiers skewing the results? (Score:4, Informative)

      by jonathan_ingram ( 30440 ) on Tuesday July 30, 2002 @09:12AM (#3977406) Homepage
      If you read the thread on HydrogenAudio [hydrogenaudio.org] (which is the message board where most of these tests / codecs are discussed), you'll find the following information from Monty, the lead developer of Vorbis:
      Ogg had a very low bitrate (in the forties) on all the classical samples, which is the way it should have been (Classical solos with their deep noise floors and simple harmonics are relatively easy). But the real reason Ogg scored so low in both (and Beauty Slept as well) was a) the tuning behind noise normalization is still not perfect. This is the very first release of that feature, and the test found flaws b) also the first release of new, more aggressive stereo modes and I think that they too need more analysis infrastructure driving them.


      I expect Ogg's performance on Liszt and Bach to be very subpar NN performance. The poor performance on BeautySlept and Waiting was most likely insufficient stereo analysis. Ogg had the infrastructure to win those four samples, but the encoder didn't know how to do it yet (because I didn't know it would be necessary).

  • A missed test? (Score:2, Interesting)

    by Alice Terry ( 233411 )
    In my experience the real test of a codec is how well it handles soft female vocals with accompaniment. For example, Jeri Southern, Nancy Wilson and Dinah Shore all have silky voices which many codecs tend to "erase". Few codecs can handle the subtle shades of a female voaclist with a soft delivery. What was once a foreground voice suddenly is relegated to the background as though a little mouse were singing. Needless to say, that is not what is wanted.

    I would hope that in future tests that samples of female vocalists are tested as well. Oh, and I'm talking standards and jazz vocals, not dance or metal. It is the soft subtle voice which is quashed by most codecs. Mens voices do not fare as badly because they tend to have higher harmonic content and a louder delivery, although some singers like Tony Bennett might not do as well.

  • I haven't seen anyone else mention this yet. At the end, he gives a table of the bitrates for each song for each codec. The one with the greatest variation appears to be oggq0. I noticed that for the songs where that codec did well, the bitrate was much higher, and where it did poorly, it was much lower. I don't realy understand how the bitrate is chosen, but as I understand it, the encoder chooses it automatically somehow, right? I wonder how effective that really is.
  • The 60 files (12 songs * 5 formats) were all compressed at between 64 and 74 kbits/sec - except LiszBMinor and BachS1007 for OGG q0. They were actually stored at 45 and 49 kbits/sec respectively. No surprise the testers rated them low.

    -
  • Does anyone else feel it would have been nice to see Red Book CD audio (16-bit 44.1KHz uncompressed) compared as a control? Seeing how "pure" audio compares to these compression standards could make the results seem more objective.
    • Comparing 'pure audio' to the compressed version is inbuilt into the test methodology. I suggest you download the program used for the tests, and try it out before commenting further. Hopefully this will give you some semblance of knowing what you're talking about.

  • Ogg on Quicktime (Score:3, Insightful)

    by heroine ( 1220 ) on Tuesday July 30, 2002 @08:37AM (#3977305) Homepage
    Been using Ogg/Vorbis/Squish on Quicktime for a year. The Ogg/Vorbis/Squish codec got much better between 1.0rc2 and 1.0. At 128k it's already better than mp3 and the managed bitrate encoding is faster than the hard drive can read. The real value is of course, the ability to read these encoded files as long as there is UNIX. Mp3 is going to die and when it does there won't be any appliance makers interested in paying the $10,000 royalty to support mp3.

  • I use flac (lossless compression). What was the problem again ? ;)

    Seriously, the reason I use flac, even though it takes up a shit load of space is that in the future, inevitably we will have more space to store everything. When we do, thousands will be cursing their crappy mp3s that they ripped at 128 to save space.

    Of course, ahem, if you kept your original CDs to rip from then you can just re-rip them to flac or another lossless compression then, but still, why do it all twice ?

    graspee

  • Personally, I can hardly tell the difference between MP3 and CD and my old vinyl. When I play something on CD, then MP3, then CD and listen carefully for the 'crappy bits' I can hear them - but they don't bother me in the slightest when listening to them.

    Are my ears just a bit shite? Are most of you guys able to tell the difference - or are the audiophiles just more vocal?
  • I think, that for anyone who would actually be interested in which codec does best on which kind of music, it's a moot point, since by now they delete anything below 128kbps on sight
  • I am truly impressed with the selection of music on this test.

    For those of you that don't know, Opeth's Blackwater Park is one of the most earth-shattering CDs I've ever been privy to witness. They are my favorite band. Check out the last song, Blackwater Park. Wow.

    You can get a taste of them in #mp3_metal or #mp3_death in dalnet. type @locator opeth blackwater park and you'll get plenty of results.

    Caution - very harsh grunting vocals. May take some time to get used to, but their musicianship is absolutely brilliant.

  • Newer algorithms of limited use to many (Score:3, Insightful)

    by dh003i ( 203189 ) <dh003i&gmail,com> on Tuesday July 30, 2002 @09:43AM (#3977568) Homepage Journal
    To thsoe of us who just want to listen to music on a PC, the newest greatest best algorithms are always good (mp3pro, oggs, wma8). But for many, the goal is to put that music on a MP3Player and listen to it anywhere. I'll summarize the support of these various codecs by MP3Players, as well as mention whether or not my MP3Player (RioVolt SP100) supports them.

    MP3PRO -- little support on MP3Players. Not supported by RioVolt SP100.

    Oggs -- little/no support on MP3players. Not supported by RioVolt SP100.

    WMA8 -- little support on MP3players, though many support older WMA's. Not supported by RioVolt.

    So, in summary, all of these new formats are completely useless to me on my MP3Player. The one option they present is if I want to encode something in two formats -- one for my computer, and another for the MP3Player.

    Personally, I think more work should go into fractal endcoding, as most music has fractal patterns in it (especially Bach's music).
    • Please define little support since WMA8 is supported by a bunch of devices less than 2 years old through a firmware upgrade. I've tried looking for a RioVolt that didn't support and the only ones were the Coke brand and the SP50. So there you go, the vast majority of players support both MP3 and WMA.
      • Duh, they all support MP3. But few yet support MP3Pro. Even if they did support MP3Pro, that's also of limited use as we only have access to 64kbps compression w/ MP3Pro -- which sucks.

        NONE support OGG that I can think of. OGG is the best format I've come accross -- and I'm speaking from listening experience here.

        As for WMA8 -- so what? WMA8 sucks anyways. I'll be the first one to tell you I gave it a try, but after trying it out, I realize that for the most part its crap: like everything else produced by M$.
  • Just worth mentioning that the single best audio format is "LIVE" - it sounds so good, it'll seem that you can reach out and touch the performers (please don't).

    I know that the thread is about compression formats, but hey - go to a bar/club with "LIVE" music, pay $10 at the door, have a drink and a good time.

    Hopefully, the guys playing are getting a percent of the door, and they'll be happy to see you in the audience. Feel that bass!

  • Check out Hydrogen Audio [hydrogenaudio.org]

    Its pretty much the best audio discussion you can find on the 'net.
  • This is an interesting and relatively well done test (although it appears that the listeners knew which format they were listening to, so it wasn't truly double-blind, and a anti-MS and pro-Ogg bias can't be ruled out).

    However, some discussions seem to be focusing on this saying AAC is bad or WMA is bad, when really it refers to the particular implementations in codecs of those formats.

    For example, the Apple MPEG-4 AAC-LC encoder was used for AAC. This is a Low Complexity version of the format. Also, the Apple encoder has a strange limitation where it automatically converts 44.1 stereo to 32 stereo at that data rate. This isn't required by the AAC format. Other AAC encoders yield MUCH better results, and beat MP3 Pro in double-blind testing. I haven't seen any double-blind comparisons between AAC and Ogg.

    Also, the WMA8 encoder is due to be replaced by the backwards-compatible WMA9 in early September. Of course, there may well be improved versions of the other encoders by then as well.

Slashdot Top Deals

In the realm of scientific observation, luck is granted only to those who are prepared. - Louis Pasteur

Close