Analyzing YouTube's Audio Fingerprinter 116
Al Benedetto writes "I stumbled across this article which analyzes the YouTube audio content identification system in-depth. Apparently, since YouTube's system has no transparency, the behaviors had to be determined based on dozens of trial-and-error video uploads. The author tries things like speed/pitch adjustment, the addition of background noise, as well as other audio tweaks to determine exactly what you'd need to adjust before the fingerprinter started mis-identifying material. From the article: 'When I muted the beginning of the song up until 0:30 (leaving the rest to play) the fingerprinter missed it. When I kept the beginning up until 0:30 and muted everything from 0:30 to the end, the fingerprinter caught it. That indicates that the content database only knows about something in the first 30 seconds of the song. As long as you cut that part off, you can theoretically use the remainder of the song without being detected. I don't know if all samples in the content database suffer from similar weaknesses, but it's something that merits further research.'"
music ip? (Score:5, Informative)
From what I've seen, it's pretty decent audio fingerprinting, but I'm sure would be subject to the same limitations- if you remove the first 30 seconds of a clip- it would produce a very different fingerprint.
There's no reason to believe youtube isn't using this library or a derivative. There's also no reason to believe this result isn't intended. If the first 30 seconds of a song are missing- maybe that makes youtube confident that it could be considered fairuse.
Either way, I could imagine creating a fingerprint based on different sections of a song has the same problems doing an MD5 hash would- each fingerprint would be entirely different. If you don't just compare bit-to-bit, it'll be impossible to catch ALL permutations. And the fact is, that's a lot of computing power anyhow.
Re: (Score:2)
If the first 30 seconds of a song are missing- maybe that makes youtube confident that it could be considered fairuse.
Nope. The principle of CYA says that if there's any possibility of a lawsuit, nuke it from orbit.
Re: (Score:2)
Exactly.
Re: (Score:1, Redundant)
Re: (Score:3, Interesting)
If the first 30 seconds of a song are missing- maybe that makes youtube confident that it could be considered fairuse
Or if 30 seconds of additional blank footage were tacked on to the beginning?
And FWIW, there is a very valid reason for assumming they aren't using this fingerprint system: They already had their own in-house created system that they based off of their thumbnail maker program. It is also limited to within 30 sec of a clip if I recall.
Re: (Score:1)
There's also last.fm fingerprint library which is open source: svn://svn.audioscrobbler.net/recommendation/MusicID/lastfm_fplib
Re: (Score:2)
To be honest, I'd be fairly surprised if they used a method that boiled down to a hash for exactly the problem you point out. I would make a bet they either currently use, or will use, a
Re: (Score:1, Offtopic)
Re: (Score:1)
Rommel, I read your book, you magnificent bastard!
Name that movie, using other obscure movie references. Go.
Re: (Score:2)
Re: (Score:2)
Re: (Score:1)
Re: (Score:2)
Re: (Score:1)
Re: (Score:1)
TL;DR
Screw you guys, I'm going home.
This makes it pointless right? (Score:3, Insightful)
Re: (Score:2)
So if I only use portions of the song that aren't in the first 30s I'm home free? That seems silly, the system must still be under refinement or is only there to stop the most blatant offenders.
I'm inclined to believe the latter. If a video doesn't use more than about 30 seconds of a recording at a time, it's likely that the video's author attempted to use the work fairly [wikipedia.org]. I guess my video [google.com] got flagged because I opened with a vocal-cut version of one of the songs on which I was commenting.
Whew! (Score:5, Funny)
Re: (Score:3, Interesting)
There-in lies the rub with the "all information should be free" mindset...EVERYONE gets to look at it.
I think that is a feature, not a bug.
Re: (Score:1, Funny)
Which leads to the metaphysical question bugging us all: does a WHOOSH count as information?!
Re: (Score:2)
WHOOSHes are overrated.
In this case the reply was more interesting than the joke.
Slashdot brainstorm here (Score:5, Insightful)
Here's an idea. Start out the video with a useless narrative for the first thirty seconds "blah blah blah skip until :30 and ignore this intro blah blah" then start the music. That way everybody is happy. All google employees are too elitist to read slashdot, right?
Re: (Score:1)
That may not be the case as it may be too computationally intensive to process the whole five minute video that is uploaded.
Re:Slashdot brainstorm here (Score:5, Funny)
Re:Slashdot brainstorm here (Score:5, Funny)
Heh. I think people have already tried that.
Hi, I'm am amateur movie critic. Today I'm going to show you an example of poor film-making. blah blah blah ...
*Plays entire Star Wars: Episode I*
So, as you can see by the [cinematography jargon] and [screen writing jargon], this movie sucked and I hope you learn from it in making your own movies.
One week later:
"No! You can't take down my video. This is CLEARLY fair use, since I have OBVIOUSLY used it for educational commentary, and the entire clip was VITAL for showing how much Episode I sucked."
Re:Slashdot brainstorm here (Score:4, Funny)
Here's an idea. Start out the video with a useless narrative for the first thirty seconds "blah blah blah skip until :30 and ignore this intro blah blah" then start the music.
Like on the radio?
Re: (Score:3, Insightful)
...and then you realize, that YouTube changed the algorithm, and that its compression makes your song sound so shitty anyway, that you actually want all the uploads to be taken down. ^^
Re: (Score:3, Insightful)
I see where you're going with this: put an advertisement at the start :-)
Re: (Score:3, Insightful)
Even better - upload the video backwards.
Someone should make backward youtube plugin for firefox. It even might autodetect backward songs and play them properly.
Re: (Score:2)
pHash (Score:5, Interesting)
Re:pHash (Score:4, Interesting)
Any ideas how one would go about doing this sort of thing?
Yeah (Score:4, Interesting)
Music Genome [pandora.com]
Re:Yeah (Score:4, Informative)
Dear Pandora Visitor,
We are deeply, deeply sorry to say that due to licensing constraints, we can no longer allow access to Pandora for listeners located outside of the U.S. We will continue to work diligently to realize the vision of a truly global Pandora, but for the time being we are required to restrict its use. We are very sad to have to do this, but there is no other alternative.
If you believe we have made a mistake, we apologize and ask that you please contact us at pandora-support@pandora.com
If you are a paid subscriber, please contact us at pandora-support@pandora.com and we will issue a pro-rated refund to the credit card you used to sign up. If you have been using Pandora, we will keep a record of your existing stations and bookmarked artists and songs, so that when we are able to launch in your country, they will be waiting for you.
We will be notifying listeners as licensing agreements are established in individual countries. If you would like to be notified by email when Pandora is available in your country, please enter your email address below. The pace of global licensing is hard to predict, but we have the ultimate goal of being able to offer our service everywhere.
We share your disappointment and greatly appreciate your understanding.
Re: (Score:2, Informative)
Re:pHash (Score:4, Interesting)
The problem you're describing is known in the Music Information Retrieval (MIR) world as content-based recommendation (CBR). There are a number of ways to do it, but they're all based on measuring similarity.
The idea is that people perceive songs as similar based on the characteristics they have, which are termed features. By representing a song's features in a model you can compare the models to see how "distant" they are, and then choose songs from a set that are least-distant. The work that my research group is pursuing represents songs based on timbral features (MFCCs) and rhythmic features (bpm, pulse clarity, syncopation, etc).
If you're interested in the approach, see http://paragchordia.com/research/cbr.html [paragchordia.com]
Re:pHash (Score:5, Funny)
fucking pHashist...
funny not troll (Score:1, Funny)
Re: (Score:2)
Cool! The history of these algorithms, and of databases like CDDB, is kind of depressing to an open-source guy like me. It's great to see someone doing this as an open-source project. How stable is the algorithm? If I compute a pHash today, will it still be compatible with pHashes computed next year? Any pla
Tragic (Score:2, Insightful)
That cool tech like this is being used to prevent "piracy" instead of something more useful.
Re: (Score:2)
Put the horse before the horse shit, alright? Blatant abuse of copyright "law" caused a problem that was addressed by piracy, then in turn,the abusers of copyright "law" attempted to address THEIR problem with piracy.
And, your holier-than-thou attitude toward cigarettes goes hand in hand with the abuse of copyright. Some kid wants to smoke, but "THE MAN" has the badge, the gun, the club, and all the money to back up his dictates that the kid can't smoke.
Why don't you smoke a doobie and chill out with that
Re: (Score:3, Insightful)
At one end of the scale, you'll find those who think the world owes them entertainment. At the other end of the scale, you'll find those copyright squatters who think the world owes them a lavish living for sitting on dead men's works, and for acquiring a monopoly on distribution schemes.
On your same scale, throwing a kid in jail for wanting to hear music without paying the parasites is just about negative infinity.
Now that we have judged each other's positions, got anything constructive to offer?
How about
Doesn't really matter for most people (Score:3, Interesting)
Then a car commercial parody I made (arguably one of my better videos) was taken down because I used an unlicensed song. That pissed me off. I couldn't easily go back and re-edit the video to remove the song, as the source media had long since been archived in a shoebox somewhere. And I couldn't simply re-upload the video, as it got identified and taken down every time. I needed to find a way to outsmart the fingerprinter. I was angry and I had a lot of free time. Not a good combination.
The guy who wrote TFA is upset that his largely unviewed videos didn't pass an automated test.
My beef with the system is that when culurally significant videos such as the Chinese "Caonima [youtube.com]" get taken down because the song violates some copyright of a company I've never heard of on a song I've never in a million years think of buying.
Hope that link works, I had to copy it from Google since I can't even access Youtube anymore here in China.
Re: (Score:1)
Re: (Score:2, Insightful)
My beef with the system is that when culurally significant videos such as the Chinese "Caonima" get taken down because the song violates some copyright of a company I've never heard of on a song I've never in a million years think of buying.
So copyrights only apply to companies you've personally heard of and it's a song you'd buy? That's pretty stupid.
Re: (Score:1)
Re: (Score:1)
So copyrights only apply to companies you've personally heard of and it's a song you'd buy? That's pretty stupid.
I believe the point wasn't the significance of the company filing the complaint, but the content of the video being removed. If a video has a significant contribution, a larger company might be more willing to let a small infringement slide on the basis of good will, since they have other sources of income. A small company would be more likely to be zealous, since even small infringements represent a significant portion of potential income. Of course, that paradigm hardly applies in all situations, probably
Research? (Score:2, Insightful)
Why exactly does it merit any research? This is not riddle posed by Nature — people devised this device (ha-ha), and know all the answers perfectly already, they just don't want to tell you. You are not advancing scientific progress by figuring out somebody's scheme.
You may be advancing your own knowledge and skills, but calling it "research" has no more merit, than paparazzis' "research" into celebrities' lives...
Re: (Score:2)
So, anthropology is not research?
Re:Research? (Score:5, Insightful)
This is not riddle posed by Nature
This is one of the wonderful things about science: it doesn't matter where the puzzle comes from, the same techniques work to solve it.
Reverse engineering of this kind is one of the most useful areas of applied science, and it is as much research as any other area of scientific enquiry. It is frequently the case that there are many ways to find the answer to a puzzle, and this guy has chosen one of them based on the resources he has available. More power to him for demonstrating how good science can be used to discover what others want to keep secret.
away pedant! (Score:2)
Why exactly does it merit any research? This is not riddle posed by Nature... You are not advancing scientific progress by figuring out somebody's scheme.
Good grief. If you're trying to find out something that you can't just go look up at the library, and you're forming and testing hypotheses to do so, that could reasonably be called "research". Don't be so pedantic. :-)
(Anyway, it may well turn out that Nature is just stuff that someone already knows all the answers perfectly to and just doesn't want to tell you.)
Re: (Score:2)
The "research" only leads to working around the content filters, posting material the site operators explicitly do not want.
It would be more interesting if there was a productive application for this knowledge. Putting Rihanna songs on Youtube does not fit my idea of "productive".
Re: (Score:2)
It would be more interesting if there was a productive application for this knowledge. Putting Rihanna songs on Youtube does not fit my idea of "productive".
Why is it that so many people are so ready to condemn others because of their own lack of imagination?
My niece is a working print model and aspiring actress who has starred in a few very high profile music videos (the kind that get nominated for MTV's annual awards) and been featured in a few national commercials. She has put together a youtube channel to promote her career - the goal was to include a copy of very video work she has been in and title it so that her name was explicitly associated with the v
Re: (Score:2)
So, the copyright belongs to someone else, and they chose to use a lower-quality version. I don't see, how this gives your niece — although she appears in the video, she is not the owner of it — the right to post her own version...
Back to the point about advancing human progress, I don't think, a particular fashion model's success or
Re: (Score:2)
You are clearly unfamiliar with how hollywood works and the concepts of reels and portfolios.
Back to the point about advancing human progress, I don't think, a particular fashion model's success or failure have any effect on it...
Did I say she was a fashion model? Or are you just projecting your own pejorative attitude about the entertainment industry? Ironic you are so dismissive of the industry and yet so quick to defend an entirely bogus preconceived notion about some of their rights.
Re: (Score:1)
Did your niece receive ownership of the various content as compensation or put up the money for the production or something along those lines? I doubt it but there's always that chance.
What I find most likely, however, is that she was hired to perform a service and compensated according to a contract she agreed to. It seems fairly unrealistic that distribution rights were part of that compensation, and if that is indeed the case, she shouldn't be distributing no matter how personally beneficial she would
Re: (Score:2)
I would certainly find it lucrative if I could resell some of the software I've written under work-for-hire contracts, but I agreed to the conditions involving my employers retaining those rights in exchange for compensation.
Damn! Yet another poster without a clue as to how portfolios and reels are used within the entertainment and advertising business. Why do you guys think you know so much about something you clearly have zero knowledge of? And why are you all so fucking high and mighty about it too? Repugnant? Jesus!
Here's your clue - it is standard procedure to send copies of your reel - i.e. a dvd with significant examples of your prior work for hire to agents (when you don't yet have an agent) and casting directors.
Re: (Score:2)
So as long as somebody knows exactly how the system works, that's good enough for you? Fine, but that's not how all of us are wired [catb.org]. Google's knowledge of their audio fingerprinting scheme is useless to me if I want to know how it works and th
Yes, but who analyzes the analyzers? (Score:5, Funny)
And who fingerprints the analyzers who analyze the analyzers?
Re: (Score:2)
Simple. Make it a recursive loop.
In Haskell you would do it like this:
Of course this is verbose and not optimized, because I am still a Haskell amateur. And of course this
Re: (Score:2)
Hmm... This would give you an empty list. So it would definitely not compile, but complain about not all cases being handled.
But the missing case could easily be added with
Of course, you would then have to handle this is the code following the last line in "main" too.
Or you limit "n" to >0.
Great. (Score:1)
Re: (Score:1)
Fair Use (Score:2)
I'd rather lose the last 30 seconds (Score:5, Insightful)
An unfortunate result. The last 30 seconds of most songs are not usually as interesting as the first 30 seconds.
I wonder if he tried mangling the first 30 seconds at all. For example, keep the first 5 seconds, mess up the 6th and 7th seconds, and then continue on. Or perhaps adding in a base line that would be hard to hear. Or something at the high end of the audio frequency spectrum, to annoy all those teenagers while I listen to my free music in peace.
Re: (Score:2)
And to make your animals go crazy? And I mean pathologically crazy, as in If it were a human, it would need strong medication an a padded room.".
If you'd have to go through all that trouble... (Score:2)
...just to pass some filter, then you must think the song's very well-worth listening to.. which, to me, implies it ought to be worth buying.
If it's just some background music piece - I dunno, try another song.. plenty of royalty-free and even completely free ones (nope, they're probably not in the billboard top 100 right now - so sorry).
I'm more curious about the cases where there really IS fair use involved.. what happens in those cases.. do you get to hit a checkbox saying "I believe this is fair use, pl
Re: (Score:2)
I wonder if he tried mangling the first 30 seconds at all. For example, keep the first 5 seconds, mess up the 6th and 7th seconds, and then continue on.
If you're going to do that, you're going to ruin the song. If that's what you want to do it would be much more effective to sing it at the top of your lungs off key in the shower and put that in as your soundtrack instead. As a bonus, you're still violating copyright.
Cutting off the first 30 seconds (Score:2)
This hole doesn't really indentify a hole in the technology itself, just in the implementation. I'm more interested in hearing some audio that sounds the same while defeating the fingerprinting scheme. Much more interesting.
Re: (Score:1)
Hearing audio that sounds the same doesn't seem that interesting to me.
Being able to generate it, on the other hand...
Acapellas are flagged! (Score:1)
MusicBrainz (Score:2)
MusicBrainz works exactly as described.
Re: (Score:2)
the only reason to subvert the system is to do something illegal...
Wrong, wrong, wrong.
1: There are many fair use possibilities that this system could infringe on.
2: Copyright infringement's illegality varies depending on your local legal system.
3: The fuzzy areas of fair use can arguably be extended pretty far into mashups and remixing.
Re: (Score:1)
I get how noble (fair use just sounds noble) those reasons are, but why is it Google's responsibility to publish things they don't want to publish? Being big and really good at what they do doesn't remove their right to control what they serve.
Patent filed with explanation of fingerprinting (Score:4, Informative)
Re: (Score:2, Informative)
From the patent:
The feature vector thus consists of the mean and standard deviation of each of the trajectories (amplitude, pitch, brightness, bass, bandwidth, and MFCCs, plus the first derivative of each of these). These number
If you cut off the first 30 seconds.. (Score:2)
Its called fair use.
Auditude (Score:1)
You can test them out on myspace, I'd be interested to see how well they stack up in real world tests to YouTube's provider.
Rick Roll Tragedy (Score:1)
Re: (Score:2)
Re: (Score:2)
I once uploaded a vid with music by Ulrich Schnauss, published by Domino Recording Co. I don't know if they're a big label or not but they were cool with me using the soundtrack because they get to post an iTunes 'buy now' button and an amazon.com 'buy now' button on my movie's page.
On the plus side, an enlightened record company can use this as a means of getting free advertising and driving sales as long as they don't find the video objectionable.
On the negative side, who's to say that all record compani
Someone should do this for imeem.com (Score:3, Interesting)
imeem have been doing this for the last few years, and they don't use audible magic, they used the Snocap fingerprint system which apparently was good enough for them to buy Snocap. Their business model has always been built around using the content identification system to make sure the right people get paid for audio played on the site.
imeem is primarily used by people uploading and sharing audio, so using an audio fingerprinting system seems more appropriate than youtube relying on an audio fingerprinter for video content.
What about Shazaam? (Score:2)
If they start using whatever Shazaam uses, we're screwed. In any case I'm sure this is the start of an arms race in which the fingerprinter keeps getting more and more elaborate to counter the effects of people trying to fool it.
Obligatory cynicism (Score:1)
Any use that doesn't result in me getting obsene amounts of money is NOT FAIR!
</RIAA>
Youtube.. (Score:1)
foosic (Score:2)
Wrong tool...? (Score:2)
dude (Score:1)