YouTube Has 1 Billion Videos With Closed-Captioning (But Not All of Them Are Accurate)

Over a billion videos on YouTube are accessible to viewers with difficulties in hearing, thanks to the video giant's automated captions, it said Thursday. From a report on Variety: That certainly sounds impressive -- except when you realize that many of the site's automatically generated captions aren't completely right. The Google-owned video giant first launched captions back in 2006, and three years later introduced automatic speech recognition to add closed-captioning to YouTube content. Today, YouTube users watch video with auto-generated captions more than 15 million times per day. But the system is prone to errors. For example, the trailer for Amazon Studio's Oscar-nominated "Manchester by the Sea" (at this link) includes numerous inaccuracies in the auto-transcribed captions, sometimes to hilarious -- not to mention frustrating -- effect.

So we're talking Auto Generated Bad Lip Reading?

[DickBreath]

> thanks to the video giant's automated captions, > That certainly sounds impressive -- except when you realize that many of the site's automatically generated captions aren't completely right.

I know robots are taking over jobs. But put those two statements together and this sounds like auto-generated bad lip reading.

Now if someone could only implement all possible bad lip readings, and then auto-rate them for hilarity, we would be onto something.

Re:

[s1d3track3D]

Thank you for submitting your video RickDeath, we will get right on close captioning it.

Re:So we're talking Auto Generated Bad Lip Reading

[cayenne8]

Yeah, the auto generated CC is pretty bad.

But I did read, that it *IS* very much worth your while to put accurate CC on your videos, as that it supposedly highly figures into your Google rankings.

I found that after I transcribed my videos, my rankings did shoot up higher on plain old Google searches and I think also on YouTube suggestions, etc.....so, looks to be worthwhile to do if you want max hits.

Re:Lip Reading?

[hackwrench]

Why bad lip reading? Why not your basic garden variety bad speech recognition?

Re:

[swillden]

Why bad lip reading? Why not your basic garden variety bad speech recognition?

https://www.youtube.com/user/BadLipReading

I've noticed a lot of errors in 'Downfall'

[Hussman32]

Something is clearly wrong with the translations of the Downfall [youtube.com] videos. Sometimes it's about SAP, sometimes it's about the World Cup, but my limited German tells me it's about the fall of the Third Reich.

Re:

[XxtraLarGe]

Something is clearly wrong with the translations of the Downfall videos. Sometimes it's about SAP, sometimes it's about the World Cup, but my limited German tells me it's about the fall of the Third Reich.

We're living in the postmodern era. The interpretation is left up to the viewer!

Re:

[gnick]

Bad closed captions allowed me to grasp the idea of most scenes of a picture I wanted to watch.

When I'm watching something in a language I don't understand, inaccurate CCs are nearly as good as accurate ones, as I have no idea that they don't match up. I prefer the original language with subtitles because the dubbed audio lacks the actors' inflections plus I don't like it when lips and words don't match. I often watch movies or TV programs with subtitles on, but avoid automated CC - When my eyes and ears don't match the conflict distracts me.

...the movie keeps playing and you get to understand who is the hero and who's the villain, at least.

I can typically do that with the film muted, can't you?

Re:

[PPH]

https://www.youtube.com/watch?... [youtube.com]

Goes for comments too...

[s1d3track3D]

site's automatically generated captions aren't completely right

Maybe they are generating the comments as well

Re:

[DickBreath]

Maybe Slashdot is what was used for beta testing auto generated comments.

Re:

[aevan]

I use CC even in languages I understand. Sometimes it's the background noise or multiple people talking (in the movie, or kids running around irl), sometimes the speaker mumbles, sometimes I have the volume low/muted, and sometimes they just speak so damn slow and I'm feeling impatient. I also like when the CC is in another language so as to give to alternate impressions of the dialogue (though if the 'sub' is bad it drives me nuts 'that isn't what they said!' :P ).

So by all means, continue doing CC.

Good enough for me.

[laserhead]

And I don't need to pay extra money for that.

But not all of them are accurate

[DontBeAMoran]

Task one undergarment.

Re:

[Tablizer]

Aw, Yorb aces Arbee long "2S".

about as good as google search.

[fish_in_the_c]

I recently watched a video with closed captioning on.
'stan fortuna school of the eucharist'
lets just say google search doesn't think eucharist is a common term and has an especially hard time with it when it is a quickly spoken rap song with a Hispanic accent.

It was pretty funny what they translated it too.

It did leave me wondering if there should be a mechanism to tell them the words are wrong and really wrong.

Crowd sourced option

[Immerial]

There is a mechanism to tell them the words are wrong. YouTube has a crowd sourced option for subtitles called "Community contributions" that you can turn on as a content owner. You can make it available for anyone to sub your videos http://www.youtube.com/timedtext_cs_panel [youtube.com].

Well you would choose Manchester By the Sea

[hey!]

I grew up in Boston, and when I go back to the old neighborhood it makes me wonder how people understand me at all. Speech recognition programs never work for me.

Re:

[farble1670]

Don't worry. You are a special, unique snowflake. If people can't understand you, they are racist.

Re:

[hey!]

Don't worry. You are a special, unique snowflake. If people can't understand you, they are racist.

Actually people who don't understand me are usually idiots. Some of them are blockheads. Funnily enough racists understand me fine, they just don't like what I have to say.

couple of things

[Quirkz]

I've lost the link, but someone recently mentioned an intentionally humorous duo who:
1) write a skit, perform it, and upload to Google
2) let captioning take its best stab
3) use the captioning as a new script, and re-record the scene
4) upload and re-caption
5) record a third time, with even weirder dialogue

Then they splice it all together, and you get to watch the degeneration of language as iterative captioning makes everything nonsensical.

My wife and I also tend to watch a lot of TV when the other wants qui

Re:

[aevan]

Sounds like a variation on those 'bad translation' humor videos: use online translators to translate through several languages, before ending back in English and using that. Like Bohemian Rapsody [youtube.com].

I wonder...

[XSportSeeker]

From my personal experience, I can't help but wonder if there is even ONE out of all the auto-generated captions that is accurate at all. :P
Have you guys ever seen one? I mean, a few mistakes are ok... but so far I haven't seen any video that had auto-generated captions that was even understandable at all. More like a mish mash of guesses.
Which is great for comedic effect I guess, but not so much for viewers with difficulties in hearing.

Re:

[Harlequin80]

I dunno I use the auto CC all the time. Some of the guesses are terrible but normally seems to have the biggest issues around random nouns. So assuming that I know what the topic of the video is I can use the CC, I just have to substitute the proper noun at the right time.

I tend to use it when watching a technical video with a single talker and I don't actually want the sound on for what ever reason.

Accuracy?

[twmcneil]

Accuracy is not that important to me. We all are pretty used to inaccuracies while texting now. What is important to me is the synchronization. If the captions follow the actual speech by more than just a bit, it makes it hard for me to follow as I lip read in addition to reading the captions. Lip reading is often ok by itself, but with movies and TV, the speakers face is not always pointed to the camera or there might be something covering the speakers lips. That's when cc comes in handy.

Re:

[fish_in_the_c]

I really don't think this thing is using lip reading ( I could be wrong does anyone have a source). I'd guess it is using the same servers as google voice search and the type of errors it has seems consistent with that.

Not accurate

[TWX]

Yeah, the Swedish Chef video was quite wrong...

Make perfect the enemy of good.

[140Mandak262Jamuna]

It is true in software engineering, and it is true everywhere.

Perfection is the goal. But doing better than current version is the shipping criterion.

Auto captioning is better than no captioning for hearing impaired.

And human captioning is not perfect. I remember watching Lion King with closed captioning turned on and they had missed a crucial "o" in some dialog that had the word "count".

Re:

[140Mandak262Jamuna]

Yes, that was the line. Zazu's lines in the song "I'm going to be king". So you noticed that too? You think it was an intentional mistake?

Not all are accurate

[argStyopa]

Not always a bad thing.

Bad Lip Reading are far more entertaining than the actual text of the presidential debates, for example.

yeah

[Feneric]

Anyone who's actually watched YouTube with captions will know almost immediately what I'm getting at in the subject. Saying the system "is prone to errors" as above is being very kind. Amongst its many errors are frequent phantom occurrences of the word "yeah". While the phantom "yeah" instances are more funny than anything else, many of the other errors are much worse. Amongst other problems they have been known to convert a family-friendly video into something that no longer fits that description.

But Not All of Them Are Accurate

[nitehawk214]

Are any of them accurate? Can you manually enter captions?

Undertake this discourse, actually types. [youtube.com]

Youtube has a potty mouth

[ukoda]

For some reason Youtube thinks that people speaking with a New Zealand accent swear a lot. I was testing the Youtube product tutorial on an Android product which, unlike PC browsers, has the closed captioning on by default. A lot of the technical terms, spoken with a Kiwi accent, were being captioned with obscene words. When I recovered from laughing at just how rude it was being I warned our marketing team that made the video. They were mortified and suddenly had a large task of checking and removing the computer generated captions. It turns out all of our SFW videos had NSFW captions.

Can't transcribe a Boston accent

[RogueWarrior65]

Clearly, these algorithms don't know what a bubblah or a blinkah or a clickah is.