Slashdot is powered by your submissions, so send in your scoop

 


Forgot your password?
Close
typodupeerror
×
Books Media Software Technology

reCAPTCHA Hard At Work, Rescuing Fading Texts 112

Posted by timothy from the strange-confluence dept.
sciencehabit writes "Computer scientists have developed a program, called reCAPTCHA, which is being used in lieu of CAPTCHA by several sites, to help digitize old books and newspapers. The reCAPTCHA takes entries from old and faded texts that optical scanners and digital-text readers have trouble with. So every time you solve that string of crooked letters, you may actually be helping historians digitally reconstruct a page from the 1908 New York Times." The Science Now story links to the longer and more informative article at Ars Technica. (We last mentioned this program last year — and now it's good to get some sense of how well it's working.)
This discussion has been archived. No new comments can be posted.

reCAPTCHA Hard At Work, Rescuing Fading Texts More

reCAPTCHA Hard At Work, Rescuing Fading Texts

Comments Filter:

  • Not new (Score:4, Informative)

    by JazzyMusicMan ( 1012801 ) on Thursday August 14, 2008 @09:07PM (#24609239)
    Ticketmaster and other sites have already been doing this for a while. Go to ticketmaster and search for tickets, you'll see two words. One is known and the other is unknown. If you don't believe me, try to guess which one they know and misspell the other one on purpose (or don't, this is for historic posterity =) )

    • Re: (Score:3, Informative)

      by Dachannien ( 617929 )

      So is the US Patent and Trademark Office, as part of the process of using PAIR [uspto.gov], the Patent Application Information Retrieval system, which lets the public look at information about patent applications that have been published.

    • Re: (Score:3, Funny)

      by felipekk ( 1007591 )

      Facebook uses reCAPTCHA. I guess you can make something useful out of the millions of useless teenagers wasting their time on Facebook.

      • Re:Not new (Score:5, Funny)

        by grahamd0 ( 1129971 ) on Thursday August 14, 2008 @09:45PM (#24609587)

        Facebook uses reCAPTCHA. I guess you can make something useful out of the millions of useless teenagers wasting their time on Facebook.

        That's not fair.

        Plenty of useless adults waste their time on Facebook.

        • Re: (Score:1)

          by tuaris ( 955470 )
          Na, the useless adults are wasting their time on craigslist's casual encounters section, who think those "ads" are real men and women. At least they use reCAPTCHA their too!
        • I'd rather they waste their time, instead of pestering me. Why can't they go annoy someone in marketing for a change? I prefer to use my time in creative discussions here, and possibly try to cure those emacs zealots from their festering mental illness, and see the light of vi.

          *duck and covers with flameproof suit under the steel table* *locks and loads AK-47*

      • Re: (Score:3, Informative)

        by Firehed ( 942385 )

        Do they really? From what I was able to tell, it's not specified as reCAPTCHA anywhere in the window; having looked at the reCAPTCHA site from a development side I could swear that I read that you needed to give credit if developing a custom style for it. Either I'm remembering wrong, they've got a deal, or FB is undergoing one of the stupidest TOS violations ever.

        • Re:Not new (Score:4, Informative)

          by erbmjw ( 903229 ) on Thursday August 14, 2008 @10:21PM (#24609911)
          from reCAPTCHA FAQ [recaptcha.net]

          When showing reCAPTCHA to the user, is it possible not to show the reCAPTCHA logo? We allow you to customize the theme of reCAPTCHA with our Client API. You are still required to have text on your website which states that you are using reCAPTCHA, however with our theming API, you are free to do this in a way that blends in to your site.

        • Re: (Score:3, Informative)

          by sangreal66 ( 740295 )

          Do they really? From what I was able to tell, it's not specified as reCAPTCHA anywhere in the window; having looked at the reCAPTCHA site from a development side I could swear that I read that you needed to give credit if developing a custom style for it. Either I'm remembering wrong, they've got a deal, or FB is undergoing one of the stupidest TOS violations ever.

          They do give attribution to reCAPTCHA. You have to click on "What's this?"

          This is a standard security test that we use to prevent spammers from creating fake accounts and spamming users. Our captchas are provided by ReCaptcha

      • Re: (Score:3, Insightful)

        by Tassach ( 137772 )
        Because that's so different than the thousands of useless geeks wasting their time on /.

      • Re: (Score:2)

        by Dan541 ( 1032000 )

        It would only be useful if teenagrs knew how to spell.

    • Re: (Score:3, Insightful)

      by Kjella ( 173770 )

      I would imagine that they use multiple logins to verify one word - it's not like people don't mistype captchas in the first place.

    • Re:Not new (Score:4, Informative)

      by Your Pal Dave ( 33229 ) on Thursday August 14, 2008 @11:29PM (#24610433)

      Quoting from the NPR story [npr.org] which aired earlier today:

      more than 40,000 Web sites -- including popular ones such as Ticketmaster, Facebook and Craigslist -- are using a new kind of security program called reCAPTCHA.

      • Re: (Score:3, Interesting)

        by Random Walk ( 252043 )

        Quoting from the NPR story [npr.org] which aired earlier today:

        more than 40,000 Web sites -- including popular ones such as Ticketmaster, Facebook and Craigslist -- are using a new kind of security program called reCAPTCHA.

        That's scary. The way ReCaptcha works allows the reCaptcha server to collect the IPs of reCaptcha users (along with the reCaptcha-enabled website they are using). If many websites are using reCaptcha, it allows to track users as they are moving through the web, from one reCaptcha-enabled website to the next.

        The idea is cute, but the implementation is fundamentally broken and a huge breach of privacy.

        • Re: (Score:2, Informative)

          by Anonymous Coward

          That's scary. The way ReCaptcha works allows the reCaptcha server to collect the IPs of reCaptcha users (along with the reCaptcha-enabled website they are using). If many websites are using reCaptcha, it allows to track users as they are moving through the web, from one reCaptcha-enabled website to the next.

          Only if you actually use the JavaScript API. If you want to protect the privacy of your site's users, you are free to use the server side API of your choice. This gives them (at most) a count of how many recaptchas your users have solved. By the way, the recaptcha site provides - amongst others - ready-made server side bindings for PHP, Java, Ruby, Python and Perl.

          • Huh? Pardon me, but their website doesn't talk about a server-side API.. according to their docs, the server-side stuff (which is available for plenty of languages) is only for verifying the answer. The captcha itself is pulled by the browser from the reCaptcha site, so they know both the user IP as well as the website (which contacts them to verify the answer).

            • If you wanted to blow the extra bandwidth, you could get around that, too. Grab the image onto your server, and let the user get it from there.

              Most sites won't do this, because I think it falls way into the tinfoil hat department. :P

        • Tinfoil hat much?
          Every bigger ad agency (google) can do the same thing.

          • ...and you can block requests to images (i.e. ads) not hosted on the original website, usually without loss of functionality.

            • Smart sites are doing something to check and see if you're blocking ads. I notice that I have to disable AdBlock on imeem.com or it will only play the first song in a playlist.

              I'm sure there is a way around it, but I haven't hacked at it enough. Not really bugging me that much. Every way I can think of to implement such an "ad checker" can be defeated.

    • If you don't believe me, try to guess which one they know and misspell the other one on purpose (or don't, this is for historic posterity =) )

      They most likely require several matching readings from different people before they consider it deciphered.

  • Validate your data, guys! (Score:3, Funny)

    by Anonymous Coward on Thursday August 14, 2008 @09:16PM (#24609359)

    I can usually tell which of the two words is from a real old text. With high probability (>90%) I can correctly answer the real CAPTCHA and replace someone's OCR'd word with "penis".

    I've only ever done this maybe ten or twenty times, but it could easily become an automatic part of using the system.

    • Re: (Score:2)

      by theguru ( 70699 )

      I'm sure they send the same unknown word out to multiple people, and wait for a concensus on it.

      Now, if we ALL started entering "penis" for the obvious unknown words.. :)

    • The thing is, they're often actually both from old texts. It's just that one of them has already been verified.

      And TFA states that they do pass every word by multiple people so as to get more accuracy in what they say. I have little doubt that they're well acquainted with people who try spoofing them.

    • Re:Validate your data, guys! (Score:4, Interesting)

      by PPH ( 736903 ) on Thursday August 14, 2008 @11:54PM (#24610609)

      Since they use entries from several users to validate correct translations for OCR'ed text, this probably won't cause them major problems. OTOH, I wonder if they can track the accuracy of each user's inputs and, if it becomes evident that a user is either incompetent or attempting to screw with the system, take appropriate measures.

      When someone's karma starts dropping into the negative range, they should let us know how well this worked out. If anyone can see their posts, that is.

      • They most likely give the same word to multiple users and choose the word most often entered.

        • Re: (Score:2)

          by PPH ( 736903 )

          They most likely give the same word to multiple users and choose the word most often entered.

          Exactly. But its possible to adapt a technique used in some AI knowledge acquisition systems wherein the outcome of such scoring is 'back propagated' to rank the relative validity of various data sources, rules, etc. If one source (user in this case) consistently ranks low, they get a lower weight in future solutions. Until eventually they get dropped off the bottom of the list (like bad karma on /.).

    • Re: (Score:2, Funny)

      by Spasmodeus ( 940657 )
      As soon as I heard about this project, I figured there'd be people finding ways to abuse it.

      I can see future generations sitting down for a good read:

      MOBY COCK

      Chapturd One

      Call me LOLOLFAG...

    • Re: (Score:1)

      by Elastri ( 911062 )

      Hopefully they are only accepting a piece of text when a lot of the people give the same thing.

    • Re: (Score:2)

      by pz ( 113803 )

      Way to make the world a better place. I'm certain your parents are very proud of your accomplishments. Perhaps you can now go find someone else's sandbox to defecate in, I suggest your own, because I certainly would rather you not be here.

      I have no doubt that the reCAPTCHA folks understand that there are going to be people who find such childish behavior irresisible or entertaining, and either start discounting such answers (based on IP address) or build in filtering to discount particular words.

      But, real

      • Tracking peoples' IP addresses all across the internets? They're lucky if that's all they have their database poisoned with.

    • Re: (Score:1, Insightful)

      by Anonymous Coward

      Both words are from 'real old text'. You won't have any effect on the data output by putting 'penis' because more people will type the correct word.

  • Cool possible uses (Score:5, Interesting)

    by Irish_Samurai ( 224931 ) on Thursday August 14, 2008 @09:19PM (#24609379)

    Man, I would love to see the results if this technique was used for an ontological [google.com] purpose.

    Please type in the word from the choices below that most closely relates to this word: OLD

    HISTORIC
    LIFESPAN

    Interesting shit indeed.

    • Re:Cool possible uses (Score:5, Funny)

      by burgundysizzle ( 1192593 ) on Thursday August 14, 2008 @09:50PM (#24609639)

      Or perhaps SLASHDOT-READER:

      OVERWEIGHT

      GEEK

      SPENDS-TO-MUCH-TIME-USING-COMPUTERS

      ALL-OF-THE-ABOVE

      I fit into the category ALL-OF-THE-ABOVE. The only generalisation that is missing about slashdotters is the one about girlfriends.

      • Re: (Score:1)

        by dword ( 735428 )
        The only generalisation that is missing about slashdotters is the one about girlfriends.
        Haha, loser, you don't have a girlfriend like all the other /.ers!
        Everybody knows that all /.ers have girlfriends. I can even remember my first couple of imaginary lesbian girlfriends I made up when I first joined /.

        • Haha, loser, you don't have a girlfriend like all the other /.ers!

          LOL, I can feel quite content in the fact that you are a fellow loser who doesn't either.

    • is full of hyperbole, dogma, propaganda, and meaningless blatherings.

      • That's kinda the point moron.

        Let me introduce you to the concept of context.

        • Re: (Score:1)

          by maxume ( 22995 )

          What context could possibly rescue those writings from being full of hyperbole, dogma, propaganda, and meaningless blatherings?

    • Surely that'd be relatively easy to hack by use of a thesaurus?
      Or even google:
      old+historic: 66,500,000
      old+lifespan: 3,480,000


      pwned!

      • Re: (Score:3, Informative)

        by Irish_Samurai ( 224931 )

        The point is to see what the populace thinks the relation is.

        If you think google is the end all be all of absolute information then you already fail.

        • Re: (Score:2)

          by x2A ( 858210 )

          I bet it's not far from the truth! What's google but the indexing of the [online] expressions of the populace of which you speak?

        • Re: (Score:2)

          by fbjon ( 692006 )
          But if you think google isn't good enough for a spammer, then YOU fail. Hah!

  • Huh? 1908 New York Times? (Score:3, Funny)

    by mschuyler ( 197441 ) on Thursday August 14, 2008 @09:27PM (#24609415) Homepage Journal

    The New York Times is already online from 1851 onwards. the concept is cool, truly, but why not CAPTCHA something not already accomplished? Oh, I know. That was, like, a metaphor, right?

    • Re: (Score:3, Insightful)

      by FlyingSquidStudios ( 1031284 )
      I am almost certain that it is not all there in its entirety. There are bits that are not online specifically because of OCR errors. That is going to be true with any large volume of OCRed text.

    • Yeah I was kinda wondering about that too, but from a different perspective... I mean: "So every time you solve that string of crooked letters, you may actually be helping historians digitally reconstruct a page from the 1908 New York Times."

      What the hell is the problem with people? All text is apparently on a single page from the NY Times in 1908... I mean fuck, stop the press, cause its obviously all redundant shit anyways, just keep redistributing that one page across the world!

    • Re: (Score:2)

      by x2A ( 858210 )

      "Oh, I know. That was, like, a metaphor, right?"

      If it was like a metaphor, does that make it a simile? No wait, this means you're using using metaphors as a simile? Hmm this could get confusing... perhaps we could make a reCAPTCHA like technology but with old metaphors instead of letters and create a big database of abstraction...

      (ps: I am my own brother who wrote that above, so I am definitely confused, can someone help please?)

  • DMCA Violation (Score:5, Funny)

    by Nymz ( 905908 ) on Thursday August 14, 2008 @09:30PM (#24609449) Journal
    The feature known as FADING was designed to protect copyright works from being pirated by becoming illegible before the work could fall into the public domain.

  • gmail captchas (Score:3)

    by v1 ( 525388 ) on Thursday August 14, 2008 @09:41PM (#24609561) Homepage Journal

    a little OT I know but is anyone else having a bad time with gmail's captchas? I've tried signing up several of our customers for gmail recently and it's becoming really hard to get them right. The "audio" playback used to be the saving grace, but the last two I did it sounded like ten people were talking to me all at once with no discernible key voice. (and last I succeeded, the string to be entered was spoken in three groups, by three different voices)

    • Yep, I do the same thing, signing clients up for Google services, and I get their captchas right about once every three or four tries. :-(

    • Re: (Score:2)

      by x2A ( 858210 )

      Wow people are really goin to town with the offtopic mods *sigh* conversation nazis "you will follow STRICTLY the rules of conversation or your karma will be no more!!!". Such a waste, there are posts out there that need modding up than these slightly-offtopic posts need modding down!

      And yes I have started coming across more captchas that do seem just impossible to read, they certainly know how to make you feel stupid 'n illiterate. Apparently it's a new system in place, like the one in the article, but for

  • Image Captchas (Score:4, Informative)

    by pembo13 ( 770295 ) on Thursday August 14, 2008 @09:48PM (#24609613) Homepage
    I've found implementing a simple "please choose the name of the item seen bellow" eliminates a large amount of spam (all?) but has the problem of not being viable for blind people.

    • Re: (Score:2)

      by x2A ( 858210 )

      I find a bit of simple javascript works well, and is out of sight of genuine users. If you wanna account for people who block javascript (rather than a note saying "please turn on javascript for a sec, think of the children") you can have a captcha in a span or div etc, then use javascript to remove it and replace it with a hidden field with a name<-->value pair that can be compared server side when they post the form and have the values checked. Yes, someone could look at the page source and see what

    • Re:Image Captchas (Score:5, Funny)

      by Martz ( 861209 ) on Friday August 15, 2008 @02:55AM (#24611667)

      Just use an alt tag.

    • Re: (Score:2)

      by MobyDisk ( 75490 )

      But that is multiple choice, so it is easier to make a program that can guess the result.

  • Took me a bit to get past the new security measures, But I got a coupon 5 cents off my next shoe purchase.

  • Right about now, I'm wondering what the implications would be for including reCAPTCHA in an open source project. (a PHP-based blog I'm working on) Right now the blog is read-only, since I have yet to build my own working CAPTCHA system and putting up an unprotected reply form is sheer idiocysince it wil lbe a whole five minutes before the spam bots find it. My project is GPLv3, so would including ReCAPTCHA cause me some sort of licensing problem?

    • Re: (Score:2)

      by erbmjw ( 903229 )
      reCAPTCHA should not cause any licensing issues if all you do is link to their site via the "magic four lines of code" or use one of their plugins

      from why reCAPTHCHA [recaptcha.net]

      It's Easy. reCAPTCHA is a Web service. As such, adopting it is as simple as adding 4 lines of code on your site. For many applications and programming languages such as Wordpress and PHP we also have easy-to-install plugins available. We generate and check the distorted images, so you don't need to run costly image generation programs.

      Word

    • Re:reCAPTCHA and Open Source (Score:4, Informative)

      by corbettw ( 214229 ) on Thursday August 14, 2008 @10:35PM (#24610039) Journal

      There are multiple libraries for reCAPTCHA already published, all under the MIT License. Just see http://code.google.com/p/recaptcha/ [google.com] for a list of them.

  • Problems With ReCaptcha (Score:1, Interesting)

    by Anonymous Coward
    I've seen a number of issues with reCaptcha that I don't really know how to handle (i.e. what to enter): 1. Multiple word strings 2. Foreign characters 3. Illegible Text 4. A single word for both entries 5. Words that look like one thing initially, but are really another when you look closer
  • How are they able to tell if I've accurately solved an unknown. If the word is "Yesterday" and I enter "Fucktard", not only will the society get some very wrong data, but I'll also have passed the CAPTCHA without entering the actual letters.

    • Re: (Score:2)

      by Peyna ( 14792 )

      RTFA.

      You get two captchas. One is your standard, let's find out if you're human captcha, where the program knows the answer. The other is the scanned text. It also presents the same scanned text to many people, and then uses the results to figure out which one is the most likely correct result.

  • It turns out... (Score:3, Informative)

    by symbolset ( 646467 ) on Friday August 15, 2008 @12:02AM (#24610689) Journal

    That slashdot's Goatse troll server guy proves useful.

    Note: This is not a troll. One of the guys that offers open web services to slashdot trolls is also responsible for considerable development of CAPTCHA breakage and is an eminent Debian developer. This is why I've said that we should respect his efforts despite the unpleasant side effects. The truly brilliant we should grant exceptions from social behavior because they discover things more proper folk would not.

    • The truly brilliant we should grant exceptions from social behavior because they discover things more proper folk would not.

      No. Just no. Being brilliant is no excuse for being an asshole.

    • Re: (Score:3, Interesting)

      by argent ( 18001 )

      How is being responsible for CAPTCHA breakage useful?

      Look, just because the guy who more or less invented both trolling and automated trolling is an eminent UNIX guru and textbook author that doesn't mean his trolling on net.suicide was any less disgusting. I was appalled at the people who laughed along with Pike when he revealed that he was behind Bimmler and Shaney. This kind of thing is just not acceptable no matter who you are.

  • Recaptcha doesn't recapture context (Score:5, Interesting)

    by Mumei no koshinuke ( 1110677 ) on Friday August 15, 2008 @12:43AM (#24610969)
    When solving these I sometimes find that there's more than one possibility for an illegible word, yet I can't tell which it is without knowing the context.
    For example, in some fonts "cost" and "cast" might be indistinguishable in the image shown. But given the context of the sentence it's trivial for a human to tell the difference.
    Suppose that they found these words on which people disagreed and had another captcha system which showed the full sentence. I'd guess they could improve their accuracy significantly in this case. Since they could prescreen for ambiguous words using the current captcha system, even if fewer people were willing to solve the "large" captcha, they would still get all the solutions they needed.
  • why don't they just use whatever software is used by the crackers to bombard us with spam email to go through all of these books are whatever speed they're capable of. If compromised PCs can send tens of thousands of fake emails, why not just set a few up to figure out these words/

    How much worse is this than trusting users to correctly identify the text? I ask because I honestly don't know the succcess rate of the automated system.

    • Re:RTFA (Score:3, Informative)

      by Psychotria ( 953670 )

      The authors also tested software designed to crack CAPTCHAs against images created using reCAPTCHA, and found that they failed completely. The authors ascribe this to the fact that the letters in scanned images contain distortions that are not the result of a clean mathematical transformation. User response times were also measured, but there were no significant differences between the time it took users to handle traditional systems and that required to use reCAPTCHA.

  • Use to hide your own email addy (Score:5, Informative)

    by RJFerret ( 1279530 ) on Friday August 15, 2008 @02:58AM (#24611685)

    You can also use reCaptcha for your own email address, and be more willing to provide it "publicly" since they'd have to answer the reCaptcha to get to the mailto... reCaptcha mailhide [recaptcha.net]

  • Interesting field (Score:2, Interesting)

    by Anonymous Coward

    My company is working on digitizing a large volume of old text (19th century government documents). There are a number of problems unique to old text:
    - OCR breaks down due to archaic letter shapes, smudging, letter damage and paper deterioration.
    - we evaluated OCR versus having the entire text retyped by Indians, and ended up going with the Indians. The only way to get sufficient accuracy (>99%) was to have everything done twice and do a comparison.
    - Even then, the typed text has to be checked using both

  • I feel pretty good about opening my porn bookmarks now that they've adopted reCAPTCHA.

    • Re: (Score:1)

      by JSund ( 1268594 )
      It's quite possible that they are just harvesting the reCAPTCHA results to spam other sites that are using it for spam protection though.
  • The API for adding reCAPTCHA to your web site is fairly easy to use, and there are extensions or plug-ins for applications like MediaWiki, Joomla, Drupal, WordPress, phpBB, etc.

    Just to try it out I set up a mechanical turk [spy-hill.net] using reCAPTCHA. So if you like the idea you can keep at it, instead of just solving one of them once. It can be a bit addicting.

  • The recaptcha.net site does not have links to the OCRed text data they are accumulating. It's nowhere to be found in the F.A.Q. or the wiki. Everything just deals with implementing the API and such. If we, the public are helping to create this archive, where can we download plaintext results of the system?

Slashdot Top Deals

Two can Live as Cheaply as One for Half as Long. -- Howard Kandel

Close