Become a fan of Slashdot on Facebook

 



Forgot your password?
typodupeerror
Books Google News

How Google Is Solving Its Book Problem 58

Pickens writes "Alexis Madrigal writes in the Atlantic that Google's famous PageRank algorithm can't be deployed to search through the 15 million books that Google has already scanned because books don't link to each other in the way that webpages do. Instead Google's new book search algorithm called 'Rich Results' looks at word frequency, how closely your query matches the title of a book, web search frequency, recent book sales, the number of libraries that hold the title, how often an older book has been reprinted, and 100 other signals. 'There is less data about books than web pages, but there is more structure to it, and there's less spam to contend with,' writes Madrigal. Yet the focus on optimizing an experience from vast amounts of data remains. 'You want it to have the standard Google quality as much as possible,' says Matthew Gray, lead software engineer for Google Books. '[You want it to be] a merger of relevance and utility based on all these things.'"
This discussion has been archived. No new comments can be posted.

How Google Is Solving Its Book Problem

Comments Filter:
  • by Toe, The ( 545098 )

    But do they really have to shred all the books just to scan them?

    • Does it matter? Its not like these are one of a kind Tomes of Utter Significance. Besides, once scanned, they can be reprinted if needed.

      • Re:Rainbows End (Score:5, Informative)

        by Samantha Wright ( 1324923 ) on Thursday November 04, 2010 @08:06AM (#34123262) Homepage Journal
        Wait! I'm undoing all my mod points because I just realised that no, you're quite wrong. The printing process wouldn't be the same for the older books, and some of them have survived hundreds of years before we came along and scanned them.

        However, the story about books being cut up for scanning was about microfilm. I think it was an institution in Texas whose library was cutting them up mentioned as an aside in a submission about how they were converting their library into a lounge and computer lab.
    • Re: (Score:1, Informative)

      by Anonymous Coward

      No. Speculation on Google's process based on a patent filing. [seobythesea.com]

      I seem to recall an article that was more than speculation, but I couldn't find it while searching. The 2003 entry [google.com] for the Google books history also points toward it being a non-destructive process.

      • Re:Rainbows End (Score:5, Insightful)

        by delinear ( 991444 ) on Thursday November 04, 2010 @08:38AM (#34123522)
        I would guess (read, hope) that while the process means books which are commonly available might be handled in the quick yet destructive manner, books which are more rare or have historic significance beyond the data would be treated much more carefully (at the lower end of the scale, someone with a hand scanner maybe, at the upper end perhaps even people manually transcribing). Ultimately, though, while I think it's a crime for a book to be destroyed, if it's a choice between it mouldering away in a basement somewhere until it falls apart or Google destroying it early in the interest of preserving the data, surely it's better that the ideas rather than the physical object are preserved (I appreciate in reality it's not just a black and white either/or choice).
    • Slashdot doesn't have a "+1 Obscure" moderation, probably because nothing is obscure on /., so I'm just gonna drop you a shout and friend you.

      • Hm, it doesn't have a "+1 Thanks for the conundrum" moderation, either. Oh, well. :-)

        • Google greater scooch-a-mout [google.com] ...

          • That was a pretty quick spoiler. I guess you young'uns just don't understand how it is when yer brain gets all fuzzy and slow.

            (I didn't even notice the comment title. I am embarrassed to admit that I actually remember that book being one of the last dead-tree science fiction books I bought, but I have never gotten around to reading it, and am not sure if I could even still find it, because my life suddenly mutated in unexpected ways quite soon after the purchase. A good excuse, I suppose, to go looking for

    • actually in rainbow's end one of the protagonists mentions that he prefer google's non-destructive scanning over the new shredding method.
      so its not google who are shredding books. they are the ones saving the books.

    • Re: (Score:3, Informative)

      But do they really have to shred all the books just to scan them?

      No. A book scanning machine is capable of scanning a book non-destructively. My unsubstantiated guess is that they are less harmful to the book than your average reader.

      You can build one if you'd like. Instructable [instructables.com] The automated page turners on the commercial models are awesome. Youtube video [youtu.be]

  • in the Atlantic?
  • For a second I thought they were merely using VSM: http://en.wikipedia.org/wiki/Vector_space_model [wikipedia.org] . As I read further, I was happily proven wrong. :)
    • by mcgrew ( 92797 ) *

      Their book search has improved greatly since they started. A year ago I was looking for Huckleberry Finn and the first result was amazon.com. This was annoying, as that book is in the public domain.

      They seemed to have fixed it. The first result now is wikipedia, the second a study guide, the third is the book itself hosted at the University of Virginia.

      Shopping is at the bottom of the page. I'm pleased!

      • Re:VSM (Score:4, Insightful)

        by delinear ( 991444 ) on Thursday November 04, 2010 @08:42AM (#34123558)
        I suspect this is as much to do with the uptake in ebook readers as any change to the search indexing. Previously, if you were searching for this book you probably had a very specific interest in it and often wanted to buy a copy, now the people searching are more likely looking for free reading material, so the ranks have adjusted to accommodate that (since "people looking for free stuff" is a much wider market than "people with interest in a particular book", so it's easy to swing the ranking in favour of the former).
  • I think it should work well for scientific monographies as they contain a lot of references to each other, but don't usually get reprinted. [citattion needed]

    • Re: (Score:3, Informative)

      by jank1887 ( 815982 )

      they already do that via Google Scholar. Scientific paper searches often (maybe not often enough) bring up textbook references. I know searching through regular Google does quite frequently.

  • by bogaboga ( 793279 ) on Thursday November 04, 2010 @07:25AM (#34122988)

    I have always wondered why the text in these books is not clear. The blurry fonts make my eyes hurt and surely, Google can create a better interface for the main page. Just 1 million dollars can do so much if some expert were hired to revamp the site. Come on Google!

    • by AdmiralXyz ( 1378985 ) on Thursday November 04, 2010 @07:37AM (#34123076)
      It's because the book-scanning process is completely automated. I can't find a look to it, but a remember a Slashdot or Wired article about Google's automatic book-scanning machine. Basically it's too difficult to adjust for perfect focus for every book.

      I wouldn't worry about it though: Google is doing OCR on all these books, and they'll presumably replace the images with plain-text equivalents at some point (more searchable, portable, etc.) That's my hope, anyway.
      • I would indeed like that, but it'll be interesting to see how they could OCR my copy of DaVinci's manuscripts. Particularly when the pages alternate between latin and english, with illustrations.

      • Re: (Score:3, Insightful)

        by grumbel ( 592662 )

        It's because the book-scanning process is completely automated.

        I doubt it, it is not exactly hard to get a book that is at a rather fixed distance into focus. Anyway, the reason why the fonts are blurry isn't the focus to begin with, the images that Google shows are simply extremely low resolution. Why they are in such a low resolution I have no idea.

        • It's because the book-scanning process is completely automated.

          I doubt it, it is not exactly hard to get a book that is at a rather fixed distance into focus. Anyway, the reason why the fonts are blurry isn't the focus to begin with, the images that Google shows are simply extremely low resolution. Why they are in such a low resolution I have no idea.

          Imagine the storage required for that many hi-res images when low-res works well enough. That's why.

        • by tlhIngan ( 30335 )

          It's because the book-scanning process is completely automated.

          I doubt it, it is not exactly hard to get a book that is at a rather fixed distance into focus. Anyway, the reason why the fonts are blurry isn't the focus to begin with, the images that Google shows are simply extremely low resolution. Why they are in such a low resolution I have no idea.

          Well, actually it is. Google's book scanners use two digital cameras to take photos of both pages at once, rather than a much clearer scanner system. Those pho

          • the book-scanning process is completely automated.

            Well if it really is automated, how comes it that some of the scanned pages show part of the hand (complete with finger rings!) of the person who was doing the scanning? It looks as if the scanning was done by someone who didn't realise that the text can't be read if there's a hand between the page and the scanner-glass!

            I reckon that's a manual process.

            -wb-

      • by ortholattice ( 175065 ) on Thursday November 04, 2010 @08:44AM (#34123578)
        As someone studying certain specialized math books from the 1800's and early 1900's, I had great expectations for Google books, since they offer downloadable PDFs for public domain works. However, the focus quality of many (most?) of them is so incredibly poor that things like tiny subscripts are illegible blobs, making them essentially useless.

        While plain text solves this problem for novels, it is useless for math books, because OCR renders the equations (which are the essence of the book) as garbage characters. And it's not clear how one would communicate them as plain text anyway, unless the OCR was extremely sophisticated and generated say LaTeX output.

        Thankfully, some of the ones I need are in the University of Michigan Historical Mathematics Collection [umich.edu], with a much higher quality. But for the ones that are not there, I've used the Google pdf as a last resort - at least I can get an overview, if somewhat unpleasant to read. But for books I actually want to study, I've ended up making my own scan from a library copy (which, if done with care, is better quality than even the U Mich. version) when Google's is the only one I can find on-line.

        However, scanning physically stresses these old books. I think it is sad that I have to repeat what Google has done, when they (presumably) could have scanned them with high quality with a little more effort or better equipment with automatic focusing. In some cases, the books have been in the rare book section of the university library, which can't be checked out, and making copies of the whole book locally is frowned upon because of possible damage and sometimes, depending on the book's condition, not allowed.

        • by kryliss ( 72493 )

          Sounds like a two person project. One person holds the book up shaped like an L. Use a digital camera to take pictures of each page. If a page is too "curved" try using the glass from a picture frame to hold it down. Make sure to use indirect lighting and no flash, if needed set the ISO speed on your camera to a good setting. You could even setup some kind of tripod that points the camera straight down and set the timer.

        • There are a couple of projects which OCR math properly:

          http://inftyreader.org/ [inftyreader.org]

          http://research.cs.queensu.ca/drl//ffes/ [queensu.ca]

          William

      • Basically it's too difficult to adjust for perfect focus for every book.

        Huh? Autofocus is not a new technology.

    • by bouldin ( 828821 )
      Maybe the font is intentionally blurry so you can't use your own OCR to scrape the book from Google.
  • Google Book Metadata (Score:1, Interesting)

    by Anonymous Coward

    I'm not sure Google can correlate the kinds of data they are talking about because their book metadata (author, title, edition, etc.) is so inaccurate. I often find Google books based on text search that can't be located in author or title searches.

  • I hope they aren't trying to get experts-exchange as 8 of my top 10 book results.

  • by unitron ( 5733 ) on Thursday November 04, 2010 @08:59AM (#34123750) Homepage Journal

    Shouldn't that be "are fewer data"?

  • Books don't link to each other?

    What are citations and footnotes?

    • Re: (Score:1, Informative)

      by Anonymous Coward

      CS Lewis and Tolkien weren't really known for their citations.

      (Now Tolkien might well be known for his appendices., but that is totally different.)

  • The Tegra2 kit I messed with was 1GHz with 1GB of RAM and it wasn't optimized but run Ubuntu great. can't wait

    LoB

In 1750 Issac Newton became discouraged when he fell up a flight of stairs.

Working...