How Google Is Solving Its Book Problem 58
Pickens writes "Alexis Madrigal writes in the Atlantic that Google's famous PageRank algorithm can't be deployed to search through the 15 million books that Google has already scanned because books don't link to each other in the way that webpages do. Instead Google's new book search algorithm called 'Rich Results' looks at word frequency, how closely your query matches the title of a book, web search frequency, recent book sales, the number of libraries that hold the title, how often an older book has been reprinted, and 100 other signals. 'There is less data about books than web pages, but there is more structure to it, and there's less spam to contend with,' writes Madrigal. Yet the focus on optimizing an experience from vast amounts of data remains. 'You want it to have the standard Google quality as much as possible,' says Matthew Gray, lead software engineer for Google Books. '[You want it to be] a merger of relevance and utility based on all these things.'"
Rainbows End (Score:2, Funny)
But do they really have to shred all the books just to scan them?
Re: (Score:1)
Does it matter? Its not like these are one of a kind Tomes of Utter Significance. Besides, once scanned, they can be reprinted if needed.
Re:Rainbows End (Score:5, Informative)
However, the story about books being cut up for scanning was about microfilm. I think it was an institution in Texas whose library was cutting them up mentioned as an aside in a submission about how they were converting their library into a lounge and computer lab.
Re: (Score:1, Informative)
No. Speculation on Google's process based on a patent filing. [seobythesea.com]
I seem to recall an article that was more than speculation, but I couldn't find it while searching. The 2003 entry [google.com] for the Google books history also points toward it being a non-destructive process.
Re:Rainbows End (Score:5, Insightful)
Re:Rainbows End (Score:5, Informative)
Re: (Score:2)
Slashdot doesn't have a "+1 Obscure" moderation, probably because nothing is obscure on /., so I'm just gonna drop you a shout and friend you.
Re: (Score:1)
Hm, it doesn't have a "+1 Thanks for the conundrum" moderation, either. Oh, well. :-)
Re: (Score:2)
Google greater scooch-a-mout [google.com] ...
Re: (Score:1)
That was a pretty quick spoiler. I guess you young'uns just don't understand how it is when yer brain gets all fuzzy and slow.
(I didn't even notice the comment title. I am embarrassed to admit that I actually remember that book being one of the last dead-tree science fiction books I bought, but I have never gotten around to reading it, and am not sure if I could even still find it, because my life suddenly mutated in unexpected ways quite soon after the purchase. A good excuse, I suppose, to go looking for
Re: (Score:1)
actually in rainbow's end one of the protagonists mentions that he prefer google's non-destructive scanning over the new shredding method.
so its not google who are shredding books. they are the ones saving the books.
Re: (Score:3, Informative)
But do they really have to shred all the books just to scan them?
No. A book scanning machine is capable of scanning a book non-destructively. My unsubstantiated guess is that they are less harmful to the book than your average reader.
You can build one if you'd like. Instructable [instructables.com] The automated page turners on the commercial models are awesome. Youtube video [youtu.be]
How does one write ... (Score:1)
Re: (Score:3, Funny)
With a pen.
Re: (Score:1, Informative)
With a fountain pen.
Re:How does one write ... (Score:5, Funny)
there's a whole branch of science that studies writing and drawing while in an ocean, it's called oceanographics
Re: (Score:2)
scratch it on an iceberg.
Re: (Score:2)
in the Atlantic?
using underwater paper like this [infohub.com].
Re: (Score:2)
With a watered-down representation [theatlantic.com] of a niche [theatlantic.com], minority [theatlantic.com], or extreme [theatlantic.com] viewpoint, apparently.
VSM (Score:1)
Re: (Score:2)
Their book search has improved greatly since they started. A year ago I was looking for Huckleberry Finn and the first result was amazon.com. This was annoying, as that book is in the public domain.
They seemed to have fixed it. The first result now is wikipedia, the second a study guide, the third is the book itself hosted at the University of Virginia.
Shopping is at the bottom of the page. I'm pleased!
Re:VSM (Score:4, Insightful)
Scientific books (Score:2)
I think it should work well for scientific monographies as they contain a lot of references to each other, but don't usually get reprinted. [citattion needed]
Re: (Score:3, Informative)
they already do that via Google Scholar. Scientific paper searches often (maybe not often enough) bring up textbook references. I know searching through regular Google does quite frequently.
Why can't the text of these books be clearer? (Score:3, Interesting)
I have always wondered why the text in these books is not clear. The blurry fonts make my eyes hurt and surely, Google can create a better interface for the main page. Just 1 million dollars can do so much if some expert were hired to revamp the site. Come on Google!
Re:Why can't the text of these books be clearer? (Score:5, Informative)
I wouldn't worry about it though: Google is doing OCR on all these books, and they'll presumably replace the images with plain-text equivalents at some point (more searchable, portable, etc.) That's my hope, anyway.
Re: (Score:1)
I would indeed like that, but it'll be interesting to see how they could OCR my copy of DaVinci's manuscripts. Particularly when the pages alternate between latin and english, with illustrations.
Re: (Score:2)
Yes, and a significant fraction of the books they are going to scan are DaVinci's Manuscripts!
Re: (Score:3, Informative)
One way they do it is through reCaptcha [google.com]. When you're typing them, you're also helping the OCR process.
Re: (Score:3, Insightful)
It's because the book-scanning process is completely automated.
I doubt it, it is not exactly hard to get a book that is at a rather fixed distance into focus. Anyway, the reason why the fonts are blurry isn't the focus to begin with, the images that Google shows are simply extremely low resolution. Why they are in such a low resolution I have no idea.
Re: (Score:1)
It's because the book-scanning process is completely automated.
I doubt it, it is not exactly hard to get a book that is at a rather fixed distance into focus. Anyway, the reason why the fonts are blurry isn't the focus to begin with, the images that Google shows are simply extremely low resolution. Why they are in such a low resolution I have no idea.
Imagine the storage required for that many hi-res images when low-res works well enough. That's why.
Re: (Score:2)
Well, actually it is. Google's book scanners use two digital cameras to take photos of both pages at once, rather than a much clearer scanner system. Those pho
Re: (Score:2)
the book-scanning process is completely automated.
Well if it really is automated, how comes it that some of the scanned pages show part of the hand (complete with finger rings!) of the person who was doing the scanning? It looks as if the scanning was done by someone who didn't realise that the text can't be read if there's a hand between the page and the scanner-glass!
I reckon that's a manual process.
-wb-
Re:Why can't the text of these books be clearer? (Score:5, Interesting)
While plain text solves this problem for novels, it is useless for math books, because OCR renders the equations (which are the essence of the book) as garbage characters. And it's not clear how one would communicate them as plain text anyway, unless the OCR was extremely sophisticated and generated say LaTeX output.
Thankfully, some of the ones I need are in the University of Michigan Historical Mathematics Collection [umich.edu], with a much higher quality. But for the ones that are not there, I've used the Google pdf as a last resort - at least I can get an overview, if somewhat unpleasant to read. But for books I actually want to study, I've ended up making my own scan from a library copy (which, if done with care, is better quality than even the U Mich. version) when Google's is the only one I can find on-line.
However, scanning physically stresses these old books. I think it is sad that I have to repeat what Google has done, when they (presumably) could have scanned them with high quality with a little more effort or better equipment with automatic focusing. In some cases, the books have been in the rare book section of the university library, which can't be checked out, and making copies of the whole book locally is frowned upon because of possible damage and sometimes, depending on the book's condition, not allowed.
Re: (Score:1)
actually windows 7 math input panel works very nice.
Re: (Score:1)
Sounds like a two person project. One person holds the book up shaped like an L. Use a digital camera to take pictures of each page. If a page is too "curved" try using the glass from a picture frame to hold it down. Make sure to use indirect lighting and no flash, if needed set the ISO speed on your camera to a good setting. You could even setup some kind of tripod that points the camera straight down and set the timer.
Re: (Score:2)
There are a couple of projects which OCR math properly:
http://inftyreader.org/ [inftyreader.org]
http://research.cs.queensu.ca/drl//ffes/ [queensu.ca]
William
Re: (Score:2)
Huh? Autofocus is not a new technology.
Re: (Score:1)
Google Book Metadata (Score:1, Interesting)
I'm not sure Google can correlate the kinds of data they are talking about because their book metadata (author, title, edition, etc.) is so inaccurate. I often find Google books based on text search that can't be located in author or title searches.
Re:Books Contribute to Global Warming (Score:5, Informative)
You're not taking into consideration the energy required to make the book, or to transport it to the marketplace. The amount of carbon sequestered in the physical pages of a book is insignificant in comparison.
The production of a book releases 8.85 lbs. of CO_2:
http://latimesblogs.latimes.com/emeraldcity/2008/06/paper-vs-paperl.html [latimes.com]
Here's a page which indicates most CO_2 production is for energy:
http://www.eia.doe.gov/oiaf/1605/ggrpt/carbon.html [doe.gov]
And here's a page which indicates that CO_2 production is a much larger problem for the manufacturing of electronics:
http://www.energybulletin.net/node/49730 [energybulletin.net]
w/ a ratio of 12 to 1 for energy usage to weight, so my PRS-505 weighs roughly 9 ozs., so presumably required 108 ounces of fuel to manufacture (on-going energy usage is trivial and not considered)
http://www.epa.gov/oms/climate/420f05001.htm [epa.gov]
gives us a figure of 19.4 pounds of CO_2 per gallon of gasoline which equals roughly 16.36875 pounds of CO_2 to make the ebook reader.
So getting two books for the Sony should make it roughly break even, and each printed book beyond that which is not purchased should result in a net reduction of CO_2 emissions, since the energybulletin.net page indicates that the embodied energy usage for electronics is much greater than the lifetime usage.
Re: (Score:1, Funny)
My printing press is fueled by the frantic posts of trolled know-it-alls.
Re: (Score:2)
Did you include the energy cost of the manufacturing and disposal of the batteries that will power your e-reader? How many batteries and e-readers do you expect to consume during the lifespan of a typical book?
Re: (Score:2)
Batteries are included in the initial production weight and the battery is a small fraction of that weight --- an e-ink screen reader uses so little power that one needs to recharge every week or so, so batteries last for _years_ --- if one does replace the battery the old one contains materials which are valuable enough to warrant recycling, so the environmental impact is minimal as stated in my post.
An ebook reader which used typical batteries would be a really bad idea and if there are any such, I hope t
aiming for "standard Google quality"? (Score:2, Funny)
I hope they aren't trying to get experts-exchange as 8 of my top 10 book results.
" 'There is less data about books than web pages.. (Score:3, Informative)
Shouldn't that be "are fewer data"?
Link == citation (Score:2)
Books don't link to each other?
What are citations and footnotes?
Re: (Score:1, Informative)
CS Lewis and Tolkien weren't really known for their citations.
(Now Tolkien might well be known for his appendices., but that is totally different.)
what's also nice is it's on a dual core Cortex-A9 (Score:2)
LoB