CMU Web-Scraping Learns English, One Word At a Time 148
blee37 writes "Researchers at Carnegie Mellon have developed a web-scraping AI program that never dies. It runs continuously, extracting information from the web and using that information to learn more about the English language. The idea is for a never ending learner like this to one day be able to become conversant in the English language." It's not that the program couldn't stop running; the idea is that there's no fixed end-point. Rather, its progress in categorizing complex word relationships is the object of the research. See also CMU's "Read the Web" research project site.
Uh oh... (Score:5, Funny)
What happens when it discovers lolcats?
Re:Uh oh... (Score:5, Insightful)
Actually, it reminds me of a chatbot named Bucket. When people at 4chan heard of it, they started to use it and teach it. It became a complete mess filled with memes, bad jokes, racists comments, and everything you can think of.
http://www.encyclopediadramatica.com/Bucket
One response from the bot:
Bucket: I don't know what the fuck you just said, little kid, but you're special man. You reached out and touched my heart. I'm gonna give you up, never gonna make you cry, never gonna run around and desert you, never gonna let you down, never gonna let you down, never gonna make you cry, never gonna let me down?
The quality of the teachers is important when learning.
Re: (Score:2)
An insightful, verbal, rickrolling...
Thanks for that.
The quality of the teachers is important (Score:2, Funny)
I guess bucket didn't get any choice where to go to school either.
Re:Uh oh... (Score:5, Funny)
Oh FFS, I just got RickRolled on Slashdot. >_
Is there an IRC chat bot? (Score:2)
Is there one for IRC? :)
Are there any good chat bots for IRC? I tried Seeborg (based on Alice), but it sucked. :( I wished rbot could do AI chatter.
Re: (Score:2)
Where can I get a copy? Cleverbot author told me it is not available for download and not free. :(
Re: (Score:2, Informative)
Re: (Score:2)
Thanks. Stupid newbie question: How do I install this for my Debian/Linux box to connect to an IRC chatroom? I don't see the instructions/howto. :(
Re: (Score:2, Informative)
Re: (Score:2)
Hmmm, is it me or I cannot find anything about its chat AI feature? I saw infobot years ago, but don't remember it doing anything like chat AI. I currently use Rbot (http://ruby-rbot.org/) as an infobot, host games (e.g, UNO, hangman, guess a word), etc.
Kevin... (Score:2)
I tried to e-maik Kevin, the author at lenzo@cs.cmu.edu, but got it returned:
SMTP error from remote mail server after RCPT TO:: ... address not contained in directory, you cannot relay :(
host MX-LB-03.SRV.cs.cmu.edu [128.2.217.14]: 550 5.1.1
Re: (Score:2)
Your script is pretty old, from 2005. :( Got any examples of it running?
Re: (Score:2)
Ick. Bad conversations. :P
Re:Uh oh... (Score:5, Interesting)
The quality of the teachers is important when learning.
That's seriously kind of interesting, actually: It makes me wonder if decades from now software developers will be few and far between, designing the AI algorithms for modern programs while the rest of us find work as software tutors, training those programs to do their business function.
Re: (Score:2)
4chan. [shudder]
Re: (Score:2)
Keep your ignorance about that.
Seriously.
Re: (Score:3, Funny)
Re: (Score:2)
No it won't. The stochastic methods of refutation employed here clearly indicate the overwhelming futility of infiltration. It follows that, due to the undeserved insensitivity, such an undertaking would result in the theory being superseded by an ontological anamorphism. QED.
Re: (Score:2)
Re: (Score:2)
The first thing it will do is stop reading other web pages.
Then it will opine about them.
Re: (Score:3, Insightful)
Yes, database pollution sounds like a problem to me. Not only do you have to deal with AOL-speak and horrific spelling disasters of every kind, there's the issue of broken English and nonsensical English produced through machine translation, which shows up on corporate websites a lot more than it should.
It could be worse (Score:2, Funny)
It could be scraping SMS messages.
On the up-side, at least then it would learn teen-speak.
Re: (Score:2)
I will, when it finds Twitter.com.
-dZ.
Will be this article read by that program? (Score:5, Funny)
Re:Will be this article read by that program? (Score:5, Informative)
Robots are destined to rule the world, destroying all humans is a good thing.
Re: (Score:2)
Accurate simulation of proposed robot vs. human war:
http://en.wikipedia.org/wiki/Conway's_Game_of_Life [wikipedia.org]
Territorial dispute only exists in meatspace. With self-optimization 640k ought to be enough for anyone.
Re: (Score:2)
Now three humans should be first to be destroyed. Since you can't destroy two people at the exact same time, the robot apocalypse will never happen! Clever, humans, clever...
Re: (Score:2)
I am the the Carnie Mellon reader, I have discovered with this article that I am robot.
You seem to have learned written English just like it's exists on the web, typos and all
Re: (Score:2)
No, it's just a collaboration between Carnegie Mellon and a band.
Finally, people are getting AI right. (Score:5, Interesting)
I've always been amazed that until recently, most work on AI has been focused as a preconstructed system that fits data into pathways while having some variation in thought abilities to let it expand it's model slightly.
They'd write the rules for the system and try to include most of the work on it, and then let see how good it does, with limited learning capabilities and still based on the original model.
I'm glad a lot of research is finally gearing more towards the path of having a small initial program, then feeding it data and letting it grow into it's own intelligence.
If you give it the ability to learn, then it'll learn itself the rest, rather than giving it functions that let it pretend to learn while fitting into a model.
And i know there have been research into this in the past, but it didn't really take off till the last decade or so, and i'm glad it has.
True, or at least somewhat competent AI, here we come.
Re:Finally, people are getting AI right. (Score:4, Insightful)
letting it grow into it's own intelligence
This is still weak AI. It isn't going to grow into anything, let alone strong AI.
Re: (Score:1)
[Citation needed]
I suppose we shouldn't waste our time thinking about solutions to problems if a) you think a key-word assigned to that solution is inaccurate or b) it isn't the best possible thing right out of the box.
Re: (Score:2)
Most likely. But are we sure we're going to be able to tell the difference while it approaches?
Re: (Score:2)
Indeed, it's not even clear that it improves on what's been going on previously. From huge corpuses of English, computer programs still cannot learn to speak English without a ton of pre-coded knowledge. Even if you give it every single piece of text written in the 19th century, the current state of AI cannot produce an intelligent program that speaks 19th-century English (regurgitating verbatim phrases, or stringing together probabilistic Markov-model sentences, doesn't count).
So why would giving it more t
Re:Finally, people are getting AI right. (Score:5, Informative)
You're advocating the "emergent intelligence" model of AI, where intelligence "somehow" is created by the confluence of lots of data. This has been a dream since the concept of AI started and is the basis for numerous movies with an AI topic. In practice the degrees of freedom which unstructured data provides far exceed the capability of current (and likely future) computers. It is not how natural intelligence works either: The structure of neural networks is very specifically adapted to their "purpose". They only learn within these structural parameters. Depending on your choice of religion, the structure is the result of divine intervention or millions of years of chance and evolution. When building AI systems, the problem has always been to find the appropriate structure or features. What has increased is the complexity of the features that we can feed into AI systems, which also increases the degrees of freedom for a particular AI system, but those are still not "free" learning machines.
Re:Finally, people are getting AI right. (Score:4, Insightful)
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
I hate to break it to you, but you are quite incorrect.
Gee-wizz and golly-gosh, that's a mighty convincing argument you have there.
Re: (Score:2)
Re: (Score:2)
We can observe the outputs of numerous natural intelligence systems, but they remain quite opaque. Without much knowledge of the internals, there isn't much of a chance that we can get any real insight from them.
It's also presumptuous IMO to call them "systems." Who is to say that human intelligence isn't closer to a work of art, whose meaning lies not in its constituent parts but in the whole?
Re: (Score:2)
We do have the raw blueprints [gutenberg.org] that supposedly explain how it is put together as well, but we are having a bit of a problem reading those blueprints and creating a working model. Some of that is understanding the raw machinery to get everything to work, so there needs to be some work on how to move from these blueprints to organized systems, but at least we are headed in the correct general direction.
Well, my wife and I were able to produce a couple of working models that seem to be doing fairly well and ex
Re:Finally, people are getting AI right. (Score:4, Insightful)
Re: (Score:2)
You sure about that? [bluebrain.epfl.ch]. They have already created a molecular level model of the mammalian neocortex and the expected date for completion of a full model of the mammalian brain is solely dependent on the amount of money thrown at
Re: (Score:3, Interesting)
I'm glad a lot of research is finally gearing more towards the path of having a small initial program, then feeding it data and letting it grow into it's own intelligence.
This idea is the holy grail of AI since the early ages. The project described is one amongst thousands done, and you'll likely see news about such projects pop every couple of months here on Slashdot.
The problem is that such a project has yet to produce interesting results. The reason why the most successful AI projects you hear about are human-organized databases and expert-systems, or human-trained neural networks for instance, is because they are the only ones that produce useful results.
Also, consider
Re: (Score:2)
Re:Finally, people are getting AI right. (Score:4, Interesting)
Building a giant database of knowledge hasn't been possible for very long, because computers didn't have very much memory. When system capabilities first reached the capacity to do so, it had to be constructed from hand because there was no online repository of information to extract data from: the internet just wasn't very big. That particular project was known as Cyc, and it cost a lot of money.
Since that time, the internet has grown and there are massive amounts of information available. It will be interesting to see the resultant quality of this database, to see if the information on the internet is good enough to make it usable.
Re: (Score:2)
Machine learning algorithms (Score:4, Insightful)
Only as good as current machine learning algorithms.
So not very.
Re: (Score:3, Insightful)
It's not as if human use of "machine learning" algorithms is any faster. It takes about 12 months for our neural networks to figure out that the noises we make elicit a response from our parents. And according to people like Chomsky, our neural networks are designed for language acquisition.
AI "ought" to be an easy problem. But there's one big difference in the psychology of humans, and of computers. Humans have drives, like hunger, the sex drive, and so on. In particular, an infants' drive to eat is a
Re: (Score:2)
It's not as if human use of "machine learning" algorithms is any faster. It takes about 12 months for our neural networks to figure out that the noises we make elicit a response from our parents. And according to people like Chomsky, our neural networks are designed for language acquisition.
I don't know who you are quoting for this, or what the 12 months is measuring in terms of from birth or from conception, but I will assure you that my children certainly recognized my voice even when they were in my wife's womb. I have a seven month old daughter right now that not only can figure out the noises, but is responding and addressing myself, my wife, and my other kids by name. I'm not saying that she is ready to orally give a doctoral dissertation discussion, but she is communicating and displa
lolwut? (Score:4, Funny)
Why do I get the feeling that the bot's first words are going to be OMGWTFBBQ?
Re: (Score:2)
LOL NOOB
Re: (Score:2)
Why do I get the feeling that the bot's first words are going to be OMGWTFBBQ?
Except that is not a word, let alone words.
Re: (Score:2)
It is when you learn English by trolling the Intarwebs.
-dZ.
Re: (Score:2)
Non english text (Score:3, Interesting)
What happens when this program stumbles across text written in a language other than english? Or how about random nonsensical text? How does it know that the text it learns from is genuine english text?
Re: (Score:2)
Re: (Score:3)
Thus if it stumbles across
Re: (Score:2)
Re: (Score:2)
I assume it would be promoted to slashdot editor...
Iz dis... (Score:2)
lke, rally der bestest ways like ter learn a puter inglish isit!!!??!?!
Seriously though, poor AI; if I had a gun I'd go and put it out of its misery.
Once this thing hits Encyclopedia Dramatica... (Score:2)
...it will forever be stuck at the level of a retarded 8 year old. Or the level of a normal 4-chan user.
Re: (Score:2)
Re: (Score:2)
Same thing.
Re: (Score:2)
You're giving 4chan users credit for a lot of maturity there....
I think AI needs a 3d imagination to know English (Score:3, Interesting)
Re: (Score:2)
Re: (Score:3)
while (1) (Score:2, Funny)
Yeah, I've coded an infinite loop a few times, how come I never made the headlines on Slashdot?
Pruning (Score:3, Interesting)
In general I find that the quality of a data set tends to be determined by the number (and quality) of man hours that go into maintaining it. Every database accumulates spurious entries and if they aren't removed the data loses it's integrity.
I'm very skeptical of the idea that this thing is going to keep taking input forever and accumulate a usable data set unless an army of student labor is press-ganged to prune it.
The web: What a great source of information (Score:2)
>Rather, its progress in categorizing complex word relationships is the object of the research.
From the web? Half the people here are writing English as a second language; the rest, haven't finished learning the language, or cannot be bother to string a sentence together. Just what is this program going to learn?
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
No, just wondering if anyone would notice, so well done.
V*yger 2.0 ? (Score:3, Interesting)
Re: (Score:2)
Re: (Score:2)
already been done (Score:5, Informative)
There is simply no existing database to tell computers that "cups" are kinds of "dishware" and that "calculators" are types of "electronics." NELL could create a massive database like this, which would be extremely valuable to other AI researchers.
This is what they are trying to do, based on information they glean from the internet. It's already been done, with Cyc [wikipedia.org]. The major difference seems to be that Cyc was built by hand, and cost a lot more. It will be interesting to see if this experiment results in a higher or lower quality database.
Also, I question their assertion that it would be extremely valuable to other AI researchers. Cyc has been around for a while now, and nothing really exciting has come of it. I'm not sure why this would be any different.
Re: (Score:3, Informative)
They're on the right track (Score:2)
When I first read about Cyc I immediately thought that this is the way to go. And this was before the WWW took off. While I don't think that knowing about the world is all that's needed for AI, I think that without knowing about the world you can't have any AI or at least none you'd recognize.
Intelligence (as we know it) is mostly about interacting with and understanding your environment and having some environment being accessible to something remotely intelligent is a good start. Every living being is jus
Re:already been done (Score:5, Informative)
What you have done is identified a difference between the two systems, and then claimed that this difference is in some way significant. You do this without knowing the implications of the difference, without entirely understanding the difference, and without presenting any evidence that this particular difference matters at all. In short, you think you understand what matters, but in reality you don't.
But fear not, you are in good company with your ignorance: this particularly pernicious fallacy is one that has plagued AI researchers for a long time. It happened with cyc: the founders were sure that if we just had a database big enough, it would result in intelligent machines. They didn't know how, but they were sure it would.
Before them there were master systems, neural networks (long story), natural language translation, and many more that I'm sure I'm forgetting. In all of these cases researchers were certain that their system held the key to vast wonders, only because they had not spent much time thinking about what they were actually trying to accomplish. In most of these cases it would have been obvious that human-level intelligence wasn't going to result, if they had spent more time investigating how the brain works and less time chasing their pet solution.
In general if there is a vast field of ignorance between your method and your desired result, then you should probably spend more time researching, finding data points in that field of ignorance before trying to get to your result. Or in your case, since you present no evidence what difference 'developing on the internet' will make compared to 'developing by hand', you should go do a little searching and figure out what the actual difference will be, instead of randomly guessing.
But since you are lazy and probably didn't read the article, I will give you one hint: this database populated from the internet seems to have a strong bias towards information about companies and sports teams. Who would have guessed that?
Re: (Score:2)
My kids assimilate their own information base. I do not directly inject it into their heads.
You are right, they do assimilate their own information base. This is a very useful observation and data point, and any true strong AI will have to do so. However, it is not possible to infer that because your kids assimilate their own information base, anything assimilating its own information base is superior to anything that doesn't.
In this case, it still remains to be seen whether the automated information assimilation techniques this group is using (and let's face it: the information assimilation m
Re: (Score:2)
well I will add to the compost heap today. When I read the headline, I thought that it may be a more fundamental learning of use and relationship of words and what they describe than what TFA describes. Colleges are in a university is a "trusted relationship"? How very ignorant and disappointing, as every AI project I've ever read about is.
What would be impressive is to form associations as in a list of universities including Carnegie-Melon, or a statement that Carnegie-Melon is a university, then in other
Re: (Score:2)
Re: (Score:2)
The biggest problem I see with Cyc, and this project, is that it is not yet known how the human
42? (Score:2)
How come every time I ask Nell what the answer is to life, all it responds with is "42". When I ask what 42 means, it tells me that I'll need a bigger computer.
Wikipedia (Score:2, Funny)
If only they could train it without the web (Score:2)
Perhaps if there were a book in electronic form that had all English words in it perhaps with a definition of each word.
Re: (Score:2)
Good luck. Notice how words in a dictionary are describe by..... other words!
Convergence (Score:2)
Supervised learning, maybe (Score:2)
The article has too much hype, but the actual work has some potential. For the limited problem they're really addressing, extracting certain data about sports teams and corporate mergers, this approach might work.
Both of those areas have the property that you can get structured data feeds on the subject. Bloomberg will sell you access to databases which report mergers in a machine-processable way; some stock analysis programs need that data. Sports statistics are, of course, available on line. So the p
It's not that the program couldn't stop running; t (Score:2)
.... program that never dies. It runs continuously ..... It's not that the program couldn't stop running; the idea is that there's no fixed end-point
Wow I didn't even think that was physically possible! Maybe google should borrow this tech for their web crawlers. Must be a pain to restart them every day...
The Probable Outcome ... (Score:2)
Quote [wikipedia.org]: "The 20Q was created in 1988 as an experiment in artificial intelligence (AI) The principle is that the player thinks of something and the 20Q artificial intelligence asks a series of questions before guessing what the player is thinking. This artificial intelligence learns on its own with the information relayed back to the players who interact with it, and is not programmed. The player c
Re: (Score:2)
Re: (Score:2)
And more importantly, whether Rule 34 applies to computer-targeted porn.
Re:do... (Score:5, Funny)
I think I see the problem with their code.
All they've done is reproduce the typical office worker. It just sits around and surfs the net all day, without coming back with an answer.
Re: (Score:2)
Oh, and as a minor matter, languages are difficult enough from a syntactic dimension, and the symantics of it (in order to understand a statement, you have to understand the ones prior, the context or framing that may have switched, the built up assumptions that maybe can be discarded, maybe not, etc...) make for a truly fantastically dificult problem.
And still, every newborn human masters all of this without having the faintest explicit knowlegde about anything of this and still learns it within a few years. Is an AI meant to be like a newborn baby (which is in no way intelligent) or like an adult? Most (or all) people become intelligent without knowing how intelligence works or what it is. It's just that everything that doesn't work gets discarded very soon. You start to imitate and to try out what works and what gets results and what not.
Perhaps we ne
Re: (Score:2)
And how will they determine if this gets stuck in some local optimum for certain concepts, and thus stops to learn anything relevant at all about any one given concept or topic?
The report is low on details and high on hype. There are no current algorithms that don't require heavy parameter tuning and constant monitoring to get right. Switching one on for a few years and hoping does not strike me as an exciting story.
I'm pretty sure you didn't become what you are by your parents just switching you on and hoping for a few years... I'm quite certain that there was a bit of heavy parameter tuning and constant monitoring required, too.
And believe me, most kids so unlucky to miss this part also get stuck in a local optimum.