Want to read Slashdot from your mobile device? Point it at m.slashdot.org and keep reading!

 


Forgot your password?
Close
typodupeerror
×
The Internet AI Communications Social Networks The Media Twitter Technology

Researchers Develop an Internet Truth Machine 87

Posted by timothy from the how-to-create-better-liars dept.
Hugh Pickens writes "Will Oremus writes that when something momentous is unfolding—the Arab Spring, Hurricane Sandy, Friday's horrific elementary school shooting in Connecticut—Twitter is the world's fastest, most comprehensive, and least reliable source of breaking news and in ongoing events like natural disasters, the results of Twitter misinformation can be potentially deadly. During Sandy, for instance, some tweets helped emergency responders figure out where to direct resources. Others provoked needless panic, such as one claiming that the Coney Island hospital was on fire, and a few were downright dangerous, such as the one claiming that people should stop using 911 because the lines were jammed. Now a research team at Yahoo has analyzed tweets from Chile's 2010 earthquake and looked at the potential of machine-learning algorithms to automatically assess the credibility of information tweeted during a disaster. A machine-learning classifier developed by the researchers uses 16 features to assess the credibility of newsworthy tweets and identified the features that make information more credible: credible tweets tend to be longer and include URLs; credible tweeters have higher follower counts; credible tweets are negative rather than positive in tone; and credible tweets do not include question marks, exclamation marks, or first- or third-person pronouns. Researchers at India's Institute of Information Technology also found that credible tweets are less likely to contain swear words (PDF) and significantly more likely to contain frowny emoticons than smiley faces. The bottom line is that an algorithm has the potential to work much faster than a human, and as it improves, it could evolve into an invaluable 'first opinion' for flagging news items on Twitter that might not be true writes Oremus. 'Even that wouldn't fully prevent Twitter lies from spreading or misleading people. But it might at least make their purveyors a little less comfortable and a little less smug.'"
This discussion has been archived. No new comments can be posted.

Researchers Develop an Internet Truth Machine More

Researchers Develop an Internet Truth Machine

Comments Filter:

  • Cultural bias? (Score:5, Insightful)

    by Anonymous Coward on Sunday December 16, 2012 @09:30AM (#42306737)

    This is really interesting research, but it's also based on one event in one country.

    Conclusions based on what may be language or cultural norms (such as "did you phrase in the positive or the negative") might not translate to other locales well (e.g. Hurricane Sandy in the US).

    But, then, that's what's great about science. Testable predictions we can apply to data.

    • You could use the same algorithm to derive credibility indicators for any language and region and use multiple verified events and facts to train the system.

      By the way, what about no pronouns vs. 1st-person and 3rd-person? What about no emoticons?

      What about links to known-unreliable sources as opposed to nominally credible sources?

      • You could use the same algorithm to derive credibility indicators for any language and region and use multiple verified events and facts to train the system.

        But what if its results leak, and bird song adapts to meet expectation, but without actually being more reliable?

        • You could use the same algorithm to derive credibility indicators for any language and region and use multiple verified events and facts to train the system.

          But what if its results leak, and bird song adapts to meet expectation, but without actually being more reliable?

          Arms race.

  • I wonder... (Score:2, Interesting)

    by Anonymous Coward

    How effective would this be on real media? I bet it'd put those bastards in their place! :)

    • You've used a question mark, an exclamation mark and a positive smiley. Thus you lost any credibility, according to the cited criteria.

  • Rating individual tweets, accurate? (Score:5, Insightful)

    by JaredOfEuropa ( 526365 ) on Sunday December 16, 2012 @09:35AM (#42306759) Journal
    So it provides a first opinion on first posts, sort of. Neat, but I do wonder how accurate this is going to be to vet individual tweets. Twitter trolls may get wise to this and game the system to get their stuff past this filter. A bit like phishers learning how to spell. In the end, the best check is still independent verification, for example by other people tweeting the same thing (not just retweeting of course). If this system could automatically group and cross-verify tweets from multiple sources on the same subject, that would be a step in the right direction.

    • Re: (Score:2)

      by Hentes ( 2461350 )

      The best check is the site of an actual seismologist. Tweets shouldn't be trusted in emergency scenarios.

  • Couldn't some enterprising douche programmer use simular programs to write better misleading tweets.
    • Comment removed based on user account deletion

    • Re: (Score:2, Insightful)

      by artor3 ( 1344997 )

      Sure, but most people tweeting false info in a disaster are just stupid kids (or man-children) who think its funny. They're probably not going to put lots of effort into it, because then it wouldn't be fun.

    • Re: (Score:2)

      by SeaFox ( 739806 )

      Of course. We have SEO programmers.
      Coming up next, TTO -- Twitter Trustworthiness Optimizers.

      We start with lots of sock-puppet follower accounts, add a pessimistic spin and frowny faces. Also use links that will probably lead to astroturfing sources, and finally give the tweet a healthy copy-edit before it's posted to made it in a first or second-person perspective and make it a declarative, expletive-free message.

  • Chile's Earthquake (Score:5, Interesting)

    by thejynxed ( 831517 ) on Sunday December 16, 2012 @09:40AM (#42306773)

    It's interesting to note, that a seismology student at a university in Chile finally had enough nonsense from false information over Twitter, etc about earthquakes, that he directly wired a big batch of seismographs to directly post their results via Twitter. The last I knew, they had over 1 million followers, and this particular student has been getting big thank yous from residents of the country.

  • Reliable (Score:5, Funny)

    by Anonymous Coward on Sunday December 16, 2012 @09:44AM (#42306781)

    Twitter is the world's fastest, most comprehensive, and least reliable source of breaking news

    Twitter has dethroned Fox News?!?

    • It went beyond that. It is even less reliable than MSNBC.

    • Re: (Score:1)

      by Anonymous Coward

      With Fox News, you can reliably conclude that the opposite of what they say is true.

    • No political bias there. Uh huh.

  • this is pretty stupid (Score:1)

    by Anonymous Coward

    let me FTFY:

    "... the results of Twitter misinformation can be potentially deadly... team at Yahoo has analyzed tweets... to automatically assess the credibility of information... A machine-learning classifier developed by the researchers uses 16 features to assess the credibility... : credible tweets tend to be longer and include URLs; credible tweeters have higher follower counts; credible tweets are negative rather than positive in tone; and credible tweets do not include question marks, exclamation marks

  • This is crap (Score:3, Interesting)

    by Anonymous Coward on Sunday December 16, 2012 @10:20AM (#42306859)

    One of the criteria in their algorithm seems to be that credible tweets were

    ... significantly more likely to contain frowny emoticons than smiley faces.

    They were evaluating tweets about a disaster; not a lot of smiley faces there.

    The algorithm seems to have a bias toward bad news. So, if my buddy tweets that a rare Belgian beer will be available at the local liquor store, the algorithm will decide that it isn't credible because of the smiley face.

    We just had the above case. Beer that you usually have to cross the Atlantic to get became available for about 30 minutes locally. Some of us lined up starting at 3:00 AM. I would have been really ticked off if some algorithm had made me miss the news.

  • Who stops to type emoticons in the middle of a natural disaster (including switching to the alternate keyboard to get those characters)?

    • Who stops to type emoticons in the middle of a natural disaster (including switching to the alternate keyboard to get those characters)?

      the same people who sing in the rain?

    • Re: (Score:3)

      by grcumb ( 781340 )

      Who stops to type emoticons in the middle of a natural disaster (including switching to the alternate keyboard to get those characters)?

      It happens. When the Rabaul Queen [wikipedia.org] capsized[*] in heavy seas, killing an estimated 321 people, there were dozens of tweets and facebook posts from people on board. They used emoticons because it's a lot easier to write :-( than it is to write 'I'm really frightened right now.' Let me tell you, when I was assigned to write about the disaster, it was very, very difficult to read those posts and remain unmoved.

      Moral: Don't make assumptions about people's state of mind unless you have some insight into what they

      • So you're saying that you believe that people who are facing their last moments on Earth, if given wifi/cell access during that time would/should NOT call their loved ones to say good-bye/"I love you", but should/would post on twitter instead, and include emoticons?

        ps -- Thanks for your de rigueur introduction of victimhood into the discussion. :-O
  • There's two topics here, one is use of potentially valuable information by, say, emergency responders (leads, evidence, etc.). The program could be useful. The second (e.g. "don't use 911") is "a headline", i.e. it is aimed at spreading news (or troll farts) as media to the social public. These are definitely two completely separate problems to solve. The second problem is best solved by evolution, as people who get their "news" off of social media become even stupider than they were to begin with and die off.

  • Twits (Score:2, Funny)

    by blagooly ( 897225 )
    It is Twitter, not Tweeter. Therefore Twits. Not Tweets. Twits.

  • Gaming Reliability/Credibility Assessment (Score:5, Interesting)

    by girlinatrainingbra ( 2738457 ) on Sunday December 16, 2012 @11:02AM (#42307009)
    Of course, in just the same way that spammers can game Bayesian spam filters or rule-matching pattern filters by knowing what the rules are, given a known set of rules that attempt to assess credibility of tweet allows someone to tweak their tweets in order to be assessed as having high credibility:
    1 -- max out your tweet length
    2 -- include an URL [doesn't say whether to use a link shrtnr ;>(]
    3 -- use a Twitter account with a high number of followers
    4 -- use a negative tone
    5 -- no question marks or exclamation points
    6 -- use 2nd person (same as don't use 1st or 3rd person)
    7 -- don't use swear words
    8 -- use a sad emoticon
    .
    Example to maximize this:
    a - break into / hack a high follower account (e.g. justinbieber) and tweet: cat > finaltweet
    You should know Mayan Calendar sez: world ending this week. Confirmed@ http://netcraft.calendar.mayan/ [netcraft.calendar.mayan] you go hug loved 1s now. :>( beebs
    wc finaltweet
    1 20 139 finaltweet

    First iteration was:
    gia@sodium$ cat > count2
    You should know that Mayan Calendar says : world ending within week. Confirmed by http://netcraft.calendar.mayan/ [netcraft.calendar.mayan] , you should hug loved ones now. :>( -- beebs
    gia@sodium$ wc count2
    1 25 159 count2
    Please note that the "[netcraft.calendar.mayan]" was inserted by /.'s /-code and is not part of the wc wordcount :>(

    • Or what about this:

      The world is going to end. :-( It will be eaten by a black hole approaching Earth, reaching us on Dec 21. See http://gaotse.cx for details.

      (Link intentionally misspelled)

  • Looking for disaster, just look at car commercials (Score:3, Insightful)

    by DarthVaderDave ( 978825 ) on Sunday December 16, 2012 @11:46AM (#42307161)
    If you weren't aware that Hurricane Sandy, Irene or whatever occurred, just tune into the local television and watch the car commercials. If I see one more Maxon, Salerno Dwayne, Rutherford Ford or Honda Hurricane Sandy stimulus event, I'm going to throw up. THAT is how you know something bad has happened.

  • There's a basic problem here. (Score:3, Insightful)

    by __aaltlg1547 ( 2541114 ) on Sunday December 16, 2012 @11:58AM (#42307221)

    The basic problem with any such approach is that tweets are individual opinions and you cannot arrive at the truth or falsehood of objective facts by analyzing a collection of he-saids and she-saids.

    The hospital is either on fire or it is not on fire, regardless of what anybody says.

  • Researchers...also found that credible tweets are less likely to contain swear words (PDF) and significantly more likely to contain frowny emoticons than smiley faces

    .

    Hey, that's pretty cool! :)

    I mean, that's pretty cool! :(

    • I think for the first one you wanted to write: "Hey, that's fucking cool! :)"

      And for the second one, you don't want the exclamation mark. That was also claimed to be a sign of non-credibility.

  • credible tweets do not include question marks, exclamation marks, or first- or third-person pronouns

    Don't write "Help!" (exclamation mark" or "please help me" (first person pronoun).

  • How to know they are real valuable is when they are censored

    My tweets were censored because they had URLs, even to Twitlonger.
    So I resorted to these tweets instead @ ~140 characters limits (how long a tweet can be):

    #taxes 1) The Declaration of independence recognized the peoples rights & duty to ... remove budgeting & accounting failed tasks from Gov't.
    #taxes 2) for proper representation, given all the budgeting & accounting fails, &more, the people must direct where their taxes R 2 B used.
    #

  • Snopes needs to borrow this algorithm and create a subsection devoted to Twitter. It will highlight the unreliable posts and list which criteria made them fail the sniff test. Then, if there's time and resources, a human being might follow up the most significant ones and flesh out the stories.

  • How combining learning algorithms with community moderation?
  • Credible tweets are negative? How is "Coney Island hospital is on fire" or "don't use 911, the lines are maxed out" positive in tone?
  • This reminds me of the anecdote about a DOD learning "AI" program to identify tanks in images that worked perfectly in the lab. We they took it into the field it didn't. They taught it by showing it pictures of landscapes with and without tanks. As it turns out, all of the tank pictures also had clouds and all of the no tank pictures didn't have clouds. So the AI was working, doing exactly what it was taught, identifying clouds.

  • Headline should read "Researchers Develop Tool For Twitter Trolls To Improve Plausibility Of Their Tweets"

Slashdot Top Deals

Beware of Programmers who carry screwdrivers. -- Leonard Brandwein

Close