Forgot your password?
typodupeerror
Social Networks Stats Twitter United States News Politics

Geomapping Racism With Twitter 409

Posted by samzenpus
from the following-the-trail dept.
Hugh Pickens writes "Megan Garber writes that in the age of the quantified self, biases are just one more thing that can be measured, analyzed, and publicized. The day after Barack Obama won a second term as president of the United States, a group of geography academics took advantage of the fact that many tweets are geocoded to search Twitter for racism-revealing terms that appeared in the context of tweets that mentioned 'Obama,' 're-elected,' or 'won,' sorting the tweets according to the state they were sent from and comparing the racist tweets to the total number of geocoded tweets coming from that state during the same time period. Their findings? Alabama and Mississippi have the highest measures followed closely by Georgia, Louisiana, and Tennessee forming a fairly distinctive cluster in the southeast. Beyond that cluster North Dakota and Utah both had relatively high scores (3.5 each), as did Missouri, Oregon, and Minnesota. 'These findings support the idea that there are some fairly strong clustering of hate tweets centered in southeastern U.S. which has a much higher rate than the national average,' writes Matthew Zook. 'But lest anyone elsewhere become too complacent, the unfortunate fact is that most states are not immune from this kind of activity. Racist behavior, particularly directed at African Americans in the U.S., is all too easy to find both offline and in information space.'"
This discussion has been archived. No new comments can be posted.

Geomapping Racism With Twitter

Comments Filter:
  • Actually Measured (Score:5, Insightful)

    by kenh (9056) on Sunday November 11, 2012 @10:18AM (#41949137) Homepage Journal

    How did they account for multiple racists tweets from one "tweeter"?

    One racist sending 100 racist tweets is not the same as 100 different racists each sending one racist tweet each.

    • Re:Actually Measured (Score:5, Informative)

      by stranger_to_himself (1132241) on Sunday November 11, 2012 @10:39AM (#41949273) Journal

      How did they account for multiple racists tweets from one "tweeter"?

      One racist sending 100 racist tweets is not the same as 100 different racists each sending one racist tweet each.

      Reading the article it doesn't look like they bothered. And they only found a total of 395 tweets which will lead to appalling precision in any of their findings. Sadly 'information scientists' don't always appear to be the best statisticians.

      • by Sir_Sri (199544) on Sunday November 11, 2012 @10:57AM (#41949341)

        With only a couple of days work this isn't bad. But it's not science, it's interest and a proof of concept for doing actual research.

        • by stranger_to_himself (1132241) on Sunday November 11, 2012 @11:11AM (#41949431) Journal

          With only a couple of days work this isn't bad. But it's not science, it's interest and a proof of concept for doing actual research.

          I accept they didn't work very hard on this but in that case its irresponsible to be promoting the findings among people who clearly won't bother to understand the (immense) limitations of the method. It's slightly irritating that as far as the general public is concerned this kind of back of the envelope calculation is indistinguishable from proper science. I wouldn't publicise any findings until I'd had them peer-reviewed and published. But then maybe I'm old-fashioned (and maybe this is why I don't have an academic blog)

          • by Sir_Sri (199544) on Sunday November 11, 2012 @12:11PM (#41949835)

            I wouldn't publicise any findings until I'd had them peer-reviewed and published.

            Then you'd never get funding for a project like this.

            They're demonstrating that there might be something interesting to study, the press lets them ask for money rather than beg, and they're not all that invested in a project that might not go anywhere.

            its irresponsible to be promoting the findings among people who clearly won't bother to understand

            I hate to break it to you, but the press doesn't understand peer reviewed work any better. Whenever media ever looks at any academic work they completely misrepresent it. That's something you get used to.

            • Re:Actually Measured (Score:5, Informative)

              by stranger_to_himself (1132241) on Sunday November 11, 2012 @01:09PM (#41950213) Journal

              I hate to break it to you, but the press doesn't understand peer reviewed work any better. Whenever media ever looks at any academic work they completely misrepresent it. That's something you get used to.

              You are right but this means that the peer review filter is even more important so that what gets out to the media and beyond has at least some chance of being right. Also, having been through the process a few times I'd say academics are at least as guilty of overstating their findings as journalists. We want the headlines and the 'impact' as much as journalists to.

              • Re:Actually Measured (Score:4, Interesting)

                by Sir_Sri (199544) on Sunday November 11, 2012 @05:15PM (#41951789)

                what gets out to the media and beyond has at least some chance of being right

                I don't think that has ever worked for anyone in the last 20 years I have no reason to believe it will start now.

                Not too long ago /. had posts from the communications department of the university of western ontario, which is where I am a researcher, and from our own university the document was a poor characterization of what the research actually was (HIV vaccine stuff in this case, though I'm in comp sci and they don't do our work any better). Somewhere along the line someone decided that the 'public' only understand high level concepts, so everything we communicate is written as thought it was for a 16 year old to understand. It doesn't matter than dozens of other research papers and groups will actually have to do the work to make the thing the 16 year old understand though, we talk about pieces of a puzzle as though they are a solution to the puzzle. And there's no central media authority who might change it.

                • Re: (Score:3, Informative)

                  by SomePgmr (2021234)

                  Somewhere along the line someone decided that the 'public' only understand high level concepts, so everything we communicate is written as thought it was for a 16 year old to understand.

                  I don't know if it's true or not, but I've heard that it's common for news sources to target somewhere near a 6-8th grade reading and comprehension level. I imagine that's a pragmatic approach if you'd like to get a message out and reach is more important than detail.

                  As for this little report, I noticed that you see dots where there are lots of people, and a shotgun pattern in the deep south. Not particularly surprising.

                  I'd like to see something like this that figures racists tweets by unique persons, as a

        • by eldavojohn (898314) * <eldavojohn.gmail@com> on Sunday November 11, 2012 @11:15AM (#41949463) Journal

          With only a couple of days work this isn't bad. But it's not science, it's interest and a proof of concept for doing actual research.

          I think it's absolutely horrible and the fact that these states names but not their numbers have found their way into headlines and a Slashdot summary makes me sick. They might have been right to indict the Southern states that we already know have issues along these lines but their map of tweets [geocommons.com] lists precisely one tweet for Utah and one tweet for North Dakota. The really appalling thing about the North Dakota tweet is that it is geolocated to Minot, a town that has seen an explosive growth in transient workers from states like Oklahoma and Texas in order to meet the demand for workers with oil specialties in the oil fields near there. It's probably a fifty/fifty shot the tweet was from an actual permanent resident of North Dakota.

          Basically if a low population states hits the top of your study and the data is that sparse (one tweet!) then I think you should omit that as an outlier and stricken those names from your press release. It's great to recognize these things in your data and to talk about them in your analysis. It's unjust to propagate just their names throughout the news making people think that North Dakota is not only cold and sparsely populated but it's also racist.

          Someone in Salt Lake City could have been joking in one tweet and suddenly Utah is one of the most racist states in a Slashdot summary. A transient worker who feels like lost his job in OK and had to use his CDL in Minot, ND because a black man was president could fire off an ignorant tweet and suddenly North Dakota is full of racists.

          • by GrumpySteen (1250194) on Sunday November 11, 2012 @11:58AM (#41949755)

            The floatingsheep page specifically says, "we are measuring tweets rather than users and so one individual could be responsible for many tweets and in some cases (most notably in North Dakota, Utah and Minnesota) the number of hate tweets is small and the high LQ is driven by the relatively low number of overall tweets." It's not their fault that the author of the Atlantic article left out those details.

            • The floatingsheep page specifically says, "we are measuring tweets rather than users and so one individual could be responsible for many tweets and in some cases (most notably in North Dakota, Utah and Minnesota) the number of hate tweets is small and the high LQ is driven by the relatively low number of overall tweets." It's not their fault that the author of the Atlantic article left out those details.

              It is their fault for publishing crap that they know will be headline grabbing, no matter how many caveats they put in.

              • by jamstar7 (694492)

                The floatingsheep page specifically says, "we are measuring tweets rather than users and so one individual could be responsible for many tweets and in some cases (most notably in North Dakota, Utah and Minnesota) the number of hate tweets is small and the high LQ is driven by the relatively low number of overall tweets." It's not their fault that the author of the Atlantic article left out those details.

                It is their fault for publishing crap that they know will be headline grabbing, no matter how many caveats they put in.

                Funding. That's how they get it. Post raw data that suggests a problem, apply for funding to monitor and/or fix the problem, rinse and repeat. Academia lives on funding. Without it, those statisticians would have to go get real jobs.

                • by GrumpySteen (1250194) on Sunday November 11, 2012 @01:28PM (#41950351)

                  LOL

                  You clearly did not look at the website. The guys who did this are in academia, but the website is not an academic publication nor would any of the stuff they post garner funding. Their top articles include such things as "The Beer Belly of America", "The Price of Weed", "Church vs. Beer? on Twitter" and "Church, bowling, guns and strip clubs."

                  They're doing this as a fun hobby, not serious academic research. There is no funding, no grants, nothing.

                  • by fatphil (181876)
                    Did you miss the section of the website titled "Publications"? And are you familiar with the concept "publish or perish"? I suspect they're having fun whilst trying not to perish.

                    As for the "no academic research", that's certainly true - I don't think I've seen any less scientific crap since some students spray-painted some grey squirrels red in order to balance the size of the two populations.
                    • "Publish or perish" refers to publishing papers in an academic journals, not putting stuff on a personal website. The fact that you don't understand the difference is sad.

          • by TubeSteak (669689)

            According to the census, North Dakot's population is 90.4% white and 1.3% black.

            The State may or may not be full of racists, but we definitely know that it is not full of black people.

        • by mothlos (832302)

          Actually, it is ridiculously terrible. All it shows is that geolocated timestamped messages can be searched, but either their search criteria was awful or there aren't enough people creating these things to draw any conclusions about a meaningful population. The fact that they then tried to draw state-level conclusions on this dataset shows a feeble grasp of statistics.

        • I can't believe you got modded up. It's basically the definition of science. You can argue the validity of the results, discuss the high potential of error. But it's science.
      • Re:Actually Measured (Score:4, Informative)

        by Hatta (162192) on Sunday November 11, 2012 @11:36AM (#41949593) Journal

        BTW, I just checked out a sample size calculator [surveysystem.com]. For a 95 percent confidence level with a +- 5% confidence interval, and a population of 400 million, guess what your sample size needs to be.

        384.

        Now this calculation for a survey is a little different from what the researchers are doing here, but it illustrates my point. You can do a lot with small sample sizes if the differences between groups are large.

        • by stranger_to_himself (1132241) on Sunday November 11, 2012 @12:55PM (#41950109) Journal

          BTW, I just checked out a sample size calculator [surveysystem.com]. For a 95 percent confidence level with a +- 5% confidence interval, and a population of 400 million, guess what your sample size needs to be.

          384.

          Now this calculation for a survey is a little different from what the researchers are doing here, but it illustrates my point. You can do a lot with small sample sizes if the differences between groups are large.

          That's if they're only trying to estimate a grand rate. To make state-by-state estimates they need this number *per state*.

      • by ultranova (717540)

        Reading the article it doesn't look like they bothered. And they only found a total of 395 tweets which will lead to appalling precision in any of their findings. Sadly 'information scientists' don't always appear to be the best statisticians.

        What findings? Apart from actual racism, racist tweeds correlate with Internet penetration, Twitter penetration, trolls, teenagers, and locale-specific slang. In other words, the study is utterly worthless, regardless of the skill of whoever interprets the data, becau

        • Finding which parts of the country are more infested with trolls is actually more interesting to me. BTW this isn't a troll.

    • by Sulphur (1548251)

      How did they account for multiple racists tweets from one "tweeter"?

      One racist sending 100 racist tweets is not the same as 100 different racists each sending one racist tweet each.

      Not a woofing.

    • Re:Actually Measured (Score:5, Interesting)

      by Anonymous Coward on Sunday November 11, 2012 @11:12AM (#41949437)

      The story also does not count African-American ("black") prejudice/racism toward Latino/Hispanic ("brown"). In reading many articles and comments, most people are unaware of this "black-on-brown" racism but, for those who live and work in minority areas, it is noticeable.

      The Washington Times published a story (three weeks in the weekend issue, IIRC) about "black-on-brown" racism in some major metropolitan settings. One was Memphis (where I lived for 16 years as an adult). The article was full of African-Americans making interesting/telling complains about Latinos/Hispanics. Statements such as "they don't look like us"; "they don't talk like us"; "we can't understand what they say"; they don't eat the same food as we eat"; even, "they don't smell like us". Quite interesting and enlightening articles. Either African-Americans are just as racism as "white" people or the noticing of differences is a normal function of being a human and part of a group.

      African-American are just as prejudiced against people who are not like them or are not a part of their group as any other group.

      Personally, I am at a quandary. Since my ethnicity includes European (northern and southern), African (north and central), Asian (near, middle, and far), and the new "Latino" and older Hispanic, who should I disdain? Which part of me is less than the other individual parts? Quite a problem in our race-oriented political culture. Thankfully, the Knoxville News Sentential ran article on "white" Southerners quoting experts who said all had 5% African blood. This means all Southerners are African-American and can legally claim to be "black" and joint the NAACP, the New Black Panthers, the Democrat Party; they can also change their EEO status and qualify for Food Stamps, Scholarships, etc., etc. much, (Oops, which part of me am I ragging on now?) LOL!!!

      • by CajunArson (465943) on Sunday November 11, 2012 @11:20AM (#41949493) Journal

        STOP POINTING OUT FACTS! We just want to hear reinforcement of our stereotype that all white people are evil racists and all minorities are racially superior since they are completely incapable of being bigots towards anyone!

        Now excuse me while I go to the Black Panther meeting where we discuss how we will be "poll volunteers" again in 2016 to make sure that [insert name of Democrate here] wins because any other vote is automatically racist.

        • Re: (Score:3, Insightful)

          by timeOday (582209)

          all minorities are racially superior since they are completely incapable of being bigots towards anyone!

          Racism isn't just about mere feelings. It's about a group wielding power against others in ways that cause real harm.

          • by Solandri (704621)

            Racism isn't just about mere feelings. It's about a group wielding power against others in ways that cause real harm.

            That's what has me face-palming. From a purely academic standpoint it's an interesting result. However, what they've done is created a report looking for discrimination based on race, but reported it in a way which could be used by people to justify discrimination based on state of origin (e.g. "those stupid redneck racist Southerners").

            It's just as wrong to propagate stereotypes based

            • Re:Actually Measured (Score:4, Informative)

              by Talderas (1212466) on Monday November 12, 2012 @09:07AM (#41955531)

              395 racist tweets from a 0.05% sample works out to 790,000 racist tweets for the country. Even if you assume each racist only posted one racist tweet max, that's 0.25% of the country overall being racist. For the state with the highest rate (Alabama) that's 2%. In an effort to root out racism, this study is presenting their results in a careless fashion which could be used to justify discrimination and anti-Southerner stereotypes against 98% of Alabama residents because of a small minority of bad apples. It is doing the very thing it is criticizing.

              395 that contained search terms that the searcher decided indicated racist intent. There's no indication that any of the tweets that came up were filtered for actual racist intent. For example "Obama won bitch, niggers back" would have came up as a "racist" tweet. However it's likely that such a tweet was not a racist and that the author of it was probably black. If you look at the clustering of where tweets were coming from you see that a high clustering was occurring in cities like Atlanta, Georgia or Montgomery, Alabama which has a majority of a minority population (54% black and 56.6% black respectively).

              The study is flawed and was never properly scrubbed, but the results were posted because it fits a preconceived notion (the south is racist) when the majority of the effect that causes that effect to show may originate from the very population which is the victim of racism against blacks.

      • by codegen (103601)
        While I understand the black on brown racisim exists, why would there be any black on brown racist tweets about Obama being re-elected? He isn't Latino. They were looking for racist comments about Obama. Just curious.
      • Marion Barry was first racist against Asians

        http://www.politico.com/news/stories/0412/74866.html [politico.com]

        We got to do something about these Asians coming in and opening up businesses and dirty shops. They ought to go. Iâ(TM)m going to say that right now. But we need African-American businesspeople to be able to take their places, too.

        He then fake apologized for his racism
        http://www.washingtonpost.com/local/dc-politics/dcs-marion-barry-widely-rebuked-for-comments-about-asian-business-owners/2012/04/05/gIQA27SVy [washingtonpost.com]

      • My first thought upon reading the summary was how in the world did they control for hip-hop/rap's use of the "n-word"?

    • by Shavano (2541114)
      They didn't. It's stated in the article. So prolific racists tweeters may have influenced the results some. I don't know if they also accounted for the common deliberate misspelling of the President's name (0bama), or referring to him as "Hussein" or other such references. I don't want to speculate on how racists on twitter usually refer to the President, but among haters on other forums, I've seen those two references at least as commonly as I've seen the man's name spelled correctly.
    • Jezebel is about presenting everything in a way that makes liberal women outraged.

      Also, like most people in gender studies, they wouldn't know the scientific process or logic even if it pinched their ass.

    • One would hope they would take into account the name of the original tweeter so as not to double count.

      • However, the content and frequency of the posts is also relevant, albeit less so in a quantitative analysis. In some parts of the world, it's an uphill battle to be an overt, extreme racist, and such people must resort to subtle and implied racism. In other parts of the world, with a critical mass of racists, it's much easier to be overt.

        In other words, the comments of the racists say just about as much about the authors as they do of their silently supportive peers.

  • by eldavojohn (898314) * <eldavojohn.gmail@com> on Sunday November 11, 2012 @10:19AM (#41949151) Journal
    I love data porn and tried to play around with this interactive map [geocommons.com]. I lived in Minnesota for 23 years and do not recall it to be very racist -- even in the rural areas. So according to that map there are five red dots in Minnesota which are strangely all centered around the twin cities area (the most populated and liberal part of the state). And that data puts Minnesota mentionably close to the top of the list? But if I look at Virginia, I can't even count the number of red dots there's so many and it's not even halfway up the list? What the hell?

    Do each of these red dots indicate a single tweet? What are the numbers and tweets that they're looking at here, I feel like the LQ value is not doing the best job of reflecting "racism."
    • The dots indicate single tweets.

      The reason Virginia isn't very high up on the list despite the large number of racist tweets is that it was offset by an even larger number of non-racist tweets which aren't shown on the map. If you look at the floatingsheep page linked in the summary, they explain "we aggregated the 395 hate tweets to the state level and then normalized them by comparing them to the total number of geocoded tweets coming out of that state in the same time period."

      As a result, a single racis

    • by bfandreas (603438)
      This whole thing should be taken with a grain of salt.
      You need to take population density into account. More people, more assholes, more asshole tweets.
      Also the sample size is awfully small. So small in fact that multiple counts may weigh in much more significantly than they should.
      Also not all twits are tied to a geolocation.
      You also need to compare it to how the population is composed.

      In fact you shouldn't use this data if you want to find out which states are more unpleasant than others. You could
  • Careful (Score:3, Insightful)

    by sideslash (1865434) on Sunday November 11, 2012 @10:21AM (#41949163)
    They're only looking for racism directed against Obama, so they won't find (for example) black against white racism in Philadelphia or Latino against Caucasian racism in California. It is truly regrettable that certain organizations like the SPLC dilute their otherwise honorable mission by turning a blind eye to hate in some of its notable forms.
    • Re: (Score:2, Interesting)

      by nopainogain (1091795)
      We all know the numbers are skewed. You can make statistics do anything. I live in Philadelphia. We live with Andy Reid and had Donovan McNabb for many years.. Statistically, McNabb threw the fewest interceptions in the NFL..... Factually, he couldn't hit a receiver's hands with a ball if he was given military targeting lasers.
    • Re:Careful (Score:5, Informative)

      by postbigbang (761081) on Sunday November 11, 2012 @10:27AM (#41949209)

      The SPLC does in deed execute their honorable mission. Go to http://truthy.indiana.edu/ [indiana.edu] for other meme propagation and dissemination graphics so you can see that this is one lens to the output of a much larger engine.

    • by kenh (9056)

      SPLC catergorized the New Black Panther Party and it's leader as a Right-Wing Extremist group [splcenter.org], because, apparently, they never allowed for the possibility of a Left-Wing Extremeist group.

  • by jeffb (2.718) (1189693) on Sunday November 11, 2012 @10:32AM (#41949243)

    "Make Trolls Stay On-Topic"

  • by thetoadwarrior (1268702) on Sunday November 11, 2012 @10:58AM (#41949343) Homepage
    The data only accounts for racism specifically targetting Obama by the looks of it. So not surprisingly the states that lost the civil war have the most. But it appears to be counting tweets vs accounts. That makes a huge difference because it only takes one mouthy retard to drive your state up the ranks.
    • by kenh (9056)

      Wow, I wonder if there is a correlation between the way the state voted and their uncovering of racisim in tweets?

      There are several kinds of racists:

      a) Racists that don't know about/don't have accounts on Twitter
      b) Racists that have accounts on Twitter, but refuse to post racists tweets
      c) Racists that have accounts on Twitter AND post racist tweets

      Their "analysis" only picks up the third type of racist.

      Question - would their sophisticated analysis software detect a re-tweet pointing out someone else's racis

      • if I had to guess there is probably no sophistication. I also think it's likely it doesn't pick up most racists because most of them are likely to be older people who don't use twitter. Also the example tweets in one of the links show it's generally all young people (surprisingly a lot of girls too - I thought they were more likely to keep their racism quiet) so I guess it's kind of cool what you can do with twitter's API but it doesn't mean much.
    • Lost the Civil War and rank among the poorest and least educated.
  • by Kr1ll1n (579971) on Sunday November 11, 2012 @11:16AM (#41949467)

    http://campaignstops.blogs.nytimes.com/2012/06/09/how-racist-are-we-ask-google/ [nytimes.com]
    "The state with the highest racially charged search rate in the country was West Virginia. Other areas with high percentages included western Pennsylvania, eastern Ohio, upstate New York and southern Mississippi."

    Different study, different results.

  • by alexmin (938677) on Sunday November 11, 2012 @11:22AM (#41949507)

    Did they really expect clusters in Rocky Mountains?

  • by snspdaarf (1314399) on Sunday November 11, 2012 @11:27AM (#41949539)
    You can find assholes almost anywhere
  • by Jiro (131519) on Sunday November 11, 2012 @11:37AM (#41949603)

    Aside from the problems people already mentioned, if you look in their FAQ, one correction they didn't even mention is correcting it for total Obama support. Obviously if a place has twice as many Obama opponents it's also going to have twice as many racist Obama opponents. But that doesn't prove that it has "more racist Obama opponents" in the sense we normally think of. If you want that you need ratios.

    And 395 is a very small number. They mention that it's not a sample, but it's all the geocoded racist tweets they found, but since it is such a small number, they failed to account for the possibility that there just isn't a lot of racism in the first place, and even if they did look for ratios, "very small percentage compared to another very small percentage" isn't interesting.

    And they mention they didn't bother checking all the hateful comments about Romney (they did check for anti-white comments, but they didn't check for comments reflecting other stereotypes). Their excuse is basically "we were trying to find out about racism, which that's not". The trouble with that reasoning is that while anti-Romney tweets are not germane to what they literally claim to be looking for, they are germane to the subtext of what they're looking for, which is that racism is a big problem--if there are a lot of anti-Romney tweets, that can show that the number of anti-Obama tweets is not really such a big deal. "Blacks called names almost as much as Mormons" makes a bad headline, after all.

  • It's not mapped, but NoHomophobes.com [nohomophobes.com] have a live stream of tweets containing homophobic language. Write up over at the Guardian's Data Store [guardian.co.uk]
  • by nimbius (983462) on Sunday November 11, 2012 @12:01PM (#41949777) Homepage

    if the results arent strictly scientific from a statisticians observation. The overarching point is to draw attention to this kind of behavior and place it in contexts such as the "you lie" incident during healthcare reform, the birther rhetoric and even the tea party itself. This was more an exercise in the patterns and processes of social inequality than it was a mathematical endeavor of quantification in my opinion, and it deserves further research into questions like what are the causes and solutions.

  • by retroworks (652802) on Sunday November 11, 2012 @01:59PM (#41950563) Homepage Journal
    I work internationally, and when the term comes up in a context like, say Africa, where people feel like underdogs but are not minorities, or have multiple splits within the race as in China, it gets really complicated. The term has more meaning in USA or European contexts, perhaps, but since this is "Geomapping" it is a geographic study and I don't think this will work. t would be similar if you were trying to track "Classist" tweets across a geopolitical line where the economic strata are different. They should be measuring "aggression" or "separatism" or something. It would be easier if Twitter got people to add hashtags #Imatroll or #fromadickweed or #aggressiveshithead etc.

Swap read error. You lose your mind.

Working...