Math Indicates Pollster Is Forging Results 319
An anonymous reader writes "Nate Silver suggests the political pollster Strategic Vision is 'cooking the books. And whoever is doing so is doing a pretty sloppy job.' Silver crunched five years worth of their polling data, and found their reported results followed a suspicious pattern which traditionally suggests fraud. The five-year distribution of the numbers 'is not random. It's not close to random.' The polling firm had already been reprimanded by the American Association for Public Opinion Research for failing to disclose their methodology, though the firm argues they did comply with the organization's request. Their response to Silver's accusation? 'We have a call in to our attorney on this and fully intend to take action that will vindicate us.'"
Re:Why should I care? (Score:5, Insightful)
On important topics such is more dangerous than on less important ones; but its mere existence makes the world a less knowable place either way. Either you have people believing false data, or you have people falling into the essentially nescient "all data are just source biases" position.
Re:Why should I care? (Score:4, Insightful)
Pretend I know nothing about Pollster (which happens to be true). Why should I care whether they've faked results? By that, I mean: do they research options of favorite flavors of cotton candy, or public support for health care reform, or the best style of car, or...? In other words, do they do stuff that actually matters?
Faked polls = astroturfing.
Need I say more?
Re:Why should I care? (Score:4, Insightful)
First of all, I don't think "What do I care" is anything but flamebaiting. Who cares if you don't care?
Second, if they're the same "strategic vision" that the article is talking about, their webpage says
"Strategic Vision has worldwide experience developing tools to measure decision-making, human behavior, attitudes and perceptions. Its globally relevant, comprehensive theory of human behavior creates the most effective strategies addressing decision-making in product development and communications in the widest variety of fields, including automotive, customer service, government and politics, medicine and healthcare, organizational and jury, travel and leisure, food and beverages, and education." So they probably report on anything you will pay them to poll on, or rather, anything you will pay them to make a graph from nothing.
Their self-reported client list [strategicvision.com]. Granted, they may have just made that list up as well.
Lastly, a quote in TFA by the company gives you plenty of reason to care:
There you go: the company is mad about being uncovered and is doing the next step any stupid assholes do when their misdeeds come to light: sue in a vain attempt to keep the information from becoming well known. Therefore, -everyone- should know they're faking the results. I'm tempted to e-mail all their clients with a link to the article. If they go out of buisiness, maybe other shitty companies will finally realize you don't sue people who expose you as charlatans.
Bwhahahah, sometimes I say ridiculous things.
Re:Why should I care? (Score:2, Insightful)
Re:Evolution in Action (Score:3, Insightful)
BREAKING NEWS:
The AP is reporting a major fuckup at Slashdot. The web site cannot even do the most basic task essential to its operation, allows readers to leave comments on articles. No comments were available from anyone employed by the web site. Phones rang and rang and rang. Several other Sourceforge properties had their numbers disconnected due to non-payment.
It is apparent no one in charge of the place gives even a sliver of a fuck, or even reads the front page after articles are posted, as it is 2009 and there are 50 fucking ways to notify the readership of the nature of the problem and the expected timeline for resolution. And that 50 is just from a fucking cell phone. If a person had an actual computer and an internet connection, even a netbook at a Starbucks, the number rises into the 1000s.
Long gone are the days when the popular geek web site devoted to technology actually worked. Long gone are the days when there were actual technical explanations of outages. Instead its more stories about politicians arguing over traffic ticket revenue posted as "Your Rights Online", iPhone slashvertisements, slashvertisements masquerading as book reviews, and links to people's blogs about blogs about news stories, and/or tweets about tweets about press conference summaries.
Re:Why should I care? (Score:5, Insightful)
Well said.
I find it disturbing, too, that the media just reports the polling companies' results, without reporting things like what questions were asked, in what order, how the poll was conducted or who commissioned it, all of which can have a big effect on the results. A lot of "push polling" goes on, especially when the polls are commissioned by special interest groups, business associations, unions or political parties themselves.
I'm not in the US, so I don't know this polling company, but I've had a municipal, provincial and federal election in the past 12 months (with another possible federal election imminent) and I think polling and radio call in shows have a great deal of effect on people's opinions these days, more so than traditional newspaper and television newscasts.
If Strategic Vision was conducting fraudulent poles, I would be looking at their client list and going after whoever paid for them as well.
Re:improbable (Score:3, Insightful)
But here's the deal:
You do the poll. You have to; you can't just make up the numbers. Sooner or later someone would figure out you don't have a phone bank.
But the poll numbers come up as 46 for, 43 against, and the rest undecided.
Now you can't go and say, 98 for, 1 against, and 1 undecided; that's what the communists do and everyone knows they're lying.
But you report it as 47 for, 42 against, and the rest undecided. Now you've falsified your data, but you think in a way that's hard to catch. You bump the numbers one or two or three points in favor of your position.
However, I'm unconvinced that this is some sort of smoking gun; Silver needs to really run this sort of simplistic analysis on a lot of other polls and see if there in fact is a bias towards a 47 - 43 split with 10% undecided. That actually sounds about right for a lot of the polls I remember in the last election.
Re:Use stats, not laws (Score:3, Insightful)
Lawyers don't make a reputable firm appear any less reputable.
Lawyers don't make a reputable firm appear any more reputable.
Re:Why should I care? (Score:4, Insightful)
If the vote is to reflect public opinion, people should vote their own opinion. They don't need to try to help the system by guessing the most popular option.
Re:Why should I care? (Score:2, Insightful)
you are so naive
our democracies aren't founded on who's the best candidate, it is on the most popular.
voting isn't about getting the right person for the job, it is all so often trying to make sure the wrong person doesn't get the job... whomever that may be.
if you intend to vote against someone, it is often best to vote for someone that is otherwise popular... that's strategic voting.
Not voting is also strategic, in the sense that your vote won't help anyone but the most popular. It's good to know where your non-vote is going.
Re:Why should I care? (Score:1, Insightful)
They are a partisan, Republican-oriented polling company.
Let me respond to your factual attack with Republican-like grace: BUT ACRON! BLAH BLAH BLAH!
Re:Why should I care? (Score:2, Insightful)
Pretty good on the explanation of the "who" was polled, but not the questions.
Just a silly example: "Are you in favor of decreasing the speed limit on Main Street to 5 MPH?" vs. "Are you in favor of saving cats and squirrels on Main Street?". I know silly example, but it is non-political and illustrates the point the the wording of the question, as well as the sequence of each question, contributes to determining the results of the poll. Even just the tone of voice can push someone in a direction. Think of a good salesperson.
I've not found a link, but I do recall this some years ago when Zogby started up, and was much more accurate than the other pollsters. They explained that their success was due to how openly they asked their questions, trying to word and order them so as to not provoke or create emotions or guide someone to an answer.
So, any poll without the questions and their order is of little value to me, other than infotainment.
Re:Why should I care? (Score:4, Insightful)
I firmly believe that the insurance industry has been paying these pollsters to lower their numbers for the democrats to push them to drop health care reform.
Yeah, you go ahead and cling to the belief that the insurance industry doesn't want the health care bill to go through. Why would they possibly look at 30 Million people who aren't buying their product and support a bill that will require everyone, by force of law, to buy their product?
I'd certainly like to see some numbers regarding who the insurance industry as a whole is contributing to.
rural places need guns to protect from criminals (Score:3, Insightful)
the police are too far away. so we have a status quo here currently in the usa where hundreds of urban dwellers die every year from thugs with guns for the sake of a law which serves only the rural minority. but as the usa continues to urbanize further, and begins to equal european urban/rural ratios, political status quo will fall in line inevitably
and instead of HUNDREDS of urban dwellers dying every year for the sake of rural-friendly laws as we currently have, DOZENS of rural folks will die instead for the sake of urban friendly laws
inevitable. deal with it
"I am not FRINGE because I don't vote."
that's true. your SELF-DISENFRANCHIZED because you don't vote. your vote is your voice in your society. if you seek to not vote, you have willfully removed your own voice, you have chosen to be irrelvant. so why are you still fucking talking? you seek to not be a member of society. which is fine, drop out if you like: in which case, shut up and stop commenting on a society you freely choose not to belong to. if you want your opinion to be considered by us in this society, try to be a part of it by voting, and make your voice heard
but you don't get to drop out of society by your own choice and still think anything you say is relevant
if you want to be relevant, vote, and consider yourself to be a member of the same society as me. or don't, and, in logical coherence with that choice of yours, shut the fuck up
otherwise, there is absolutely zero for me to respect about anything you say, because by your own admission, you choose to not matter to me by not voting
oh you have your gun. awesome: why solve problems with voting when you can shoot, is that your point of view? fucking shizophrenic loser
Re:Why should I care? (Score:3, Insightful)
Actual fraud in any form or sense should not be tolerated.
Many people made decisions based on those polls, including politicians. If the results are not random samples but where cherry picked, it could influence those politicians to support bills and policies that they think the public wants (Patriot Act, Warrentless Wiretapping, Waterboarding, Wars, etc) but in reality they might not actually want as a majority.
This applies for anything using statistics including scientific theories, the same fraud detecting method can be used on scientific theories to weed out the problems and fraud in science.
Executive Summary: (Score:4, Insightful)
Web 2.0
Re:Strategic Visions Inc. != Strategic Visions, LL (Score:4, Insightful)
I love how the ".biz" TLD is effectively the "evil bit".
Re:rural places need guns to protect from criminal (Score:3, Insightful)
You really think that the only people who want guns legal are rural? And that the laws are "rural friendly" in that regards? I've got news for you, the vast majority of gun owners and enthusiasts are urban dwellers, and that isn't looking like it's going to change anymore now than it has in the past couple of decades.
In addition, you really think that the majority of murders with weapons wouldn't happen without weapons? People murdered each other before guns were invented, removing them might make a few cases go away but won't impact the vast majority of homicides.
Good luck voting to stop that bear from getting you by the way, I'm sure he'll listen to your excellently thought out democratic system of determining who he should eat next.
-Someone who owns no guns but isn't dumb enough to think guns are the root of problems humanity has had to deal with for centuries before the discovery of gunpowder.
Re:Random vs Even Distribition (Score:2, Insightful)
Oh shut the fuck up. The guy's a statistician and has proven quite a good one at that. You barely grasp the law of large numbers [wikipedia.org]. Fucking know-it-alls thinking they get all about anything and dismiss claims of experts without even a grasp of the basics of the topic at hand, get the fuck off my Internet.
Re:Not statistically significant (Score:1, Insightful)
Serious question: could all of this just be accounted for by a really bad coding error or data preparation algorithm somewhere when it comes time to round numbers off, such that the trailing digits are non-random? I'm thinking about something like not carrying enough significant figures through the calculation or doing a "floor()" or "ceil()" instead of a proper rounding operation. There's a lot of potential for mangling the results between taking a (supposedly) representative poll across a whole country or region and then trying to scale it up to the full value.
In other words, even if the data is weird, are there innocent explanations? Is it an example of the all-too-common "don't attribute to malice what can be attributed to stupidity" scenario?
Re:Why should I care? (Score:3, Insightful)
When will people learn? (Score:2, Insightful)
You don't need fraud to lie using statistics!
Re:Not statistically significant (Score:3, Insightful)
First, the example he gives where he looks at polls from ALL sources is an example of a plausible distribution of real results because, assuming the majority of pollsters are not cooking their data, the data should be dominated by randomness.
Here is the thing. Did he begin with the theory that Strategic Vision was fraudulent, or did he begin with the theory that some pollsters were fraudulent?
After all, he was churning a lot of pollsters data.
Isnt it quite possible that he was simply mining his massive dataset for something, anything, that made any pollster look bad?
In short, how likely is it for one legitimate pollster out of many legitimate pollsters to have data that isn't quite normal (pun intended?)
Re:Why should I care? (Score:2, Insightful)
Re:Random vs Even Distribition (Score:2, Insightful)
You're a dumbass. RTFA and you'll see how likely these "patterns" are given the huge sample size (tens of thousands of numbers).
I mean seriously, what fucking arrogant cunt would think that a well regarded expert would fall for the most basic of mistakes of his domain of expertise, and that you'd be the one to point it out? Goddamn basement intellectuals...
Re:Why should I care? (Score:1, Insightful)
And I'll be just as blunt: I think the same thing when I see a CBS poll that is wildly different from the consensus. Get over it. :)
News flash: politicians of BOTH PARTIES hire pollsters not just to find out what's going on, but also to ALTER public opinion (it's called "push polling"). Where you will lose me is when you start insisting that it's primarily Republicans or Democrats who does it. BOTH do it. Politics is one giant, gelatinous and festering stew of circus-like improbabilities.
As for it influencing my opinion, anyone with half a brain knows that polls will vary depending on where they're done. Ex: do a poll on gun control and gay marriage in the Deep South and the numbers will be something like 90% against in both cases. :)
I also depends on whom they ask, what time of day the poll is done, and -- as others here have pointed out -- the wording of the questions.
Different pollsters have different methodologies. Speaking as a conservative, I've discounted Strategic Vision's numbers for years because they do seem unbelievable at times. But then I'll look at Gallup, Zogby, Rasmussen, and a couple of others that I trust and look for TRENDS. I don't look at actual numbers, I look at TRENDS.
And I hate to tell you this ... but on the health care issue (since you brought it up), Strategic Vision is not the ONLY one showing that it's losing traction, especially amongst seniors. It's not just "republican pollsters" showing this. Look at the consensus: the numbers may go up a little one day, down a little the next, but the overall trend among ALL pollsters has been headed straight down since the actual proposed bill was released online and people had a chance to read the thing.
Bottom line: if you are a partisan Democrat who believes that some giant conspiracy keeps thwarting them, *OR* a partisan Republican who believes that some giant conspiracy keeps thwarting them, I'll (ahem) be as blunt as possible: grow a brain. Grow up. Realize that both sides are self-centered, self-absorbed crooks and that, at the end of the day, you are voting for your crooks only because you believe they are marginally better than the others. :)
Re:Why should I care? (Score:1, Insightful)
You know WHY they were so surprised?
Because THEIR precious pollsters were assuring them that there was "broad-based support" FOR the health care bill. They honestly believed that the people complaining at those meetings were just noisy troublemakers who didn't represent the consensus.
By the time they figured out that THEIR pollsters had been telling THEM what they wanted to hear, the damage had been done. They lost a great deal of support for their health care scheme in the PR department -- calling some mild-mannered Granny or a young couple with two children "Nazis" ain't exactly a great way to sway minds and win public opinion, because the average American can watch TV and think, "yeah, sure, that old biddy looks JUST LIKE a paid 'rent a protestor' from some fringe militia movement."
Jesus help us, people. GET A BRAIN. BOTH parties are guilty of this crap, and have been for years.
Grow up and stop being naive! The Christian Right has allowed Republicans to cynically use them and depend on their vote for years ... just as Blacks and young intellectuals are completely taken for granted by the Democrats. Both sides need to rent a clue.
Re:Not statistically significant (Score:3, Insightful)
Actually, I think most of the large polling organizations are pretty good about releasing their methodology. From the sound of it, this one is kind of an exception, and has taken a lot of flak for it.
You're certainly correct, if you go around comparing things long enough you're likely to get a false positive, unless you correct for multiple comparisons.
I've never actually seen any second-digit analysis before, but election and poll fraud isn't my field. I expect a lot of election monitoring would use similar techniques. A fair bit of work has been done looking at distributions of digits, including Benford's law and I believe work that shows that the fourth and on digits are actually uniformly distributed. There is also some psychological research looking at patterns in numbers that people tend to select. I seem to recall that if you ask a large group of people to pick a number between one and a hundred, a disproportionate number will pick either 32 or 36. It's a trick used by psychics - ask a large audience to pick a number between one and one hundred. Then ask whoever picked 32 to put up their hands. An impressive number of hands go up. Next pretend to be a little uncertain and say, wait, I'm also getting a strong signal on 36... and suddenly a bunch more hands go up. In a big audience it suddenly looks like most of the hands are up and now you can take their money.
Re:Why should I care? (Score:2, Insightful)
A) I agree with much of what you said. I also did not accuse republican politicians of anything. I accused R pollsters and the insurance industry of working together to mislead everyone...including republican politicians.
B)I agree with you it is much better to look at trends. So my point was that although the trend has been downwards for the democrats. The slope for Rasmussen and Strategic vision and has been very different than the slope of the over all trend.
C) If you already agree that Strategic Vision tends to be overly biased is it that big if a leap to think a few else have been. I am honestly mostly concerned with Rasmussen who is one of the big names. He previously have been "close" to the norm but more recently he has gone on TV and publicly endorsed republican ideas and since than his polls have gone on to be more and more extreme.
"And I hate to tell you this ... but on the health care issue (since you brought it up), Strategic Vision is not the ONLY one showing that it's losing traction" Please don't think you corrected me. I already stated that in my first post my point was their results have been much different than the downward trend of other pollsters.
To your last comment about crooks. I cant disagree more. Yes their are crooks in both parties. Ill point out how i think very very lowly of Max Bacus and Ben Nelson right about now. But for the most part I think they are good hearted power hungry people. The reason to me why "R" get labled as corrupt more often is due to the fact they general have some of the richest industrial friends.