Cheaters Exposed Analyzing Statistical Anomalies 437
Hugh Pickens writes "Proctors and teachers can't watch everyone while they take tests — not when some students can text with their phones in their pockets, so with tests increasingly important in education — used to determine graduation, graduate school admission, and — the latest — merit pay and tenure for teachers, Trip Gabriel writes that schools are turning to 'data forensics' to catch cheaters, searching for data anomalies where the chances of random agreement are astronomical. In addition to looking for copying, statisticians hunt for illogical patterns, like test-takers who did better on harder questions than easy ones, a sign of advance knowledge of part of a test or look for unusually large score gains from a previous test by a student or class. Since Caveon Test Security, whose clients have included the College Board, the Law School Admission Council, and more than a dozen states and big city school districts, began working for the state of Mississippi in 2006, cheating has declined about 70 percent, says James Mason, director of the State Department of Education's Office of Student Assessment. 'People know that if you cheat there is an extremely high chance you're going to get caught,' says Mason."
Headline misleading (Score:4, Insightful)
This doesn't prove anything (Score:4, Insightful)
If I fall into the anomaly category without cheating, I'll be screwed. What can I demonstrate in my defense? Not much. I find it hard to believe they can prove that you cheated without actually video-taping you cheating or something along those lines.
Anomalies are what they are, data anomalies, nothing more and nothing less.
Re:This doesn't prove anything (Score:5, Insightful)
If I fall into the anomaly category without cheating, I'll be screwed. What can I demonstrate in my defense? Not much.
But this is good. Kids should learn that in the real world they'll be arbitrarily punished for doing well merely to further the career of the person they're working for.
Re:This doesn't prove anything (Score:5, Insightful)
In a sane and rational world, yes. In a school? The instant the business flags someone the school is going to treat that as the voice of god himself coming down from the heavens to demand blood and expulsion.
That's not even getting into the blindingly obvious conflict of interest here. Just like turnitin these guys are a business that relies on there always being cheaters to catch, it's in their interest to produce as many "catches" as possible without losing credibility. Hell at least with turnitin you're dealing with something you can prove, with these guys the student literally has no defense.
It's a witch hunt, pure and simple. If they're not failing they must be cheating somehow, if they are failing obviously they're innocent. No matter what the student is fucked.
Re:False Positives (Score:5, Insightful)
That is why they don't automatically assume you are a cheater ... You get flagged and they do a further investigation.
Being accused of cheating in the academic world is kind of like being accused of a sex crime in the world at large -- the burden of proof is essentially on you to prove your innocence no matter what the law says, it's very difficult to prove you didn't do it, there are people who will go to insane lengths to get you convicted, and even if you're cleared the damage to your reputation is done.
Catching cheaters is missing the point (Score:5, Insightful)
There seems to be an increasing emphasis by schools on "catching cheaters". This seems to be missing the point.
We send our kids to school not so they can pass tests. I honestly do not care if my kid gets an "A" or an "F" on the test; I care that he actually learns the material. Tests are a tool that educators can use to help them determine if a child is learning the material but passing grades shouldn't be the goal. If students are cheating on tests then you need to look at the reason why. Is the material being presented in a way that is too hard for the child to understand? Is it not being presented in a way that interests the student? If a student is intererested, he will learn. If he learns, then he has no need to cheat.
Stop spending money on anti-cheating technologies. Spend money on improving the methods of education.
Re:Wouldn't this require specially designed tests? (Score:4, Insightful)
Reading comprehension FAIL. Caveon is too busy doing work to freely publish their methods, as you say for peer review. However, if you pay them (thus becoming a client), they have someone available to explain it to you.
Logic FAIL. If there is someone at the company who can take the time to explain the methods to clients (and I mean really explain, not just spout a bunch of buzzwords) then that same person can also take the time to write an article suitable for publication in a peer-reviewed journal.
Also, methodology FAIL. Writing up results for publication is not just something you do when you have time, when you get around to it. In any real research field, it is an integral part of doing the work.
Re:Wouldn't this require specially designed tests? (Score:3, Insightful)
Also, methodology FAIL. Writing up results for publication is not just something you do when you have time, when you get around to it. In any real research field, it is an integral part of doing the work.
Believing there's no world outside of Academia/Research FAIL? Last I checked companies selling a product don't get paid for research. Their "integral part of doing the work" is selling the product. At the end of the day nothing else matters. Publishing their methods in a peer reviewed journal would necessitate the marketing gains from proving their work to outweigh the advantage they'd be giving competitors insofar as the ability to duplicate their methods. And yes, any competing company worth their salt would definitely read the published papers and implement the methods if they were found to be better.
Re:Headline misleading (Score:4, Insightful)
The real problem is that people are lazy and want to get the best return for the smallest investment.
One man's laziness is another man's efficiency. Take morality and ethics out of the equation and the two are virtually synonymous.
Re:Wouldn't this require specially designed tests? (Score:2, Insightful)
Then why didn't they just say "we don't want our competitors to be able to use this" in the first place instead of lying? Honesty FAIL?