Turnitin's Different Messages To Students, Teachers 306
Economist David Harrington (spotted via Tyler Cowan's Marginal Revolution) charges anti-plagiarism service Turnitin with "playing both sides of the fence, helping instructors identify plagiarists while helping plagiarists avoid detection." Turnitin analyzes student papers for suspicious elements in order to spot the plagiarism, scanning for things like lifted quotations or clever rephrasing. However, the same company offers a counterpart — a scanning service called WriteCheck which essentially lets the writer of a submitted paper know whether that paper would pass muster at Turnitin, and thus provides a way to skirt it (by tweaking and resubmitting). Harrington gave these two systems an interesting test, involving several New York Times articles and a book he suspected of having lifted content from those articles.
Re:dunno (Score:5, Informative)
Your idea falls apart for the simple fact that Turnitin.com doesn't suggest false positives.
For the last year and a half, I was a Teaching Assistant assigned to a senior-level engineering ethics course at a major university. We had about 650+ students every semester, and each of them would submit 3 essays via Turnitin.com. Out of the 14 TAs and 4 professors associated with the course, it fell to me to check for cheaters via Turnitin.com for all of those essays.
To make a long story short, rather than the essays going into a black box with Turnitin.com spitting out a list of of cheaters on the other side, which is what you seem to think happens, Turnitin.com ranks the students by telling the instructor what percentage of the student's paper is a match with other sources. It doesn't label them as a cheater or automatically give them a 0. Instead, the instructor can click on the essay via Turnitin.com, and Turnitin.com will highlight each portion of the essay for which it found a match, showing the instructor the original text side-by-side with the essay. That allows the instructor to make an informed decision on whether or not the student is guilty of plagiarism.
Had they not structured it that way, you'd be absolutely correct. In my time with the course, it wasn't uncommon to see average scores for matches in the 15-25% range (i.e. 15-25% of every student's paper could be identified as coming from another source). Most of that, however, was either a result of quotations, coincidental phrasing (there are only so many unique ways to discuss ideas on a narrow topic, so there's quite a bit of overlap between 650 students), or bibliographies (Turnitin.com can be told to ignore bibliographies, but if you don't have it do that, then they'll oftentimes show up as a match with the other students citing the same references).
Anyway, because of the severity of academic dishonesty allegations (I saw one estimate that failing a class due to academic dishonesty costs a student about $100,000 over their career), a responsible instructor would never rely on a black box to tell them who was cheating. Every academic council, honor council, review board, or whatever else I've heard of demands to see evidence before punishing a student, and a responsible instructor should have had that prepared whenever they made the initial allegation anyway. Turnitin.com's job, rather than labeling cheaters, is merely to identify possible plagiarism and put the information in the hands of the instructor so that they can make an informed decision, and it does that job well.
Turnitin as a teaching tool (Score:3, Informative)
I was a technical advisor to a committee creating policy for Turnitin style service use on the university campus I work on. Turnitin isn't a plagiarism detection service: they're being disingenuous when they say that. It is a text matching service. The difference is significant: a first-year history paper might be 75% matched, but not plagiarized because the student correctly attributed all their quoted passages.
The committee recommended against using it for detecting plagiarism, and for encouraging its use as a teaching tool to make students aware of proper citation techniques and the importance of avoiding plagiarism.
Some service like this also happen to be quite good at the most common kind of plagiarism: someone on campus submitting someone elses paper from the previous year to a different prof... but that's a special clear-cut case of cheating, not what people commonly think of as plagiarism.