Bringing Auto-Graders To Student Essays 227
fishmike writes with this excerpt from a Reuters report:
"American high school students are terrible writers, and one education reform group thinks it has an answer: robots. Or, more accurately, robo-readers — computers programmed to scan student essays and spit out a grade. The theory is that teachers would assign more writing if they didn't have to read it. And the more writing students do, the better at it they'll become — even if the primary audience for their prose is a string of algorithms. ... Take, for instance, the Intelligent Essay Assessor, a web-based tool marketed by Pearson Education, Inc. Within seconds, it can analyze an essay for spelling, grammar, organization and other traits and prompt students to make revisions. The program scans for key words and analyzes semantic patterns, and Pearson boasts it 'can "understand" the meaning of text much the same as a human reader.' Jehn, the Harvard writing instructor, isn't so sure. He argues that the best way to teach good writing is to help students wrestle with ideas; misspellings and syntax errors in early drafts should be ignored in favor of talking through the thesis."
How stupid can you get? (Score:5, Insightful)
This is one area where automatic grading will cause massive skill decrease, as no auto-grader can actually assess contents.
Excellent (Score:3, Insightful)
No way this could backfire whatsoever (Score:3, Insightful)
...thus producing an entire country whose writing-skills were conditioned to game the auto-grader.
Give them something they enjoy writing about. (Score:2, Insightful)
The best way to get students to write is give them something they enjoy writing about.
No thank you. (Score:4, Insightful)
My wife worked for Pearson as a "second tier" grader (or whatever they call them).
In her case, the tests went through the algorithm and were assigned a grade, then the grade and test were passed along to a human to read and check. Invariably, she would come home complaining about tests where the students had obviously studied specifically to answer the way the algorithm wanted: the algorithm would score the paper high, while the actual content of the test answer would leave a LOT to be desired. The answers would score high, but were more or less gibberish as read by a human.
This was about two years ago, so obviously the algorithms could have improved since then, but I have severe reservations about them becoming the sole arbiter of grading.
Re:How stupid can you get? (Score:5, Insightful)
This is one area where automatic grading will cause massive skill decrease, as no auto-grader can actually assess contents.
My thoughts exactly.
Auto graders could check spelling and grammar, and to some extent plagiarism, but without a human reviewing the content, students will learn be gaming the algorithms from day 1.
This is *half* right. (Score:5, Insightful)
The problem is not that the students don't get enough practice. The problem is that the students don't get feedback until they get their grade.
Having an auto-grader grade your work is a terrible idea because auto-graders can't handle complex English. I thought it might be a good idea to run a grammar checker across my novels before publishing them just to have an extra set of eyes, so to speak. So as an experiment, I fed some fragments of one novel (that I knew contained no grammatical errors) into about a dozen of these so-called grammar checkers, along with a list of deliberately broken sentences to see if they actually caught problems.
I just about died laughing at the ludicrous suggestions that the grammar checkers made, mostly stemming from them incorrectly guessing the parts of speech for words that could have more than one meaning. The best of these algorithms correctly reported about 80% of the correct sentences as correct, though many of those algorithms also failed to flag a lot of the incorrect sentences. The worst algorithms flagged more like 80% of the correct sentences as incorrect (and still failed to flag the actual errors in many of the incorrect sentences).
Based on that, I'd say that having someone's grade depend upon such poor algorithms is a really, really bad idea, I'm guessing it will be at least another 1-2 decades before I would trust a computer-based grader to actually perform grading that counts.
However, making those auto-graders available to students for online pre-screening of their writing before they hand in the final version would be a good thing, provided they can make them a lot better. Such software is great at catching simple errors, and anyone with poor writing skills can probably benefit from such software pointing those mistakes out, allowing them to correct their own mistakes before handing the assignments in. This allows the students to learn from the mistakes. A well-designed checker could even keep track of what mistakes a student makes regularly and point out the pattern so that the student can learn to watch for that type of mistake in the future. Unlike robo-grading, such software can actually teach students to improve their skills usefully.
This makes writting essays demeaning (Score:5, Insightful)
If I knew that a machine gets to grade my work I would feel like my time and efforts are worth so little that humans can't be bothered to read it. It defeats the purpose of even writing the thing.
When you write something you are trying to convey an idea. Knowing that the machine doesn't give a fsck proves my efforts are useless.
Re:Why not fewer students and more face-to-face ti (Score:5, Insightful)
Lucky you. For me English class, fro 7th grade through undergrad was a constant string of "infer the hidden meaning behind this text" with nobody ever trying to teach us the process for inferring that hidden meaning. This lead to me being a C student in English for my entire academic career.
Despite all my efforts, in 8 years of English classes, I was never even able to get a single teacher or professor to explain to me how he knew there were hidden meanings behind the text that was assigned. Nor could I get anyone to tell me why they would put hidden meanings into text, when they could put the meaning the want in the literal text.
The funny thing is, my English is fine. IIRC I got a 760 on the English portion of the SAT. I always got As on papers in classes other than English, and complements on my writing were common. It seems to me that the way English classes are normally taught, they have nothing to do with English at all.
Re:Why not fewer students and more face-to-face ti (Score:0, Insightful)
Re:Ridiculous (Score:2, Insightful)
Gladly. Let's expire the Bush tax cuts and put all of it towards teacher salaries. This can't be the only thing we do, but it would help.