How Good Are Robo-Graders? 157
Posted
by
samzenpus
from the reading-robots dept.
from the reading-robots dept.
stoolpigeon writes "With a large study showing software grades essays as well as humans, but much faster, it might seem that soon humans will be completely out of the loop when it comes to evaluating standardized tests. But Les Perelman, a writing teacher at MIT, has shown the limits of algorithms used for grading with an essay that got a top score from an automated system but contained no relevant information and many inaccuracies. Mr. Perelman outlined his approach for the NY Times after he was given a month to analyze E-Rater, one of the software packages that grades essays."
Re:More importantly (Score:5, Funny)
How quickly will students learn to game the system to get perfect scores with perfect gibberish?
Noooooooooo.
I had to deal with a Robo grader once during an exam. Time was up and I was still writing. Several large automatic weapons appeared and in a robotic voice it said, "Drop your pen!"
I did immediately and it said, "Thank you for your cooperation."
Or that might have been when I was taking an art class taught by Peter Weller [ew.com] .... I don't remember now.
Re:More importantly (Score:4, Funny)
Several large automatic weapons appeared and in a robotic voice it said, "Drop your pen!"
I did immediately and it said, "Thank you for your cooperation."
You were lucky. You should see what happened to the guy in this documentary [wikipedia.org] when the robo-grader didn't hear the pen hit the floor.
Re:Sorry, human intervention required (Score:4, Funny)
When I was in high school, we read A Portrait of the Artist as a Young Man. This is literally the worst alleged novel I have ever read. I actively despised it with my entire soul. So I skipped huge chunks of it wherever I figured I could get away with doing so and still pick up the threads of the mostly nonexistent plot.
When we (finally) finished the thing, we had to write a series of short essays responsive to several prompts. One of the prompts told us to describe the symbolism and significance of the "rose."
Having skipped huge portions of the book, I had never encountered this purported rose. And I certainly wasn't going to go back and pick through the dense, sophomoric prose to find it. Instead, I figured I could probably pick up some partial credit by saying some random insightful-sounding thing. So I started spewing what English teachers love. I used words like "juxtaposition" and "antithesis" and compared the rose to some other random symbolic object in the book. It was pure, unadulterated, Grade A, premium All-American BS.
I got an A on the paper. The teacher was particularly profuse in her praise of my short essay on the "rose," commented that I had captured the symbolism of the "rose" perfectly. I couldn't have agreed more.
Re:More importantly (Score:4, Funny)
What kind of result do you think Mr. Churchill might have received if he had stated, "Them Nazis is bad, we gots to beat em."
Here in the US, we'd just elect him president.