Want to read Slashdot from your mobile device? Point it at m.slashdot.org and keep reading!


Forgot your password?
Education Technology

Grading Software Fooled By Nonsense Essay Generator 187

An anonymous reader writes "A former MIT instructor and students have come up with software that can write an entire essay in less than one second; just feed it up to three keywords.The essays, though grammatically correct and structurally sound, have no coherent meaning and have proved to be graded highly by automated essay-grading software. From The Chronicle of Higher Education article: 'Critics of automated essay scoring are a small but lively band, and Mr. Perelman is perhaps the most theatrical. He has claimed to be able to guess, from across a room, the scores awarded to SAT essays, judging solely on the basis of length. (It’s a skill he happily demonstrated to a New York Times reporter in 2005.) In presentations, he likes to show how the Gettysburg Address would have scored poorly on the SAT writing test. (That test is graded by human readers, but Mr. Perelman says the rubric is so rigid, and time so short, that they may as well be robots.).'"
This discussion has been archived. No new comments can be posted.

Grading Software Fooled By Nonsense Essay Generator

Comments Filter:
  • by litehacksaur111 ( 2895607 ) on Wednesday April 30, 2014 @08:39PM (#46885707)
    I though most schools don't even care about the essay. Also the elite schools nowadays prefer the ACT and SAT II subject tests to demonstrate real knowledge. The SAT is really a dumb test, especially with all the coaching resources available now.
  • Irrelevant (Score:3, Insightful)

    by Ol Olsoc ( 1175323 ) on Wednesday April 30, 2014 @08:50PM (#46885771)
    As long as Precious gets an "A', Helicopter Daddy, and Blackhawk Mommy won't try to have the school president fired for ruining Precious's permanent record.
  • by Anonymous Coward on Wednesday April 30, 2014 @08:56PM (#46885793)

    ... because Slashdot shows that humans already make evaluations about articles without reading them.

  • Quid pro quo (Score:4, Insightful)

    by Opportunist ( 166417 ) on Wednesday April 30, 2014 @08:57PM (#46885797)

    When you're too lazy to read my essay to grade me and let software do it, I don't really see no moral problem with doing the same to write the essay.

  • by MrBigInThePants ( 624986 ) on Wednesday April 30, 2014 @09:05PM (#46885851)
    Not being from the USA, every article I ever read about your education system just leaves me scratching my head.

    How on earth did you guys let it get so ridiculous??
  • by sootman ( 158191 ) on Wednesday April 30, 2014 @09:37PM (#46885997) Homepage Journal

    Artificial intelligence, while seemingly tasty on the surface, tends to be underwhelmed by insufficient fish, with regard to warrantless searches.

  • Re:Quid pro quo (Score:5, Insightful)

    by Anubis IV ( 1279820 ) on Wednesday April 30, 2014 @09:56PM (#46886087)

    As someone who graded hundreds of essays while serving as a teaching assistant for a senior-level engineering ethics course, I have to say that I find your lack of integrity rather appalling. Your moral obligation to write the essay yourself is independent of the method they use for grading it. Just because someone else is doing a lousy job does not mean that you suddenly have a license to short-change them for what you're obligated to do.

    I would guess that I graded around 300-400 essays during the three semesters I served as a TA, and that I probably averaged around 20 minutes per essay, since I was a strong believer in providing useful feedback over things the students could improve, even if they weren't necessarily incorrect. That said, other TAs spent as little as a minute or two per essay, and barely provided any feedback at all. Regardless of how much time the TAs did or didn't spend on the essays, however, the students had the same obligations, and rightfully so.

  • by Anonymous Coward on Wednesday April 30, 2014 @10:07PM (#46886139)

    If I've been hired to build a Potemkin village, then it would be unethical of me to spend time constructing interiors for the buildings.

    The English department has some nice courses on compositional writing where I can get real feedback on my progress on those skills. As far as the machine-graded essays for any other Department -- either I understood the topic before writing the essay or I didn't and if I didn't then a no-feedback essay isn't going to fix the problem.

  • Re:Quid pro quo (Score:4, Insightful)

    by number17 ( 952777 ) on Wednesday April 30, 2014 @10:24PM (#46886201)

    Your moral obligation to write the essay yourself is independent of the method they use for grading it.

    Students pay big bucks and expect to have experts in the field teach them and grade their work. It sounds like these schools are off-shoring their marking so that they can do other work (ie Research). If the school was upfront, before paying tuition, that they were going to just send your essay to Bangladesh for marking then I would be ok with having a moral obligation to write the essay myself.

  • by ceoyoyo ( 59147 ) on Wednesday April 30, 2014 @10:28PM (#46886219)

    Odd you choose math as an example, a subject where your grammar must be perfect or what you've written is nonsense.

  • by AK Marc ( 707885 ) on Wednesday April 30, 2014 @10:40PM (#46886257)
    I got an 800 on my math (old SAT, back in 1990), and I still count on my fingers. They allowed calculators (only a few approved ones), and, of course, I used it. The SAT math was about speed, efficiency, and answering the right question. Most people had a problem with the latter. I don't know all of my multiplication tables, it wasn't my thing, but seeing the question and figuring the best way to word it to find the answer was my thing. Didn't miss a question.

Beware of Programmers who carry screwdrivers. -- Leonard Brandwein