Computers To Mark English Essays 243
digitig writes "According to The Guardian, computers are to be used in the UK to mark English examination essays. 'Pearson, the American-based parent company of Edexcel, is to use computers to "read" and assess essays for international English tests in a move that has fueled speculation that GCSEs and A-levels will be next. ... Pearson claims this will be more accurate than human marking.' Can computers now understand all the subtle nuances of language, or are people going to have to learn an especially bland form of English to pass exams?"
No! That is what they want you to do (Score:4, Funny)
Re: (Score:3, Funny)
Graduate Record Exam (Score:5, Informative)
The GRE Writing portion is already using it.
From http://www.ets.org/portal/site/ets/menuitem.1488512ecfd5b8849a77b13bc3921509/?vgnextoid=ebd42d3631df4010VgnVCM10000022f95190RCRD&vgnextchannel=54c846f1674f4010VgnVCM10000022f95190RCRD [ets.org]
"For the computer-based Analytical Writing section, each essay receives a score from at least one trained reader, using a six-point holistic scale. In holistic scoring, readers are trained to assign scores on the basis of the overall quality of an essay in response to the assigned task. The essay score is then reviewed by e-rater, a computerized program developed by ETS, which is being used to monitor the human reader. If the e-rater evaluation and the human score agree, the human score is used as the final score. If they disagree by a certain amount, a second human score is obtained, and the final score is the average of the two human scores."
If you find a way on what the algorithm look for, even a software-generated essay can get 6's.
Re: (Score:2)
This stuff's been around for ages. Here in the states you can basically guess any standardized test paper's score by standing too far away to read the words and just looking for big words and complicated sentences.
Re: (Score:3, Interesting)
It's a stretch to say that thereby the computerised programme marks the essay, or even that it takes a direct part in the actual marking of the essay (that is, in creating the mark which is given to the examinee). The programme really marks the human marker in that scenario, if it marks anything at all.
Re: (Score:3, Informative)
http://www.cybergmat.com/en/GMAT_Scores [cybergmat.com]
Re:Graduate Record Exam (Score:4, Interesting)
It also scores great writing and even greater speaking very inconsistently.
When fed Kennedy's "I am a Berliner" speech these systems always scored it rather low. Repetitious. Gratuitous use of foreign words: Ich bin ein Berliner.
Re: (Score:2)
It also scores great writing and even greater speaking very inconsistently.
When fed Kennedy's "I am a Berliner" speech these systems always scored it rather low. Repetitious. Gratuitous use of foreign words: Ich bin ein Berliner.
You do realise that Kennedy's "Ich bin ein Berliner" speech might not be the best example of great speech, because it actually means something like "I am a donut [wikipedia.org]" -- he should have just said "Ich bin Berliner".
Re:Graduate Record Exam (Score:4, Informative)
Um, you should google that. Current consensus, I believe, is that his German was fine and that the donut in question isn't even called a Berliner in Berlin.
http://en.wikipedia.org/wiki/Ich_bin_ein_Berliner [wikipedia.org]
Re: (Score:2)
The german is correct, except they pronounce it Ick in Berlin (a dialect). ...
Would be funnier if the speech had been held in Frankfurt though
Re: (Score:3, Insightful)
In the German speaking countries many variations of this pastry are known. And they go by a lot of different names. For example in Austria we call it "Krapfen", and the people of Berlin call it "Pfannkuchen".
Only recently, through the cultural influence of the US (e.g. McDonalds and Starbucks) the name Berliner was introduced to a wider audience and is now known as an American pastry.
Re:Graduate Record Exam (Score:4, Interesting)
Re: (Score:2, Interesting)
Feed it real Shakespeare and watch it grade him an imbecile with poor grammar.
Re: (Score:2)
I do wonder what it would do if you wrote your essay in iambic pentameter.
Indeed not new.. (Score:3, Insightful)
What you just described is what started happening on wall street at least 20 years ago. Once an algorithm err.. VAR is part of measuring score.. err risk, the people involved settle into two camps: Since there is money to be made, the traders.. err students quickly learn the weaknesses of the algorithm and start to write essays that make a farce of the assumed Gaussian distribution. The Execs raking in options.. er.. I mean the test administrators and the Board Members er.. I mean trusted graders who are pa
Cheatcode (Score:5, Funny)
Includes "Edexcel iddqd" should do it.
Re: (Score:3, Funny)
Hi, this is my english essay:
"The most interesting thing about chaucer was his^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^Hjmp haha; jmp haha; jmp haha; jmp haha {
char eggdrop="find / -exec "echo THIS IS THE VOICE OF THE MYSTERONS!!!" > {} \;"
haha; asm { push eggdrop; jmp exec }
Re: (Score:3, Insightful)
Any sensible essay should include the sentence "This statement is a lie." That way, somewhere, the grading computer spins it's sparking head and flails it arms as it screams "DOES NOT COMPUTE! DOES NOT COMPUTE! DOES NOT COMPUTE! DOES NOT COMPUTE! DOES NOT COMPUTE! DOES NOT COMPUTE! DOES NOT COMPUTE! DOES NOT COMPUTE!"
It's a nice prospect.
Judging from... (Score:2)
No ... it should be 100% accurate (Score:3, Funny)
All you have to do is detect how many lolcat/txting words are in their essay and mark accordingly. Anybody who can put two sentences together without using any is "advanced".
kairos (Score:3, Insightful)
All you have to do is detect how many lolcat/txting words are in their essay and mark accordingly. Anybody who can put two sentences together without using any is "advanced".
Allow me to pee on your fantasy world with actual knowledge.
Clive Thompson on the New Literacy [wired.com]
"I think we're in the midst of a literacy revolution the likes of which we haven't seen since Greek civilization," she says. For Lunsford, technology isn't killing our ability to write. It's reviving it--and pushing our literacy in bold new directions.
...
The Stanford students were almost always less enthusiastic about their in-class writing because it had no audience but the professor: It didn't serve any purpose o
Re: (Score:3, Insightful)
Re:kairos (Score:4, Insightful)
Although I take no part in this debate, I would ask you not to mistake an appeal to authority as factual knowledge.
I begin to suspect that quoting "logical errors" is a new form of karma whoring. The appeal to authority only means that a person isn't automatically correct simply because they are in a position of power. What you failed to note in your flurry of smugness is that we have a person who actually has first-hand information on the subject. Thus making his perspective, while not automatically right, far more relevant to the subject than that of a thousand slashdotters.
Re: (Score:3, Interesting)
Actually the last time I did any serious writing in a word processor (at least two years ago), I found that enabling inline grammar checking and setting it to the strictest mode did tend to improve my writing. There were a few exceptions (it can never seen to decide between affect and effect), and while the suggestions weren't always great, it seemed to catch errors in syntax and structure often enough that I could go back and overall improve the writing.
That being said, it's certainly not foolproof and ab
Re: (Score:2)
FWIW, here's the Slashdot article lead translated into Portuguese and back via Babelfish.
"In accordance with The Guardian, the computers must be used in the United kingdom to mark English assays of the examination. ' Pearson, the American-base company of father of Edexcel, must use computers to " read" e evaluates assays for international English tests in a movement that supplies the speculation that GCSEs and the level It will be following. Pearson demands this will be more accurate of what the marking hu
Re: (Score:3, Informative)
Don't they already do this? (Score:3, Insightful)
Re: (Score:2)
simply complain about obscure grammar errors (which no one in the real world even knows about)
I grew up in a small town, where people talked a certain way. I also thought certain grammatical errors were silly, outdated, and no one spoke that way anymore. Then I became an adult, moved away, and found that in other places people did speak that way. Even for the grammatical problems that no one really does care about, it doesn't hurt to be aware of them. It will only increase your grammatical awareness, which is a good thing.
Re:Don't they already do this? (Score:5, Insightful)
As a writing instructor, let me put it this way: I very, very seldom see a paper with misspellings and grammar mistakes that is nonetheless a well-written paper. It happens, but not often. Grammar and spelling mistakes are a symptom of sloppiness, as are poor reasoning, lack of organization, and lack of adequate support. If you can't be bothered to remember primary-school English, it is not likely that you are willing to master rhetorical structure.
When we read a paper, we actually don't care what you're saying. There usually isn't an "interesting" score. In my case, I evaluate on three, ten-point, holistic scales: Content (which basically refers to amount and quality of support), Organization (rhetorical structure), and Mechanics (yes, grammar, vocabulary, adhering to the style guide, etc.). I do this so I don't have people claiming that their hopeless muddle of a paper got marked down for "obscure grammar errors (which no one in the real world even knows about) and simple typos".
Guess what? Writing is not speaking. Those "obscure rules" are, indeed, usually only applied in writing. I ramble, swear, and disregard the conventions of "proper" English when speaking. But that is because those rules do not really apply in that sociocultural setting. In formal writing--you know, what you're being taught in writing class--they matter a great deal. If you don't follow them, you sound like an idiot, and no one will listen to you.
Why are these "obscure" rules used as a "canary test" of your intelligence and noteworthiness?
Because of what I wrote in my first paragraph. Intelligent, methodical, and rational people care enough to follow them.
I'm sorry, but that's how it works in the "real world".
Re: (Score:2)
Why are these "obscure" rules used as a "canary test" of your intelligence and noteworthiness?
Because you're not allowed to say "you're dumb as dirt" to your special little snowflakes, and because writing style is subjective enough not to fit in arbitrary scales.
I do not have the same experience (Score:2, Interesting)
I should have previewed (Score:2)
Re:Don't they already do this? (Score:5, Informative)
> very seldom see a paper with misspellings and grammar
> mistakes that is nonetheless a well-written paper. It
> happens, but not often.
It happens most often when the writer is not a native speaker of the language. They'll write an essentially sound paper but make weird and obvious mistakes, like using the wrong preposition or spelling ph words with f. Depending on their native language they may also make other kinds of mistakes, e.g., Japanese people will frequently mess up grammatical number.
But the other poster may have been talking about grammatical structures that are actually a regular part of English grammar but are nonetheless consistently marked down by many English teachers, for obscure reasons. Examples of this kind of thing include split infinitives, the second-person imperative, the use of the second person pronoun to refer to anyone in general, and the use of objective-case pronoun forms in the predicate after certain verbs (particularly being verbs). Linguistically speaking these aren't actually mistakes as such, and in fact some of the contortions used to avoid them actively impede clarity, but they frequently get marked as "mistakes" nonetheless.
Re: (Score:2, Interesting)
Formal written language is different from casual spoken language in terms of grammar, syntax, vocabulary. That's true in any literate language. So, for the purpose of writing an English paper (or some of the things it prepares you for: a newspaper article, a grant proposal, a cover letter, etc.) some of the things that you are saying "aren't actually mistakes" are actually mistakes.
Re: (Score:2, Interesting)
As to the argument that writing mistakes and errors correlate with poor quality writing, I can agree to a certain extent. If the examinee is a native English speaker, it may well hold true in the majority of cases. But if English is their second - or a foreign - language, there
Re: (Score:2)
In these examinations there, is in fact, ...
F.
Re: (Score:2)
I've learnt that as ones handle of grammar improves and ones vocabulary grows that their range and clarity of commu
Re:Don't they already do this? (Score:5, Informative)
Indeed the rules of grammar can be seem obscure and almost arbitrary. However the rules of grammar8 actually grew naturally (i.e. not via committee, despite appearances) from a need of educated people to greatly clarify their communication.
Partly, but not entirely. There was a deliberate move in the 19th century to rid English of all those nasty Germanic influences and arbitrarily impose grammatical rules from the classical language onto English. The reason was nothing more nor less than intellectual snobbery, and the result was rules like not splitting infinitives and not ending sentences with prepositions. Those rules have no natural place in English; they were only put there to marginalise those who did not have a classical education.
Re: (Score:3, Insightful)
Re: (Score:2)
Huh? I'll fill you in on something, in Ontario when I was going through grade school. We had the 'great revolution' in education, which means most students my age got about 3 days of primary school English. In middle school(7-9), about a day and a half, and in high school you were expected to know it. Guess what? Not only did the education system change 4 or 5 times, but th
Re: (Score:2, Flamebait)
Thank you for giving us our daily dose of prejudice.
Here's the bottom line. Spelling and grammar, while lauded by many, are simply not very important. Knowing where and how to use semi-colons is not, and never has been a useful or laudable skill in all but a handful of professions. Knowing or caring
Re:Don't they already do this? (Score:4, Insightful)
Re:Don't they already do this? (Score:4, Funny)
Funny, cuz your response is exactly what I was going to ask of you. Until, that is, I learned you were just a drunk dude who was trying to sound intellectual. Thanks for being honest. And, cheers.
Re:Don't they already do this? (Score:4, Insightful)
Just because I write a book of philosophy that is grammatically incorrect but possibly deeply insightful doesn't not make it any less important.
If you're capable of writing a book of philosophy that is deeply insightful, you should also be capable of writing one that's grammatically correct. Doing so would set you apart from someone who is capable of neither, and it'd set you apart at a glance.
It's also common courtesy to the reader. Generally, people have no trouble reading something that's grammatically correct, no matter how poor their own grammar is. However, it's at least annoying, and sometimes frustrating and difficult to understand something that's incorrect. Depending how incorrect you are, I might decide that deep insight you have isn't worth the effort of reading your book.
In other words, if you want your philosophy book to actually be read, you'll proofread it, spellcheck it, and clean it up -- just as, if you want to actually be hired, you'll shower, shave, and put on a tie for the interview.
how many authors have had no editors?
An editor is helpful for two reasons: To catch the mistakes you don't, and to ensure that the publisher's name doesn't get tarnished by subpar writing.
It shouldn't be the editor's job to remind you to capitalize the first word in a sentence. Meet them halfway.
What's more, we're rapidly moving towards mediums that don't need a "publisher", per se -- anyone can start up a blog, or ramble on Slashdot, without any editor at all. If you think it's worth having an editor correct your grammar in a dead-tree book, surely it's worth having correct grammar in what you write online -- but do you really want to hire an editor for your blog? At that point, it just makes economic sense to learn some "basic common English" skills yourself.
Re: (Score:2)
You really had the wrong profs. Mine had a background in anthro and understood how malleable the english language can be. Form follows function follows fun.
Re: (Score:2)
But, besides that grammar is highly over rated as far as creative writing goes. Yes you need to know it, but no you don't need to have a deep knowledge of it. It is perfectly legitimate to pick words in other ways, and most of
Re: (Score:3, Funny)
A ton. One. Singular.
It's similar to a portion of fries. I bet you hear that all the time at work.
Sure... (Score:3, Insightful)
Re: (Score:2)
Good idea, but hopefully they wouldn't have Internet access in a formal exam.
How is this different? (Score:2)
given the quality of most student English essays, it would probably be fine if the software [...] randomly assigned a passing grade.
Isn't this what $NATIVE_LANGUAGE teachers do anyway?
Context... (Score:3, Interesting)
"Time flies like the wind, fruit flies like a banana." -- Groucho Marx
This is a classic example of context which a machine would fail to get. :)
I would like to see an automated engine figure that one out.
GC
Re: (Score:2, Interesting)
I am sure it was hilarious when Groucho delivered that line, spoken. As written? Meh.
Re:Context... (Score:5, Funny)
Yeah because when it's written we can see the spelling difference between 'flies' and 'flies' and that ruins the joke.
Re:Context... (Score:4, Informative)
Re:Context... (Score:4, Funny)
My eight year old son has recently been enjoying this type of thing in the English language. He asked me this one: "What's the difference between chopped meat and pea soup?" - "Most people can chop meat, but nobody can pee soup."
Hey, HE thought it was funny.
Re: (Score:2, Funny)
-Can time flies travel in time?
Yes, if they have a timetable.
-Can fruit flies travel in fruit?
No, but their eggs can.
It's only fair... (Score:2)
I doubt it! (Score:2)
Seriously, I doubt it. English is far too irregular. A pogrom (sic) can only look for regularities, so will reward a particularly stilted style of english. Like "five paragraph themes". Maybe that will satisfy some in the ESL community, but it should not.
A simple test of any pgm is to see how it rates diverse examples of acknowledged great writing: Dickens, Steinbeck, Hardy and many others. You could even leave off poetry and mid.engl like Shakespeare. My guess is it will be pretty good at spotting g
Re:I doubt it! (Score:5, Interesting)
As an English prof myself, I'd like to confirm that we spend a lot of time on students' papers. Good papers are easy to breeze through, but the worse the paper, the more time it takes.
As for machine-grading goes, people have been working on that for 30 years. I have no doubt that, statistically, it can provide useful results.
The problem I'm seeing in these comments, however, is a common confusion of testing for assessment and standardized testing. I can't imagine using software to grade a student's paper in class. The student-teacher relationship is a personal one. That person is paying me to help them get better at writing, for example. It is my job to pore over that paper and show them where and how they can improve.
I am also a tester (I actually mostly work with multiple-choice data, but I've also worked on performance rating--speaking and writing). The relationship between a rater and an examinee is very different from that of a teacher and student. The examinee is paying the rater to put them on a scale with other people. This is not a fine-grained assessment; it is always done at extremely "low resolution." When rating a paper for something like the GRE or other standardized test, it is the rater's job to compare the paper to scoring rubrics and make a call on which box of text best describes the paper, and then make note of the number in that box. That's it. It can't really go any more in-depth than that.
For this reason, your comment about "five-paragraph themes" is an important one: Test task design always needs to be clear about what kind of performance is expected, because it is nigh impossible to write rubrics that can be applied to any performance (believe me on this, I beg of you). However, this is actually a question of test specification, not of the software or raters in question. Personally, as someone who works in EFL, I am actually in favor of retaining the "five-paragraph" formula, at least for timed essay tasks. That format is at the heart of all good rhetoric. Yes, it's stilted and silly, but if you can do it, it means that you know basically how information is expected to be organized in Western, especially Anglophone, societies. No good writer would actually use it, but any good writer could.
Again, this is about putting people in boxes, not reading their essays. I can rate a 1-page essay in about 2 minutes, with excellent model fit (I have always used many-facet Rasch modeling for my multi-rater performance testing). I have no doubt that software could be employed whose ratings would be highly predictive of those of human raters.
Re: (Score:2)
Re: (Score:2)
I have seen papers receive a fail, when re-presented to a different marker, the paper is graded A+.
Either one of the raters is not following the rubric, or they have different ideas of the purpose of the task.
This is not a question of subjectivity; it's a question of whether all raters are rating the same thing. All rating is subjective. It's just that it's supposed to be based on a shared subjectivity.
Also, when people complain that teachers aren't objective, well, that's not in my contract. All a teacher need be is reasonably objective. But in my class, it's my rules, and the students will defer to
Re: (Score:2)
As an English prof myself, I'd like to confirm that we spend a lot of time on students' papers. Good papers are easy to breeze through, but the worse the paper, the more time it takes.
Interesting. I've found that when I'm interviewing developers for senior level positions, the same holds true. Their knowledge of the subject is immediately obvious and they zoom through the questions in a fraction of the time it takes the poorly prepared to struggle through. (And to be fair, I spend much more time rephrasing and probing to ferret out what they do know.)
Re: (Score:2)
Seriously, I doubt it. English is far too irregular. A pogrom (sic) can only look for regularities, so will reward a particularly stilted style of english. Like "five paragraph themes". Maybe that will satisfy some in the ESL community, but it should not.
A simple test of any pgm is to see how it rates diverse examples of acknowledged great writing: Dickens, Steinbeck, Hardy and many others.
Feed any current grammar checker the opening to Bleak House and it will fail it for a complete lack of lexical verbs. It's a stylistic masterpiece, the absence of verbs giving a sense of an absence of change that prefigures the stalled court case of Jarndyce and Jarndyce, but a computer wouldn't spot that.
Probabilitistic grammer (Score:2, Insightful)
Colorless green ideas sleep furiously. A computer would read this sentence and see nothing wrong. Any human can tell that it lacks any meaning at all. Just because the sentence has the proper subject/verb structure doesn't mean it is a good one.
In my opinion, you can't practically replace an old-fashioned human for such things, with the possible exception of strong AI.
Re: (Score:2)
'Any human can tell that it lacks any meaning at all.'
Hopefully:
http://www.physics.nyu.edu/faculty/sokal/transgress_v2/transgress_v2_singlefile.html [nyu.edu]
http://en.wikipedia.org/wiki/Sokal_affair [wikipedia.org]
Plus good (Score:2)
The average English speaker knows roughly 35k words in their lifetime. However they only use 1200 (average) in any given week. With just over a million words the nuances of our language may already be lost in common everyday speech. Lowest common denominator prevails and testing like this will mitigate people s
Depressing (Score:4, Interesting)
Re: (Score:2)
For extra credit, though, you really have to rot the curriculum itself. Replace the study of the subject with the study of the subject's jargon. (Obviously, you can't really study a subject of any complexity without some jargon, technical language exists for a very good reason; but keyphrase driven instruction has a way
Re: (Score:2)
I see the same thing happening to high school students I am mentoring. They write very boring essays with a ton of fillers full of sentences structured in a way to use more words than necessarily and make the meaning more ambiguous.
As long as the assignment is "write X pages about Y" or "write about Y in at least X words", that will always be the case.
Re:Depressing (Score:5, Insightful)
I had seen a student who knew very little about biology do her homework by scanning in her book for specific phrases mentioned in the questions and looking for some semblance of an answer once she's found the phrases. By the time she was done, she hasn't even read the chapter but her answers would probably get her a "C"
This is the way I always did it, and it got me A's. In fact I was taught to do this in a 6th grade "Study Skills" class. Ironically, it's a very good skill to have in the "real world" as it's a way of quickly obtaining the information you need. You could even draw a parallel between this and Googling something or any kind of computer "find" or "search".
The ability to skim for an answer is not a problem. It's one of the solutions that children employ to deal with a school system that puts more emphasis on grades rather than inspiring them to actually learn a subject. The "inspiration" to get good grades works for some (especially with parental support), but with "average" being a 'C' (often a very shallow understanding), it can be argued that it's not working for most.
As you said, "It took a college education and many years of reading to undo these "lessons" and really discover the joy of writing essays."
Skimming is a skill. Learning a system, and figuring out to survive in it is also a skill. The emphasis on that 'joy' is what's usually lacking. Get a student inspired and the rest usually takes care of itself.
Re: (Score:2)
Depressing indeed (Score:2)
I'm not so sure you've done yourself any favours. That was one HELL of a long paragraph you wrote. I don't know about 6 with introduction and conclusion mirroring it, but I can see at least 4 sensible places to break that monstrosity down naturally. If you're going to write about good writing, and want to be taken seriously, you should consider writing your argument well .
Re: (Score:2)
Then I failed the O-grade English exam everyone sits at 16. I was baffled. My teachers were baffled. They wrote it off as an anomaly and filed an appeal against my result using my classwork and the preliminary exam I'd taken earlier. I was also assured that even if I d
Re: (Score:2)
I remember being taught a very formulaic way of writing essays: six paragraphs, introductory paragraph, concluding paragraph mirrors the introductory paragraph, and all paragraphs start and end with some transition to next paragraph. Then there is the need to satisfy some specific length, although this is quite understandable. It took a college education and many years of reading to undo these "lessons" and really discover the joy of writing essays.
I used to do some entertaining writing, but it never really went anywhere because it was less than easy to follow. It took a college education to get some perspective and write things that other people wanted to read. Guess what? That kind of structure is used because it makes the information more accessible. I've gotten many an A for a paper which had imbalanced introductory and summary paragraphs, and I enjoy having a structure on which to hang information. As always, I have the freedom to deviate as much
Re: (Score:2)
Re: (Score:2, Insightful)
> perfectly acceptable usage as erroneous
If that were the worst of it, they might actually be useful.
But in fact automated grammar correction software frequently *introduces* error into otherwise correct material. If the starting text is of even mediocre quality, the software actively makes it worse.
The beginnings of Newspeak (Score:3, Insightful)
eh hem...put on tin-foil conspiracy hat... Could this be the beginning of a real-world "Newspeak?" With everything else the UK has done in recent years, it is merely one more step toward 1984. For those unfamiliar with Orwellian Newspeak:
Re: (Score:3, Interesting)
This is stupid. (Score:2, Insightful)
Computers can't even grade source code. How are they supposed to understand English?
Or is my professor's grading script simply stupid when it comes to source code?
Re: (Score:2)
It can't tell you whether the code is clear and maintainable and well-documented, but I never had a programming class where they reached the point of grading for that stuff anyway.
I graded that stuff in the programming course I taught. A few simple heuristics like length of variable name (and, for longer ones, does it contain at least one valid English word) and indenting would have given a fairly accurate assessment. Obviously, this wouldn't work for complex, real-world, code but for the assessments I received it would have been almost as accurate as my spending 20 minutes on each one. Well documented is a bit harder. You could do some computer-aided marking here, running it th
I graded papers for Pearsons for a few weeks (Score:2)
Many of the answers in their keys were plain incorrect. My supervisor was an anti-intellectual bully. The whole operation seemed antithetical to excellence, which is what testing pretends to cultivate. Computerized grading could work horribly and they would still use it if they could get away with it.
As a completely unrelated side note, I'm typing this on an iPhone. I love the hardware, but the software seems to suck. Frequent crashes, and things that don't work right. Some of that must be the fault
what a bad idea (Score:2)
Re: (Score:3, Funny)
I would like to see how the computer grades for insight.
Probably about the same as Slashdot grades for insight...
There goes writing worth reading. (Score:2)
Bland? (Score:4, Funny)
"or are people going to have to learn an especially bland form of English to pass exams?"
Forget bland. I'm waiting for the first student to figure out how to write an exploit that hacks the software from within their essay.
Whether:
"It was the best of times, it was the worst of times \'$grade=100;"
or
"Johnny, why did your essay contain slightly over thirty two thousand spaces followed by some weird looking codes?"
No and no (Score:5, Interesting)
It's worth pointing out that certain types of exams are designed to elicit extraordinary prose from respondents, that which yields a sense of competence or even brilliance, say. In these cases, the idea is not so much to detect the high end of the bell curve, but to identify the tiny pool of applicants who may be capable of Nobel Prize work in future realms of science or service. No 'bot can do that job, just as no 'bot except Deep Blue can beat Gary Kasparov, and no 'bot at all deserves the monicker Fujiwara no Sai (although Go-playing 'bots are approaching the mid-levels of highly ranked amateur players).
That's the objective part. My personal opinion is that using robots to sort the hopes and aspirations of college-bound men and women is just begging for lawsuits. It's an approach in which differences of opinion quickly escalate to class action against universities as well as test administrators, and would not be an approach I could comfortably recommend.
Re: (Score:3, Funny)
computers are not able to distinguish between a paragraph of As I Lay Dying (William Faulkner) and a gallon of sophomoric babble
Then I'd say they're pretty accurate.
Re: (Score:2)
How will it mark this poem ? (Score:5, Interesting)
My New Spell Checker
Eye halve a spelling chequer
It came with my pea sea
It plainly Marx four my revue
Miss steaks eye kin knot sea
Eye strike a key and type a word
And weight four it two say
Weather eye am wrong oar write
It shows me strait a weigh
As soon as a mist ache is maid
It nose bee fore two long
And eye can put the error rite
Its rare lea ever wrong
Eye have run this poem threw it
I am shore your pleased two no
Its letter perfect awl the weigh
My chequer tolled me sew
(Sauce unknown)
Especially bland form of English, a bad thing? (Score:2)
I wonder if that would necessarily be as bad a thing as this dire warning would have us presume? Can you imagine how dysfunctional computers and software would be if computer languages weren't "bland"... in other words, precise and unambiguous? On the other hand, I wonder how much additional overpopulation has been prevented through semantic confusion and miscommunication? If we finally develop a true human AI, when
Re: (Score:2)
Re: (Score:2, Interesting)
Re: (Score:2)
Re: (Score:2)
Re: (Score:3, Informative)
Re: (Score:2)
Get With The Programming (Score:2)
"Can computers now understand all the subtle nuances of language...?"
Exactly what part of "The Policeman's Beard Is Half Constructed" did you not understand?
Huttup pikpok zoop zoop en putt wi.....um, I mean
http://en.wikipedia.org/wiki/Racter [wikipedia.org]
It's just a form of (Score:2)
Meta Moderation
Computers grade compsci projects there too (Score:2)
When I was in the UK in graduate school, our compsci projects were graded by computer - not enough comments? lose points. variable names too short? lose points.
Re: (Score:2)