Anti-Cheating Technology Challenged at Dartmouth Medical School (yahoo.com) 85
Dartmouth college switched to remote tests when the coronavirus ended in-person exams — then accused 17 medical students of cheating, reports the New York Times:
At the heart of the accusations is Dartmouth's use of the Canvas system to retroactively track student activity during remote exams without their knowledge. In the process, the medical school may have overstepped by using certain online activity data to try to pinpoint cheating, leading to some erroneous accusations, according to independent technology experts, a review of the software code and school documents obtained by The New York Times.
Dartmouth's drive to root out cheating provides a sobering case study of how the coronavirus has accelerated colleges' reliance on technology, normalizing student tracking in ways that are likely to endure after the pandemic. While universities have long used anti-plagiarism software and other anti-cheating apps, the pandemic has pushed hundreds of schools that switched to remote learning to embrace more invasive tools. Over the last year, many have required students to download software that can take over their computers during remote exams or use webcams to monitor their eye movements for possibly suspicious activity, even as technology experts have warned that such tools can be invasive, insecure, unfair and inaccurate.
Some universities are now facing a backlash over the technology....
While some students may have cheated, technology experts said, it would be difficult for a disciplinary committee to distinguish cheating from noncheating based on the data snapshots that Dartmouth provided to accused students. And in an analysis of the Canvas software code, the Times found instances in which the system automatically generated activity data even when no one was using a device. "If other schools follow the precedent that Dartmouth is setting here, any student can be accused based on the flimsiest technical evidence," said Cooper Quintin, senior staff technologist at the Electronic Frontier Foundation, a digital rights organization, who analyzed Dartmouth's methodology.
Seven of the 17 accused students have had their cases dismissed. In at least one of those cases, administrators said, "automated Canvas processes are likely to have created the data that was seen rather than deliberate activity by the user," according to a school email that students made public. The 10 others have been expelled, suspended or received course failures and unprofessional-conduct marks on their records that could curtail their medical careers... Tensions flared in early April when an anonymous student account on Instagram posted about the cheating charges. Soon after, Dartmouth issued a social media policy warning that students' anonymous posts "may still be traced back" to them.... The conduct review committee then issued decisions in 10 of the cases, telling several students that they would be expelled, suspending others and requiring some to retake courses or repeat a year of school at a cost of nearly $70,000...
Several students said they were now so afraid of being unfairly targeted in a data-mining dragnet that they had pushed the medical school to offer in-person exams with human proctors. Others said they had advised prospective medical students against coming to Dartmouth.
Dartmouth's drive to root out cheating provides a sobering case study of how the coronavirus has accelerated colleges' reliance on technology, normalizing student tracking in ways that are likely to endure after the pandemic. While universities have long used anti-plagiarism software and other anti-cheating apps, the pandemic has pushed hundreds of schools that switched to remote learning to embrace more invasive tools. Over the last year, many have required students to download software that can take over their computers during remote exams or use webcams to monitor their eye movements for possibly suspicious activity, even as technology experts have warned that such tools can be invasive, insecure, unfair and inaccurate.
Some universities are now facing a backlash over the technology....
While some students may have cheated, technology experts said, it would be difficult for a disciplinary committee to distinguish cheating from noncheating based on the data snapshots that Dartmouth provided to accused students. And in an analysis of the Canvas software code, the Times found instances in which the system automatically generated activity data even when no one was using a device. "If other schools follow the precedent that Dartmouth is setting here, any student can be accused based on the flimsiest technical evidence," said Cooper Quintin, senior staff technologist at the Electronic Frontier Foundation, a digital rights organization, who analyzed Dartmouth's methodology.
Seven of the 17 accused students have had their cases dismissed. In at least one of those cases, administrators said, "automated Canvas processes are likely to have created the data that was seen rather than deliberate activity by the user," according to a school email that students made public. The 10 others have been expelled, suspended or received course failures and unprofessional-conduct marks on their records that could curtail their medical careers... Tensions flared in early April when an anonymous student account on Instagram posted about the cheating charges. Soon after, Dartmouth issued a social media policy warning that students' anonymous posts "may still be traced back" to them.... The conduct review committee then issued decisions in 10 of the cases, telling several students that they would be expelled, suspending others and requiring some to retake courses or repeat a year of school at a cost of nearly $70,000...
Several students said they were now so afraid of being unfairly targeted in a data-mining dragnet that they had pushed the medical school to offer in-person exams with human proctors. Others said they had advised prospective medical students against coming to Dartmouth.
Activity tracking without consent? (Score:5, Insightful)
You know, with the GDPR such a thing would simply be a criminal act unless they have a court order.
Also sounds like they have both trouble with the data collected and with its interpretation.
Re: (Score:3)
"Without consent"? You are required to submit yourself to scrutiny when you take a test. That's a literal point of having a test in the first place. Here, they cannot scrutinize you because you can't take a test in controlled environment, therefore scrutiny has to come in a different fashion to maintain control over the test environment.
Re: (Score:3)
You have to give informed consent. That means they have to explain, in detail, any and all data they will collect and then they have to get consent specifically tied to that explanation. Otherwise it is "without consent" and criminal. This is about data though, physical monitoring without computers is not subject to the GDPR.
Re: (Score:3)
And the data generated by a student logging into a server and staying logged in to class material - even passively - is the school's data and not yours. This is pretty much a script reading a fancy sign in/sign out sheet. GDPR wouldn't likely apply here, since the school is using their data, not yours.
Re: (Score:3)
You really, really have no clue. Recording your activities without your informed consent is a criminal act under the GDPR. The GDPR very much applies to such things.
Re: (Score:2)
To which you would have agreed to when signing up for using the software to begin with. It isn't like this is a GOTCHA moment.
Re: (Score:3)
The USA doesn't have a GDPR. The only state with something similar is California, and this college is not in that state, so the data is legal.
If the interpretation of the data is screwed up, then I think the school should invest in something meant to monitor cheating. Obviously with online classes you'll need some form of anti-cheating involved, otherwise the issued degree will become almost worthless. As an example of a proper system; for Salesforce exams, you need 2 independent cameras on you. If you use
Re: (Score:2)
The problem is not the monitoring, The problem is the missing information and the lack of making sure there is informed consent. That is what would make this illegal in the EU.
Welcome to our robot overlords (Score:2)
Re: (Score:2)
The system flags a student for suspicious behaviour and a
for 70K they need to get an lawyer & let an co (Score:2)
for 70K they need to get an lawyer and let an court judge / jury make the call with full rights to all evidence.
Re: (Score:2)
-- Benjamin Franklin.
I would put over reliance on automation, machine learning, or AI, squarely in the camp of security, ie 'temporary safety'.
Re: (Score:2)
In other words, "Those would would not be able to prevent having their essential liberties stripped from them by people in power, to give those people in power a bit more power, deserve better but let's blame the students anyway because I want to use this tired ass quote and get some karma, baby!"
Re: (Score:3)
Perhaps I'm wrong, but I think you mis-read what GP wrote.
They were saying, if I'm reading it correctly, that this software is used at the expense of liberty, and shouldn't be.
I think you're saying the same thing, albeit in a more nasty tone.
Investigated != Guilty (Score:3)
At least it used to be that way in the past. Just because some software flags some irregularity doesn't mean you are guilty. 7 of the 17 accused have been found not guilty. Let the process take its course, complain only if innocent people are found guilty, not just accused (unless the accusation itself leaves some permanent mark on the record). Perhaps rethink the "accused means guilty" social media attitude.
Re: (Score:3)
At least it used to be that way in the past.
Yup. A lot of common sense existed in the past. Not anymore.
Just because some software flags some irregularity doesn't mean you are guilty. 7 of the 17 accused have been found not guilty. Let the process take its course, complain only if innocent people are found guilty, not just accused.
If shit software is shit, then get rid of it. That is what needs to be properly evaluated here. Those spending $70K/year in the future shouldn't have to rely on a "lucky" outcome. Even if that happens to be the case this time around, it may not be next.
(unless the accusation itself leaves some permanent mark on the record).
Cheating in pursuit of a degree in the medical field? Kinda ranks up there with teachers falsely accused of sexual assault against a student. Sadly, I'm thinking even the accusation carries we
Re:Investigated != Guilty (Score:5, Insightful)
If shit software is shit, then get rid of it. That is what needs to be properly evaluated here..
This to my nose has a similar stink to the post office horizon scandal here in Britain.
You're proved to be embezzling funds from the post office because the software says so. Many innocent peoples' lives have been ruined over this and I suspect the same might be happening with this exam software.
Re: (Score:2)
Perhaps society should stop being taught that fucking tactic, by their own mainstream media.
AFAIK, 99% of commercial media is in it for the money, the clicks, and eyeballs. Just like much journalism has been since the invention of the printing press. And if the potential customer's curiosity isn't piqued, the media companies don't get their money. So, it comes down to what the public wants. As so many on here claim, it's a free market. If you don't like it, set up your own newspaper/web site/blog. I don't see much more than societal irresponsibility, on the part of all involved.
Re: (Score:2)
If you don't like it, set up your own newspaper/web site/blog. I don't see much more than societal irresponsibility, on the part of all involved.
Um, why in the hell would you even bother to suggest the former, when the latter implies just how fucked the entire system is?
Go ahead. Stand up the Blog of Righteous Truth and Fact in condemnation of the idiocy that is "reporting". Bask in your glory of a whopping 17 bot hits a day while Mass Ignorance continues to feed their Bad News addiction.
When it comes to fixing this, there is no easy way. Ignorance only learns the hard way; usually with violence. We're not quite there yet, but we are allowing a
Re: (Score:3)
Being flagged by this technology means you're accused of wrongdoing.
"Where there's smoke, there's fire": false accusations have a proven history of harm, especially with crimes people feel strongly about, like rape, child abuse, fraud/lying/cheating, etc. More so with our current social media culture, where exonerating evidence usually doesn't get spread around sufficiently.
This is (one of) the reason(s) libel and slander are also crimes.
And then there is the problem with proof & defense in such investi
Re: (Score:2)
The problem with this phrase is that the people saying it are often making up their own definition of what constitutes 'smoke.'
Re:Investigated != Guilty (Score:5, Insightful)
I work in an institute which also uses Canvas. Don't get me wrong, it has a lot of great features and I don't know how we could have taught over the last year without it, but we've also found a few bits of jank in the way it operates, which have sometimes required us to remark work, rejig how we handle submitted work, or not use the built-in system for something because under the hood it conflicts with how we actually want the system to work. I'd definitely not want to come down hard on someone just because Canvas flagged something, without some serious manual checking to make sure this wasn't also a quirk of Canvas. We ended up giving the students the benefit of the doubt when it comes to exams, and haven't used any serious proctoring/anti-cheating systems. Yes, we've had a bit of grade inflation and a few students probably got away with things they otherwise couldn't have, but given the privacy issues, the hassle getting both students and staff up to speed on how to use the software, and the potential for this sort of thing happening we thought it just wasn't worth it. The really serious stuff which separates good and bad students is project work, which is much more individual and harder to cheat.
Re: (Score:2)
+1 especially for " The really serious stuff which separates good and bad students is project work, which is much more individual and harder to cheat.". It makes sense to use the good evidence you have rather than rely on very flake bad evidence.
Re:Investigated != Guilty (Score:5, Informative)
Canvas admin here. Canvas doesn't have any built-in cheat detection. Some 3rd party system like Honorlock, etc. may have sent up an alert about potential cheating though.
The thing that Canvas does, that I've busted many a student for, is that it does track student clicks in a course with a most recent click time/date stamp for all items/tools/content in the course
When the tracking log shows Johnny started the test at 12:33, then looked at the course modules at 12:45, then submitted the test at 12:56, it is pretty obvious that he was looking at stuff he shouldn't have been during the exam.
Fortunately, I just state the facts, show it to instructor, dept. chair, maybe the VP for student affairs, and to the student. Let them decide whether it is cheating or not. Not my call to make.
Re: Investigated != Guilty (Score:2)
Is your claim that anyone who finishes in 11 minutes is cheating? How is that obvious? What about multiple tabs? What happens with the back button processing? What about if the system is under load and your shitty JS is getting throttled by the browser?
How is it obvious the guy is cheating in this situation?
Re: (Score:2)
You need to read what he wrote again. Test started at 12:33 and finished at 12:56. But at 12:45, in the middle of the test, the student clicked on the course module, which presumably has the information in it that they are being tested on. That is the electronic version of opening the text book in the middle of an exam.
I know nothing about Canvas and am not defending it, just pointing out that you are inappropriately attacking the poster.
Re: Investigated != Guilty (Score:2)
There's no attack of anyone anywhere.
You're probably right, but it's not clear to me that a course module would be cheating.
I don't know who is administering tests that aren't open book. Every course I ever took we were free to use the resources available. Tests were written based on this fact.
So, this is software that "catches" someone that may have bookmarked courseware in their browser, but fails to catch someone who opens the book during the test?
Nothing about this sounds obvious to me.
Re: (Score:3)
I don't know who is administering tests that aren't open book. Every course I ever took we were free to use the resources available. Tests were written based on this fact.
And I've pointed this out to our faculty repeatedly over the past 20 years. And when I teach (Linux administration and the occasional programming course) I make it so that projects and lab work and such make up 70% of the grade with only 20% coming from quizzes (that are open resource but 30 questions in 30 minutes, so while you can look up one or two you need to know most of it) and the final 10% from that nebulous "participation" category.
But the same people that think a 3 hour talking head lecture is a
Re: (Score:1)
The thing that Canvas does, that I've busted many a student for, is that it does track student clicks in a course with a most recent click time/date stamp for all items/tools/content in the course
Could an automated page refresh be recorded that way (as I think is being suggested)?
Re: (Score:2)
When the tracking log shows Johnny started the test at 12:33, then looked at the course modules at 12:45, then submitted the test at 12:56, it is pretty obvious that he was looking at stuff he shouldn't have been during the exam.
Fortunately, I just state the facts, show it to instructor, dept. chair, maybe the VP for student affairs, and to the student. Let them decide whether it is cheating or not. Not my call to make.
Sounds like a bunch of FUD to me. Nothing about those timestamps says anything "OBVIOUS" other than that is when they clicked.
You're assumption is the problem, you are judging people based on your "feels" not actual evidence.
Re: (Score:2)
When people think about problems, the look to the ceiling, left or right.
Eye tracking says absolutely nothing about looking something up or not.
Make the tests in a way that the student can use all materials he would/could use in real conditions as well. Memorizing CS stuff is mostly plain stupid. You learn the concepts, and learn how to look up details.
And as the medical example keeps popping up: a future doctor will most likely often check sources anyway. His own knowledge is useful to edge out unlikely (i
Re: (Score:2)
It did, they probably were, from the summary::
Sounds like the students should go class action against the software company for misrepresenting the data as cheating, if they didn't cheat.
Re: (Score:2)
How do you know the 10 students were innocent? Every year before Covid and online exams some students were caught cheating, what makes you think that because everyone went online, suddenly the there was no more cheaters? 7 of 17 were absolved, which means humans investigated. It seems the 10 students looked guilty not just to the software, but also to people watching the recorded exam footage, and could not explain why the footage showed what it did. How is this different from watching security camera foota
Re: (Score:2)
That'd be a person looking visually for concrete evidence of cheating or rule breaking, software can be well written or it can be some flaky algorithms looking at odd undisclosed metrics that may or may not be the result of cheating. I didn't say they didn't cheat, what is important is knowing with a high degree of certainty whether or not they cheated before applying harsh costly penalties.
Re:Investigated != Guilty (Score:4, Insightful)
complain only if innocent people are found guilty, not just accused
No... It is quite unreasonable for a school to be charging innocent people without clear and convincing evidence. Not just "anomalous or suspicious data" being detected.
Hassling or even merely questioning a student is a harmful/damaging thing which is potentially disruptive to their lives, etc. There must be adequate justification for doing it. Suspicious activity should be investigated to the extent possible without doing so; It is Not acceptable to lay charges without adequate specific evidence - which some counter value such as "clicking too fast, clicking too slow, blinking too much, or breathing too often" is Not.
Does live proctor testing centers flag people for (Score:2)
Does live proctor testing centers flag people for stuff like that?
And for schools with testes this big that costs 70K / year they can have live proctors.
Re: (Score:2)
Re: (Score:2)
Perhaps we need more people who turn out innocent to cancel the "cancel culture". Maybe if enough accused people turn out innocent, society will finally start ostracizing cancel culture, call out those who assume accused==guilty as the cancer they are to society. Stop giving in to the ignorant vocal minority. Personally I am starting to see this happen, the silent majority is slowly choosing not to remain silent anymore.
Re: (Score:3)
Indeed. I have many thoughts about this case. I am a faculty in Computer Science at $LOCALSCHOOL.
Going through an academic integrity violation process is a massive drain on my time. So I don't go through with it unless I am damn sure the case is rock solid.
Having only 48 hours for students to consider their options is actually standard practice. We do two business days in the first stage of the process: the stage where you decide to fight or fess up. During that time, we do advise students that they can tal
Re: (Score:2)
a lot to think about here, and I'm not sure I fully understand and/or have a grasp of all the salient info
so for an integrity violation, the school gets all the time they need to carefully weigh their end of things before proceeding; gather evidence, run it by a number of people, think about it, etc.
the student gets 48 hrs to either confess or if they maintain their innocence, to talk to some school staff or hire an attorney to set up a defense
hardly seems fair to give someone only 48 hrs to fully understan
It must be awful (Score:2)
to be an automated mass-surveillance-era student.
When I see what's happening today - and it's just just the virus, this was in the making before - I'm so glad I could to do all my studies with real teachers bothering to organize real exams (not multiple-choice) in real classrooms with real proctors, giving real marks and listening to me for real when I had an objection.
Those days are long gone.
Does not look like a good school to me (Score:2)
This sucks. And, if they were so worried they could establish a zoom conference with students so that the teacher literally has their eye on them, for a lot less than whatever this nasty software cost them. After this news of automated judgements it would only be prudent to ask for in-person testing. It is hard to imagine medical school students not studying for the final and cheating on it after putting in so much work, but the surveillance software and its designers does not have empathy or even good judg
Re: (Score:3)
Re: (Score:2)
They also use eye-tracking software to figure out if you are looking up and away at something other than the screen (if you have a habit of staring into space while thinking, don't do it).
I mean that right there is fucked. I probably would have failed out of HS calculus if I couldn't stare out the window or put my head down on the desk and have a think during tests, never mind some of my more difficult college courses. Thank god that was 20 years ago and I don't have to go through school now. I despair for the post-Columbine, post-COVID tracking environment my kids will have to go through though.
Re: (Score:2)
Re: (Score:2)
If you own a Dell, you could give someone sufficient permissions in DRAC for them to see your screen. No external hardware required.
Re: (Score:2)
Re: (Score:3)
I doubt they could detect it at all. DRAC effectively runs at ring 0. No BIOS settings.
Re: (Score:2)
How are these students going to do in a job interview after being trained by years of school to always stare directly forward like Stepford students (or the creepy Heaven's Gate cult guy)?
Re: (Score:2)
(if you have a habit of staring into space while thinking, don't do it)
Everyone has that habit.
And people still manage to cheat.
The interesting issue on cheating on that level is: if people would spent the same amount of time on working on the subjects, they spent on figuring how to cheat, they would be way better off in the end.
Go elsewhere (Score:1)
Re: Go elsewhere (Score:1)
If you can get into med school at Dartmouth
All three Dartmouth alumni I can think of off the top of my head are, by far, the dumbest Ivy Leaguers I've ever met.
Retake the test in-person with live proctoring (Score:2)
Outsourcing responsibility (Score:4, Interesting)
This kind of software would not pass even the most basic User Acceptance Testing, yet the Universities pay for it without any testing.
The main reason to pay for this is to outsource responsibility of proctoring remotely: Computer says you cheated, if you have a problem take it up with the computer, not with us.
Same with everyone moving their e-mail to the Azure cloud. If a leak happens it is Microsoft's fault, and they are experts at defleting any responsibility.
It was all laid bare in the early 2000's when Linux has become mature enough to be used in various corporate systems. The management refused to migrate to Linux as they did not get an answer to the question:
"If something goes wrong, who do I sue ?"
Re: (Score:2)
There was no cheating detection software. The software involved was the software used to access course materials in normal day to day work. Some prof noticed that students were "accessing" it during exams, so some IT monkey wrote a script to process the activity logs automatically. But, the problem is that the course access software generated activity autonomously as long as it was not logged out, and kids typically just stayed logged in all the time, hence were also logged in during exams.
Sigh. (Score:2)
Play silly games, win stupid prizes.
Accuse me of cheating in an exam and see a lawsuit for not only the cost of the tuition, but future damages including reputational and professional, and I'll work to invalidate the entire system that you spent all that money on, and thus open the avenue for compensation for all such students, ever.
And I'll go elsewhere in the meantime..
Re: Sigh. (Score:3)
Re: (Score:2)
Not only will you lose, you will go broke in the process.
Re: (Score:2)
They don't have 'evidence of cheating.' That's the point. They have a report from a piece of software that they think is evidence of cheating. Now they have to prove in court that they're correct.
Saying it doesn't make it so, much like DUI charges then to mysteriously go away when the defense asks to have the breathalyzer independently analyzed, audited, and verified.
Re: (Score:2)
Now they have to prove in court that they're correct.
In a civil court, YOU have to prove they are wrong as well.
much like DUI charges
You are comparing a civil suit to criminal prosecution and they are very different things with different standards of evidence. The report from the software is considered evidence of cheating. YOU will have to prove it is not evidence of cheating.
Re: (Score:2)
Probably not. The undisclosed settlement that will be reached will prevent setting precedence, so it won't matter that much. They'll likely reimburse tuition and make sure the door hits you in the ass on your way out so hard no other school will enroll you in that program.
Something I didn't like (Score:1)
Re: (Score:2)
It is the students' own fault. (Score:2)
The school obviously don't care about the students (Score:1)
Screw it (Score:2)
Re: (Score:1)
The evidence was verified... by the software that created the evidence from whole cloth. Why do you see a problem with this?
Extra device (Score:1)
Culture Rot (Score:5, Interesting)
Dartmouth Alum here. We used to operate on the Dartmouth Honor Code. Want to do your calculus exam out on the Green? No problem, just be back in time.
About ten years later they stopped favoring "the well-rounded individual" and started looking for "An Incan trapeze artist/cellist with an interest in entomology " in Admissions. They started to try to engineer the well-roundedness of the collective rather than the individual. I mentored some of these kids. They stood no chance of passing Calculus without cheating.
The software is a countermeasure to the symptom but doesn't address the cause. Getting back the culture of Honor once it was gone is a Herculean task. Something that took centuries to build can be destroyed in half a decade.
Re: (Score:1)
Destroying the culture of Honor has been a overarching goal of the Communist Left for decades. They've finally succeeded. What we have now is a low-trust society of individuals that are far less likely to unite to oppose the commie takeover.
Re: (Score:2)
Destroying the culture of Honor has been a overarching goal of the Communist Left for decades.
That sentence does not compute:
a) you have no communsts in the US
b) you have not even lefties in the US - by any measure of "left" in the rest of the world
c) lefties have the same honour codes as anyone else - you are just an idiot.
Re: (Score:2)
Commie Lefty detected!!!
Must suck to pay $70,000 to not be trusted (Score:1)
Why are they doing this at all? (Score:3)
In-person exams with vaccinated people and portable HEPA filters and good masks would have a risk level so low that even I wouldn't worry about it.
Re: (Score:2)
But then they'd have to start spending money on lights and air conditioning again. It's very nice of the student to keep paying full tuition and provide their own computer and internet connectivity while the entity spends much less on creature comforts.
i can't figure out, (Score:2)
Personal property laws do not apply??? (Score:1)
To be clear from the start I'm highly biased against spyware and forced contract signing of sketchy / slimy rules. In my eyes it's illegal. And in my country it IS illegal. You can't sign away your rights.
Can a school really force you to install spyware on your personal property that you bought with your own money? If so than it's messed up. Can a school also force you to install security cameras in your room too? It's the same thing.
Is spying on students in their private home really legal??? I have a vague