Software Goes Through Beta Testing. Should Online College Courses? (edsurge.com) 70
"Testing online courses is not standard practice at traditional colleges," points out a new article at EdSurge -- though beta-testing is part of the process for other online learning sites. jyosim summarizes their report:
Coursera has recruited a volunteer corp of more than 2,500 beta testers to try out MOOCs before they launch. Other free online course providers have set up systems that catch things like mistakes in tests, or just whether videos are confusing. Traditional colleges have shied away from checking online course content before going live, citing academic freedom. But some colleges are developing checklists to judge course design and accessibility.
"It would be lovely if universities would consider ways of adopting the practice of beta testing," says Phillip Long, chief innovation officer and associate vice provost for learning sciences at the University of Texas at Austin. One factor, though, is cost. "How do you scale that at a university that has thousands of courses being taught," he asks... How much beta testing makes sense for courses, and what's the best way to do it?
A senior instructional designer at the State University of New York says "On most campuses, instructional designers have their hands full and don't have time to review the courses before they go live... We're still trying to find the magic bullet that motivates people to review other people's courses when they're not being paid."
"It would be lovely if universities would consider ways of adopting the practice of beta testing," says Phillip Long, chief innovation officer and associate vice provost for learning sciences at the University of Texas at Austin. One factor, though, is cost. "How do you scale that at a university that has thousands of courses being taught," he asks... How much beta testing makes sense for courses, and what's the best way to do it?
A senior instructional designer at the State University of New York says "On most campuses, instructional designers have their hands full and don't have time to review the courses before they go live... We're still trying to find the magic bullet that motivates people to review other people's courses when they're not being paid."
traditional colleges don't test to if there own c (Score:1)
traditional colleges don't test to if there own classes will lead to jobs but they don't give a dam as they loans that you can't discharge.
Re: (Score:2)
Maybe you should not have tried to go to college without first learning something in high school.
I never went to high school but went to college anyway. High school is overrated anyway.
Re: (Score:2)
Well that explains why you were unemployed for years around 2009.
I was unemployed for two years (2009-10), underemployed for six months (working 20 hours per month), and filing for Chapter Seven bankruptcy (2011). That has nothing to do with my lack of high school education, an A.A. degree in General Education (1994), and an A.S. degree in Computer Programming (2007). Plenty of educated people got laid off during the Great Recession. Unlike those with or without high school diplomas, I've bounced back to where I was financially ten years ago.
Re: (Score:2)
And why he's a fat fuck
Because I was born big. No diet can fix bigness.
He never did any sport.
Not in grade school. Rode my bike all over the place in college. When I had a restaurant job and lived ten miles away, I road my bike 20 miles per day for three years. I still do cardio at the gym.
Re: (Score:2)
Why? Are you a Mormon or something?
Still, unlike Joe_Doylem you can generally string a coherent sentence or two together. Which was, I think, the point.
Re: (Score:2)
Why? Are you a Mormon or something?
I was misdiagnosed as being mentally retarded due to an undiagnosed hearing lost in one ear, spent eight years in Special Ed and got tired of being treated like a well prized idiot by the school system. Especially since I blew out the annual evaluation on the genius side of the scale, which my teachers always called a statistical fluke. My parents let me stay home during my high school years as I taught myself. I had a personal library of 800 books when I turned 18-years-old.
Re: (Score:2)
Now I'm faced with choosing how to raise my own children: home school or high school?
I'm not sure what the right answer is for you. What I've seen in my own experience is that kids want to learn but that desire to learn is often snuffed out by the school system. Society has too many adults who stopped learning once they get out of school and then get stucked in life because they can't learn their way into a better situation. The desire to learn as a kid and keep learning as an adult must always be paramount.
Re: (Score:2)
Anyone who denigrates you for missing the "high school experience" is a selfish idiot.
I had trouble getting entry level, minimum wage jobs because I didn't have a high school diploma and an associate degree didn't count. Once I got established in my technical career, no one cared about my lack of high school diploma.
Re: (Score:2)
This is true. One of my friends did civil engineering and he had to make his own.
IHAW,TTSP,DFTTYW, etc
Re: (Score:2)
What? (Score:4, Insightful)
Traditional colleges have shied away from checking online course content before going live, citing academic freedom.
What the hell sense does that make? That's like saying I don't check my texts for errors before hitting send because "Freedom of Speech, bitch!"
Re: (Score:3)
Funny you should mention that. Must be a new thing in academia. No one at the Department of Education bothered to check the texts before tweeting.
http://www.newser.com/story/238241/education-dept-tweets-typo-apologizes-with-typo.html [newser.com]
They Do. (Score:2, Interesting)
I am posting as AC for -reasons- (not sure what all is NDA, etc)
But Coursera courses go through a beta test before they are launched. I'm not sure how many 'learners'(they don't call them students) go through it, or if it is just Coursera Staff... but there -are- _some_ people who go through the course(s) early.
Their new platform sucks IMHO. It is difficult for Instructors to actually produce meaningful content since their platform is 24/7 on-demand only now and new course sections start every 2 weeks. So i
TEEX alpha and beta tests theirs, even small chang (Score:2)
I used to work at TEEX (which has some good free cybersecurity courses, btw) and they enforce a policy of alpha testing followed by beta testing. Even minor changes to already-released courses require an appropriate degree of testing. All changes must be approved by a separate department, a curriculum department which is independent of the departments which run the various types of courses.
Re: (Score:2)
Why should the instructor provide the "right" answer? You are given unlimited attempts at the weekly quizzes and any course projects.
A mistake is something to be learned from -- having access to a worked answer allows you to diagnose your own errors. This is important enough in face-to-face teaching, but in the Coursera model there is no personal tutelage, and no scope for individual or common student errors to be addressed in later sessions of the course (because the content is fixed). This makes it absolutely vital that students can spot and work on their own weaknesses.
No. (Score:2)
No. [wikipedia.org]
Re: (Score:2)
And happy little unicorns will frolic in my yard.
Why is this different from traditional classes? (Score:5, Insightful)
In many cases, the beta testing of classes taught in traditional settings is just to teach them. As someone who is teaching an online science course and doing so for the first time, I'm building my class based on lectures when I taught the same class in a traditional setting. Although it's a different instructional format, I'm not sure that beta testing would have been helpful. My class is an elective requirement, and although my students might never care about meteorology again, I try to teach them a bit about how science works (in general) and force them to do higher order thinking (~80% of the grade is based on levels of thinking above recall in Bloom's Taxonomy).
It's a legitimate science class, and any good instructor should be constantly testing their teaching and trying to improve. Every semester I've taught, I've learned from what I've done well and what needed improvement throughout the semester. There isn't a single beta test, but any worthwhile instructor should always be working to improve the quality of instruction. That's been no different this semester, even though it's an online class. I have made a couple of changes to my instruction this semester, which I believe are for the better. You can't simply prepare a class from the beginning and expect that you won't have to change anything during the semester. Instead, you need to have clear objectives for your class, and make changes as needed to either assess whether those objectives are being met (assignments and tests) and how to prepare the students to meet those objectives (instruction).
Any good instructor is always working to improve a class, even during the class. Instead of a single beta test, each week of my class is like Patch Tuesday, where I learn from how the students did the previous week, and always make minor or major tweaks. It might be as simple as explaining concepts again that students didn't understand as well or as complex as making changes to my learning assessments.
For a good instructor, a beta test isn't as important as the continual revisions during the semester. I can certainly say that I've gotten better at teaching this class as the semester has gone on. If I'd prepared the entire class prior to the semester and beta tested it, I might have missed some things that could be improved upon during the semester. Beta tests are no substitute for periodic updates to software. And they're no substitute for continually improving courses.
Re: (Score:2)
Re: (Score:1)
That may work for you, but I believe that would have been an epic failure for my class. The bulk of my students in these classes are second semester freshmen. I suspect that your ability to succeed in that environment is not a good indication of how freshmen would perform in such a class. I have new lectures and assessments each week, and many of my students wait until the day it's due to start on the work. Most of those students also get lower grades and don't learn the material nearly as well as those who
Re: (Score:2)
Students who try to cram in their learning in a short time, like to prepare for an exam, generally don't learn the material nearly as well.
That depends on the class. I had a biology class without a lab to meet the requirements but I wasn't interested in retaining that knowledge once I meant the requirement. I quite literally slept through that class since the content matter made me queasy. On the day before the exam, I read a 1,200-page biology text in 12 hours. I got a B for that course — and a hell of a headache after the final.
Re: (Score:2)
Okay enough with the self-serving lies.
Donald Trump is not relevant to this discussion.
You are a post-middle aged desktop tech making 30k less than average for where you live.
Based on the various job titles I have, I'm actually a computer engineer doing system admin tasks at a desktop technician pay rate. When I requested a cost of living adjustment, I got a lot of hemming and hawing. But, hey, that's government IT for you.
You complain about this often while lauding yourself with grandeur like reading a textbook at 36 seconds per page while an average A student takes 5 minutes [...]
When I was in the eighth grade, I had a college level reading comprehension. By the time I turned 18-years-old, I had personal library of 800 books. If I can read ten novels a month (note that the average America
Re: (Score:2)
I guess the main difference is that if you take a course marked "beta" it looks worse on your CV.
Re: (Score:2)
Yup. The key that our accreditation body (SACS) looked for when we did a substantiative change review to start offering online courses back in '98 was "equivalence". Did an online ENC101 course give the same experience, etc. as a F2F one? Did a student who took ENC101 online do just as well in the next course down the road compared to one who took it face to face?
Of course my sarcastic comment about it all is "of course, they all suck equally".
Re: (Score:2)
I'd go one step further and be more scientific, why not use automated A/B testing? Like you make a new revision of a lecture, half the class gets the new one and half the class the old one. Then you run some form of quiz on the material. If you have at least a few hundred students or ideally a few thousand you should pretty quick get statistically significant answers. And you could test with short/medium/long explanations to see whether you're beating down open doors or areas they'd benefit from more detail
Re: (Score:2)
Interesting post.
So, to summarize, you...
Planned
Designed
Deployed
Maintained
I wish they would... (Score:3)
Re: (Score:3)
Yeah, I've been to my fair share of technical classes where the self practice projects were broken to the point where they couldn't be completed.
Some proper QA testing would probably catch that, but most companies seem to be cheaping out on proper QA now. Sure, why not try a "public beta" first to work out the bugs.
Re: (Score:2)
Yeah, I've been to my fair share of technical classes where the self practice projects were broken to the point where they couldn't be completed.
The CIS dean that taught most of the programming courses at my community college gave out extra credit for anyone who identified that the self practice assignment was broken, explained why the self practice assignment was broken, and then complete the self practice assignment with a fix in code. Most students didn't bother. Since my day job at that time was being a lead video game tester, I got the extra credit because I could identify, explain and fix the problem.
Re: (Score:2)
Re: (Score:2)
If I'm dropping $3,500 on a 16 week course, I don't want extra credit for improving their quality of the product I just laid out $$ for - but I'll take a $500 rebate my tuition!
When I went back to community college to learn computer programming, Uncle Sam picked up the tab with a $3,000 tax credit that George W. signed into law after 9/11. I don't mind getting extra credit on a free education. ;)
Re: (Score:2)
Yeah, I've been to my fair share of technical classes where the self practice projects were broken to the point where they couldn't be completed.
Some proper QA testing would probably catch that, but most companies seem to be cheaping out on proper QA now. Sure, why not try a "public beta" first to work out the bugs.
That seems to be a common problem. I've seen online course where you had to type in an answer, and if you didn't type exactly what they wanted verbatim, even if your answer was correct, it was marked wrong. Online coursework have become a crutch for lazy professors to avoid as much work as possible while teaching.
Re: (Score:2)
This depends on the course delivery system, and how much your instructor both knows how to use the system and its quirks and how much your instructor cares about doing a decent job.
The platform I use (Canvas) is pretty good about a lot of stuff, but instead of entering possible answers that are an exact match of what a student might type in (for questions like "On a machine running Debian Jessie what command would you use to display the routing table?") and having to hunt down each occurrence of the questio
Re: (Score:2)
This depends on the course delivery system, and how much your instructor both knows how to use the system and its quirks and how much your instructor cares about doing a decent job.
The platform I use (Canvas) is pretty good about a lot of stuff, but instead of entering possible answers that are an exact match of what a student might type in (for questions like "On a machine running Debian Jessie what command would you use to display the routing table?") and having to hunt down each occurrence of the question across 40 exams and check to be sure the student didn't "out think me", I simply don't enter ANY correct answers, and the system marks it as "needs grading" which lets me get to it with a single click on each exam.
Sounds like you are using it in a way that supports learning. My experience has been the prof simply lets Canvas grade the paper for them since scores get posted as soon as the test is complete.
Re: (Score:2)
Mine's a graduate course, so the rules and expectations are a little different. But in general, we have a 2-hour lecture each week, 4-5 chapters of reading in one book, another 4 chapters of reading in another book, and exercises that can take from a couple of hours (that was a good week) to a lot more (16 hours one week, for me, 20 hours another). In my case the instructor has specifically said he expects students to fill in "holes" in the material/assignment instructions on their own, which is ok with m
Re: (Score:2)
I try to focus about 80% of the grade on higher orders of thinking than recall (see Bloom's Taxonomy for what this means).
Bloom's taxonomy is a wonderful paradox. It takes the idea of higher order thinking and presents it in an easy-to-digest form that requires no higher-order thinking whatsoever. Higher order thinking skills exist outwith and beyond the taxonomy, and any explanations I've seen tied to the taxonomy are weaker than those independent of it.
So please, can we kill Bloom's Taxonomy now? It has outlived it's usefulness by a long chalk.
Getting a job (Score:2)
How about college students be taught what skills would be useful at a job? I mean, nobody wants to hire people straight out of college, so why not provide the work experience during college? Actually, I would let college students sit in on meetings in my company. I would do it for free, of course I am not paying them either since they aren't working for me. I mean, even if there is no work for unskilled people .. colleges should hook up deals so that individual students (maybe no more than 3 at a time) coul
Re: (Score:2)
That said, I am always very dubious of "skills useful for a job", because the more you ta
Length of courses more important than content (Score:3)
Faulty premise (Score:5, Insightful)
I think I've spotted the flaw in this plan. Anyone else?
Re:Faulty premise (Score:4, Insightful)
I think I've spotted the flaw in this plan. Anyone else?
Damn. I knew there was a flaw somewhere; but I have a fix: give away the course free to the first 1000 participants in exchange for feedback; then ignore the feedback and publish it as is because fixing it would cost to much and you're already in the red from the free beta test. Of course, the as soon as you say "give it away for free to the first..." no one will like the idea anyway.
Yester year (Score:2)
The traditional ones got beta tested (Score:2)