Want to read Slashdot from your mobile device? Point it at m.slashdot.org and keep reading!

 



Forgot your password?
typodupeerror
Education The Internet

Software Goes Through Beta Testing. Should Online College Courses? (edsurge.com) 70

"Testing online courses is not standard practice at traditional colleges," points out a new article at EdSurge -- though beta-testing is part of the process for other online learning sites. jyosim summarizes their report: Coursera has recruited a volunteer corp of more than 2,500 beta testers to try out MOOCs before they launch. Other free online course providers have set up systems that catch things like mistakes in tests, or just whether videos are confusing. Traditional colleges have shied away from checking online course content before going live, citing academic freedom. But some colleges are developing checklists to judge course design and accessibility.

"It would be lovely if universities would consider ways of adopting the practice of beta testing," says Phillip Long, chief innovation officer and associate vice provost for learning sciences at the University of Texas at Austin. One factor, though, is cost. "How do you scale that at a university that has thousands of courses being taught," he asks... How much beta testing makes sense for courses, and what's the best way to do it?

A senior instructional designer at the State University of New York says "On most campuses, instructional designers have their hands full and don't have time to review the courses before they go live... We're still trying to find the magic bullet that motivates people to review other people's courses when they're not being paid."
This discussion has been archived. No new comments can be posted.

Software Goes Through Beta Testing. Should Online College Courses?

Comments Filter:
  • traditional colleges don't test to if there own classes will lead to jobs but they don't give a dam as they loans that you can't discharge.

    • they don't give a dam

      This is true. One of my friends did civil engineering and he had to make his own.

      IHAW,TTSP,DFTTYW, etc

  • What? (Score:4, Insightful)

    by alzoron ( 210577 ) on Saturday February 18, 2017 @06:50PM (#53893505) Journal

    Traditional colleges have shied away from checking online course content before going live, citing academic freedom.

    What the hell sense does that make? That's like saying I don't check my texts for errors before hitting send because "Freedom of Speech, bitch!"

  • They Do. (Score:2, Interesting)

    by Anonymous Coward

    I am posting as AC for -reasons- (not sure what all is NDA, etc)

    But Coursera courses go through a beta test before they are launched. I'm not sure how many 'learners'(they don't call them students) go through it, or if it is just Coursera Staff... but there -are- _some_ people who go through the course(s) early.

    Their new platform sucks IMHO. It is difficult for Instructors to actually produce meaningful content since their platform is 24/7 on-demand only now and new course sections start every 2 weeks. So i

    • I used to work at TEEX (which has some good free cybersecurity courses, btw) and they enforce a policy of alpha testing followed by beta testing. Even minor changes to already-released courses require an appropriate degree of testing. All changes must be approved by a separate department, a curriculum department which is independent of the departments which run the various types of courses.

  • by Striek ( 1811980 )

    No. [wikipedia.org]

  • by Anonymous Coward on Saturday February 18, 2017 @07:15PM (#53893575)

    In many cases, the beta testing of classes taught in traditional settings is just to teach them. As someone who is teaching an online science course and doing so for the first time, I'm building my class based on lectures when I taught the same class in a traditional setting. Although it's a different instructional format, I'm not sure that beta testing would have been helpful. My class is an elective requirement, and although my students might never care about meteorology again, I try to teach them a bit about how science works (in general) and force them to do higher order thinking (~80% of the grade is based on levels of thinking above recall in Bloom's Taxonomy).

    It's a legitimate science class, and any good instructor should be constantly testing their teaching and trying to improve. Every semester I've taught, I've learned from what I've done well and what needed improvement throughout the semester. There isn't a single beta test, but any worthwhile instructor should always be working to improve the quality of instruction. That's been no different this semester, even though it's an online class. I have made a couple of changes to my instruction this semester, which I believe are for the better. You can't simply prepare a class from the beginning and expect that you won't have to change anything during the semester. Instead, you need to have clear objectives for your class, and make changes as needed to either assess whether those objectives are being met (assignments and tests) and how to prepare the students to meet those objectives (instruction).

    Any good instructor is always working to improve a class, even during the class. Instead of a single beta test, each week of my class is like Patch Tuesday, where I learn from how the students did the previous week, and always make minor or major tweaks. It might be as simple as explaining concepts again that students didn't understand as well or as complex as making changes to my learning assessments.

    For a good instructor, a beta test isn't as important as the continual revisions during the semester. I can certainly say that I've gotten better at teaching this class as the semester has gone on. If I'd prepared the entire class prior to the semester and beta tested it, I might have missed some things that could be improved upon during the semester. Beta tests are no substitute for periodic updates to software. And they're no substitute for continually improving courses.

    • by creimer ( 824291 )
      When I went back to school to learn computer programming, I took some online courses to fill out the requirements. Some instructors were very insistent on meeting virtually every week to go over assignments. Other instructors gave out the material and wished us good luck. My HTML class was the latter. Since I already knew HTML, I waited until the day of the final exam to complete all the assignments in six hours, upload them to the website, and then take the final exam. Like all my programming courses, I go
      • by Anonymous Coward

        That may work for you, but I believe that would have been an epic failure for my class. The bulk of my students in these classes are second semester freshmen. I suspect that your ability to succeed in that environment is not a good indication of how freshmen would perform in such a class. I have new lectures and assessments each week, and many of my students wait until the day it's due to start on the work. Most of those students also get lower grades and don't learn the material nearly as well as those who

        • by creimer ( 824291 )

          Students who try to cram in their learning in a short time, like to prepare for an exam, generally don't learn the material nearly as well.

          That depends on the class. I had a biology class without a lab to meet the requirements but I wasn't interested in retaining that knowledge once I meant the requirement. I quite literally slept through that class since the content matter made me queasy. On the day before the exam, I read a 1,200-page biology text in 12 hours. I got a B for that course — and a hell of a headache after the final.

    • by AmiMoJo ( 196126 )

      I guess the main difference is that if you take a course marked "beta" it looks worse on your CV.

    • Yup. The key that our accreditation body (SACS) looked for when we did a substantiative change review to start offering online courses back in '98 was "equivalence". Did an online ENC101 course give the same experience, etc. as a F2F one? Did a student who took ENC101 online do just as well in the next course down the road compared to one who took it face to face?

      Of course my sarcastic comment about it all is "of course, they all suck equally".

    • by Kjella ( 173770 )

      I'd go one step further and be more scientific, why not use automated A/B testing? Like you make a new revision of a lecture, half the class gets the new one and half the class the old one. Then you run some form of quiz on the material. If you have at least a few hundred students or ideally a few thousand you should pretty quick get statistically significant answers. And you could test with short/medium/long explanations to see whether you're beating down open doors or areas they'd benefit from more detail

    • Interesting post.

      So, to summarize, you...

      Planned
      Designed
      Deployed
      Maintained

  • by anegg ( 1390659 ) on Saturday February 18, 2017 @07:22PM (#53893595)
    I'm in a new online course at the University of Maryland. There are definitely some pain points (for me, at least) that could be eliminated with some beta testing, as well as some standards for what its reasonable to expect students to do in terms of work outside the classroom. Beta testing (or at least offering the course at a reduce cost for the first one or two offerings) would make being a guinea pig a bit more palatable. Instead, I have a grade in my degree program dependent upon the whim/good sense of an instructor. I don't know what outside observation there is to ensure that the treatment is reasonable.
    • Yeah, I've been to my fair share of technical classes where the self practice projects were broken to the point where they couldn't be completed.

      Some proper QA testing would probably catch that, but most companies seem to be cheaping out on proper QA now. Sure, why not try a "public beta" first to work out the bugs.

      • by creimer ( 824291 )

        Yeah, I've been to my fair share of technical classes where the self practice projects were broken to the point where they couldn't be completed.

        The CIS dean that taught most of the programming courses at my community college gave out extra credit for anyone who identified that the self practice assignment was broken, explained why the self practice assignment was broken, and then complete the self practice assignment with a fix in code. Most students didn't bother. Since my day job at that time was being a lead video game tester, I got the extra credit because I could identify, explain and fix the problem.

        • by anegg ( 1390659 )
          If I'm dropping $3,500 on a 16 week course, I don't want extra credit for improving their quality of the product I just laid out $$ for - but I'll take a $500 rebate my tuition!
          • by creimer ( 824291 )

            If I'm dropping $3,500 on a 16 week course, I don't want extra credit for improving their quality of the product I just laid out $$ for - but I'll take a $500 rebate my tuition!

            When I went back to community college to learn computer programming, Uncle Sam picked up the tab with a $3,000 tax credit that George W. signed into law after 9/11. I don't mind getting extra credit on a free education. ;)

      • Yeah, I've been to my fair share of technical classes where the self practice projects were broken to the point where they couldn't be completed.

        Some proper QA testing would probably catch that, but most companies seem to be cheaping out on proper QA now. Sure, why not try a "public beta" first to work out the bugs.

        That seems to be a common problem. I've seen online course where you had to type in an answer, and if you didn't type exactly what they wanted verbatim, even if your answer was correct, it was marked wrong. Online coursework have become a crutch for lazy professors to avoid as much work as possible while teaching.

        • This depends on the course delivery system, and how much your instructor both knows how to use the system and its quirks and how much your instructor cares about doing a decent job.

          The platform I use (Canvas) is pretty good about a lot of stuff, but instead of entering possible answers that are an exact match of what a student might type in (for questions like "On a machine running Debian Jessie what command would you use to display the routing table?") and having to hunt down each occurrence of the questio

          • This depends on the course delivery system, and how much your instructor both knows how to use the system and its quirks and how much your instructor cares about doing a decent job.

            The platform I use (Canvas) is pretty good about a lot of stuff, but instead of entering possible answers that are an exact match of what a student might type in (for questions like "On a machine running Debian Jessie what command would you use to display the routing table?") and having to hunt down each occurrence of the question across 40 exams and check to be sure the student didn't "out think me", I simply don't enter ANY correct answers, and the system marks it as "needs grading" which lets me get to it with a single click on each exam.

            Sounds like you are using it in a way that supports learning. My experience has been the prof simply lets Canvas grade the paper for them since scores get posted as soon as the test is complete.

  • How about college students be taught what skills would be useful at a job? I mean, nobody wants to hire people straight out of college, so why not provide the work experience during college? Actually, I would let college students sit in on meetings in my company. I would do it for free, of course I am not paying them either since they aren't working for me. I mean, even if there is no work for unskilled people .. colleges should hook up deals so that individual students (maybe no more than 3 at a time) coul

    • One of the problems in modern education is the blurring of the lines between "academic" and "vocational" education. In vocational courses, work release, sandwich courses etc were the norm, but more and more, people are being pushed into universities rather than technical colleges/trade schools. Really, the university sector is too large, and we should be attempting to rejuvenate the vocational education sector instead.

      That said, I am always very dubious of "skills useful for a job", because the more you ta

  • by laughingskeptic ( 1004414 ) on Saturday February 18, 2017 @09:00PM (#53893921)
    In some cases, I'm guessing when a college or a state writes a contract for creating these courses, the length of time of the video content must be the most important clause in the contract. My son signed up for Texas' online high school physics class last summer in order to avoid taking it during the school year. It was very clear that the objective of the course materials was to consume required amount of time and not really to teach physics. It was more like a remedial drivers ed class one takes as punishment for speeding than anything that resembled a real class. I imagine that quite a few online classes come into being based on this "time content" model.
  • Faulty premise (Score:5, Insightful)

    by Hognoxious ( 631665 ) on Saturday February 18, 2017 @09:03PM (#53893933) Homepage Journal

    We're still trying to find the magic bullet that motivates people to review other people's courses when they're not being paid.

    I think I've spotted the flaw in this plan. Anyone else?

    • Re:Faulty premise (Score:4, Insightful)

      by Registered Coward v2 ( 447531 ) on Sunday February 19, 2017 @10:26AM (#53895629)

      We're still trying to find the magic bullet that motivates people to review other people's courses when they're not being paid.

      I think I've spotted the flaw in this plan. Anyone else?

      Damn. I knew there was a flaw somewhere; but I have a fix: give away the course free to the first 1000 participants in exchange for feedback; then ignore the feedback and publish it as is because fixing it would cost to much and you're already in the red from the free beta test. Of course, the as soon as you say "give it away for free to the first..." no one will like the idea anyway.

  • In the 1960s there were false answers listed in the back of some text books to catch students who simply copied answers rather than actually thinking out the solution to a problem. Actually a similar tactic could be employed to catch teachers by placing false information in the texts. I have seen professors with lofty credentials repeat a false fact that they were taught when they were in college and it tends to get handed down from generation to generation. the reason why is quite simple. Even at
  • One of the roles of postgrads is often to give feedback on course material before is it put into practice.

The moon is a planet just like the Earth, only it is even deader.

Working...