(Coursera engineer here) It's great to see thoughtful critiques like this.
Thankfully, we have a data analytics team now (3 out of our 17 engineers), and they are studying our retention statistics and the factors that affect it. They're also running A/B experiments to see what increases it and getting some interesting results.
We do see some big advantages of the timed model for learning, particularly in classes with peer-to-peer grading and evaluations, but there's obviously a big desire for the self-study mode, which is enabled for a few of our classes currently.
We're also introducing things like Signature Track, which some students sign up for just to encourage themselves to make it to the end (and it seems to work for many of them).
We'll keep experimenting to see what makes students both happy and successful. :-)
Hi, Pamela. I am a long-time user of Coursera and I love the site.
I have feedback for you as regards retention of students.
The number one thing is due dates and late penalties. If I sign up for a course and work through the first few weeks and get everything done, and then I miss a deadline because life happens, suddenly I am no longer able to get a 100% in the class. I do not mind getting an A- in a class if it happens naturally because the material is difficult, but losing my A+ or A because I am 12 hours late on an assignment ruins the experience for me. I usually drop a class when this happens and wait for it to come around again.
A counter-example to prove the rule: Robert Sedgewick's Algorithms course. The last time he gave it, he had late penalties and I only had time to get through the first two weeks of the class without getting behind. This time around, there were no late penalties. Being enormously busy with other things, I was not able to keep up with the class as it went along. However, because I lost no credit for doing things late, I was able to complete the entire class in the final 10 days of course and I got an A. I have no problem staying up all night to finish interesting programming assignments and quizzes, but I need to be allowed to do this on my schedule.
I strongly encourage you to encourage your professors to do away with late penalties. It should be up to us as students to determine what our work schedule is. I know for some courses, those that use peer-evaluation, this is impossible, but many more could take this route than currently do.
I agree that Signature Track does have the effect of encouraging one to keep up with the class.
I do hope that you will open up most (if not all) of your courses for self-study mode. There are a few courses I really wanted to take, but due to the lack of time I really couldn't commit myself at the time they opened (I wanted to do them in the summer instead!).
Taking a Coursera course is hard work. I've taken 3 courses so for (one on databases and two on algorithms) over the past year and a half. This is about as much as I can manage (I work full time and have a family), even though I would like to take more (so many interesting to choose from). But the main reason for this limit is the amount of work it takes, not how the course is presented, or how well written the mails are.
For me, the pace of the course is actually a plus. If there were no deadline, I simply would not get around to doing it. I remember when I first found all the MIT courses on-line (several years ago). I really wanted to take some of them, but because there was no schedule and no deadline, I never got around to taking any of them. It's only with Coursera that I have actually taken (and finished) any.
I don't see how something can be a "dirty secret" when everyone already knows about it. This is not news, and I think the author is well aware of that and just wanted an eye-grabbing headline.
As for courses being too fast: they're college courses. That's a core part of their value proposition. If they were slower, or did not follow a rigid schedule at all (like the "better" examples presented in the article), they would be a fundamentally different product. The author simply doesn't understand the concept behind MOOCs, and probably isn't their intended audience. He would be better served watching lectures on Youtube (I don't mean that sarcastically, there are fantastic courses available there).
Synchronous learning is not, as another poster claims, an anachronism. It simply isn't necessary or valuable for everyone. For some, myself included, it is a significant benefit. That is the market that Coursera and edX are targeting, and one shouldn't criticize a company simply because one isn't a member of their target market.
Or should you not criticize dissenters just because it happens to fit you? There could be a better way for more people, regardless of your personal experience.
My problem was mainly the availability of the materials. Codecademy and duolingo give you access to as much as you require from the start and you can go through as quickly as you like. The university driven sites limit access to so much per week (though I'm not sure how courses will operate the second time) and demand you stick to their schedule, though granted this may be due to their need to peer-review the more demanding assignments.
Unfortunately my free time isn't available in nice predetermined six-week chunks, but even if I am able to catch up three weeks or more in a weekend the courses gave a very negative vibe about continuing to progress as soon as you miss a single one of their deadlines (i.e.- "you missed our deadline for this multiple-choice computer marked test, so your effort no longer counts"). I've 'failed' several coursera sessions in the fourth or fifth week for that reason.
Timetabling seems a very traditional educational view, and it contrasted sharply with codecademy and sites like duolingo where I spent Jan and Feb learning the basics of new languages - computer and human. I finished the courses I took because I did them at my own pace.
The way that Udacity does it is perfect for me -- work through at your own pace. Coursera has a couple of self-paced classes, I believe, and I wish that more of the courses had that option.
I think the majority of MOOC course developers want to run their online courses as closely to their university versions as possible, since that's what they're used to working with. It also provides a handy way for them to go off-duty, in a sense, if the course has a finite end date. Using their current pacing structure (which is incredibly difficult, as the OP points out, for people that aren't full-time students) allows the teachers to do something other than devote themselves solely to the course, assuming they don't want to just post an archive and leave it alone -- which would meet a lot of people's needs, but misses the whole teacher-student interaction, which is pretty much missing from MOOCs anyway. If they want to provide an environment that's like a classroom, with students interacting with the instructors and with each other, you kind of need everybody at the same pace. It would be nice for us if they could slow that pace down, but that would probably increase the workload for the instructors.
We're still early in this game. I'm glad that so many professors have been willing to invest the time into developing the courses, and I understand why they are currently set up to be conveniently structured for them. I think we'll start to see some improvements if/when the money appears in the MOOC game. Once it's no longer basically charity work for the instructors, there will probably be more efforts to work around student schedules.
I agree completely on the Udacity model allowing you to work through at your own pace, as well as keeping the course open After it finishes. The strict scheduling thing with other sites drives me mad!
In addition to having a normal job, I'm a traditional university student, so any of the online classes I take are simply out of interest in the subject. I dip into the class when I have time, almost like a leisure activity. I just don't currently have the time to fit in more strict course work on top of my already over loaded class schedule and work week.
Everyone needs to understand that the low completion rates are not as bad as they at first seem. The way to try out these classes for free is to sign up. There is no obligation at all: completely voluntary, no cost, no time commitment and it will not show up on permanent record.
These completion rates are actually more like conversion rates for free trials. As many of us know these are almost always quite low. How many people actually ever read books they "Look Inside" on Amazon? How many people finish long articles on the web? Now many people sign up for paying accounts after free trials of your new website? Not very many.
They are not the same as dropout rates in high schools or universities despite what some online education haters say[1]. Like any disruptive technology the existing players are threatened so you need to pay attention to the source of the criticism.
Instead completion rates are a metric that can be used to improve the class such as the suggestions here. The developers of online classes should and are using A/B testing to improve completion rates. However like almost all conversion rates they will likely remain low.
That's because online education tries to simulate traditional education.
Back awhile ago free online lecture videos (e.g. Berkeley) were simply uploaded to a page and anyone could go watch them without signing up for a class or otherwise jumping through hoops.
This resulted in me watching a few random lectures about subjects I know little or nothing about. Like econ classes, engineering, science, or similar.
This was good because it was short bursts of information without any hassle, commitment, and similar.
These days everyone has locked their content behind sign up/registration pages, and you're expected to commit to a tradition semester of a class in order to get a worthless certificate of completion.
They also dictate the speed at which you learn. No more learning at your own pace, no, you have to do one a week for as many weeks as it goes on for like it was a traditional class.
And why? Why indeed. Why is online education simulating University education when it actually has no relationship to it? You aren't getting a University qualification by completing a "class," there is no reason why a "class" has to be X number of weeks or you should have to complete the rereq's in order to take it.
Places like Khan Academy have got this right. I can go to Khan Academy right now, click on a video series, watch a few bites of information and then stop when I'm ready/bored. Coursera's class system is just pointless, stressful, and annoying.
If Coursera was transferable then it would absolutely make sense to do. But that doesn't look likely and while that isn't the case they're just making access to learning material more difficult with no obvious benefit to anyone.
I agree with many of your points but the main reason I use Coursera is because it's an actual course, with structure and quizzes and assignments. I personally find Khan Academy to be useless, and I've tried to use it (Edit: useless for me, but I do think what he has created is fantastic). I think I would find it useful as a supplement to an existing course, and that's what it was originally designed as. Very few people learn anything of moderate complexity without doing exercises throughout.
With regards to your use of the word 'worthless' relating to the Coursera certificate of completion.
Many university courses aren't significantly different to Coursera so I think that if the courses keep up their standards and enough people have knowledge of them then it is possible that that certificate will not remain worthless.
We were discussing Coursera at work last week and I think that if a job candidate appeared with Coursera completion certificates on their CV then it wouldn't be looked at unfavourably.
I actually often "never really intended to seriously take the class in the first place." Coursera doesn't let you view the materials unless you sign up for the class, so I sign up for every class I might be interested in browsing a little. Then I can view the materials at leisure, even long after the class is finished.
On the other hand, I mostly completed the first algorithms course and Model Thinking, and while Daphne's class kicked my ass the first time, I'm already reviewing to tackle it again and try to finish it this time.
It is true though that the time commitment is horrendous, especially for Daphne's class, which estimates 15-20 hours per week! If I fall behind, I may finish slower than their schedule and miss the homeworks and final exam, in which case I'll officially be an incomplete but I'll still have gotten through all the material.
In short, retention is an irrelevant metric for online classes, especially for Coursera which has such a strong incentive to sign up "unseriously." My suggestion would be to keep accepting and grading homeworks and exams at any time, and give extra props on the certs for completing on time with the other students.
You should title this article "How to cheapen the online education experience". You may be right, that edX and Coursera require a big commitment, but that's what learning requires. Do you want to play games for an hour, or do you want to advance your skills and understanding?
(I've completed the Codeacademy Javascript track, and a full Coursera class (livin' in the 5%, wooo!)).
You say Codeacademy got it right, but I think you're dead wrong. As other articles today have pointed out, at best you're going to learn syntax on codeacademy, and the most basic programming principles, but you'll be miles away from being a capable programmer. The jump from the online editor with tiny little exercises to setting up your own environment and programming your own project is huge. Many of their lessons give misleading or outright incorrect information, not to mention teaching you bad practices.
What this article says is the huge barrier to entry, is not a barrier to entry to online education, but a barrier to entry of learning. What the author proposes is solutions to hide the barrier of learning, by gamifying the platforms and making them more "fun" with a disregard for the depth of effectiveness, and the consequences of pursuing such techniques. (Obviously I'm delving into opinion land, but I strongly believe gamification cheapens and platform and doesn't produce the deep engagement necessary for learning. This is my assessment from my experiences, I'm not going to tell someone their wrong for feeling differently about this. But I think if you wanted to create a deeper experience from gamifying education, it would take a tremendous effort, far different from the typical "Wow you completed 5 exercises! ZOMG KEEP GOING!" joke of rewards that other websites have.
I don't want to rail on the online education effort either, but this is plain cheap thinking. Coursera and EdX are doing an excellent job, but they certainly have many problems to surmount. They've definitely solved the problem of higher education accessibility, now the problems they have are in the effectiveness of education, and on administration and grading that gets the f#$% out of the way of learning. But making it "easier to learn"? I call utter bulls$%t.
I think I agree with most of your points, but disagree with your disapproval of "easier to learn". The presentation of material really makes a gigantic difference on how well you learn the material (if at all). While I agree that coursera and edX shouldn't hasten towards "gamification" of any sort, I think there is plenty of work in making the classes of better quality, and I think that _will_ make it easier to learn.
I was taking a course on coursera where I thought the professor was talking too quickly --- so I actually slowed it down to 75% the pace in the video. Awesome! I was happy to be able to do that.
Otherwise though, I don't think the course was presented that well. For one, lectures were shitty recordings of slides; I was looking at nothing but text and static images for entire lectures (never seeing the professor's face), and they were blurry, hard to read, and just hard to understand. Of _course_ that makes it harder to learn! I have to focus on figuring out what the hell is written there! And there was no (free) textbook for the course.
Anyway, I think the point I am just trying to make is that while the author of this article might be suggesting to Coursera to take steps in questionable directions, I DO think a lot of those courses could be better polished, and that would make it easier to learn from them.
Why do people think retention is a good thing?! I think retention is actually bad even for classical education.
The lower the retention rate, the more the likelihood that they are following lots of courses at the same time or multitasking with something else. This means that people are "exploring" more! I think that "exploratory learning" has always been stifled by classical organized education and is one of the reasons why I hate most academic environments (yeah, they're cool if you're in one of the top 10% unis or in a "privileged" position", but not for the rest of cases...). I believe that, after a certain level of baseline knowledge, in any field, it's actually more important that someone learns "what they want/need/have inclination for/find more interesting" than that they "learn more"! Maybe more time exploring and less time actually uploading things to your mind is better (not "productive", just "better", and I said "fuck productivity" a lot lately because I found that it just doesn't lead to better anything). All the new ways of doing education make exploration easier (as in you can explore a lot without really "wasting" that much time), and we should take advantage of this!
Pacing is incredibly important to motivation. The thing is, there's no single answer. Even in traditional, full-time, in person classrooms where everyone puts in the same class hours, natural ability, different skills at the beginning and better attention mean that students will always learn at different speeds.
The ideal model is one of personalized education. What's kind of shocking is that online education so far has basically ignored this altogether. There's been zero investment in ways to continually pace and encourage students to learn at a speed that is both challenging and within reason.
This isn't too surprising: it's a hard problem, and it's much easier to have users paying a subscription fee forever (for those services that are paid) and blame themselves when they fall behind.
At Thinkful (http://www.thinkful.com/) we see evidence that our learning model, which pairs experts and students together much more like you'd expect from tutoring, sees 10x the retention and completion as other online course options.
My mother used to tell me that my eyes were bigger than my stomach when I took food more food than I could eat. I have the same problem with MOOCs.
The future me always has more time, is always more dedicated to learning, is always more focused. So I sign up, and when the class starts I'm no longer the better, stronger, faster, future, me, I'm just me.
Still I manage to finish some, and learn a little from the ones that I drop, so I don't see it as a problem.
Another week, another article complaining that teaching a class which 5-10 thousand people successfully complete, and which cost less than $10 per successful student to provide, is an abysmal failure.
Meanwhile, the schools that the authors hold up as more successful examples of education have 20-500 successful graduates per session offered, nearly all of whom are idle young wealthy white people living in rich western countries, and who pay up to several thousand dollars per session each for this privilege.
The article seems to imply that, in an ideal world, all of these student would stay throughout the course. But I don't think it's clear that the churn is bad.
There is some lower bound to the amount of time that it takes to learn something (absent some educational revolution). Many people aren't even willing to pay that lower bound simply because they have other priorities in life. But they do like to sample (which is good), and potentially finish out a course if nothing else gets in the way.
Online courses are great because they lower the cost of sampling, so we shouldn't be surprised that there are more samplers, and fewer people finishing out the courses. If 1000 people complete an online course, then that's great, even if 99000 people signed up and disappeared a week later.
That being said, I'm pleased to see the specific criticisms offered in the article, and I hope they lead to a better balance for more people.
Thankfully the article is much better than its title. Low retention is not a problem per se - since there's no cost to signing up for (and subsequently dropping) these online courses, it's to be expected that retention won't be great. However, I agree that retention could probably be improved among the marginal subscribers by using some of the techniques covered in the post.
For the past year, I've only been seriously interested in three Coursera courses - Scala, Ng's Machine Learning, and Probabilistic Graphical Networks.
All three overlap at the same time!! And they don't give assurances of when they will be offered again, or even if. Argh. I've started Scala but doubt I'll be able to continue if it I attempt PGN again. This happened to me last fall and I ended up completing none of the three.
Plus, it is apparently "known"... somewhere... that certain courses are easier if you take other courses first... but good luck finding that information when you want to refer to it.
I agree the scheduling makes it difficult. I'm doing the NLP class right now, and scheduled to start the PGM class in a couple of weeks. I'm worried about the overlap, but I'll make it work out.
But I also wanted to do the Scala course. I didn't start it because I know I'd not be able to keep up with all three when they overlapped for a few weeks. It's a shame, but I had to prioritize.
My wife also wanted to take that Introductory Human Physiology course. She's a freelance medical translator, so improving her knowledge of that subject matter will have a direct positive impact on her career. She's not among the people who Daphne Koller said "never really intended to seriously take the class in the first place" - she bought the suggested books ahead of time, marked it all down on her calendar, and planned out time to work on it.
What she found was that the course was way more difficult than the description suggested, and proved to be well above what she could understand with her current background. She watched the first video and found that it was going to take way more time than the course description predicted, because she would have to try to fill in a tremendous amount of background knowledge that the teachers presumed. On the plus side, from the course forums she found about resources (videos, etc.) that were more at her current level, and has been working through these.
The comments made elsewhere about the synchronous model for these courses is probably right for her case - she had to wait a number of weeks in order to find out it wasn't the right class for her. This is what you expect when you're in college, at which point you either drop the class and maybe find a different one to take, or tough it out in a class that's not as interesting or useful as you'd hoped. I hope online courses can do better than this.
> What she found was that the course was way more difficult than the description suggested, and proved to be well above what she could understand with her current background. She watched the first video and found that it was going to take way more time than the course description predicted, because she would have to try to fill in a tremendous amount of background knowledge that the teachers presumed.
It's quite possible that this is a function of having so little open content out there right now. I can imagine a future where the background your wife needed would be easy to get from the platform itself, so that the profs' demanding teaching could serve audiences both with and without that background. Right now many of these courses must feel like skyhooks but once they're embedded in a web of knowledge (an earlier commenter called these 'nodes') they'll be more accessible i think.
There's still a huge problem of collecting and sequencing these nodes and I think many students will need direct contact with instructors of some sort to do this.
One of the main problems the author has with Coursera or EdX is that it is "too fast". To address this problem, I urge the author to try Udacity [0]. You can set learning at your own pace.
That said, not all material on Coursera is "amazing". Some of the classes have very high completion rate like Functional Programming with Scala has a 19.2% completion rate [1]. Similarly, the class taken by Andrew Ng on Machine Learning is fantastic. However, many of my peers had bad reviews on Daphne Koller's Probabilistic Graphical Models class. Last year, I myself registered for one of these MOOC's and found that half of the course was good while the other half was quite bad - both halves had different professors.
At some point of time, universities would have to realize that great researchers do not make great teachers. Some excel in both - researching & teaching while some in just one of those two fields.
PS - Other problems on reengagement do stand though.
Koller's class is difficult, but it's a graduate-level CS class with almost all the same material as the class at Stanford. I thought she taught it well. It was rough, and I didn't make it through the whole thing the first time, but I'm back to try again.
We could come up with a bunch of excuses about why online education will always have worse retention, or we could figure out ways to fix it.
Or we could ask whether it is a problem. I'm not sure that it is.
I don't think the numbers are surprising or unnatural at all. The thing to compare them against is not traditional education but rather the patterns of all people who take the autodidactic route.. every person who has ever picked up a book and not finished it. Even then, the comparison isn't perfect.
In fact, we could argue that low retention numbers are actually a good thing for non-credit courses. It means that the content is challenging and focused, and that people are able to discover early whether it is or isn't for them or cherry-pick the content that they need and move on.
If there are usability problems, fix them. But if participant retention got dramatically higher, I'd worry.
[+] [-] pamelafox|13 years ago|reply
Thankfully, we have a data analytics team now (3 out of our 17 engineers), and they are studying our retention statistics and the factors that affect it. They're also running A/B experiments to see what increases it and getting some interesting results.
We do see some big advantages of the timed model for learning, particularly in classes with peer-to-peer grading and evaluations, but there's obviously a big desire for the self-study mode, which is enabled for a few of our classes currently.
We're also introducing things like Signature Track, which some students sign up for just to encourage themselves to make it to the end (and it seems to work for many of them).
We'll keep experimenting to see what makes students both happy and successful. :-)
[+] [-] jahmohn|13 years ago|reply
I have feedback for you as regards retention of students.
The number one thing is due dates and late penalties. If I sign up for a course and work through the first few weeks and get everything done, and then I miss a deadline because life happens, suddenly I am no longer able to get a 100% in the class. I do not mind getting an A- in a class if it happens naturally because the material is difficult, but losing my A+ or A because I am 12 hours late on an assignment ruins the experience for me. I usually drop a class when this happens and wait for it to come around again.
A counter-example to prove the rule: Robert Sedgewick's Algorithms course. The last time he gave it, he had late penalties and I only had time to get through the first two weeks of the class without getting behind. This time around, there were no late penalties. Being enormously busy with other things, I was not able to keep up with the class as it went along. However, because I lost no credit for doing things late, I was able to complete the entire class in the final 10 days of course and I got an A. I have no problem staying up all night to finish interesting programming assignments and quizzes, but I need to be allowed to do this on my schedule.
I strongly encourage you to encourage your professors to do away with late penalties. It should be up to us as students to determine what our work schedule is. I know for some courses, those that use peer-evaluation, this is impossible, but many more could take this route than currently do.
I agree that Signature Track does have the effect of encouraging one to keep up with the class.
[+] [-] zura|13 years ago|reply
So successfully completion of the course won't require watching the videos.
[+] [-] mikevm|13 years ago|reply
I do hope that you will open up most (if not all) of your courses for self-study mode. There are a few courses I really wanted to take, but due to the lack of time I really couldn't commit myself at the time they opened (I wanted to do them in the summer instead!).
I also think this guy makes valid points: https://news.ycombinator.com/item?id=5469909
[+] [-] henrik_w|13 years ago|reply
For me, the pace of the course is actually a plus. If there were no deadline, I simply would not get around to doing it. I remember when I first found all the MIT courses on-line (several years ago). I really wanted to take some of them, but because there was no schedule and no deadline, I never got around to taking any of them. It's only with Coursera that I have actually taken (and finished) any.
I've written about my experience of all three courses on my blog. The latest was Algorithms: Design and Analysis, part 2 http://henrikwarne.com/2013/02/18/coursera-algorithms-course...
[+] [-] nbouscal|13 years ago|reply
As for courses being too fast: they're college courses. That's a core part of their value proposition. If they were slower, or did not follow a rigid schedule at all (like the "better" examples presented in the article), they would be a fundamentally different product. The author simply doesn't understand the concept behind MOOCs, and probably isn't their intended audience. He would be better served watching lectures on Youtube (I don't mean that sarcastically, there are fantastic courses available there).
Synchronous learning is not, as another poster claims, an anachronism. It simply isn't necessary or valuable for everyone. For some, myself included, it is a significant benefit. That is the market that Coursera and edX are targeting, and one shouldn't criticize a company simply because one isn't a member of their target market.
[+] [-] lawnchair_larry|13 years ago|reply
[+] [-] manifold|13 years ago|reply
Unfortunately my free time isn't available in nice predetermined six-week chunks, but even if I am able to catch up three weeks or more in a weekend the courses gave a very negative vibe about continuing to progress as soon as you miss a single one of their deadlines (i.e.- "you missed our deadline for this multiple-choice computer marked test, so your effort no longer counts"). I've 'failed' several coursera sessions in the fourth or fifth week for that reason.
Timetabling seems a very traditional educational view, and it contrasted sharply with codecademy and sites like duolingo where I spent Jan and Feb learning the basics of new languages - computer and human. I finished the courses I took because I did them at my own pace.
[+] [-] viveutvivas|13 years ago|reply
I think the majority of MOOC course developers want to run their online courses as closely to their university versions as possible, since that's what they're used to working with. It also provides a handy way for them to go off-duty, in a sense, if the course has a finite end date. Using their current pacing structure (which is incredibly difficult, as the OP points out, for people that aren't full-time students) allows the teachers to do something other than devote themselves solely to the course, assuming they don't want to just post an archive and leave it alone -- which would meet a lot of people's needs, but misses the whole teacher-student interaction, which is pretty much missing from MOOCs anyway. If they want to provide an environment that's like a classroom, with students interacting with the instructors and with each other, you kind of need everybody at the same pace. It would be nice for us if they could slow that pace down, but that would probably increase the workload for the instructors.
We're still early in this game. I'm glad that so many professors have been willing to invest the time into developing the courses, and I understand why they are currently set up to be conveniently structured for them. I think we'll start to see some improvements if/when the money appears in the MOOC game. Once it's no longer basically charity work for the instructors, there will probably be more efforts to work around student schedules.
[+] [-] goostavos|13 years ago|reply
In addition to having a normal job, I'm a traditional university student, so any of the online classes I take are simply out of interest in the subject. I dip into the class when I have time, almost like a leisure activity. I just don't currently have the time to fit in more strict course work on top of my already over loaded class schedule and work week.
[+] [-] spikels|13 years ago|reply
These completion rates are actually more like conversion rates for free trials. As many of us know these are almost always quite low. How many people actually ever read books they "Look Inside" on Amazon? How many people finish long articles on the web? Now many people sign up for paying accounts after free trials of your new website? Not very many.
They are not the same as dropout rates in high schools or universities despite what some online education haters say[1]. Like any disruptive technology the existing players are threatened so you need to pay attention to the source of the criticism.
Instead completion rates are a metric that can be used to improve the class such as the suggestions here. The developers of online classes should and are using A/B testing to improve completion rates. However like almost all conversion rates they will likely remain low.
[1] http://mitsloan.mit.edu/shared/ods/documents/High-Costs-of-F...
[+] [-] UnoriginalGuy|13 years ago|reply
Back awhile ago free online lecture videos (e.g. Berkeley) were simply uploaded to a page and anyone could go watch them without signing up for a class or otherwise jumping through hoops.
This resulted in me watching a few random lectures about subjects I know little or nothing about. Like econ classes, engineering, science, or similar.
This was good because it was short bursts of information without any hassle, commitment, and similar.
These days everyone has locked their content behind sign up/registration pages, and you're expected to commit to a tradition semester of a class in order to get a worthless certificate of completion.
They also dictate the speed at which you learn. No more learning at your own pace, no, you have to do one a week for as many weeks as it goes on for like it was a traditional class.
And why? Why indeed. Why is online education simulating University education when it actually has no relationship to it? You aren't getting a University qualification by completing a "class," there is no reason why a "class" has to be X number of weeks or you should have to complete the rereq's in order to take it.
Places like Khan Academy have got this right. I can go to Khan Academy right now, click on a video series, watch a few bites of information and then stop when I'm ready/bored. Coursera's class system is just pointless, stressful, and annoying.
If Coursera was transferable then it would absolutely make sense to do. But that doesn't look likely and while that isn't the case they're just making access to learning material more difficult with no obvious benefit to anyone.
[+] [-] lawnchair_larry|13 years ago|reply
[+] [-] shawabawa3|13 years ago|reply
You can. But you wont.
The sad fact is that deadline motivate people. Without having deadlines and structure coursera would have gone nowhere.
[+] [-] adaml_623|13 years ago|reply
Many university courses aren't significantly different to Coursera so I think that if the courses keep up their standards and enough people have knowledge of them then it is possible that that certificate will not remain worthless.
We were discussing Coursera at work last week and I think that if a job candidate appeared with Coursera completion certificates on their CV then it wouldn't be looked at unfavourably.
[+] [-] DennisP|13 years ago|reply
On the other hand, I mostly completed the first algorithms course and Model Thinking, and while Daphne's class kicked my ass the first time, I'm already reviewing to tackle it again and try to finish it this time.
It is true though that the time commitment is horrendous, especially for Daphne's class, which estimates 15-20 hours per week! If I fall behind, I may finish slower than their schedule and miss the homeworks and final exam, in which case I'll officially be an incomplete but I'll still have gotten through all the material.
In short, retention is an irrelevant metric for online classes, especially for Coursera which has such a strong incentive to sign up "unseriously." My suggestion would be to keep accepting and grading homeworks and exams at any time, and give extra props on the certs for completing on time with the other students.
[+] [-] Jormundir|13 years ago|reply
(I've completed the Codeacademy Javascript track, and a full Coursera class (livin' in the 5%, wooo!)).
You say Codeacademy got it right, but I think you're dead wrong. As other articles today have pointed out, at best you're going to learn syntax on codeacademy, and the most basic programming principles, but you'll be miles away from being a capable programmer. The jump from the online editor with tiny little exercises to setting up your own environment and programming your own project is huge. Many of their lessons give misleading or outright incorrect information, not to mention teaching you bad practices.
What this article says is the huge barrier to entry, is not a barrier to entry to online education, but a barrier to entry of learning. What the author proposes is solutions to hide the barrier of learning, by gamifying the platforms and making them more "fun" with a disregard for the depth of effectiveness, and the consequences of pursuing such techniques. (Obviously I'm delving into opinion land, but I strongly believe gamification cheapens and platform and doesn't produce the deep engagement necessary for learning. This is my assessment from my experiences, I'm not going to tell someone their wrong for feeling differently about this. But I think if you wanted to create a deeper experience from gamifying education, it would take a tremendous effort, far different from the typical "Wow you completed 5 exercises! ZOMG KEEP GOING!" joke of rewards that other websites have.
I don't want to rail on the online education effort either, but this is plain cheap thinking. Coursera and EdX are doing an excellent job, but they certainly have many problems to surmount. They've definitely solved the problem of higher education accessibility, now the problems they have are in the effectiveness of education, and on administration and grading that gets the f#$% out of the way of learning. But making it "easier to learn"? I call utter bulls$%t.
[+] [-] asafira|13 years ago|reply
I was taking a course on coursera where I thought the professor was talking too quickly --- so I actually slowed it down to 75% the pace in the video. Awesome! I was happy to be able to do that.
Otherwise though, I don't think the course was presented that well. For one, lectures were shitty recordings of slides; I was looking at nothing but text and static images for entire lectures (never seeing the professor's face), and they were blurry, hard to read, and just hard to understand. Of _course_ that makes it harder to learn! I have to focus on figuring out what the hell is written there! And there was no (free) textbook for the course.
Anyway, I think the point I am just trying to make is that while the author of this article might be suggesting to Coursera to take steps in questionable directions, I DO think a lot of those courses could be better polished, and that would make it easier to learn from them.
[+] [-] nnq|13 years ago|reply
The lower the retention rate, the more the likelihood that they are following lots of courses at the same time or multitasking with something else. This means that people are "exploring" more! I think that "exploratory learning" has always been stifled by classical organized education and is one of the reasons why I hate most academic environments (yeah, they're cool if you're in one of the top 10% unis or in a "privileged" position", but not for the rest of cases...). I believe that, after a certain level of baseline knowledge, in any field, it's actually more important that someone learns "what they want/need/have inclination for/find more interesting" than that they "learn more"! Maybe more time exploring and less time actually uploading things to your mind is better (not "productive", just "better", and I said "fuck productivity" a lot lately because I found that it just doesn't lead to better anything). All the new ways of doing education make exploration easier (as in you can explore a lot without really "wasting" that much time), and we should take advantage of this!
[+] [-] darrellsilver|13 years ago|reply
The ideal model is one of personalized education. What's kind of shocking is that online education so far has basically ignored this altogether. There's been zero investment in ways to continually pace and encourage students to learn at a speed that is both challenging and within reason.
This isn't too surprising: it's a hard problem, and it's much easier to have users paying a subscription fee forever (for those services that are paid) and blame themselves when they fall behind.
At Thinkful (http://www.thinkful.com/) we see evidence that our learning model, which pairs experts and students together much more like you'd expect from tutoring, sees 10x the retention and completion as other online course options.
[+] [-] lawnchair_larry|13 years ago|reply
[+] [-] KC8ZKF|13 years ago|reply
The future me always has more time, is always more dedicated to learning, is always more focused. So I sign up, and when the class starts I'm no longer the better, stronger, faster, future, me, I'm just me.
Still I manage to finish some, and learn a little from the ones that I drop, so I don't see it as a problem.
[+] [-] droithomme|13 years ago|reply
Meanwhile, the schools that the authors hold up as more successful examples of education have 20-500 successful graduates per session offered, nearly all of whom are idle young wealthy white people living in rich western countries, and who pay up to several thousand dollars per session each for this privilege.
[+] [-] jeffdavis|13 years ago|reply
There is some lower bound to the amount of time that it takes to learn something (absent some educational revolution). Many people aren't even willing to pay that lower bound simply because they have other priorities in life. But they do like to sample (which is good), and potentially finish out a course if nothing else gets in the way.
Online courses are great because they lower the cost of sampling, so we shouldn't be surprised that there are more samplers, and fewer people finishing out the courses. If 1000 people complete an online course, then that's great, even if 99000 people signed up and disappeared a week later.
That being said, I'm pleased to see the specific criticisms offered in the article, and I hope they lead to a better balance for more people.
[+] [-] waterlesscloud|13 years ago|reply
I signed up for the Hinton neural network class when it was almost over, and completed it well after it ended.
Was I "retained"? Did I not count as completing the class?
Just because there's deadlines doesn't mean you have to finish it by the deadline.
[+] [-] kvb|13 years ago|reply
[+] [-] tunesmith|13 years ago|reply
All three overlap at the same time!! And they don't give assurances of when they will be offered again, or even if. Argh. I've started Scala but doubt I'll be able to continue if it I attempt PGN again. This happened to me last fall and I ended up completing none of the three.
Plus, it is apparently "known"... somewhere... that certain courses are easier if you take other courses first... but good luck finding that information when you want to refer to it.
[+] [-] waterlesscloud|13 years ago|reply
But I also wanted to do the Scala course. I didn't start it because I know I'd not be able to keep up with all three when they overlapped for a few weeks. It's a shame, but I had to prioritize.
[+] [-] andrewem|13 years ago|reply
What she found was that the course was way more difficult than the description suggested, and proved to be well above what she could understand with her current background. She watched the first video and found that it was going to take way more time than the course description predicted, because she would have to try to fill in a tremendous amount of background knowledge that the teachers presumed. On the plus side, from the course forums she found about resources (videos, etc.) that were more at her current level, and has been working through these.
The comments made elsewhere about the synchronous model for these courses is probably right for her case - she had to wait a number of weeks in order to find out it wasn't the right class for her. This is what you expect when you're in college, at which point you either drop the class and maybe find a different one to take, or tough it out in a class that's not as interesting or useful as you'd hoped. I hope online courses can do better than this.
[+] [-] eah13|13 years ago|reply
It's quite possible that this is a function of having so little open content out there right now. I can imagine a future where the background your wife needed would be easy to get from the platform itself, so that the profs' demanding teaching could serve audiences both with and without that background. Right now many of these courses must feel like skyhooks but once they're embedded in a web of knowledge (an earlier commenter called these 'nodes') they'll be more accessible i think.
There's still a huge problem of collecting and sequencing these nodes and I think many students will need direct contact with instructors of some sort to do this.
[+] [-] denzil_correa|13 years ago|reply
That said, not all material on Coursera is "amazing". Some of the classes have very high completion rate like Functional Programming with Scala has a 19.2% completion rate [1]. Similarly, the class taken by Andrew Ng on Machine Learning is fantastic. However, many of my peers had bad reviews on Daphne Koller's Probabilistic Graphical Models class. Last year, I myself registered for one of these MOOC's and found that half of the course was good while the other half was quite bad - both halves had different professors.
At some point of time, universities would have to realize that great researchers do not make great teachers. Some excel in both - researching & teaching while some in just one of those two fields.
PS - Other problems on reengagement do stand though.
[0] https://www.udacity.com/ [1] http://www.katyjordan.com/MOOCproject.html
[+] [-] DennisP|13 years ago|reply
[+] [-] michaelfeathers|13 years ago|reply
Or we could ask whether it is a problem. I'm not sure that it is.
I don't think the numbers are surprising or unnatural at all. The thing to compare them against is not traditional education but rather the patterns of all people who take the autodidactic route.. every person who has ever picked up a book and not finished it. Even then, the comparison isn't perfect.
In fact, we could argue that low retention numbers are actually a good thing for non-credit courses. It means that the content is challenging and focused, and that people are able to discover early whether it is or isn't for them or cherry-pick the content that they need and move on.
If there are usability problems, fix them. But if participant retention got dramatically higher, I'd worry.