Whilst the rise of the MOOCs has gained a lot of publicity, it is slightly more controversial to suggest that they have been a resounding success from an educational perspective.  Accusations of sky high drop out rates and poor learning takeup have marred the industry.

A recent study, published in the International Review of Research in Open and Distance Learning, suggests however that MOOCs have actually done a pretty good job, and provides a ringing endorsement of their pedagogical merits.

“It’s an issue that has been very controversial,” the researchers say. “A number of well-known educators have said there isn’t going to be much learning in MOOCs, or if there is, it will be for people who are already well-educated.”

When the researchers tested both MOOC students on a MIT physics class and their offline peers, they found that the knowledge learned via the MOOC was greater than that in the traditional, lecture based course.

What’s more, they found that even those MOOC students that were not at all well prepared for their course (ie those with low scores before the course started), ended up learning just as well as their fellow students.  The rate of improvement was almost equal regardless of the skill level at the outset of the course.

The researchers believe their methodology is a sound one, as it has been used previously to study around 65 classroom based courses at MIT.  Therefore they have a high degree of confidence in their pre and post testing approach to measuring learning.  They reveal however that this is the first time such an approach has been deployed on the MOOC environment.

In addition to their before and after testing, the researchers also looked at the regular testing performed during the MOOC, and the homework that was requested of each student.  This analysis was done using item-response theory, which is similar in style to the method used to ensure results from standardized tests remain consistent.

With both approaches, the results remained the same.  Online courses were giving equal learning opportunities to all students, regardless of their abilities at the start of the course.

Whilst the researchers would love to extend their approach across the 1,000 or so classes currently available via the MOOC platforms, they are well aware of the work involved in doing this, plus the commitment of the instructors to include the additional testing procedures within their courses.

They are however quite confident that their study will form the bulkhead of a new wave of analytical study of the effectiveness of MOOC based learning.  The quantitative nature of the courses give researchers a treasure trove of data they can use, such as their access to video content, how much of a digital text book they have read, how long they spend watching lectures and so on.  All of this can be analyzed to gain a better understanding of what works and what doesn’t.