Presentation from OpenEd13 (Utah 6-8 Nov 2013) where I talk about my research looking at free online courses and responsibilities toward the learners.

1. Free online courses and social
responsibility toward learners.
A systematic literature and blogature review.
Dr Vivien Rolfe
University of the West of England, Bristol UK
National Teacher Fellow
Science Open Educator
CC BY Vivien Rolfe

2. Presentation for
Open Education Conference
(OpenEd 2013)
November 6-8, Utah Park City

3. Why this question?
• I would argue that campus and distance learning
programmes have the following ethical stance:
– Students have to meet entry criteria for enrolment
– Curricula place great emphasis on student experience and
outcomes
– Strategies address academic quality, accessibility,
inclusivity and diversity
– Academics care for student welfare
• What is the academic and ethical stance surrounding
newer free online courses? xMoocs, cMoocs?

4. Aims
• The aim was to systematically review the research literature and
“blogature” to identify evidence (and the gaps) relating to open
online courses and the student experience. Areas of interest:
–
–
–
–
Student experience and success
Social and digital inclusion
Accessibility
Quality and governance
• Did not achieve a second aim to survey academic and student
opinion – moved house, jobs and cities over the summer!
CC BY Vivien Rolfe

5. #Searching “NOT Minnie the Moocher”
SOURCES
•
•
•
•
Six research databases – keywords & MeSH
Google Scholar
Google Blogs (first 200 blogs of 2 searches)
Peers, Twitter and Google alerts for six months
KEYWORDS
• cMOOX*, xMOOC*, massive online open
courses….
• Student experience, social responsibility, ethics,
diversity, accessibility……
• NOT molybdenum (MoOCl)
• NOT “Minnie the Moocher”
Wikimedia Commons, Public Domain

6. Lit and Blog Analysis
IDENTIFICATION
•
•
Read titles and abstracts, and retrieved full document.
Searched blogs / web documents / PDFs for keywords using the “search option”.
EXCLUSION LEG 1
•
Duplicates were excluded. Those with no keyword matches within the article were excluded. No
research was excluded on the basis of quality (which would be the norm for a systematic review).
(Rolfe and Gray 2011)
EXCLUSION LEG 2
•
Articles, reviews, blogs were then excluded on fuller examination of the text (e.g. if a blog about a
MOOC, personal experience of a MOOC, keyword matched but out of context with my themes).
ANALYSIS
•
•
•
Search results were recorded on a spreadsheet.
Separate analyses for 1) peer-reviewer research studies; 2) peer-reviewed journal review articles; 3)
reports, eBooks and blog articles.
Review and blog themes were clustered, and new relevant themes identified.

7. Lit and Blog Identification
BLOGS,
75
(20122013)
Results
ARTICLE SOURCES
ARTICLES,
141
(20042013)
Medline
12
30
Web Of
Knowledge
IEEE Explorer
44
Scopus
40
Google Scholar
15
ARTICLE TYPES
4%
2%
Conference
abstracts/proceedins
Journal articles
0
New Literature
8%
32%
5%
18%
31%
Journal
editorial/comment
Journal literature
review
White Paper/horizon
report
Systematic review
Other
Scholar did not
retrieve a single
JOURNAL paper.

8. Lit and Blog Identification
BLOG SOURCES
5%
Twitter
Google Blog
Search
95%
BLOG AUTHORS
1
11
Students
4
Dolls
20
8
Nuns
Bloggers
12
Journalists
Technologists
26
Educators
Others

9. BLOGS,
75
(20122013)
ARTICLES,
141
(20042013)
EXCLUSION
LEG 1
75
129
EXCLUSION
LEG 2
39
38

10. Puntastic!
The MOOCs of Hazard
I’m in the mood for MOOCs
Some MOOC points
MOOC or not to MOOC
Wikimedia
Commons, http://en.wikipedia.org/wiki/File:Erroll_Geo
rge.jpg

11. Lit Analysis – Leg 2; 129 Articles down to 38.

12. •
•
•
•
Only 1 “controlled” academic study.
3 gained ethical approval to use student data/blog narratives for research.
1 study from Sri Lanka looked at digital and social inclusion.
Learner data not the learner!

13. Systematic Reviews
• Liyanagunawardena et al (2013) – systematic account of
MOOC literature (research + other documents); 45 items
and 9 matched for student experience. Not evaluated for
quality further.
• Means et al (2010); search of 1132 and 45 included for
analysis.
“The most unexpected finding was that an extensive initial search
of the published literature from 1996 through 2006 found no
experimental or controlled quasi-experimental studies”
“Elements such as video or online quizzes do not appear to
influence the amount that students learn in online classes. The
research does not support the use of some frequently
recommended online learning practices”.

14. Blog Analysis – Leg 2; 75 Articles down to 39.

15. INCLUSION
“And what does “primarily free” mean?
MOOCs are only available to people with
excellent broadband internet access and fast
computers”
OUR ACADEMIC SOCIAL RESPONSIBILITY
“Drop out rates. But what about the drop outs?”
“I would hate to see people get a bad taste of
university because there's just too many
students in there to get personal attention.”
“The question is if students without the
literacies required to take a Higher Education
degree are able to succeed”

16. IP AND PRIVACY
“Then there are the sticky matters of
plagiarism, identity fraud, peer grading, and a
lack of creditable assessment of learning
outcomes.”
“There will be no private, “safe” spaces for
learning.”
“Assuming you have the right tech, you’ll also
have to be comfortable with being tracked and
monitored.”
ACADEMIC QUALITY ASSURANCE / PEDAGOGY
“Who will validate student learning?”
“Fairly unaltered in relation to the important
stuff like instructional design, instructional
delivery, and authentic assessment.”

17. Summary – learner experience?
• Concerns for learner experience and our social
responsibility was reflected in blog
opinions, but was not the subject of good
quality research.
• A lack of concern with who drops out and
why, and the impact on their future learning.

18. Summary – pedagogy?
• Insights into online learning design from Means et al’s review (2010) are
not reflected in xMOOC design. Students online performed slightly better
than face-to-face (although there was no mention of time invested).
Learning outcomes were enhanced by:
– Collaborative or instructor-directed learning rather than independent.
– Giving learners control of their interactions and time for reflection.
• Learning was not enhanced by:
– Use of video or online quizzes.
– Providing guidance to groups as opposed to individuals.
• Discussions about pedagogy were reflected in blog opinions but not
empirical research.

19. Summary – unethical?
• What are the user rights - for privacy? A safe
learning space? Is there any data protection?
Who owns the user data, and how will it be
used next? Affiliate marketing?
• Where are the ethical standards? Blog and
personal learning narratives readily used in
research. Where is the informed consent?

20. Summary – scope of research?
• The current work concurs with past systematic
reviews on the lack of good quality
educational research in online learning (Dean
2010, Rolfe 2011).
• Research emerging over the last 5 years is
interested in learner DATA not the learner.

21. Recommendations for future research.
ANY!

22. Let’s not do education by numbers!
CC BY Vivien Rolfe

23. Thank you!
For further information go to:
vivrolfe.com
• Slides on Slideshare
• Reference Lists on Google Docs
• Methods blogs – searching literature and “blogature”
• Learning by numbers. Pitfalls of pursuing targets.
• Full paper – results of systematic review, research findings
(under way).

24. References
• Liyanagunawardena TR, Adams AAA and Williams SA (2013) MOOCs: A
systematic study of the published literature 2008-2012. The International
Review of Research in Open and Distance Learning, Volume 14(3), 202-227.
http://www.irrodl.org/index.php/irrodl/article/view/1455/2531
• Means B, Toyama Y, Murphy R, Bakia M and Jones K (revised 2010)
Evaluation of evidence-based practices in online learning. A meta-analysis
and review of online learning studies. US Department of Education.
http://www2.ed.gov/rschstat/eval/tech/evidence-basedpractices/finalreport.pdf
• Rolfe VE and Gray D (2011) Are Multimedia Resources Effective in Life
Science Education? A Meta-Analysis. Bioscience Education, Volume 18.
http://journals.heacademy.ac.uk/doi/abs/10.3108/beej.18.5