MOOC Research Conference 2013, Arlington TXPanel on Certifying MOOCs http://moocresearch.com

1. MOOC Research Conference 2013, Arlington TX
Panel on Certifying MOOCs
moocresearch.com
Certifying the hard stuff
(and where learning analytics might help)
Simon Buckingham Shum
Knowledge Media Institute, The Open University, UK
simon.buckinghamshum.net
learningemergence.net
@sbskmi
1

2. The final educational C…
may be about to go Open
content +
curriculum
certification
community

3. Assessment regimes drive certification
“What you assess is what you
get; if you don’t test it, you won’t
get it.
To improve student performance
we must recognize that essential
intellectual abilities are falling
through the cracks of
conventional testing.”
Lauren Resnick
3

4. Certification
Assumes we know what quality
looks like
That we know how to assess it,
reliably
That certification is a brand
(but based on scarcity)
4

5. The problem of Massive
How do we avoid just
counting what’s easiest
to count?
5

6. Big Data and Analytics to the rescue!
Finally, we can let the data speak
for itself
Learning analytics and eassessment make it possible to
objectively evaluate performance
at scale
6

7. Data and Analytics are infused with human
values
Finally, we can let the data speak
for itself
Learning analytics and eassessment make it possible to
objectively evaluate performance
at scale
7

8. Learning technology is not neutral
Any technology embodies
epistemological, pedagogical
and assessment assumptions
“Our learning analytics are our
pedagogy”
epistemology
assessment
pedagogy
Knight S, Buckingham Shum S and Littleton K. (2013) Epistemology, Pedagogy, Assessment and Learning Analytics. Proc. 3rd
International Conference on Learning Analytics & Knowledge. Leuven, BE: ACM, 75-84 Open Access Eprint: http://oro.open.ac.uk/36635
8

9. The challenge of authenticity:
of learner, and of learning
Are you really doing the work?
Did you really write this?
Plagiarism is not a new problem
>>> “authentic assessment”
9

10. Authentic Assessment strategies
resist plagiarism, promote deep learning and build real world competencies
 Oral Interviews Teacher asks student questions about personal background, activities,
readings, and other interests.
cf. Jim Groom’s
keynote
 Story or Text Retelling Student retells main ideas or selected details of text experienced
through listening or reading.
 Writing Samples Student generates narrative, expository, persuasive, or reference paper with
clear reference to their own experience and views.
 Projects/Exhibitions Student works with other students as a team to create a project that
often involves multimedia production, oral and written presentations, and a display.
 Experiments/Demonstrations Student documents a series of experiments,
illustrates a procedure, performs the necessary steps to complete a task, and documents the results of the
actions.
 Constructed-Response Items Student responds in writing to open-ended
questions.
 Teacher Observations Teacher observes and documents the students attention and
interaction in class, response to instructional materials, and cooperative work with other students.
 Portfolios A focused collection of student work to show progress over time.
10

11. Learning dispositions
“In the growth mindset, people believe
that their talents and abilities can be
developed through passion,
education, and persistence …
It’s about a commitment to …
taking informed risks …
surrounding yourself with people
who will challenge you to grow”
Carol Dweck
See the MRI mindset
intervention projects
Interview with Carol Dweck:
http://interviewscoertvisser.blogspot.co.uk/2007/11/interview-with-carol-dweck_4897.html
11

12. Learning dispositions
“We’re looking at the profiles of
what it means to be effective in the
21st century. […] Resilience will
be the defining concept. When
challenged and bent, you learn and
bounce back stronger.”
“Dispositions are now at least as
important as Knowledge and
Skills. …They cannot be taught.
They can only be cultivated.”
John Seely Brown
US Dept. of Educ. http://reimaginingeducation.org conference (May 28, 2013)
Dispositions clip: http://www.c-spanvideo.org/clip/4457327
Whole talk: http://www.c-spanvideo.org/program/SecD
12

13. Learning dispositions
“It’s more than knowledge and skills.
For the innovation economy,
dispositions come into play:
readiness to collaborate;
attention to multiple
perspectives; initiative;
persistence; curiosity.
The purpose of learning in the 21st century
is not to recite inert knowledge but to
transform it.
It’s time to change the subject.”
Larry Rosenstock
LearningREimagined project: http://learning-reimagined.com
Larry Rosenstock:
http://audioboo.fm/boos/1669375-50-seconds-of-larry-rosenstock-ceo-of-hightechhigh-on-how-he-would-re-imagine-learning
13

14. How Learning Analytics may help
14

15. Dispositional
Learning
Analytics
Workshop
http://learningemergence.net/events/lasi-dla-wkshp
15

16. 1st International Workshop on
Discourse-Centric Learning Analytics
solaresearch.org/events/lak/lak13/dcla13
analytics that look
beneath the surface,
and quantify linguistic
proxies for ‘deeper
learning’
http://www.glennsasscer.com/wordpress/wp-content/uploads/2011/10/iceberg.jpg
Beyond number / size / frequency
of posts; ‘hottest thread’

17. Social Network Analysis to gain insight into
peer-peer and peer-mentor dynamics
(SNAPP tool)
http://www.slideshare.net/aneeshabakharia/snapp-20minute-presentation
Bakharia A and Dawson S. (2011) SNAPP: a bird's-eye view of temporal participant interaction. Proceedings of
the 1st International Conference on Learning Analytics and Knowledge. Banff, Alberta, Canada: ACM, 168-173.
17

18. Epistemic Gaming
Shaffer, D. W., Hatfield, D., Svarovsky, G. N., Nash, P., Nulty, A., Bagley, E., Franke, K., Rupp, A. A., Mislevy, R.
(2009). Epistemic Network Analysis: A prototype for 21st Century assessment of learning. The International
Journal of Learning and Media. 1(2), 33-53.
18

19. Badging framework
Open Learning Design Studio MOOC — Badging Framework: http://www.olds.ac.uk/blog/moocbadgingandthelearningarc
19