Se ha denunciado esta presentación.
Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.

Moving through MOOCs: Pedagogy, Learning and Patterns of Engagement

4.950 visualizaciones

Publicado el

Presentation for ECTEL 2015, Toledo, Spain (the detailed version).

The related, shorter, presentation is at http://www.slideshare.net/dougclow/moving-through-moocs

Publicado en: Educación

Moving through MOOCs: Pedagogy, Learning and Patterns of Engagement

  1. 1. Moving through MOOCs: Pedagogy, Learning and Patterns of Engagement Rebecca Ferguson, Doug Clow (OU) Russell Beale, Alison J Cooper (Birmingham) Neil Morris (Leeds) Siân Bayne, Amy Woodgate (Edinburgh)
  2. 2. Current context Students seek not merely access, but access to success “ John Daniel, 2012 % complete from: www.katyjordan.com/MOOCproject ”
  3. 3. Patterns of engagement: Coursera ● Sampling learners explored some course materials ● Auditing learners watched most videos, but completed assessments rarely, if at all ● Disengaging learners completed assessments at the start of the course and then reduced their engagement ● Completing learners completed most assessments MOOC designers can apply this simple and scalable categorization to target interventions and develop adaptive course features “ ” Coursera study Kizilcec, R., Piech, C., and Schneider, E., 2013. Deconstructing disengagement: analyzing learner subpopulations in massive open online courses. LAK13
  4. 4. 4 Replication Open University FutureLearn data Replication MOOC1 MOOC2 MOOC3 MOOC4 Subject area Physical sciences Life sciences Arts Business M 51% 39% 32% 35% F 48% 61% 67% 65% Participants 5,069 3,238 16,118 9,778 Fully participating 1,548 684 3,616 1,416 Participation rate 31% 21% 22% 14%
  5. 5. 5 Calculating an activity profile Replicating the method ● T = on track (3) undertook the assessment on time ● B = behind (2) submitted the assessment late ● A = auditing (1) engaged with content but not assessment ● O = out (0) did not participate Replication
  6. 6. 6 Replication Identifying dissimilarity between engagement patterns Assigned numerical value to each label ● On track = 3 ● Behind = 2 ● Auditing = 1 ● Out = 0 Calculated L1 norm for each engagement pattern Used that as the basis for one-dimensional k-means clustering Repeated clustering 100 times and selected solution with highest likelihood Focused on extracting four clusters Replication
  7. 7. 7 Replication Coursera and FutureLearn results were different ● Sampling learners explored some course materials ● Auditing learners watched most videos, but completed assessments rarely, if at all ● Disengaging learners completed assessments at the start of the course and then reduced their engagement ● Completing learners completed most assessments √ √ x x They also differed when we tried ●Different values for k ●A one-dimensional approach ●Running k means directly on engagement profiles Replication
  8. 8. 8 FutureLearn is different Conversation is a central feature Sharples, M., & Ferguson, R. (2014). Innovative Pedagogy at Massive Scale: Teaching and Learning in MOOCs. ECTEL 2014.
  9. 9. 9 Revising the numeric values OU FL study 1 only visited content (for example, video, audio, text) 2 commented but visited no new content 3 visited content and commented 4 did the assessment late and did nothing else that week 5 visited content and did the assessment late 6 did the assessment late, commented, but visited no new content 7 visited content, commented, late assessment 8 assessment early or on time, but nothing else that week 9 visited content and completed assessment early / on time 10 assessment early or on time, commented, but visited no new content 11 visited, posted, completed assessment early / on time
  10. 10. 10 Typical engagement profiles These profiles apply to an eight-week course ● Samplers visit only briefly [1, 0, 0, 0, 0, 0, 0, 0] – 1 means they visited content ● Strong Starters do first assessment [9, 1, 0, 0, 0, 0, 0, 0] – 9 means they visited content and did assessment on time ● Returners come back in Week 2 [9, 9, 0, 0, 0, 0, 0, 0] ● Mid-way Dropouts [9, 9, 9, 4, 1, 1, 0, 0] – 4 means they submitted assessment late ● Nearly There drop out near the end [11, 11, 9, 11, 9, 9, 0, 0] – 11 means full engagement, 8 means submission on time ● Late Completers finish [5, 5, 5, 5, 5, 9, 9, 9] – 5 means they viewed content and submitted late ● Keen Completers do almost everything [11, 11, 9, 9, 11, 11, 9, 9] OU FL study
  11. 11. 11 Samplers & Strong starters Samplers (1, 0, 0, 0, 0, 0, 0, 0) ● The largest group in all MOOCs ● Typically accounted for 37% – 39% of learners ● Visited the materials, but only briefly ● Active in a small number of weeks ● 25% – 40% joined after Week 1 ● Very few Samplers posted comments (6% – 15%) ● Almost no Samplers submitted any assessment Strong starters (9, 1, 0, 0, 0, 0, 0, 0) ●All Strong Starters submitted the first assignment ●Engagement dropped off sharply after that ●A little over a third of them posted comments ●Typically posted fewer than four comments OU FL study
  12. 12. 12 Returners & Mid-way dropouts Returners (9, 9, 0, 0, 0, 0, 0, 0) ● Completed the assessment in the first week ● Completed the assessment in the second week ● Then dropped out ● Over 97% completed those two assessments, although some submittted late ● No Returner explored all course steps ● Average amount of steps visited varied (23% – 47%) Mid-way dropouts (9, 9, 9, 4, 1, 1, 0, 0) ●A much smaller cluster (6% of learners on MOOC1, 7% on MOOC4) ●These learners completed three or four assessments ●They dropped out around halfway through the course ●Mid-way dropouts visited about half the steps on the course ●Just under half posted comments ●Posted just over six comments on average OU FL study
  13. 13. 13 Nearly There Nearly there (11, 11, 9, 11, 9, 9, 0, 0) ● Another small cluster (5% – 6% of learners) ● Consistently completed assessments ● Dropped out just before the end of the course ● Visited around 80% of the course ● Submitted assignments consistently (>90%) and typically on time until Week 5 ● Activity then declined steeply ● Few completed the final assessment ● None completed the final assessment on time OU FL study
  14. 14. 14 Late completers & keen completers Late completers (5, 5, 5, 5, 5, 9, 9, 9) ● Submitted the final assessment ● Submitted most other assessments ● However, either submitted late or missed some assessments ● Each week, more than 94% of this cluster submitted their assessments ● More than three-quarters submitted the final assessment on time (78% – 90%) ● Around 40% of them posted comments (76% did so on MOOC3) Keen completers (11, 11, 9, 9, 11, 11, 9, 9) ●Accounted for 7% – 23% of learners ●All Keen Completers submitted all assessments ●More than 80% of these were submitted on time ●Typically, Keen Completers visited more than 90% of course content ●Over two-thirds contributed comments (68% – 73%) ●Mean number of comments varied from 21 to 54 OU FL study
  15. 15. 15 Cross-university dataset FutureLearn data from four universities Cross-university Name Duration University Discipline Active learners LongMOOC1 8 OU Hard science 5,069 LongMOOC2 7 Edinburgh Hard science 10,136 TalkMOOC3 6 Edinburgh Politics 6,141 ShortMOOC4 3 Birmingham Medical science 6,839 ShortMOOC5 3 Leeds Medical science 4,756
  16. 16. 16 Values for k used in this study Different values were used for the three study phases Cross-university Name Phase 1 Phase 2 Phase 3 LongMOOC1 7 – – LongMOOC2 7 – – TalkMOOC3 – 7 3 ShortMOOC4 – 7 4 ShortMOOC5 – 7 5 Phase 1: Best-fit value for k aligned with OU study Phase 2: Testing k=7 where this was not the best fit Phase 3: Most suitable value for k in each set of data
  17. 17. 17 Phase two: k ≠ 7 Why k≠7 in Talk MOOC3 Phase two The absence of assessment in TalkMOOC3 limited its coding profile 1 only visited content (for example, video, audio, text) 2 commented but visited no new content 3 visited content and commented 4 did the assessment late and did nothing else that week 5 visited content and did the assessment late 6 did the assessment late, commented, but visited no new content 7 visited content, commented, late assessment 8 assessment early or on time, but nothing else that week 9 visited content and completed assessment early / on time 10 assessment early or on time, commented, but visited no new content 11 visited, posted, completed assessment early / on time
  18. 18. 18 Phase two: k ≠ 7 Why k≠7 in ShortMOOC4 and ShortMOOC5 Phase two Three clusters are indistinguishable in a three-week MOOC Returners who come back in Week 2 Mid-way Dropouts who drop out mid-course Nearly There who drop out near the end With only three opportunities for late submission, there are no Late Completers (who typically submit assessment late five times) The three-week course design means other clusters emerge, such as: Surgers concentrate their effort after the first week of a three-week course Improvers fall behind in Week 1, begin to catch up in Week 2 and complete on time
  19. 19. 19 Phase three: suitable values for k TalkMOOC3: k=3 Phase three Quiet (1, 0, 0, 0, 0, 0) ● The largest cluster ● Visit a quarter of course steps ● Do not comment in first week ● Only 7% comment at all ● Only 9% engage with second half of course Contributors (3, 1, 1, 0, 0, 0) ● 19% of cohort ● Visit 38% of course steps ● Every cluster member posts in first week of course ● Half do not comment again Consistent engagers (3, 3, 3, 3, 1, 1) ● 11% of cohort ● Visit 82% of course steps ● Engage throughout course ● Every cluster member posts a comment ● 95% contribute more than three comments ● 7% contribute more than 100 comments
  20. 20. 20 Phase three: suitable values for k ShortMOOC4: k=4 Phase three Very weak starters (2, 1, 0) ● The largest cluster ● Visit 20% of steps ● 20% do not engage in first week Strong starters (truncated) (10, 1, 0) ● 17% of cohort ● Submit week 1 assessment ● Do not submit another assessment ● Almost half post comment Returners (truncated) (3, 3, 3, 3, 1, 1) ● Most submit week 1 assessment ● All submit week 2 assessment ● Half submit at least one comment Keen completers (truncated) (9, 9, 9) ● Visit more than 90% of steps ● Submit work on time ● Engage throughout
  21. 21. 21 Phase three: suitable values for k ShortMOOC5: k=5 Phase three Samplers (truncated) (1, 0, 0) ● Visit few steps ● Includes many latecomers (>25%) ● Very few submit assessment Strong starters (truncated) (9, 1, 0) ● Submit week 1 assessment ● Do not submit another assessment Returners (truncated) (8, 8, 2) ● Most submit week 1 assessment ● All submit week 2 assessment Keen completers (truncated) (9, 9, 9) ● Visit more than 90% of steps ● Submit work on time ● Engage throughout Improvers (5, 6, 9) ● Activity increases each week ● Final assessment submitte on time
  22. 22. 22 Learning design and pedagogy ● The Coursera Study suggested that MOOC designers would be able to apply the four engagement patterns they had identified ‘to target interventions and develop adaptive course features’ ● These subsequent studies show that this is not necessarily the case –engagement patterns are not consistent across MOOCs ● Changes to the basic pedagogic elements of a course are associated with shifts in patterns of engagement. ● Shifts in pedagogic approach can change the elements of a course that can be regarded as key ● Changes to some elements of learning design can change learners’ patterns of engagement with a MOOC
  23. 23. 23 Shorter courses ● Reducing course length does not necessarily increase engagement ● Many learners do not approach a three-week course in the same way as an eight-week course ● Many focus their attention on later weeks and may miss out the content and activities in the first week ● A three-week course offers limited opportunities to get ahead of, or behind, the cohort ● It is possible to dip in at different points without losing the sense of being a cohort member
  24. 24. 24 Improving learning and learning environments Closing the loop ● Previews of course material would allow Samplers to make a more informed decision about whether to join the course ● Sign-up pages could draw attention to the problems experienced by those who are out of step with the cohort ● Discussion steps for latecomers could support those who fall behind at the start ● Prompts might encourage flagging learners to return and register for a subsequent presentation ● Bridges between course weeks could indicate links and point learners forward
  25. 25. 25 View these slides at www.slideshare.net/R3beccaF Rebecca Ferguson @R3beccaF Doug Clow @dougclow

×