Se ha denunciado esta presentación.
Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.

Using data to provide personalised feedback at scale

519 visualizaciones

Publicado el

The current state of higher education has increasing pressure over academics to offer high quality experience at scale. But what could be the actions that can be deployed to achieve this increase? What would be a good guiding principle to decide these actions? In this talk we explore first the possibility of using feedback and a coach mentality to provide student support, and then how data can help us scale that technique. There are examples of potential scenarios to deploy this at the level of a course, program or overall student experience.

Publicado en: Educación
  • There is a useful site for you that will help you to write a perfect and valuable essay and so on. Check out, please ⇒ www.HelpWriting.net ⇐
       Responder 
    ¿Estás seguro?    No
    Tu mensaje aparecerá aquí
  • See how I make over $7,293 a month from home doing REAL online jobs! ▲▲▲ http://ishbv.com/ezpayjobs/pdf
       Responder 
    ¿Estás seguro?    No
    Tu mensaje aparecerá aquí
  • Sé el primero en recomendar esto

Using data to provide personalised feedback at scale

  1. 1. Using data to provide personalised feedback at scale Prof Abelardo Pardo
 Division of IT, Engineering and the Environment slideshare.net/abelardo_pardo Twitter: @abelardopardo NickMacMillan
  2. 2. Abelardo Pardo Using data to provide personalised feedback at scale 2 JonnyCaspari Under
 Pressure Can
 Data Help? Which
 Actions? Examples
  3. 3. Abelardo Pardo Using data to provide personalised feedback at scale 3 JonnyCaspari Under
 Pressure
  4. 4. Abelardo Pardo Using data to provide personalised feedback at scale 4 Simple information transfer is not working Mazur, E. (2009). Farewell, lecture. Science, 323(5910), 50-51. Krugazorflickr.com
  5. 5. Abelardo Pardo Using data to provide personalised feedback at scale 5 SamAbrahamflickr.com Blended Learning Frontier between physical and virtual spaces is blurring
  6. 6. Abelardo Pardo Using data to provide personalised feedback at scale 6 JansonHewsflickr.com
  7. 7. Abelardo Pardo Using data to provide personalised feedback at scale 7 Use a theoretical model
  8. 8. Abelardo Pardo Using data to provide personalised feedback at scale 8 “… robust correlations between student involvement in a subset of ‘educationally purposive activities’, and positive outcomes of student success and development, including satisfaction, persistence, academic achievement and social engagement” Trowler, V. (2010). Student engagement literature review. York, UK: The Higher Education Academy. shuaGandersonflickr.com
  9. 9. Abelardo Pardo Using data to provide personalised feedback at scale 9 JonnyCaspari Under
 Pressure Which actions?
  10. 10. Abelardo Pardo Using data to provide personalised feedback at scale 10 “… teaching in higher education will necessarily shift the balance of its efforts towards a greater investment in design as a way of coping with otherwise intolerable pressures on staff and resources.” Goodyear, P. (2015). Teaching as Design. HERDSA Review of Higher Education, 2, 27-50.
  11. 11. Abelardo Pardo Using data to provide personalised feedback at scale 11 “There is no such thing as a neutral design” JeremyBrooksflickr.com Thaler, R. H., & Sunstein, C. R. (2008). Nudge. Great Britain: Yale University Press.
  12. 12. Abelardo Pardo Using data to provide personalised feedback at scale 12 Thaler, R. H., & Sunstein, C. R. (2008). Nudge. Great Britain: Yale University Press. “People make good choices in contexts in which they have experience, good information, and prompt feedback” DerekBruffflickr.com
  13. 13. Abelardo Pardo Using data to provide personalised feedback at scale 13 Bjork, R. A., Dunlosky, J., & Kornell, N. (2013). Self-regulated learning: beliefs, techniques, and illusions. Annu Rev Psychol, 64, 417-444. doi:10.1146/annurev-psych-113011-143823 • Understand human memory and learning • Know useful study techniques MatthewMaillet • Know how to monitor • Understand existing biases
  14. 14. Abelardo Pardo Using data to provide personalised feedback at scale 14 Hattie, J. A. (1999). Influences on student learning. Inaugural professorial address, University of Auckland, New Zealand If You Could Choose One… • More than 500 meta-analyses of student achievements • 100 factors with potential influence • Feedback in top five • (74 meta-analyses) Most effective form: video, audio, computer-assisted instructional feedback, and/or related goals
  15. 15. Abelardo Pardo Using data to provide personalised feedback at scale 15 Pardo, A. (2017). A feedback model for data-rich learning experiences. Assessment & Evaluation in Higher Education, 1-11. doi: 10.1080/02602938.2017.1356905 FarukAteşflickr.com “Feedback is a process to positively influence how students engage with their work in a learning experience so that they can improve its overall quality with respect to an appropriate reference and increase their self-evaluative capacity”
  16. 16. Abelardo Pardo Using data to provide personalised feedback at scale 16 JacksonLavarnway
  17. 17. Abelardo Pardo Using data to provide personalised feedback at scale 17 JonnyCaspari Under
 Pressure Can
 Data Help? Can Data Help?
  18. 18. Abelardo Pardo Using data to provide personalised feedback at scale 18 Large number of events per user 2016-08-09 00:05:43.124199+00,989,129.78.56.144,"{""outcome"": ""incorrect"", ""assessment"": ""summative"", ""score"": 44.4444444444444, ""exercise"": ""/data2u/static/exco_exc/DRM/ problem_13_c.html"", ""sequence"": ""DRM-exco-C""}",abelardopardo,196619,exco-answer 2016-08-09 00:05:44.140307+00,989,129.78.56.144,"{""score"": 44.4444444444444, ""exercise"": ""/data2u/ static/exco_exc/DRM/problem_03_c.html"", ""sequence"": ""DRM-exco-C""}",abelardopardo,196620,exco-view 2016-08-09 00:05:47.76122+00,968,49.182.128.186,"{""question_id"":""COD-numberofwires-eqt_1"",""answer"": 1}",abelardopardo,196621,embedded-question 2016-08-09 00:05:48.861036+00,806,129.78.56.151,"{""question_id"":""COD-encodeintegers-section- eqt_4"",""answer"":0}",abelardopardo,196622,embedded-question 2016-08-09 00:05:48.959791+00,989,129.78.56.144,"{""outcome"": ""correct"", ""assessment"": ""summative"", ""score"": 50.0, ""exercise"": ""/data2u/static/exco_exc/DRM/problem_03_c.html"", ""sequence"": ""DRM-exco-C""}",abelardopardo,196623,exco-answer 2016-08-09 00:05:49.539162+00,989,129.78.56.144,"{""score"": 50.0, ""exercise"": ""/data2u/static/ exco_exc/DRM/problem_14_c.html"", ""sequence"": ""DRM-exco-C""}",abelardopardo,196624,exco-view 2016-08-09 00:05:49.571138+00,1311,129.78.56.159,"{""url"":""https://flip.ee.usyd.edu.au/elec1601/ Material/COD/index.html""}",abelardopardo,196625,resource-view 2016-08-09 00:05:50.050035+00,1069,129.78.56.199,"{""question_id"":""COD-integerencoding-videoeqt- eqt_3"",""answer"":0}",abelardopardo,196626,embedded-question 2016-08-09 00:05:51.709295+00,806,129.78.56.151,"{""question_id"":""COD-encodeintegers-section- eqt_4"",""answer"":0}",abelardopardo,196627,embedded-question 2016-08-09 00:05:51.962474+00,1069,129.78.56.199,"{""question_id"":""COD-integerencoding-videoeqt- eqt_3"",""answer"":0}",abelardopardo,196628,embedded-question 2016-08-09 00:05:52.0819+00,806,129.78.56.151,"{""question_id"":""COD-encodeintegers-section- eqt_4"",""answer"":""-1""}",abelardopardo,196629,embedded-question 2016-08-09 00:05:53.025356+00,1069,129.78.56.199,"{""question_id"":""COD-integerencoding-videoeqt- eqt_3"",""answer"":1}",abelardopardo,196630,embedded-question 2016-08-09 00:05:54.756229+00,1311,129.78.56.159,"{""url"":""https://flip.ee.usyd.edu.au/elec1601/ Material/COD/COD_notes.html#range-accuracy-and-precision-of-the-floating-point- representation""}",abelardopardo,196631,resource-view 2016-08-09 00:05:54.856333+00,989,129.78.56.144,"{""outcome"": ""incorrect"", ""assessment"": ""summative"", ""score"": 50.0, ""exercise"": “"/data2u/static/exco_exc/DRM/problem_03_c.html"", ""sequence"": ""DRM-exco-C""}",abelardopardo,196623,exco-answer
  19. 19. Abelardo Pardo Using data to provide personalised feedback at scale 19 4187, 2016-07-14 00:56:46.341946+00, {“time”":0,"id":"xEJtdMQMcrs","event":"PLAY"}, https://mycourse.com/Material/HLP/HLP_notes.html, embedded-video 4187, 2016-07-14 00:56:46.341946+00, VIDEO, xEJtdMQMcrs, PLAY, COD, W2 Combine logs with design Learner played a video about topic COD from Week 2
  20. 20. Abelardo Pardo Using data to provide personalised feedback at scale 20 4187, 2016-07-14 00:56:46.341946+00, {“time”":0,"id":"xEJtdMQMcrs","event":"PLAY"}, https://mycourse.com/Material/HLP/HLP_notes.html, embedded-video 4187, 2016-07-14 00:56:46.341946+00, VIDEO, xEJtdMQMcrs, PLAY, COD, W2 Combine logs with design Learner played video about COD from W2 during W6 For the first time during
 the course — The student is catching up For the sixth time during
 the course — The student is reviewing
  21. 21. Abelardo Pardo Using data to provide personalised feedback at scale 21 Combine events to obtain tactics Tactic 1 • Review videos and immediately answer questions (or annotate). • Review suggested additional material. • Engage with some practice exercises. • Engage in discussion in forum Tactic 2 • Try to solve the summative assessment questions. Most attempts are incorrect • Review course structure (assessment, requirements) • Consult a small subset of available resources
  22. 22. Abelardo Pardo Using data to provide personalised feedback at scale 22 Identify learning strategies Strategy 1 • Numerous short sessions with comprehensive levels of engagement. • Sessions spread across time. • Frequent self-reflection elements. • Numerous engagements with discussion forum. Strategy 2 • Infrequent long sessions only focused in one type of resource. • Sessions always close to submission deadlines. • Poor variety of tactics.
  23. 23. Abelardo Pardo Using data to provide personalised feedback at scale 23 JonnyCaspari Under
 Pressure Can
 Data Help? Which
 Actions? Examples
  24. 24. Abelardo Pardo Using data to provide personalised feedback at scale 24 MikeWilson Course Degree or program Student experience
  25. 25. Abelardo Pardo Using data to provide personalised feedback at scale 25 Pardo, A., & Mirriahi, N. (2017). Design, Deployment and Evaluation of a Flipped Learning First-Year Engineering Course. In C. Reidsema, L. Kavanagh, R. Hadgraft, & N. Smith (Eds.), Flipping the classroom. Practice and Practices in Higher Education (pp. 177-191). Singapore: Springer. doi:10.1007/978-981-10-3413-8_11 Blended Learning Experience • 13 week experience • 1 lecture (2hr), 1 tutorial (2hr), 1 laboratory session (3hr) per week • Videos with immediate MCQ for recall (formative assessment) • Resources with embedded formative assessment items • Preparation exercises required (summative) before lectures, and then available for practice (formative) • Self-reflection and awareness elements available
  26. 26. Abelardo Pardo Using data to provide personalised feedback at scale 26 Fincham, E., Gašević , D., Jovanović, J., & Pardo, A. (In Press). From Study Tactics to Learning Strategies: An Analytical Method for Extracting Interpretable Representations. IEEE Transactions on Learning Technologies. Detecting Learning Tactics in Sessions 1. 60% video, 40% content 2. Almost exclusively summative assessment (90%) 3. Almost exclusively formative assessment or practice (90%) 4. Content browsing (90%) 5. Meta-cognitive actions (90%). Browsing the self-reflection elements, checking pages about how the course is organised, suggested strategies, etc. 6. Content access and formative assessment (90%) 7. Access to content and meta-cognitive elements. 8. Formative assessment, summative assessment and content access.
  27. 27. Abelardo Pardo Using data to provide personalised feedback at scale 27 Fincham, E., Gašević , D., Jovanović, J., & Pardo, A. (In Press). From Study Tactics to Learning Strategies: An Analytical Method for Extracting Interpretable Representations. IEEE Transactions on Learning Technologies. Identifying Learning Strategies
  28. 28. Abelardo Pardo Using data to provide personalised feedback at scale 28 Jovanović, J., Gašević, D., Pardo, A., Mirriahi, N., & Dawson, S. (In Press). An analytics-based framework to support teaching and learning in a flipped classroom. In J. M. Lodge, J. Cooney Horvath, & L. Corrin (Eds.), Learning analytics in the classroom: translating learning analytics research for teachers. United Kingdom: Taylor & Francis. Support teaching and learning in a flipped classroom
  29. 29. Abelardo Pardo Using data to provide personalised feedback at scale 29 Instructor — per task Technology — per student
  30. 30. Abelardo Pardo Using data to provide personalised feedback at scale 30
  31. 31. Abelardo Pardo Using data to provide personalised feedback at scale 31 ontasklearning.org Pardo et al. OnTask: Delivering Data-Informed, Personalised Learning Support Actions. Journal of Learning Analytics. In Press
  32. 32. Abelardo Pardo Using data to provide personalised feedback at scale 32
  33. 33. Abelardo Pardo Using data to provide personalised feedback at scale 33
  34. 34. Abelardo Pardo Using data to provide personalised feedback at scale 34 • Support instructors to manage personalised feedback processes • Simple rule-base knowledge encoding • Provide appropriate view of data sources • Scale to large and highly diverse cohorts • Open-source project • Pilots running in 2018 • Contact us if interested ontasklearning.org
  35. 35. Abelardo Pardo Using data to provide personalised feedback at scale 35 Focus groups • “It helps me to validate where I am; do I need to freak out right now?” • “…gives you a nudge — Stop procrastinating and playing games!” • A reminder to study “across the board” (flow-on effect) • “The wording makes you want to do it. Like an encouragement.”
  36. 36. Abelardo Pardo Using data to provide personalised feedback at scale 36 Program
 objective 1 Program
 objective 2 Program
 objective 3 Program
 objective4 1st year course 1 Basic Basic 1st year course 2 Basic 2nd year course 1 Proficient Proficient Basic 2nd year course 2 Proficient Proficient 3rd year course 1 Expert Expert Expert Expert Support at the Degree/Program Level
  37. 37. Abelardo Pardo Using data to provide personalised feedback at scale 37 Program
 objective 1 Program
 objective 2 Program
 objective 3 Program
 objective4 1st year course 1 Basic Basic 1st year course 2 Basic 2nd year course 1 Proficient Proficient Basic 2nd year course 2 Proficient Proficient 3rd year course 1 Expert Expert Expert Expert Support at the Degree Level • Indicators of student engagement with respect to program objectives • Tactics adopted in courses assessing each objective • Provide personalized feedback to support achievement
  38. 38. Abelardo Pardo Using data to provide personalised feedback at scale 38 Program
 objective 1 Program
 objective 2 Program
 objective 3 Program
 objective4 1st year course 1 Basic Basic 1st year course 2 Basic 2nd year course 1 Proficient Proficient Basic 2nd year course 2 Proficient Proficient 3rd year course 1 Expert Expert Expert Expert Identifying Degree Trajectories • Provide personalized feedback to support trajectory
  39. 39. Abelardo Pardo Using data to provide personalised feedback at scale 39 James, N., Kovanovic, V., Marshall, R., Joksimovic, S., & Pardo, A. (2018). Examining the value of learning analytics for supporting work-integrated learning. Paper presented at the National Conference on Work Integrated Learning, Brisbane, Queensland, Australia. SalzburgGlobalSeminar Preparing graduates for work • Identify indicators of progress in: social integration, soft skills, community awareness. • Connect with institutional resources • Provide suggestions on how to engage or increase effectiveness
  40. 40. Abelardo Pardo Using data to provide personalised feedback at scale 40 RyoYoshitake • Institutions are under pressure to provide quality at scale. • Knowledge must be connected with actions at course, program overall student experience levels. • Data captured through technology mediation can provide valuable insights for decision making. • Deliver personalise actions based on knowledge extracted from data. Conclusions
  41. 41. Using data to provide personalised feedback at scale Prof Abelardo Pardo
 Division of IT, Engineering and the Environment slideshare.net/abelardo_pardo Twitter: @abelardopardo NickMacMillan

×