Se ha denunciado esta presentación.
Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.

PASTEL: Evidence-based Learning Engineering Method to Create Intelligent Online Textbook at Scale

57 visualizaciones

Publicado el

Presented at the First Workshop on Intelligent Textbooks (Chicago, IL, US; June 25, 2019)

Publicado en: Educación
  • Inicia sesión para ver los comentarios

  • Sé el primero en recomendar esto

PASTEL: Evidence-based Learning Engineering Method to Create Intelligent Online Textbook at Scale

  1. 1. COMPUTER SCIENCE PASTEL: Evidence-based Learning Engineering Method to create Intelligent Online Textbook at Scale Noboru Matsuda & Machi Shimmei Center for Educational Informatics Department of Computer Science North Carolina State University
  2. 2. COMPUTER SCIENCE Current MOOC Challenge • Lack of individualization – Ineffective learning (no learning!) – Disengagement / drop-out • Lack of systematic content creation & validation – Where should we start from? – How can we iteratively make it better? Matsuda & Shimmei CyberBook @ Intelligent Textbook WS 2019 2
  3. 3. COMPUTER SCIENCE Current ITS Challenge • Scalability / Generality – Too expensive to build – Mostly good for procedural skill acquisition • What about conceptual learning? • Robustness of Learning – Luck of learning to solve with justifications Matsuda & Shimmei CyberBook @ Intelligent Textbook WS 2019 3
  4. 4. COMPUTER SCIENCE Summary of Challenges • To overcome the issues of MOOC and ITS, there is a critical need to innovate a technology that – provides adaptive instruction while promoting synergetic learning • An evidence-based curriculum development is essential – to build a large scale online course Matsuda & Shimmei CyberBook @ Intelligent Textbook WS 2019 4
  5. 5. COMPUTER SCIENCE Our Solution • Evidence-based learning engineering methods – PASTEL (Pragmatic methods to develop Adaptive and Scalable Technologies for next generation E-Learning) • Adaptive Online Courseware – CyberBook = MOOC + Cognitive Tutors + Adaptive Control Matsuda & Shimmei CyberBook @ Intelligent Textbook WS 2019 5
  6. 6. COMPUTER SCIENCE CyberBook • Online courseware that provide macro- adaptive scaffolding – Problem balancing – Mastery practice (cognitive tutoring) – Proactive detection of unproductive failure – Scaffolding on failure • Incorrect quiz attempt and/or wheel-spinning Matsuda & Shimmei CyberBook @ Intelligent Textbook WS 2019 6
  7. 7. COMPUTER SCIENCE Matsuda & Shimmei CyberBook @ Intelligent Textbook WS 2019 7 CyberBook: Example
  8. 8. COMPUTER SCIENCE CyberBook: Adaptive Scaffolding Matsuda & Shimmei CyberBook @ Intelligent Textbook WS 2019 8
  9. 9. COMPUTER SCIENCE Skill Name Association Matsuda & Shimmei CyberBook @ Intelligent Textbook WS 2019 9
  10. 10. COMPUTER SCIENCE CyberBook: Cognitive Tutor Integration Matsuda & Shimmei CyberBook @ Intelligent Textbook WS 2019 10
  11. 11. COMPUTER SCIENCE Technological Challenges • Automatic validation of courseware content • Rapid creation of high-quality skill-models • Affordable creation of cognitive tutors • Automatic creation of formative assessments • Reliable prediction of unproductive failure Matsuda & Shimmei CyberBook @ Intelligent Textbook WS 2019 11
  12. 12. COMPUTER SCIENCE Technology Innovations • PASTEL: Evidence-based, iterative learning engineering methods Matsuda & Shimmei CyberBook @ Intelligent Textbook WS 2019 12 SMART WATSON QUADL RADARS RAFINE Skill model discovery Cog tutor authoring Assessment generation Unproductive failure prediction Content validation
  13. 13. COMPUTER SCIENCE Evaluation Study • Does macro-adaptive scaffolding promote learning? • Middle School Science—Newton’s Law – 11 units / 17 videos / 83 quizzes / 0 cog tutor • High School Geometry—Coordinate Geometry – 23 units / 0 videos / 179 quizzes / 14 cog tutors • In-service teacher as a curriculum developer 13Matsuda & Shimmei CyberBook @ Intelligent Textbook WS 2019
  14. 14. COMPUTER SCIENCE Evaluation Study (Cont.) • Participants – 175 middle school science students – 168 high school geometry students • Pre-test, 3 intervention days, post-test • Stratified RCT on pre-test 14Matsuda & Shimmei CyberBook @ Intelligent Textbook WS 2019
  15. 15. COMPUTER SCIENCE Results: Test Scores 15Matsuda & Shimmei CyberBook @ Intelligent Textbook WS 2019
  16. 16. COMPUTER SCIENCE Results – Num. Assessments • Adaptive >> Non-adaptive for Science, but not for Math • No correlation between # assessments and post-test when pre-test was controlled Matsuda & Shimmei CyberBook @ Intelligent Textbook WS 2019 16
  17. 17. COMPUTER SCIENCE Results – Dynamic Link • Surprisingly low use of dynamic link – 0.4±1.8 in average • Students figured out that corresponding resources were almost always on the same page Matsuda & Shimmei CyberBook @ Intelligent Textbook WS 2019 17
  18. 18. COMPUTER SCIENCE Results – Video Use • Adaptive == Non-adaptive • Doer/Non-doer effect was not observed Matsuda & Shimmei CyberBook @ Intelligent Textbook WS 2019 18 (Koedinger et al., 2015)
  19. 19. COMPUTER SCIENCE Results – Hint Use • Hint on Failure Ratio • Adaptive >> Non-adaptive for Science, but not for Math. In particular low-prior students Matsuda & Shimmei CyberBook @ Intelligent Textbook WS 2019 19
  20. 20. COMPUTER SCIENCE Conclusions • CyberBook with the macro-adaptive scaffolding facilitated learning – Controlling the amount of formative assessment and Dynamic link – Only for the middle school Science, but not for high school Math • Hint use on failure among low-prior students was associated with learning Matsuda & Shimmei CyberBook @ Intelligent Textbook WS 2019 20
  21. 21. COMPUTER SCIENCE Future Works • Redo the study with the balanced online courseware – Add cognitive tutors to Science – Add videos to Math • Redesign Dynamic Link Matsuda & Shimmei CyberBook @ Intelligent Textbook WS 2019 21

×