Developing 21st-Century Skills with Multimodal Learning Analytics
•Descargar como PPTX, PDF•
1 recomendación•265 vistas
Collaboration, communication, creativity, critical thinking and problem-solving are among the skills that are needed to study and work in this 21st century. As important as they are, evaluating, assessing and teaching them in a practical, scalable and efficient way is still a challenge not fully met by current pedagogical-technological practices. Multimodal Learning Analytics (MmLA), the processing and analysis of multiple sources of data to better understand and improve learning processes, has been posed as a possible solution to augment the natural capabilities of both instructors and students to provide and receive feedback to support the development of those skills. During this session, we will have a hands-on demo of two systems to automatically generate feedback for communication and collaboration skills; then, we will explore the affordances that low-cost sensors and current advances in artificial intelligence provide to automatically record and analyze face-to-face, complex learning processes as those involved for the development of 21st-Century Skills. Finally, we will discuss and ideate practical MmLA tools that could be built to augment your current teaching and learning practices. Presentation at NYU - November 2019.
Recomendados
Recomendados
Más contenido relacionado
Similar a Developing 21st-Century Skills with Multimodal Learning Analytics
Similar a Developing 21st-Century Skills with Multimodal Learning Analytics (20)
Más de Xavier Ochoa
Más de Xavier Ochoa (20)
Último
Último (20)
Developing 21st-Century Skills with Multimodal Learning Analytics
- 1. Developing 21st-Century Skills with Multimodal Learning Analytics Xavier Ochoa
- 3. Demo- Time Douglas Engelbart, during the “Mother of all Demos”, 1968
- 4. Two options – 10 minutes • OPAF – Oral Communication • CLiF – Collaboration
- 5. Two options – 10 minutes • Perform a flawed presentation • Do not look at the audience • Have a close posture • Speak too softly • Introduce filled pauses • Solve “Lost at Sea” task • Be aware of the collaboration • Try to remember as much as possible
- 7. Discussion -Time Vannavar Bush, description of the Memex, 1945
- 9. Increasing the capability of humans to approach a complex problem situation, to gain comprehension to suit their particular needs, and to derive solutions to problems. Douglas Engelbart Augmenting Human Intellect 1962
- 10. Learning analytics is the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs.
- 12. Fundamental
- 13. Images in public domain. Obtained from Wikimedia Foundation.
- 16. Difficult to provide personalized feedback
- 17. Multimodal Learning Analytics MmLA Capturing learning traces wherever they happen
- 18. Humans communicate (and leave traces) in several modalities What you say is as important as how you say it
- 20. We need to analyze the traces with variable degrees of sophistication And we have to do it automatically as humans are not scalable
- 22. We need to provide feedback in the real world Often in a multimodal way too
- 24. 4-Easy-Steps for Provide Feedback for Fundamental Skills with MmLA
- 25. Step 1: Capture relevant data
- 26. Step 1: Capture relevant data • Recommended Sensors: • Cameras • Normal • Fisheye (horizontal 360 degrees) • Array (360 degrees) • Micropohones • Normal • Array (Directional information) • Digital Pens
- 27. Step 1: Capture relevant data • Sensors for labs: • Eye-trackers • EDA – pulse wristbands • Brain activity
- 28. Step 1: Example Collaboration around a table
- 29. Setting: Collaboration around a table
- 30. Setting requirements • Non-intrusive • Capture the same information that a human observer • Mainly faces and speech of participants • No cables • Synchronized • Easy-to-operate • Record at least 2 hours
- 31. MmLA Sensors – Collaboration around a table ≈
- 32. MmLA Sensors Fisheye Camera Microphone Array (Respeaker) Computer (Raspberry Pi) Battery Pack
- 33. MmLA Sensors $ 30 $ 40 $ 40 $ 25 ≈ $ 150
- 34. Step 2: Extract low-level modalities
- 36. Step 2: Extract low-level modalities • Video: • Body posture • Head posture • Gaze • Facial Expression • Movement • Object Recognition / Tracking • Object Interaction • Gestures • Actions
- 37. Step 2: Extract low-level modalities • Audio: • Direction of arrival • Speech to Text • Prosodic features • Speaker diarization • Text: • Word frequency • Sentiment analysis • Questions asked • …
- 38. Step 3: Extract low-level modalities • Digital Pen: • Writing mechanics • Text • Sketch recognition
- 39. Step 2: Example Collaboration modalities
- 40. MmLA Low-Level Feature Extraction - Video • Face/Body counting (How many students around the table) Library: OpenPose / OpenFace
- 41. MmLA Low-Level Feature Extraction - Video • Body posture (2D position of main body joints/features) Library: OpenPose / OpenFace
- 42. MmLA Low-Level Feature Extraction - Video • Face posture / Rough Gaze Direction Library: OpenFace / Open Pose
- 43. MmLA Low-Level Feature Extraction - Video
- 44. MmLA Low-Level Feature Extraction - Video • Face Action Units – Facial Expression
- 45. MmLA Low-Level Feature Extraction - Audio • Direction of Arrival
- 46. MmLA Low-Level Feature Extraction - Audio • Speech to Text
- 47. Step 3: Fuse low-level features into learning constructs
- 48. Fundamental
- 49. Step 3: Example Collaboration features to collaboration constructs
- 50. Feature Fusion • Participation • Direction of arrival • Text • Joint attention • Gaze fusion • Turn taking • Direction of arrival • Speaker diarization • Collaboration moments • Participation • Text analysis • Idea uptake • Turn taking • Idea words • Leadership • Turn taking • Collaboration moments • Text analysis
- 51. Step 4: Provide Feedback
- 52. Feedback requirements • Enable self-reflection • Give a path to improvement • Non-intrusive, part of the experience
- 53. Step 4: Example Feedback for oral presentations
- 55. Creation-Time Marie Skłodowska working in her laboratory at the University of Paris.
- 56. In which context we can use MmLA to provide feedback for Fundamental Skills
- 57. Let’s ideate
- 58. Moving forward • Let’s talk about your project • Build/adapt a prototype • Test it