Ecosystem Interactions Class Discussion Presentation in Blue Green Lined Styl...
Data Driven Education and Serious Games
1. Data-Driven Education &
Serious Games
Manuel Freire Morán
Grupo e-UCM de la Universidad Complutense de Madrid
www.e-ucm.es
Red eMadrid
www.emadridnet.org
3. Analytics, Learning & SGs
manuel.freire@fdi.ucm.es - OEB'18, 2018.12.06
3
Freire, M., Martínez-Ortiz, I., & Fernández-Manjón, B. (2018). Making
Understandable Game Learning Analytics for Teachers. In ICWL 2018,
Chiang Mai, Thailand, August 22-24, 2018
https://doi.org/10.1007/978-3-319-96565-9_11
4. e-UCM, RAGE, BEACONING
& IMPRESS
manuel.freire@fdi.ucm.es - OEB'18, 2018.12.06
4
210 papers since 2008
6 PhDs defended in last 7y
(3 w/ awards)
ERASMUS+, 2017-2020
5 partners
4 countries
H2020, 2015-2018
5.9 M€
16 partners
H2020, 2014-2018
8.9 M€
18 partners
10 countries
11. Design Philosophy
Use existing components whenever possible
Data stores, Authentication libraries, xAPI LRS,
Dashboard infrastructure, Analysis infrastructure, ...
Avoid platform coupling via containers
Avoid managing other's data;
make it easy to deploy locally
github.com/e-ucm/rage-analytics
Follow development best -practices
Shallow end, deep -end
Something useful out -of-the-box
Which you can customize to needs if you invest the time
11
manuel.freire@fdi.ucm.es - OEB'18, 2018.12.06
12. Case study: Conectado
manuel.freire@fdi.ucm.es - OEB'18, 2018.12.06
12
Evaluation with 257 students from 3 centers, ages 15-17
Calvo-Morata, A., Rotaru, D. C., Alonso-Fernández, C.,
Freire, M., Martínez-Ortiz, I., & Fernández-Manjón, B.
(2018). Validation of a Cyberbullying Serious Game
Using Game Analytics. IEEE Transactions on Learning
Technologies. https://doi.org/10.1109/TLT.2018.2879354
13. Customized Dashboards
manuel.freire@fdi.ucm.es - OEB'18, 2018.12.06
13
Freire, M., Martínez-Ortiz, I., & Fernández-Manjón, B. (2018).
Making Understandable Game Learning Analytics for
Teachers. ICWL 2018, Chiang Mai, Thailand, August 2018
https://doi.org/10.1007/978-3-319-96565-9_11
14. Learning Analytics Models
manuel.freire@fdi.ucm.es - OEB'18, 2018.12.06
14
Perez-Colado, I., Alonso-Fernandez, C., Freire, M., Martinez-Ortiz, I., &
Fernandez-Manjon, B. (2018, April). Game Learning Analytics is not informagic!.
https://doi.org/10.1109/EDUCON.2018.8363443
Learning
Analytics
Model
(LAM)
15. Learning Analytics Models
manuel.freire@fdi.ucm.es - OEB'18, 2018.12.06
15
M. Freire, Á. Serrano-Laguna, B. M. Iglesias, I. Martínez-Ortiz, P. Moreno-Ger,
and B. Fernández-Manjón, “Game Learning Analytics: Learning Analytics for
Serious Games,” in Learning, Design, and Technology, Cham: Springer
International Publishing, 2016, pp. 1–29.
16. Hierarchical LAMs
manuel.freire@fdi.ucm.es - OEB'18, 2018.12.06
16
Perez-Colado, I., Alonso-Fernandez, C., Freire, M., Martinez-Ortiz, I., &
Fernandez-Manjon, B. (2018, April). Game Learning Analytics is not informagic!.
https://doi.org/10.1109/EDUCON.2018.8363443
17. Hierarchical analyses
manuel.freire@fdi.ucm.es - OEB'18, 2018.12.06
17
Ivan Perez-Colado, Dan C. Rotaru, Manuel Freire, Iván Martínez-Ortiz, Baltasar
Fernández-Manjón (2018): Multi-level Game Learning Analytics for Serious
Games. VS Games 2018. Würzburg, Germany.
https://doi.org/10.1109/VS-Games.2018.8493435
18. Hierarchical dashboards
manuel.freire@fdi.ucm.es - OEB'18, 2018.12.06
18
Ivan Perez-Colado, Dan C. Rotaru, Manuel Freire, Iván Martínez-Ortiz, Baltasar
Fernández-Manjón (2018): Multi-level Game Learning Analytics for Serious
Games. VS Games 2018. Würzburg, Germany.
https://doi.org/10.1109/VS-Games.2018.8493435
19. Heterogeneous contexts
Analytics for a set of disparate activities?
Each manages their own authorization & authentication
Some have no analytics at all, while others keep their own
No plans to use LMS: LTI impractical
Case study: IMPRESS
Unified authentication & authorization via SAML
Activity Manager for
Configuration : class & activity setup
Launches activities
Allows on-the-fly changes to configuration
Analysis re-configuration requires
Kappa architecture
On -the-fly calculations specified directly in the dashboard
manuel.freire@fdi.ucm.es - OEB'18, 2018.12.06
19
20. Evaluating & Predicting
manuel.freire@fdi.ucm.es - OEB'18, 2018.12.06
20
Experiment with 227 students on a previously-validated game
Cristina Alonso-Fernández, Iván Martínez-Ortiz, Rafael Caballero,
Manuel Freire and Baltasar Fernández-Manjón, Predicting students’
knowledge after playing a serious game based on learning analytics
data, (Manuscript under review)
22. Opportunities & Dangers
Opportunities
Insights
Alerts
Predictions enable
Recommendations
Adaptation
Open ecosystems
Data sharing
Dangers
Poor data, analysis, and/or
presentation leads to
Misleading , false or
biased results
Confusion
Compounded by lack of
transparency
Privacy problems
Vendor lock-in
manuel.freire@fdi.ucm.es - OEB'18, 2018.12.06
22
23. Addressing Dangers
Privacy
All experiments conducted using pseudonymous
tokens & not collecting personally-identifiable data
Systems to be installed, maintained by data owners
Lock -in
Standard data formats: xAPI-SG
Open platforms: all our software available as open
source (github.com/e-ucm)
Quality : see next slides
manuel.freire@fdi.ucm.es - OEB'18, 2018.12.06
23
24. Improving quality
Transparency requires information visualization
Show me the data! (and how you concluded that)
1. data, 2. ???, 3. profit! … does not work
LA is not informagic!
GIGO: Garbage in, garbage out.
Is the data good? Is the analysis sound? Does it have the
data it needs?
LA is not a black-box silver bullet:
insights build upon each other
No silver bullet
But the easy should be simple, and the complex possible
manuel.freire@fdi.ucm.es - OEB'18, 2018.12.06
24
25. Learning Analytics Models
manuel.freire@fdi.ucm.es - OEB'18, 2018.12.06
25
M. Freire, Á. Serrano-Laguna, B. M. Iglesias, I. Martínez-Ortiz, P. Moreno-Ger,
and B. Fernández-Manjón, “Game Learning Analytics: Learning Analytics for
Serious Games,” in Learning, Design, and Technology, Cham: Springer
International Publishing, 2016, pp. 1–29.
26. Data-Driven Learning Design
Test -Driven Design
Better software if designed to be testable
Also allows easy early testing, when bugs are cheap
Data -Driven Learning Design
Better learning if designed to use good LA
Also allows easy early validation, when changes are quick
Reflects trends in education calling for evaluation-based LD
How to get there?
LA should be integrated into authoring - uAdventure
Deep vs shallow learning: how can we measure depth?
manuel.freire@fdi.ucm.es - OEB'18, 2018.12.06
26