Data Driven Education and Serious Games

Data-Driven Education &
Serious Games
Manuel Freire Morán
Grupo e-UCM de la Universidad Complutense de Madrid
www.e-ucm.es
Red eMadrid
www.emadridnet.org

Table of Contents
Introduction
 Analytics, Learning,
Serious Games
 e-UCM, RAGE, BEACONING
& IMPRESS
Recent work
 Architecting Game Learning
Analytics
 Dashboards & Learning
Analytics Models
 Hierarchical models
 Heterogeneous contexts
 Prediction
Data-Driven
Education
Opportunities & Dangers
Addressing Dangers
Improving Opportunities
Data -Driven Learning Design
manuel.freire@fdi.ucm.es - OEB'18, 2018.12.06
2

Analytics, Learning & SGs
3
Freire, M., Martínez-Ortiz, I., & Fernández-Manjón, B. (2018). Making
Understandable Game Learning Analytics for Teachers. In ICWL 2018,
Chiang Mai, Thailand, August 22-24, 2018
https://doi.org/10.1007/978-3-319-96565-9_11

e-UCM, RAGE, BEACONING
& IMPRESS
4
210 papers since 2008
6 PhDs defended in last 7y
(3 w/ awards)
ERASMUS+, 2017-2020
5 partners
4 countries
H2020, 2015-2018
5.9 M€
16 partners
H2020, 2014-2018
8.9 M€
18 partners
10 countries

RAGE
5
gamecomponents.eu

BEACONING
6
beaconing.eu

IMPRESS
7
code-defenders.org
Formalz gameplay trailer
https://www.youtube.com/watch?v=LYpxz0N7TpY
Impress-project.org

Table of Contents (II)
Introduction
Analytics , Learning,
Serious Games
& IMPRESS
Recent work
Architecting Game Learning
Analytics
Dashboards & Learning
Analytics Models
Hierarchical models
Heterogeneous contexts
Prediction
Data-Driven
Education
 Opportunities & Dangers
 Addressing Dangers
 Improving Opportunities
 Data-Driven Learning Design
8

Architecting GLA
9

Architecture
10

Design Philosophy
Use existing components whenever possible
Data stores, Authentication libraries, xAPI LRS,
Dashboard infrastructure, Analysis infrastructure, ...
Avoid platform coupling via containers
Avoid managing other's data;
make it easy to deploy locally
github.com/e-ucm/rage-analytics
Follow development best -practices
Shallow end, deep -end
Something useful out -of-the-box
Which you can customize to needs if you invest the time
11

Case study: Conectado
12
Evaluation with 257 students from 3 centers, ages 15-17
Calvo-Morata, A., Rotaru, D. C., Alonso-Fernández, C.,
Freire, M., Martínez-Ortiz, I., & Fernández-Manjón, B.
(2018). Validation of a Cyberbullying Serious Game
Using Game Analytics. IEEE Transactions on Learning
Technologies. https://doi.org/10.1109/TLT.2018.2879354

Customized Dashboards
13
Freire, M., Martínez-Ortiz, I., & Fernández-Manjón, B. (2018).
Making Understandable Game Learning Analytics for
Teachers. ICWL 2018, Chiang Mai, Thailand, August 2018
https://doi.org/10.1007/978-3-319-96565-9_11

Learning Analytics Models
14
Perez-Colado, I., Alonso-Fernandez, C., Freire, M., Martinez-Ortiz, I., &
Fernandez-Manjon, B. (2018, April). Game Learning Analytics is not informagic!.
https://doi.org/10.1109/EDUCON.2018.8363443
Learning
Analytics
Model
(LAM)

15
M. Freire, Á. Serrano-Laguna, B. M. Iglesias, I. Martínez-Ortiz, P. Moreno-Ger,
and B. Fernández-Manjón, “Game Learning Analytics: Learning Analytics for
Serious Games,” in Learning, Design, and Technology, Cham: Springer
International Publishing, 2016, pp. 1–29.

Hierarchical LAMs
16
Perez-Colado, I., Alonso-Fernandez, C., Freire, M., Martinez-Ortiz, I., &
Fernandez-Manjon, B. (2018, April). Game Learning Analytics is not informagic!.
https://doi.org/10.1109/EDUCON.2018.8363443

Hierarchical analyses
17
Ivan Perez-Colado, Dan C. Rotaru, Manuel Freire, Iván Martínez-Ortiz, Baltasar
Fernández-Manjón (2018): Multi-level Game Learning Analytics for Serious
Games. VS Games 2018. Würzburg, Germany.
https://doi.org/10.1109/VS-Games.2018.8493435

Hierarchical dashboards
18
Ivan Perez-Colado, Dan C. Rotaru, Manuel Freire, Iván Martínez-Ortiz, Baltasar
Fernández-Manjón (2018): Multi-level Game Learning Analytics for Serious
Games. VS Games 2018. Würzburg, Germany.
https://doi.org/10.1109/VS-Games.2018.8493435

Heterogeneous contexts
Analytics for a set of disparate activities?
Each manages their own authorization & authentication
Some have no analytics at all, while others keep their own
No plans to use LMS: LTI impractical
Case study: IMPRESS
Unified authentication & authorization via SAML
Activity Manager for
Configuration : class & activity setup
Launches activities
Allows on-the-fly changes to configuration
Analysis re-configuration requires
Kappa architecture
On -the-fly calculations specified directly in the dashboard
19

Evaluating & Predicting
20
Experiment with 227 students on a previously-validated game
Cristina Alonso-Fernández, Iván Martínez-Ortiz, Rafael Caballero,
Manuel Freire and Baltasar Fernández-Manjón, Predicting students’
knowledge after playing a serious game based on learning analytics
data, (Manuscript under review)

Table of Contents (III)
Introduction
Analytics , Learning,
Serious Games
& IMPRESS
Recent work
Architecting Game Learning
Analytics
Dashboards & Learning
Analytics Models
Hierarchical models
Heterogeneous contexts
Prediction
Data-Driven
Education
 Opportunities & Dangers
 Addressing Dangers
 Improving Opportunities
 Data-Driven Learning Design
21

Opportunities & Dangers
Opportunities
 Insights
 Alerts
 Predictions enable
 Recommendations
 Adaptation
 Open ecosystems
 Data sharing
Dangers
Poor data, analysis, and/or
presentation leads to
Misleading , false or
biased results
Confusion
Compounded by lack of
transparency
Privacy problems
Vendor lock-in
22

Addressing Dangers
Privacy
All experiments conducted using pseudonymous
tokens & not collecting personally-identifiable data
Systems to be installed, maintained by data owners
Lock -in
Standard data formats: xAPI-SG
Open platforms: all our software available as open
source (github.com/e-ucm)
Quality : see next slides
23

Improving quality
Transparency requires information visualization
Show me the data! (and how you concluded that)
 1. data, 2. ???, 3. profit! … does not work
LA is not informagic!
GIGO: Garbage in, garbage out.
Is the data good? Is the analysis sound? Does it have the
data it needs?
LA is not a black-box silver bullet:
insights build upon each other
No silver bullet
But the easy should be simple, and the complex possible
24

25
M. Freire, Á. Serrano-Laguna, B. M. Iglesias, I. Martínez-Ortiz, P. Moreno-Ger,
and B. Fernández-Manjón, “Game Learning Analytics: Learning Analytics for
Serious Games,” in Learning, Design, and Technology, Cham: Springer
International Publishing, 2016, pp. 1–29.

Data-Driven Learning Design
Test -Driven Design
Better software if designed to be testable
Also allows easy early testing, when bugs are cheap
Data -Driven Learning Design
Better learning if designed to use good LA
Also allows easy early validation, when changes are quick
Reflects trends in education calling for evaluation-based LD
How to get there?
LA should be integrated into authoring - uAdventure
Deep vs shallow learning: how can we measure depth?
26

27

Data Driven Education and Serious Games

Recommended

Recommended

More Related Content

Similar to Data Driven Education and Serious Games

Similar to Data Driven Education and Serious Games (20)

Recently uploaded

Recently uploaded (20)

Data Driven Education and Serious Games