In this paper we analyze the process of designing and developing a Serious Game intended to train people with intellectual disabilities in moving around a city using the public transportation system. The first step in our investigation is to understand the cognitive, psychological and motor abilities of our users and their specific needs. Secondly, we translated the characteristics of the players into user requirements, with adapted mechanics to improve the understanding and to increase the probability for the user to be able to carry out the tasks to perform in the video game. Finally, due to the specific characteristics of our final users a Learning Analytics module has been included in the game to collect relevant information about how users are actually playing and to infer how the learning process of every user is occurring. We also discuss the next steps in our research and the future work related with it: design a range of experimental tests to verify the adequacy of the video game as a learning tool for this type of users

1. Downtown, A Subway Adventure: Using Learning
Analytics to Improve the Development of a Learning
Game for People with Intellectual Disabilities
Ana R. Cano
Baltasar Fernández-Manjón, Álvaro J. García-Tejedor
Grupo e-UCM: www.e-ucm.es
anarcano@ucm.es @anaruscano
ICALT 2016. Austin, TX 07/25/2016
http://www.slideshare.net/AnaRusCano/

2. 1. What is LA and GLA?
2. The GLA problem
3. Our Model: GLAID
4. Case Study: Downtown
2

3. LA & GLA 101
• Learning Analytics: Improving education based on Data Analysis
̶ Data driven
̶ Evidence-Based Education
• Game Learning Analytics application of LA to Serious Games
̶ Interaction data in a Serious Game is collected and analyzed for improving the
learning process supported by the game
̶ Educational game not as “black boxes”
But LA & GLA is not “informagic”
̶ We need to relate data with what happens in the game and with the
educational design!

4. The GLA Problem
• Ok, we are collecting ALL the interaction data in a video game but…
IT IS A HUGE AMOUNT OF DATA!
Now what?
• What are the relevant observables?
• How do I analyze the data collected?
• How do I translate interaction data
into useful information about the
learning process?

5. And the problem gets bigger…
…If the user has an intelectual condition or disability
(e.g. Down Syndrome)
User Special Needs:
• Interaction with the game (motor skills)
• Ordering thoughts and language in a “logical” layout
• Listening and taking turns in conversations
• Communication in an interactive sense
• Relating objects and actions to spoken or written words

6. H2020 Beaconing project
• BEACONING stands for ‘Breaking Educational Barriers with Contextualised,
Pervasive and Gameful Learning’
• Started in january 2016, 15 partners, 9 countries, 6M
• Global goal is learning ‘anytime anywhere’
• Exploitation of technologies for contextual pervasive games and use of gamification
techniques
• Problem based approach to learning
• Enriching the Gaming Learning Analytics data model with
the contextual, geolocalized and accessibility information
• Large pilots in real settings: formal and informal learning
across virtual and physical spaces
• GLA is a key element in the games and pilots evaluation
• Addressing accessibility for people with cognitive disability

7. Our approach: The GLAID Model
Present
Individualized
Learning Analysis
Collective
Learning Analysis
Predictive
Learning Analysis
….
Group 1
Group 2
Group 3
Game Sessions
LearningProgress
d1.a d1.n
d2.a
d3.a
d2.n
d3.n
*d = Data collected during a game session
GLAID (Game Learning Analytics for Intellectual Disabilities) Model
Analytics Framework
User 1
User 2
User n
User 1
User n
User 3 User 2
User 5
User 4
User 1
Data Handling
Designer Perspective Educator Perspective
User cognitive
restrictions
Formal
Requirements
Game & Learning
Design
Group of
Observables
Group of
Observables
Descriptive
Analytics
Clustering
Analytics
Predictive/Prescriptive
Analytics

8. First Step: From the User Restrictions to a Game Design
• Challenges:
1) Transform the user characteristics
into formal requirements
2) Develop a learning game design
adequate for users with intelectual
disabilities (such as Down Syndrome,
mild cognitive impairments, ASD
Autism Spectrum Disorders,…)
3) Select a group of
observables/variables that help to
evaluate the learning outcome of
the user for future assessment
….
*
User
User
User
User cognitive
restrictions
Formal
Requirements
Game & Learning
Design
Group of
Observables
Group of
Observables

9. 1st Level Analysis: Individualized Learning Analysis
• Goal: Describe and analyze historical
learning data from the student’s
perspective
• Outcome: Gives an overview of the user’s
learning behaviour through several game
sessions
• Observables collected individually
• Timestamps
• Level changes
• Achievements vs. Fails
• User interactions (number of clicks, heatmaps,
time between clicks,…)
Individualized
Learning Analysis
….
d1.a d1.n
d2.a
d3.a
d2.n
d3.n
*d = Data collected during a game session
User 1
User 2
User n

10. 2nd Level Analysis: Collective Learning Analysis
• Goal: Identify causes of trends and learning
outcomes for a group of users segmented
by disability or cognitive skills
• Outcome: Learning patterns
• Observables collected collectively
• Timestamps
• Level changes
• Achievements vs. Fails
• User interactions (number of clicks, heatmaps,
time between clicks,…)
Collective
Learning Analysis
Group 1
Group 2
Group 3User 1
User n
User 3 User 2
User 5
User 4

11. Data Handling: stakeholders
• 2 Data handling perspectives:
Game Designer’s Perspective
• Collect and analyze all the states that
the user can reach in a game session
• Are the mechanics of the game
appropriate for the user?
Educator’s Perspective
• Learning experience of each user
• Are the users learning or struggling
with the game?

12. Case Study: Collecting data with xAPI
• We can collect the relevant data in a standard format using xAPI
• We are working in a xAPI serious games profile with ADL
• This will simplify the analysis and visualization of data (e.g. dashboards)
xAPI

13. Case study: Downtown
• Serious Game designed and develop
to teach young people with Down
Syndrome to move around the city
using the subway
• Status: Designed and developed.
Analysis pending
• Type of game: Serious Game
• Audience: People between 15 and 30 y/o with
Down syndrome
• Platform: PC and Android (work in progress)

14. Case Study: Downtown

15. Case Study: Data Report
32
User Restrictions
51
Game & Learning
Design decisions
14
Observables
Divided by:
• Intelligence, memory
and perception (6)
• Learning experience (14)
• Personality (6)
• Biological and motor
skills (6)
Divided by:
• Intelligence, memory and
perception (10)
• Learning experience (25)
• Personality (6)
• Biological and motor skills
(8)
1. Difficulty level
2. Total time
3. Total inactivity time
4. Fails in minigames
5. Time completing missions
6. Clicks in Map
7. Clicks in Menu
8. Clicks in Help
9. Clicks in accessibility menu
10. Options in accessibility menu
11. Number of gems
(gamification)
12. Heatmaps
13. Clicks in repetition button
14. Login attemps

16. Case Study: From user requirements to a game design
User Restriction Game Requirement Game Design & Mechanics Observable
Limited
intellectual
autonomy
The game should be able to
guide the user during the
learning session through
interactive help, pop-up tips or
other mechanics
There will be a "help" button
permanently in the screen where the
user can ask for help at anytime during
the game session
Clicks in the Help
buttons during a
game session
If the user doesn't perform any
interaction for more than 2 minutes, a
pop-up aid will appear providing guide,
tips and advices
Total inactivity time
Inactivity time after
pop-up help appears
The phone will act as a help
button. If the user needs tips or
advices, he can call the police
asking for clues to complete the
ongoing task

17. Case Study: From user requirements to a game design
User Restriction Game Requirement Game Design Observable
Difficulty in the process
of abstractions,
conceptualization,
generalization and
learning transfer
The game should explain any
action to do, even the easiest,
without assuming that the
user already know how to
complete it
Tutorials: The description about how
to achieve the goals in the game will
be performed as a video explanation
before the task starts
Time consumed in
completing the task
Previous research prove that
visual explanations help to
understand the assignments
better than hearing or
reading.
Savidis, Grammenos and Stephanidis "Developing
inclusive e-learning and e-entertainment“. 2007

18. Next steps: Formative Assessment
• Identify correlations between the game and
learning design and the game experience of the
user
• Apply the 1st and 2nd level of the GLAID
Analysis
• Define dashboards for educators

19. Case study: Assessment Dashboard
User: John Doe
Segment: Group 1, Down Syndrome
Age: 19
5
Game
Sessions
15
Completed
Missions
Vs.
3
Failed
Missions
Completion Rate
95%
Overall Education
Progress
83,2%
Individualized Learning Analysis Collective Learning Analysis
0
1
2
3
4
0 1 2 3
Overall Achievement Rate
Rank by Skill (0 to 10)
0 10 0 10
0 100 10
General Comprehension Short-term memory
Spatial vision Path accuracy
8,2 9,1
5,8 10
Motivation
Engagement
Activity Time
0%
20%
40%
60%
80%
100%
Session 1 Session 2 Session 3 Session 4 Session 5
Performance Evolution

20. Thanks!
Questions?Mail: anarcano@ucm.es
Twitter: @anaruscano
GScholar: https://scholar.google.com/citations?user=8vXG8X8AAAAJ&hl=en
ResearchGate: https://www.researchgate.net/profile/Ana_R_Cano
Slideshare: http://www.slideshare.net/AnaRCano