The document summarizes two studies conducted with blind children in Malaysia to test their spatial intelligence and executive function using mobile devices. The first study involved 25 blind children ages 12-17 exploring and answering questions on a tactile mobile device. Researchers observed collaboration between students and different strategies for interacting with the device. The second study involved the Tower of London game but time constraints prevented all students from playing. Researchers concluded more study is needed on cognition, interface design, and providing appropriate feedback for blind users.
2. All Rights Reserved by Seeds of Empowerment
• The first study was conducted on November
The first study was conducted on November
11, 2010 in SMPK Setapak (Asarama) school
located at Jalan
located at Jalan Genting Kelang 53300 Kuala
Kelang,53300 Kuala
Lumpur, Malaysia
• Participants were 25 total blind children age
Participants were 25 total blind children age
12‐17 years old.
2
4. All Rights Reserved by Seeds of Empowerment
First Touch
•Then we handed in the
•Then we handed in the
devices to them. As we
didn’t have enough TMs
They shared the devices.
•They started to explore
TMs right away and were
discussing about it with
each other.
each other.
• 2 participants were
holding TMs with 90
degree rotation saying
they feel more
h f l
comfortable in that
position.
p
•We asked them simple
questions like how many
buttons does the device
have ? They started to talk
to each other and then
to each other and then
90% answered correctly .
•They were excited! 4
5. All Rights Reserved by Seeds of Empowerment
Exploration
•4 boys figured out
the up, down, right
the up, down, right
left concept of the
buttons very quickly. I
asked them if they
had any experience
h d i
working with
computers and the
answer was positive.
p
They said “they like
computers and they
want to play
computer games !
computer games”!
Later I checked with
their teacher and she
said these guys are
competent in using
computers.
5
10. All Rights Reserved by Seeds of Empowerment
Connection
The audio test was set as the default
program in TM so it started playing
the audible test after Aina turned it
on and everyone were excited
,working harder on their device to
figure out how to turn it on.
fi th t t it
Next one was the boy with gray T‐
shirt (The Wong) with 5min record.
After they solved the problem they
were guiding each other and sharing
the experience.
We talked about the process. They
said “I knew the power is in corner
but I didn’t know I should hold it”
10
12. All Rights Reserved by Seeds of Empowerment
Interaction Level1
Interaction‐Level1
Level 1 questions was described as “easy” by
q y y
the participants.
Almost 85% of them answered to the level 1
q
questions correctly. They were able to follow
y y
the instruction and answer the questions.
No Interaction and discussion happened
among the participants while were working
g p p g
on level 1 questions.
They seemed like they are enjoying it.
All of them knew the cardinal directions.
All of them knew the cardinal directions
12
13. All Rights Reserved by Seeds of Empowerment
Interaction Level 2
Interaction‐Level 2
• The moment they heard 180 degree rotation in questions, we observed
expressions like “wow, phww, mmm,oooh…” Mostly from boys.
expressions like “wow phww mmm oooh ” Mostly from boys
• 3 participants (2boy, 1girl) asked us to repeat level 2‐q2,3
• They started talking to each other and discussing the answers. More
collaboration.
• At Level 2 question, 2 girls started demonstrating the scenario. One of
them stood up and literally followed all the instructions such as 180
rotation. Facing North, turning to right, left. She answered all the
questions correctly.
questions correctly
• The other girl was following the instruction by demonstrating lines on
palm of her hand with her index finger .
• One boy (13yrs) started to show signs of frustrations, holding his head
with his hands, moving his leg, he only answered one question correctly.
i h hi h d i hi l h l d i l
• Whenever they received “correct” feedback from the device they were
excited sharing their success stories with their friends.
• Longer response time was reported compare to level 1
Longer response time was reported compare to level 1
13
15. All Rights Reserved by Seeds of Empowerment
Reflection
• We asked them who answered to all the questions correctly? 5 of them stood up,
we announced their name and everybody else applaud them and congrats them
• We interviewed them after finishing the data collection. What do you think about
the game, what was the problem,…
• “I liked it”, “I like to play science game, I don’t like geography”, “I want to answer
“I lik d it” “I lik t l i I d ’t lik h ” “I tt
English questions” “Level 3 was hard” “ I want repetition feature” “do you have
more games?” “ what does this button (yellow, green) do”
• Because of lack of time we weren t able to conduct TOL study with all of them so
Because of lack of time we weren’t able to conduct TOL study with all of them so
we decided to play the instruction on computer and receive feedback.
• They listened to the scenario carefully then gave us following comments:
– “too hard to follow”
– “why do you want to put fruits on sticks”
– "hard to imagine" , "why are we moving the fruits", "why we can only move one fruit at a time". One
girl (13yrs old asked me if I have the sticks and fruits somewhere she can touch it" “do you have any
other game?”
15
16. All Rights Reserved by Seeds of Empowerment
• Blind children collaborate with each other to solve more complicated problems
• Blinds need an authentic scenario and enjoy problem solving questions.
• Blind children can perform audible tests using TM however the Instruction and
scenario design, narration language, length, and authenticity should be further
studied.
t di d
• Usability and interface design study would add more richness to the experience.
• More study on Blinds’ cognition and comprehension process is required (spatial
intelligence , why did they started demonstrating the scenario physically, how do
intelligence why did they started demonstrating the scenario physically how do
they imagine the aural instruction? )
• The more cognition required the more response time needed.
• Participants requested for more aural feedback when they were interacting with
Participants requested for more aural feedback when they were interacting with
the device.
16
18. All Rights Reserved by Seeds of Empowerment
• Study was conducted on 16 November 2010 in
Study was conducted on 16 November 2010 in
SMPK Setapak (Asarama) school located at
Jalan Genting Kelang 53300 Kuala Lumpur
Kelang,53300 Kuala Lumpur,
Malaysia
• Participants were 12 total blind and 1 visually
Participants were 12 total blind and 1 visually
impaired children age 14‐17 years old.
18
19. All Rights Reserved by Seeds of Empowerment
Plan
• Due to School holiday number of participants
Due to School holiday, number of participants
was decrease to 12. However quantitative and
qualitative data was collected.
qualitative data was collected
• Quantitative data was collected through
TeacherMate while participants were playing
while participants were playing
with Tower of London game.
• Q li i d
Qualitative data was collected through semi‐
ll d h h i
structured interview and focus group sessions.
19
20. All Rights Reserved by Seeds of Empowerment
Entrance
• Upon our arrival we were welcomed by
Upon our arrival we were welcomed by
participants from previous study.
we said Hello guys they replied Hi, where is
we said “Hello guys”, they replied “Hi where is
your TM?”
Then they asked us to wait for them till they call
Th h k d i f h ill h ll
up their friends.
20
21. All Rights Reserved by Seeds of Empowerment
Introduction
We didn’t have much to do anymore. Our
little friends managed everything. We
just sat there and observed.
Old group gathered their friends and
passionately explained who we are and
what we do. Then they asked us to give
them some TeacherMates so they can
explain more accurately.
l i l
Next question “Do you have a new game
for us today?”
They invited us to their class and showed
us around.
21
23. All Rights Reserved by Seeds of Empowerment
New ideas
New ideas
• While we were waiting for teachers to prepare
While we were waiting for teachers to prepare
the new study setting, we started to discuss
with our friends about their last experience
with our friends about their last experience
with TM and asked them how do they want to
use it? What kind of game do they suggest?
use it? What kind of game do they suggest?
How are they planning to use TM for their
study?
23
27. All Rights Reserved by Seeds of Empowerment
“I like hearing different
I like hearing different
voices, I feel I’m getting to
p p
know new people.”
We had two English
narration with Korean and
Malaysian accent and one
sentence in Spanish, they
were excited about new
accents and were imitating
the accents and laughing
th t d l hi
among themselves .
27
28. All Rights Reserved by Seeds of Empowerment
“I want to have all my
school books in TM so I
don’t have to sit and read
all the braills. I can play
with TeacherMate and
listen to my lessons.”
Two other participants
instantly suggested they
will record the lessons for
him!
28
29. All Rights Reserved by Seeds of Empowerment
“We like audio
games. They
games ” They
patiently explained
to us how they
interact with the
environment and
emphasized on
p
auditory channel as
one of the most
important methods
important methods
of communication.
“When we listen to any instruction we imagine it in our mind You know
When we listen to any instruction we imagine it in our mind. You know
blind people have a better memory than sighted kids!!!!”
29
30. All Rights Reserved by Seeds of Empowerment
Other suggestions about using TM in class
continued as:
continued as:
“We can use TM to do our assignments.” Azli,
14years old asked us how we created the
14years old asked us how we created the
games with audible test and if it is easy for
their teacher to use the audio maker. Then
their teacher to use the audio maker. Then
problem solved.
“we will ask our teacher to record the
we will ask our teacher to record the
assignment in audio test maker and we will
submit our audio files to her”.
Currently they are using braille typewriters to do
their assignment.
30
33. All Rights Reserved by Seeds of Empowerment
New Ideas
New Ideas
Next lesson for us was learning how they replied to
g y p
the “cardinal direction” game.
“I imagined myself walking while following the
instruction”
“I imagined I’m driving a car and am following the
direction to get to the destination.
direction to get to the destination.”
“I imagined leading my friend, holding his hand and
guiding him”
We continued our discussion after playing TOL
game.
33
34. All Rights Reserved by Seeds of Empowerment
Tower of London
Tower of London
• To design the audible version of TOL game we followed participatory design
approach and changed the original game based on feedback received from our
blind designers.
• V1‐Original TOL using color to differentiate the donuts‐ Our little fiends
commented that color is not authentic to them and it s rather a label and name
commented that color is not authentic to them and it’s rather a label and name
than a comprehensive concept.
• V2‐Using fruit and skewer concept. You have to move the fruits and rearrange the
order‐ We had to justify to them “why do you want to move them” why did you
j y y y y y
put the fruit on sticks”
• V3‐Using a barbecue party and role playing concept, problem solving‐ Players are
expert chefs who are taking different orders to prepare various arrangements of
chicken, fish and lamb.
hi k fi h d l b
• V4‐ Local version of barbecue party with fish, and shrimp ball and chicken
suggested by total blind participants. (Malay translation)
34
35. All Rights Reserved by Seeds of Empowerment
Sample Scenario
Sample Scenario
• Order 1. Start position: The left skewer holds up
O de . Sta t pos t o : e e t s e e o ds up
two foods, chicken and shrimp. The chicken is at
the bottom, and the shrimp is on the chicken. The
middle skewer holds up a fish ball at the bottom.
ddl k h ld f h b ll h b
The right skewer holds up nothing.
• G l
Goal position: The left skewer holds up nothing.
iti Th l ft k h ld thi
The middle skewer holds up two foods, fish ball
and chicken. The fish ball is at the bottom, and
and chicken. The fish ball is at the bottom, and
the chicken is on top of the fish ball. The right
skewer holds up a shrimp.
35
36. All Rights Reserved by Seeds of Empowerment
Game Design
Game Design
Original TOL has 5 levels with 52 questions.
Original TOL has 5 levels with 52 questions
For this design we planned to test only 3 levels
with 21 questions.
with 21 questions
36
38. All Rights Reserved by Seeds of Empowerment
TOL Study
TOL Study
We divided the participants to two groups to be able to use
our devices and facilitate observation process.
38
39. All Rights Reserved by Seeds of Empowerment
TOL Study
TOL Study
Participants had
a t c pa ts ad
enough time to
explore the TM.
We explained the
l d h
concept and what
they have been
they have been
asked to do and
they eagerly
y g y
started to play the
game.
39
40. All Rights Reserved by Seeds of Empowerment
TOL Level 1
TOL‐Level 1
Beginning level of the TOL game. Participants were focusing on
g g g p g
the instruction to figure out how this game works.
40
41. All Rights Reserved by Seeds of Empowerment
TOL Level 1
TOL‐Level 1
The level of participation and collaboration at level 1,beginning
of the game, was low and they separately were concentrating
of the game was low and they separately were concentrating
on their tasks.
41
42. All Rights Reserved by Seeds of Empowerment
TOL Level 2
TOL‐Level 2
After they passed level 1 and understood the game concept the
p
participation and collaboration started and they looked more
p y
relaxed and seemed they are having fun with the game
42
43. All Rights Reserved by Seeds of Empowerment
TOL Level 3
TOL‐Level 3
• Level 3 questions are a bit harder and more
Level 3 questions are a bit harder and more
complicated compare to previous questions
but participants had the highest level of
collaboration and they were talking together
about their performance.
• The TOL game provides feedback to players
and informs them if they replied correctly or
not. Participants showed high level of concern
P i i h d hi h l l f
about correct/incorrect feedback.
43
47. All Rights Reserved by Seeds of Empowerment
More ideas
More ideas
• After data collection we had a chance of discussing more
g
about their feedback and learning from their suggestion.
• They labeled our game as Geography game (Cardinal
direction) and Mind game (TOL game).
di ti ) d Mi d (TOL )
• 10/12 participants found the mind game more challenging
and interesting rather than geography game
g g g p yg
• All participants asked us about our next game
• 2 participants called up their visually impaired friends to play
with TM.
• 3participants asked for science game (2boy 1 girl) and 8 asked
for more mind game and 1 asked for literacy game.
for more mind game and 1 asked for literacy game
47
48. All Rights Reserved by Seeds of Empowerment
More ideas
More ideas
• Natasha: “ I like mind game to improve my memory so I can
g p y y
remember more, it makes my brain smarter”
• All participants suggested to add repeat feature and more
control over the game flow.
t l th fl
• They asked for more group games.
They informed us
about their everyday
activities such as
activities such as
playing “goal ball”,
Chess, riding a bicycle,
listening to radio and
li t i t di d
TV.
48
49. All Rights Reserved by Seeds of Empowerment
African stories
African stories
We shared Seeds’
African stories with
f h
them (Jomopira and
Kakama), playing on
Kakama) playing on
iPad and Android
phone. They answered
phone They answered
to all the story
q
questions
correctly.
49
50. All Rights Reserved by Seeds of Empowerment
Zambia and India plan
Zambia and India plan
Next we shared Seeds plan to go to India and
Next we shared Seeds’ plan to go to India and
Zambia and design games for blind children. We
explained children s educational situation to them
explained children’s educational situation to them
and our young friends started to propose solutions.
50
51. All Rights Reserved by Seeds of Empowerment
Zambia and India plan
Zambia and India plan
They asked many questions such as how old are they, if they don’t
y yq y, y
have typewriter how do they take exam?
What language they speak?
How could they learn more about them?
Why are they facing difficulties in their life?
What courses they like?
What courses they like?
Do you have more stories from them?
51
52. All Rights Reserved by Seeds of Empowerment
Zambia and India suggestions
Zambia and India suggestions
“Their teacher can record all of their courses on TM and they can
listen to them all the time, It’s OK if they don’t know braille“
“We can record all of our lessons for them and send it to them”
“They can use TM to practice their courses do assignments take
They can use TM to practice their courses, do assignments, take
exam”
“We can design audio games for them to play after school. Do you
have more mind games?”
h i d ?”
“We can record our stories for them and send on TM”
ey o e ed us t e e p a d suppo t a d p oposed e s g ou
They offered us their help and support and proposed revising our
future games to make it more suitable for blinds!
We were asked if we can give TMs to them to play in school,
unfortunately we didn t have enough devices to share.
unfortunately we didn’t have enough devices to share
52
53. All Rights Reserved by Seeds of Empowerment
Next?
• Revising TOL game and create multi‐lingual versions
• More study on Instructional design and narration instruction design for
blinds
• Pilot testing the audio game with children from varied socioeconomic
g g
background
• Increase the sample size
• Design game in different context: science, math, literacy,…
Design game in different context: science math literacy
• Study on collaborative and participatory learning among blinds
• User interface and tactile experience studies
• Study on blinds’ cognitive intelligence, ability and development
S d bli d ’ i i i lli bili dd l
53
54. All Rights Reserved by Seeds of Empowerment
Acknowledgement
Acknowledgement
Special thanks to Nurzyanti Mohammad Khan for
Special thanks to Nurzyanti Mohammad Khan for
her generous help in translating and narrating all
the questions and great help in conducting the
q g p g
studies. Thanks to Maryam Sarkhail for her
assistance in conducting the Cardinal direction
study. We couldn’t do any of this without you.
We acknowledge our appreciation toward principle
g pp p p
and teachers of SMPK SETAPAK school specially
MS Rosi.
54