M-Assessment in situ: QuesTInSitu and cases from multiple contexts
1. Javier Melero, Interactive Technologies Group (GTI), UPF, Barcelona, Spain
Patricia Santos, Designing for Digital Learners (D4DL), UWE, Bristol, UK
M-Assessment in situ:
QuesTInSitu and cases from multiple contexts
Towards sustainable mobile learning scenarios
Wednesday 9th October, 2013
2. QuesTInSitu
“Assessment in situ” refers to a type of activity where the questions of a test have to be
answered in front of a related real location (in situ) *.
The interaction context of the test
had to be the city, a natural park...
Educational resources (questions)
have to be associated to real
positions and distributed as a route
in the physical environment
Geolocated questions and tests
(routes), can be created and
learners answer the tests using a
mobile device with GPS and 3G.
* Santos, P., Pérez-Sanagustín, M., Hernández-Leo, D. & Blat, J. (2011). QuesTInSitu: From tests to routes
for assessment in situ activities. Computers & Education, 57 (4), 2517-2534. R Q1) Impact factor: 2.617
2Javier Melero & Patricia Santos
3. M-assessment in situ scenarios
in diverse subject matters and educational levels
Barca Barca
Discovering Barcelona!
Bachelor students
explored the city observing
and interacting with the
architecture and the street
furniture with the aim of
putting in practice
urbanism and history
skills.
Assessing Botany in situ
The teacher created a route
where the university
students had to answer
questions in situ observing
the Barcelona botany
garden, and finding,
touching and measuring
specific plants.
A literature adventure
A group of senior
learners (~65 years old),
members of a literature
group, created two routes
with the aim of proposing
questions about facts of
a literature novel set in a
district of Barcelona.
3Javier Melero & Patricia Santos
4. Assessment in real situ: Videos and Findings summary
Findings
1. QuesTInSitu supports successfully an assessment in situ activity thanks to functionalities such as
the monitoring service or the provision of feedback in real time/situ.
2. Educational benefits: dynamic and practical activity; put in practice of transversal (collaborative
and communicative) and specific skills (botany, art, history, geography...).
3. The GPS location-based system makes the process of finding locations faster and helps the
students in their exploratory tasks
4. The assessment in situ activity had a notable positive impact in the attitude/motivation of the
students and teachers
5. Older learners found m-learning useful and meaningful because it enabled them to accomplish
learning activities putting in practice (in the real world) their knowledge by doing a playful activity.
Examples (videos of the scenarios):
(1) Learning History Art in situ , a m-learning activity with 62 secondary students and 2 teachers
http://bit.ly/GUW5vf (starting at 2′ 30”, in Catalan language)
(2) A literature adventure in situ, design and implementation of m-learning routes with older learners
http://www.youtube.com/watch?v=bBKvz9hYQ-k (english annotations)
4Javier Melero & Patricia Santos
5. QuesTInSitu: The Game
QuesTInSitu: The Game, an extended
gamified version of QuesTInSitu,
looks for promoting students’
reflection (avoiding being focused
just on achieving higher scores).
QuesTInSitu: The Game is based on
a metaphor that considers puzzles
and game elements*:
• Puzzles as a strategy to involve teachers
as designers, and engage students in the
subject topics, while foster students’
problem solving, analytical and memory
skills.
• Game elements: levels, hints, bonus,
punctuation mechanisms, and feedbacks.
* Melero, J., Santos, P., Hernández-Leo, D. & Blat, J. (2013) “Puzzle-Based Games as a Metaphor for Designing In Situ Learning
Activities”, In Proceedings of the 7th European Conference on Games Based Learning (pp. 674-682), Porto, Portugal.
5Javier Melero & Patricia Santos
6. Gamified In-situ Indoor and Outdoor
Scenarios
Discovering L’Hospitalet!
• Transversal activity in the school to learn about
the heritage of the city of l’Hospitalet.
• 10 levels.
• 55 questions.
• 250 points (correct answers) and 100 (incorrect
answers).
• 52 hints. Accessing to each hint subtract 100
points.
• Extra bonus is proportional to the number of
questions of a level.
• Hints are suggestions (ask people, read the
information that appears next to the statue...).
• Level’s information contains general
descriptions about the zone and particular
information about the questions.
Discovering the MNAC!
• Activity, associated to a subject, intended to
learn about different contemporary pictures
of a contemporary art museum.
• 4 levels.
• 20 questions.
• 50 points (correct answers) and 10 points
(incorrect answers).
• 19 hints. Accessing to each hint subtract 50
points.
• Extra bonus adds 50 points more.
• Hints are short texts about the context
related to the question.
• Levels’ information is short sentences of
each museum’s room.
6Javier Melero & Patricia Santos
7. Videos and Findings on QuesTInSitu: The
Game
Example:
• “Discovering L’Hospitalet Using your Smartphone” a m-learning activity with 74 secondary students
and 6 teachers (involved in the design of the gamified situated m-learning activity)
http://youtu.be/BTSsXa_e-6M
Findings
Teachers:
• The metaphor based on puzzles and game elements enables the creation of gamified in situ
learning activities depending on the teachers’ needs.
• Different design strategies: number of levels, punctuation mechanisms, hints, etc.
Students:
• Students enjoy the proposed approach.
• Not forcing students to correctly answer the different questions.
• Offering the possibility of moving to next levels without completing correctly all the questions
• Discrepancies when subtracting points.
• Students avoid to access to the hints.
7Javier Melero & Patricia Santos
8. Conclusions
• Successful combination of mobile devices and Location Based Systems to
support assessment in situ.
• M-assessment in situ can be applied in diverse subject matters (i.e. literature,
biology, history..), educational levels (i.e. primary, secondary, higher education,
lifelong learning), and educational settings (e.g museums, natural park, a city…).
• Considering puzzles and game elements enrich the in situ learning experience
and fosters students’ reflection (e.g. reflecting on wrong past choices, consulting
resources provided by the gamified application, discussing with other students,
asking people, etc.).
• The use of the puzzle-based game metaphor helps teachers understand an
unfamiliar situation (the design of gamified in situ m-learning activities) from a
familiar situation (traditional jigsaw puzzles).
• Authoring tools and re-usable formats (e.g. XML) enhance the sharing and re-
utilisation of m-learning scenarios.
8Javier Melero & Patricia Santos
This case study was evaluated through two experiments in real educational scenarios. The main experiment... The other experiment... Again we use different data sources to collect qualitative and quatintative information with the main aim of understanding the educational benefits (and limitations) of these type of tests. The main findings were: ...
This case study was evaluated through two experiments in real educational scenarios. The main experiment... The other experiment... Again we use different data sources to collect qualitative and quatintative information with the main aim of understanding the educational benefits (and limitations) of these type of tests. The main findings were: ...
