This document discusses a pilot project called Waisda? that used social tagging and gaming to annotate over 340,000 tags to audiovisual heritage collections. Players of the Waisda? video labeling game added tags that described what they saw and heard in television programs over a 7 month period. An evaluation found that 40.3% of tags matched other users' tags, and that tags for reality shows were more useful than those for documentaries. The project aims to improve tagging through linking tags to professional ontologies and exploring social gaming theory.

1. Emerging Practices in the Cultural Heritage Domain
Social Tagging of Audiovisual Heritage
Johan Oomen Lotte Belice Baltussen Sander Limonard Annelies van Ees Maarten Brinkerink Lora Aroyo Just Vervaart Kamil Afsar Riste Gligorov
VU University Netherlands Institute TNO ICT VU University Netherlands Institute VU University KRO Q42 VU University
Amsterdam* for Sound and Vision Amsterdam for Sound and Vision Amsterdam Amsterdam
Abstract Waisda?: Video Labeling Game Evaluation
To explore the impact and success criteria of social tagging Qualitative evaluation consisted of three separate activities.
in the cultural heritage domain a large-scale video labeling First of all, an online questionnaire (completed by 42 people)
pilot was executed. The game (Waisda? - ʻWhatʼs that?”) was sent out to Waisda? players. Secondly, a focus group
introduced three innovations: was organized. In a moderated discussion, five people
Using gaming as a method for annotating television elaborated on their experiences playing the game. Thirdly,
heritage. usability tests have been conducted, with five subjects that
Actively seeking collaboration with communities had never played the Waisda?. Primary aim of this test was
to evaluate the interface design.
connected to the content.
Using curated vocabularies as a means to integrate tags Quantitative evaluation was carried on all tags added
with professional annotations. between May and November 2009.
Within a period of 7 months, over 340,000 tags were added 42,068 unique tags have been added.
by the Waisda? players. An extensive evaluation was The total amount of tags added by players is 340,551, of
conducted, that provided input on the usability of the tags which 40.3% (137,421 tags) consists of matching tags.
and the game design. Based on this input, a roadmap for
future developments towards a fully operational service was Usefulness of Tags
drafted. The usefulness of the tags has been determined by a
professional cataloguer. A significant difference was found
Motivation Figure 1. Homepage
between the usefulness of tags added to reality shows
Players choose one of four ʻchannelsʼ that contain different
As the Web gets more “social” and as museums, libraries opposed to tags added to television documentaries.
programmes.
and archives are beginning to offer online access to digital
In the game environment players enter tags that decribe what they
representations of their collections, users and institutions are
see and hear.
beginning to inhabit the same, shared information space.
Players score points when their tag exactly matches the tag
This is an exciting prospect, as we are now witnessing new
entered by another player within 10 seconds
paradigms for engaging users with our shared heritage. In
Multiple other scoring mechanisms.
2005, social tagging – ad-hoc annotation by end-users – was Figure 2. Game Environment
introduced. After completion of the successful pilot fase of Figure 3. Usefulness: Figure 4. Usefulness:
steve.museum, one of the first experiments in heritage field, Reality Television shows Television Documentary
social tagging was embraced by institutions in the sector to
explore how they could benefit from what ʻthe crowdʼ has to
Future work
Comparing tags by end-users with tags in professional
offer. Social tagging offers several potential benefits for ontologies (first results look promising).
heritage institutions:
Using Linked Open Data to improve usefulness of the
Bridging the semantic gap between the terminology used
social tags.
by professionals and search strategy of end users.
Enriching collections / cultural heritage with factual and Improvements in game design (introducing game
ʻrecapʼ, multiple levels).
contextual information.
Increasing ʻconnectednessʼ with the archive. Design of retrieval interfaces.
Defining the future annotation workflows. Research in social gaming theory to generate traffic.
www.waisda.nl
* email: joomen@beeldengeluid.nl This work is part of the FP7 project PrestoPRIME. The development of Waisda? is
twitter: johanoomen supported by the digitization programme Images for the Future and the Netherlands Institute
for Sound and Vision
maandag 26 april 2010