Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
INSEMTIVES Tutorial ISWC2011 - Session4
1. Content tagging at Moon Zoo &
Tiny Planets
Carl Goodman
Pepper’s Ghost Productions Ltd
2. Background: Tiny Planets, educational television
property and virtual world
• Aired in 110 countries around the world
• Million visits per year to the various Tiny Planets websites
• TV series co-produced with Sesame Workshop in US
• Winner of two BAFTA awards in UK
• Online content distribution relationships with Yahoo! Kids and
National Geographic Kids
• Upcoming distribution relationships with Discovery Kids and
ReadMe Stories (iPad + mobile)
• Principle education advisor Ann McCormick, founder of The
Learning Company
4. Candidates for annotation in virtual world based
on a space theme
• Objective was to explore several formats with respect to annotation,
varying from straightforward game-play to educational / altruistic
• Criteria for latter was: substantial volumes of data, ideally astronomical in
nature to match space-theme of site, educational credibility and ability to
reassure parents
• COPPA compliance (US Child Online Protection and Privacy Act) requires
Tiny Planets to adhere strictly to rules concerning protection of children’s
Personally Identifiable Information
• Parental opt-out necessary to allow parents to prevent children from
submitting
5. Galaxy Zoo: Oxford University Department of
Astrophysics and the Citizen Science Alliance
• Approached OU in 2010 with a proposal to create children’s version of
existing annotation activity.
• Galaxy Zoo: ‘an online astronomy project which invites members of the
public to assist in the morphological classification of large numbers of
galaxies.’
• “Scientists are increasingly finding it difficult to cope with what has been
called the "Data Deluge", where modern research is producing vast sets of
information. Often the teams involved don't have the time, resources or
brainpower to analyse it all.” Kevin Schawinski, project co-founder.
• Classification scheme for Galaxy Zoo required users to answer a series of
quite simple questions about a particular image and respond by clicking
the appropriate icon.
• Galaxy Zoo 2 allowed for more detailed observations; it achieved
60,000,000 classifications from February 2009 to April 2010 (Wikipedia)
7. Classification of galaxies: is your image recognition
algorithm smarter than an 8 year old?
• "Most of these galaxies have been photographed by a robotic telescope
[Sloan Digital Sky Survey] and then processed by computer. So this is the
first time they will have been seen by human eyes.“Schawinski
• From a human perspective the Galaxy Zoo images seemed comparatively
straightforward to classify – easy enough that a child in the Tiny Planets
target age-range really ought to be able to out-perform a computer.
• However, CSA was in the process of launching another project around the
same theme.
• Moon Zoo would take data from the NASA Lunar Reconnaissance Orbiter
(LRO) to examine crater sizes and boulder distribution. This would require
marking images in a slightly more complex fashion – a Flash interface
appeared most appropriate.
• Moon Explorer is Tiny Planets children's version of the activity
11. Moon Explorer annotation activity as a part of the
Tiny Planets virtual world
• Activity is linked to the Tiny Planets game economy; players earn virtual
currency to buy items to decorate their space-station and for their pet
alien as a reward.
• After initial trials on a ‘satellite’ website Moon Explorer proved sufficiently
poplar to move to the Tiny Planets main site
• Concept of Moon Explorer has proven especially popular with parents,
including US home-schooling community (science / education theme, plus
the idea that the annotations performed by children are actually used in
data analysis)
• Currently averaging c.500 annotation sessions per month with an average
session length around 8 minutes. This results in between 5-8000
annotations per month. (Data from Google analytics)
12. Moon Explorer rewards players with Stars (virtual
economy) in order to buy virtual goods.
13. The Universe Game: another approach to annotation of
astronomical images through game play
14. The Universe Game: an intermediate level
strategy game
• Motivation: to attempt to develop an ‘intermediate level’ (i.e. moderately
complex but not to full simulation standard) strategy game that
incorporates annotation as part of the game play.
• Turn based game loosely following the ‘battleships’ scenario of hunting
hidden objects / hidden enemies
• Annotations of a library of astronomical images grouped into 8 top level
categories
• Purpose of annotation is two-fold; to ‘educate’ the casual player to
recognise specific categories of astronomical objects, and to obtain
annotations from advanced players as to more tightly defined sub-
categories of objects.
• Annotations reported in RDF format and submitted to DBpedia.
19. The Universe Game: Discovering hidden objects
and annotating them – players’ process
20. The Universe Game: annotations become more
complex
• Galaxy sub-categories: Elliptical, Spiral, Irregular, Barred spiral, Ring, Lens,
Dwarf, Spherical
• Nebula sub-categories: Diffuse, Reflection, Planetary, Emission, Supernova
Remnant, Dark
• Comet sub-categories: Long period, Intermediate period, Short period,
Single apparition, Periodic, Sun-grazing
• Asteroid sub-categories: C group (darkish), S group (stony), X group
(metallic), T type (reddish), D type (very dark), V type (bright)
• Meteor sub-categories: Ordinary chondrites, Carbonaceous Chondrites,
Enstatite Chondrites, Rumuruti Chondrites, Achondrites, Primitive
Achondrites
• Star sub-categories: Blue giant, Red giant, Binary, Triple, Red dwarf, White
dwarf, Neutron
21. The Universe Game: rewards and motivations: in-
game bonuses and upgrades, RDF console
22. The Universe Game: Out in the wild…
• Submitted Universe Game to Kongregate game portal late April 2011 – a
site not known for restrained comments...
• Around 900 plays in a period of 4 – 5 hours
• User comments: purpose of annotations were confusing (this was prior to
two-tier approach) and users were perplexed that if they tagged ‘Saturn’
as a ‘Galaxy’ they could still get it ‘right’.
• Typical comment: “needs more levels. needs upgrades”
• Some users did almost get it though: “rosou, its like "battleships", you
gotta destroy enemy's hidden ships (the red dots) before he destroys yours.
you cant attack the other player directly. You win by probing the green
spots and getting the questions right.”
• In 5 out of the top 9 responses the idea of getting the question ‘right’ was
deemed to be important enough to mention prominently.
23. The Universe Game: Conclusions
• Hard core gamers such as those who frequent Kongregate are going to be
a very tough audience to please (not a surprise)
• However, even the most critical feedback gave clues as to how to develop
the game. Most of this feedback has now been integrated into version 2.
• Games require and provide simplistic, definitive outcomes – which is
perhaps why they are popular alternatives to ‘real-life’
• Activities such as Moon Explorer which are presented in an entirely
different way to a game and which offer entirely different rewards provoke
a markedly different response from players – even though the underlying
purpose is the same.
• Universe Game version 2 is almost at the point where we’re ready to
upload it to games portals for comment. Watch this space...