1.
HOW TO FAIL AT
#GAMIFICATION RESEARCH
Lennart Nacke, PhD
Associate Professor
The Games Institute and
Stratford School of Interaction Design and Business
@acagamic | slideshare.net/acagamic | linkedin.com/in/nacke/
*Thanks, Henrik!

2.
OVERVIEW OF HOW TO FAIL
1. WHY FAILING IS NECESSARY
2. FAILING AT GAMIFICATION EXPERIMENTS
3. FAILING TO UNDERSTAND AND DO GAMIFICATION
4. FAILING AT WRITING GAMIFICATION PAPERS
5. SUMMARY

3.
WHY FAILING IS
NECESSARY

4.
WHAT DID GOOGLE LEARN?
• Keep expectations in check
• Be clear about your products purpose
• Do not launch before features are ready
• Offer value to customers

5.
TWITTER DEVELOPED FROM A FAILED
PODCASTING APP

6.
FROM ODEO TO TWITTER
• Only a few people interested in creating audio
• They did a design brainstorm to fix their idea of making
everyone a creator
• Writing is easier than creating audio
• Pivot to Twitter in 2006, a listening and writing tool in
140 characters

7.
SLACK EMERGED FROM A FAILED
VIDEO GAME

8.
GLITCH WAS BASED ON SOFTWARE
TOOLS THAT BECAME SLACK
• Slack used to be a game developer called Tiny Speck
• Glitch was a multiplayer game they developed
• Whimsical illustrated characters and cooperative,
nonviolent gameplay
• Of course that would never be successful ;)
• Slack used the tech Glitch used for synching
• Slackbot came from a pet rock in Glitch that explained
the game world

9.
ABRAHAM WALD AND
SURVIVORSHIP BIAS

10.
IN WHICH AREAS OF THE B-29
SHOULD WE ADD ARMOR?

11.
FAILING AT
GAMIFICATION
EXPERIMENTS

12.
DO WE FALL PRAY TO SURVIVORSHIP
BIAS IN GAMIFICATION RESEARCH?
POST HOC ERGO PROPTER HOC?

13.
WE NEED AN OPEN SCIENCE
COLLABORATION FOR
GAMIFICATION
SHARE DATA AND REPLICATE STUDIES

14.
273 EMPIRICAL GAMIFICATION
STUDIES
• What if data from all of these studies or even the review
database were shared among us?
• How to find emerging focus areas beyond
• Health
• Crowdsourcing
• Social networking
• The non-significant, non-empirical, non-successful
gamification approaches are not catalogued – we
cannot review them (Jonna, Apu?)
Koivisto, J., & Hamari, J. (2019). The rise of motivational information systems: A review of gamification
research. International Journal of Information Management, 45, 191-210.

15.
MY OWN STORY OF FAILURE
• Published BrainHex in 2014, but the analysis was flawed
• Basically a failure, even though it became highly popular and
well-cited
• Revisited the data in 2015 with my research team
• First fixing attempt in 2018:
• Gustavo Fortes Tondello, Deltcho Valtchanov, Adrian Reetz, Rina R. Wehbe, Rita Orji & Lennart E. Nacke (2018):
Towards a Trait Model of Video Game Preferences, International Journal of Human–Computer Interaction, DOI:
10.1080/10447318.2018.1461765
• Next attempt in 2019:
• Just accepted into INTERACT 2019: Tondello et al.: “I don’t fit into
a single type”: A Trait Model and Scale of Game Playing Preference

16.
FROM 7 NON-VALID TYPES TO 5 TRAITS
•7 types collapsed into:
•action orientation (Conqueror, Daredevil)
•aesthetic orientation (Socializer, Seeker)
•goal orientation (Mastermind, Achiever,
Survivor)
•Suggested additions:
•social orientation
•immersion orientation

17.
FAILING TO
UNDERSTAND AND
DO GAMIFICATION

18.
START WITH WHY
What?
How?
WHY?
Sinek’s Golden Circle

19.
Apple changing the music industry

20.
Apple catching up with Steam…

21.
WHAT DOES GAMIFICATION
RESEARCH NEED?
• We keep doing studies on points, badges, and
leaderboards – there is so much more to build with
• Let’s not turn into Apple...
• Where are our conceptual models?
• What is the purpose of our research? How do we move
past user engagement?
• Gamification for wellbeing? Improving lives?

22.
OUR GAMEFUL EXPERIENCE MODEL
• Gameful experience as fundamental state of gameplay
• Gameful systems defined by qualities of interventions
and environments that create gameful experiences
• Gameful design is the process of creating these systems
• Gamefulness is then: designer actions, system
characteristics, or the user’s psychological experience
Landers, R. N., Tondello, G. F., Kappen, D. L., Collmus, A. B., Mekler, E. D., & Nacke, L. E. (2018). Defining gameful experience as a
psychological state caused by gameplay: Replacing the term ‘Gamefulness’ with three distinct constructs. International Journal of
Human-Computer Studies.

23.
IS THERE REALLY A DARK SIDE OF GAMIFICATION?
https://darkpatterns.org

24.
WHY ARE WE NOT WORKING MORE
WITH INDUSTRY?
• There are great models out there done by gameful
designers that we can use and run studies on
• Example: Hexad (we did several studies)
• But also Gamified.UK’s 52 gamification mechanics and
elements
• Many more models out there that help us actually do
gameful designs beyond our simple implementations
• Let’s look at an example

25.
Design for Skill-Building
Your users needs and goals change over time. Know
their journey and design with the journey in mind.
26Players Journey By Amy Jo Kim, see http://amyjokim.com/blog/2014/04/08/the-players-journey/
Your users learn your systems as a visitor
first, then a newcomer, a regular, and finally
they become experts or enthusiasts.
Build a core learning loop first as a simple
version of the regular user experience you
are creating.

26.
Stage 1: Discovery
This stage is for visitors or
people who have not yet used
your service or product, here you
show your value proposition.
What do most important early
customers need to learn during
Discovery?
Amy Jo Kim. Game Thinking Explained. https://medium.com/@amyjokim/game-thinking-explained-fa6da3e8debb

27.
Stage 2: Onboarding
This stage is for newcomers, who are
using the product or service but need
to learn and get value out of it quickly.
What are the most important skills to
develop / things to learn during
Onboarding?
Amy Jo Kim. Game Thinking Explained. https://medium.com/@amyjokim/game-thinking-explained-fa6da3e8debb

28.
Stage 3: Habit-building
This is the hook or core loop in
your application that the user
keeps coming back to: a
pleasurable, repeatable activity.
What repeatable, pleasurable
activity will pull people back for
Habit-building?
Amy Jo Kim. Game Thinking Explained. https://medium.com/@amyjokim/game-thinking-explained-fa6da3e8debb

29.
FOCUS POINT: The Learning Loop
How to create habit-building experiences
• Cues and triggers
• Internal, Situational, External, Engaged
• What’s the internal trigger/urge/need that
drives someone to seek out my product?
• Repeatable pleasurable activity
• Triggered by emotion
• Internal urge or need
• In games, we call this a core mechanic (a rule
in action)
• Skill-building feedback
• Provide feedback and help players improve
• Feedback loops to help people get better at
core activity
• Progress and investment
• Show a progress and investment path
• Allow users to customize, build, collect
Amy Jo Kim. Game Thinking Explained. https://medium.com/@amyjokim/game-thinking-explained-fa6da3e8debb

30.
Stage 4: Mastery
In games, this is the late/elder game; only your
best customers experience this, building full
customization and community.
What powers, access, roles, or privileges can
they earn/unlock for achieving Mastery?
Amy Jo Kim. Game Thinking Explained. https://medium.com/@amyjokim/game-thinking-explained-fa6da3e8debb

31.
How might we address…
Put
hurdles/enablers
here
Put fitting design
lens/category
here
Put desired change here
(, using )
to achieve ?
Thanks to Sebastian Deterding for the examples.

32.
How might we address…
AVOID WORKER
ERRORS
Put fitting design
lens/category
here
WAREHOUSE SELECTION TASKS
(, using )
to achieve ?
http://www.funkydesignspaces.com/plex/

33.
How might we address…
AVOID WORKER
ERRORS
Put fitting design
lens/category
here
WAREHOUSE SELECTION TASKS
(, using )
to achieve ?
http://getmentalnotes.com/

34.
FAILING AT WRITING
GAMIFICATION
PAPERS

35.
START WITH THE END IN MIND
• Write focused on the WHY
• Consider your contribution to the field
• Read Jonna’s reviews
• Situated your work in the gamification space and be
clear about how to expand the field
• Consider reporting long-term studies
• If you can, then consider doing a randomized controlled
trial or a replication study

36.
SUMMARY
• Let’s embrace failure as scientists
• More replication of results
• Build a stronger field and stay accountable
• Rapid pivoting and better improvement similar to
industry
• Let’s get inspired by industry

37.
Questions?
Get in touch
HCI Games Group, University of
Waterloo, Ontario, Canada
Web: www.hcigames.com
Twitter: @hcigamesgroup
@acagamic
Facebook: facebook.com/hcigames
E-Mail: lennart.nacke@acm.org
Phone: (+1) 519-888-4567 x38251
BUYGamesUserResearch(OxfordUniversityPress)