Alphabet Soup Cans: Avoiding Bad Tropes of Educational Games

I am Stuart Criley, of Indelible Learning.  Thank you for coming today.  Jasminka Criley, my
co‐presenter, was unfortunately unable to attend, because she had clinical duties in the
hospital that prevented her from traveling to be here today.
We want to acknowledge the generous support of the National Institute of General Medical
Sciences, of the National Institutes of Health, who recently funded our project to build a
learning game on fostering sleep health and interest in future careers in health and STEM.
While this sleep mystery game is still in development, and I will be talking more about
another game that we have developed on presidential elections, it is relevant to this talk,
because we are in the midst of this very struggle to avoid bad tropes in educational games,
as we all must be when building a new learning game.
1
7/10/2019
©2019 Indelible Learning, Inc.
Alphabet Soup Cans: Avoiding Bad Tropes of
Educational Games.  Presented at the 2019
Serious Play Conference, Montreal, Quebec.

Graeme Devine, if you don’t know him, has been a pioneer for computer games. He
worked on one of the first CD‐ROM games in the early ‘90s, called 7th Guest.  He also
worked on Quake, and Doom, and is now with Magic Leap.  When I submitted my abstract
to this conference, and then learned it was accepted, I had no qualms about poking fun at
awkward game mechanics, including one in his game.  But then I ran into him in the Games
for Change conference earlier this summer, and felt remorse.  I asked him about the
inspiration for the game puzzles, and if it was true that they were resurrected 19th Century
puzzles and anagrams.  He said they were, and gave credit to Matthew Costello, who had a
vast collection of them.  For example, the maze in the carpet in one of the rooms was
based on a garden hedge maze in England, which they visited.  None of the puzzles were
invented for the game.  They all came from somewhere.  In ending, he said not to worry.
“All criticism is a form of love,” which I took sincerely.  And I mean it sincerely here. For
every bad learning game out there, it took just as many people, and just as many hours of
work, as it took to make a good game.  There are hardworking people behind these games,
and I want to remind you of that.
2
7/10/2019

Let’s define what we mean by Sorting Soup Cans.  It is a trope that describes any game
puzzle that sticks out like a sore thumb.  It could be a chessboard in the middle of the
room.  You are unable to cross the room until you solve the puzzle at the chess board.  In
real life, this would make no sense. But even in the game you are playing, this makes no
sense.  The puzzle not only doesn’t fit thematically into the game, but it’s not clear that it
should even be an obstacle to further progress in the game.
3
7/10/2019

The origin of this trope, is, as a matter of fact, the 7th Guest.  In the game, a house is
haunted by an evil spirit, who previously has killed six people who were invited inside.  You
are the latest guest. As you work your way through the house, it is clear that this spirit is
trying to kill you too.  So, one of the deadly traps you encounter is in the kitchen pantry,
where you must unscramble the soup cans to spell out a message.  A critic remarked,
“What evil genius would try to kill me with sorting soup cans?”
4
7/10/2019

[Video clip of the sorting soup cans scene in the 7th Guest.]
5
7/10/2019

Here is the anagram and its solution.  It spells out a message that doesn’t mean anything to
the story in the game, but it is a clever puzzle.  But does it belong in the game?
6
7/10/2019

When a chore like this appears in an learning game, it’s called “Alphabet Soup Cans.” And it
occurs more often than you think.
7
7/10/2019

[Video clip] In Math Blaster, you are an intergalactic space traveler, who flies a spaceship…
by doing arithmetic problems.  And the reward?  If you get enough 2+5 problems correct,
you get to collect garbage.  It sounds very much like what a child would be told to do at
home: first do your homework, then take out the trash.  But in this case, it’s …in space!!!
8
7/10/2019

I M Meen is an early learning game that advertises a 3D adventure, when three‐
dimensional graphics were still somewhat novel.
9
7/10/2019

[Video clip of I M Meen] In this game, you fight giant spiders in a dungeon with your bare
fists. Until you reach a door, at which point you are told to correct the punctuation on the
written scrolls in order to unlock the next level. Punctuation?
10
7/10/2019

This is my favorite of the three: released to a Japanese audience, this game has zombies
attack you who speak English. To kill them, you must pick the correct English word or
phrase.
11
7/10/2019

[Video clip of English of the Dead] The irony of the sentences that appear in the game, “The
sun rises in the East,” or “I’ll let her do my homework,” are in stark contrast with the
legions of undead that are chasing after you. I don’t know if the authors had this stark
contrast in mind, but to me, this makes the game very funny.
12
7/10/2019

While these are obviously bad learning games, the important lesson here is to recognize
that it could happen to you.  These groups did not set out to make a bad game.  But it
happened just the same.  One real hazard of developing learning games is that the false
positive rate is so high.  Students, who are a captive audience, will gladly play even a very
bad game if you let them.  Because the alternative, listening to their teacher, is often so
very much worse.  Kids will happily play away on a Chromebook that has no internet
connection, sending their little dinosaur hopping across obstacles, if you let them.  That is a
terrible benchmark for a learning game.  Don’t use this as your benchmark for success.
13
7/10/2019

The antidote for this problem isn’t something that you can shortcut: you cannot get away
with only a superficial understanding of the topic.  You must know it deeply.  You must
know it so well, that you also understand keenly the exact points where students struggle
to learn it.
Finally, to keep the game thematically sound, it is a conservative, and reasonable plan to
give the player an authentic problem to work on.  Setting the game in a dungeon, or in
space, or within a zombie apocalypse is riskier than you think. Setting the game in an actual
workplace, with professionals who must make a consequential decision, is more engaging
to students than you might imagine.
14
7/10/2019

Time in a classroom is precious. If your game is taking away instruction time from the
teacher, your game must be worth it.  This is not a price that you can afford to take lightly.
Teachers that let you into the classroom are giving you time that they often don’t have,
because they are behind on their timeline, with all the educational milestones that they are
required to hit.
Please, do not waste their time.
For this reason, ask yourself: does the learning objective I am planning to target even
belong in a game?  Sometimes the answer is no.  Think hard about what you can do, that
would be better than what is already being done in the classroom.  Simply automating
memorization is a bad use of learning games.  And will very often lead to bad games being
created.
15
7/10/2019

Dan Meyer, a former middle‐school math teacher, now Chief Academic Officer at Desmos,
used this way to describe engagement. And this level of engagement is helpful to think
about when making your learning games…
16
7/10/2019

If students are bored, there is no desire to learn, and their belief in their own power to
solve the problem is not even an issue.
17
7/10/2019

Confused is bad too: your students want to know, but for whatever reason, they believe
that it is beyond their power to learn it. But don’t overlook the important fact: you almost
have them: they want to know. But something you did, or something about the way the
problem is presented, has convinced them not to jump in and try it.
18
7/10/2019

Perplexity is where you want to be: the students want to know, and they know in their
bones that it is within their power to know it.  If you hook them here, they will walk
through fire to get to the answer.  Always aim for perplexity.
19
7/10/2019

One way to kill the desire to learn is to give student the answer too soon.  Dan Finkel, a
mathematician, who created a game called Prime Climb, says “Be less helpful.” Do not be
the answer key.  Let your students wrestle with the problem.
Do you notice a pattern here?
20
7/10/2019

Perhaps it would help if I showed a few more numbers. What do you see?  Some of you
were quick to point out that the colors represent factors of numbers.  Both composite
numbers (numbers with multiple colors in a ring around them) as well as prime numbers (a
single, unbroken color) are shown here.  How much worse would it be, if you simply gave
the definitions of prime and composite numbers?  You would be robbing your students of
the joy of discovery.
21
7/10/2019

Dragon Box is perhaps the perfect learning game. It embeds the rules of algebra into a
graphical game with a simple goal: the Dragon Box wants to be alone.  Mathematically, this
means solve for x, placing everything on the other side of the equals sign, or in this case,
the wall running down the middle of the screen.  Here the game illustrates the Properties
of Negation:  double‐clicking on a tile will cause it to become negative.  In this case,
negative 1 becomes 1, which causes the dragon box next to it to become positive, as well
as the term on the other side of the wall.
22
7/10/2019

23
7/10/2019

My sons were 10 when I discovered this game. They played it straight through, having no
idea that they were learning algebra. In fact, that is the test of good engagement: are you
learning something without knowing that you’re learning?
24
7/10/2019

There may be more reasons than I have listed here, as to why a learning game goes bad.
An example of executive meddling when the client, or the sponsor, has a fixed idea about
what the game should be about, overruling the development team.  Since they are paying
for the project, they obviously have enormous influence over its direction, if they care to
exercise it.  However, your game will not rescue a bad educational standard, or a bad idea.
Technology in search of a mission is another way of saying, when all you have is a hammer,
the rest of the world looks like a nail.  Each new technology is advertised as a panacea: for
a time it was 3D graphics, or multimedia, or more recently, VR. These technologies do have
their best applications, where they succeed like no other; but they also have a whole lot of
bad applications that time will reveal.  Be wary of new technology, only because new
technology is immature, and the world has not yet discovered where it works, and where it
does not.
Finally, the disconnect between content experts and the development team is a big reason
for failure.  It gets back to what I emphasized earlier: you need to really, really know your
content.  But content experts may know a lot about their field, but very little about games.
On the other hand, game developers may know a lot about games, but next to nothing
about the content expert’s field of work.
The last reason, disconnect between the content experts, and the game development
team, is perhaps the most challenging.  I am experiencing this as we speak.  In the past,
when we were developing medical education software, we were both the content experts
25
7/10/2019

and the developers.  But more recently, we are developing a game on sleep health with a
development team, and it has been a challenge to get the dev team up to speed on
something that we have been living and breathing for the last 3 or 4 years.  And I am sure it
has been a challenge for them to get us up to speed on what works in a game, and what
doesn’t.
7/10/2019
©2019 Indelible Learning, Inc. 25

In the ideal world, the two teams have sufficient overlap that they can come to a common
understanding, and the learning game they build together can succeed.  But the opposite is
also possible: there can be zero overlap in knowledge between the teams.  And that spells
trouble.
26
7/10/2019

Carson City, Springfield, Albany, Tallahassee, Augusta, Salem… how often will you encounter
these cities in your daily lives?
AZ, FL, CA, TX, MO, MA, MI… it is important to know which states are which; mixing them
up can have real consequences.
So…. guess which one is a state educational standard, and which one is not:
Bonus points for explaining who writes the educational standards, and where they live and
work much of the time.
By the way, this is a prime example of executive meddling.
27
7/10/2019

Here is a better learning objective. But it didn’t start out this way.
28
7/10/2019

This is what we were given instead.  I tried very hard to imagine how this activity could be
worth the loss of instructional time with a teacher.  I thought of what the kids would be
doing, and what they would learn. I realized that it would be a perfect transfer of
knowledge.  Because, in truth, this is how many adults behave when they vote in the
United States.  If that is what we want to teach future voters to do, then this is how to do it.
29
7/10/2019

So, back to the question: how do Americans elect their president?
30
7/10/2019

There’s a strong periodicity to this search term.  Care to guess when it peaks?  The day after
a presidential election.  What’s more, the height of the peak is inversely proportional to the
margin of victory.  That is to say, the closer the race in the popular vote, the greater the
interest in the Electoral College, which comes under increasing scrutiny when the race is
close.  People searching the web seem to be asking, “Will the Electoral Vote agree with the
popular vote this time around?” And in 2016, the race was as close as 2004, but as we all
know very well, the candidate who lost the popular vote ended up winning the Electoral
College, and the presidency.
31
7/10/2019

Well, that seems like a dandy learning objective.  How do we go about teaching it?  How
can it become a learning game?
32
7/10/2019

This civics exercise is a hands‐on activity that puts students in the role of a professional: a
campaign manager in a presidential election.  It is late September, with only a few weeks
left before Election Day.
• Some states are already solidly red, and will vote for the Republican candidate.
• Likewise, some states are already solidly blue, and will vote for the Democrat.
• It is the states that are still up for grabs that will determine victory or defeat.
To win, the campaigns must make difficult choices where to place their limited resources
on the states that are still up for grabs (the white states on the map).
• They must make these choices without knowing what the other side is doing
with their own campaign resources.
• Furthermore, once they have their plan in place, they cannot change it.
• And, just as in a real election, they won’t know if they were successful until the
votes roll in from the various states on election night.
33©2019 Indelible Learning, Inc.
7/10/2019

7/10/2019

Two game boards serve as campaign maps for the two opposing sides. Because the maps
are double sided, make sure that the election years that are face up match!
7/10/2019

Place the red and blue pieces as shown on the map; this way you can easily identify if any
pieces are missing.
7/10/2019

The partition allows both sides to set up their pieces in secret. You will remove the
partition once both sides are ready to play.
7/10/2019

You will need to supply a Chrome Book or tablet that can see the Internet. The digital
device will keep score during the game.
7/10/2019

Let’s go over how to set up the game, and the rules for playing it:
You will need:
• A pair of game maps (one for each side: red vs. blue)
• A partition that you set up between the two maps, hiding them from the other side
• 24 red pieces, and 5 red dice
• 24 blue pieces and 5 blue dice
• A Chromebook, or other digital device that can connect to the Internet, to keep score
automatically
• An instruction sheet, just in case you need to consult them during the game.
7/10/2019

40
7/10/2019

41
7/10/2019

Election day is a few weeks away.  But voters in most states are leaning for one candidate
or the other so strongly, that we are already pretty certain how these states will go.  The
red states will go for the Republican candidate.  The blue states will go for the Democratic
candidate.  Just looking at the map, it looks like there are a lot more red states than blue
states.  Red looks clearly ahead!  But the presidential election doesn’t work that way: the
size of the state doesn’t matter at all.  It’s the number of Electoral Votes in each state that
matters.  These Electoral Votes are based on the number of members of Congress that
each state has.*
If you add up all the red states, you get 188 Electoral Votes.  Adding up the blue states, you
get 191 votes.  The race is practically tied!
The white states on the map are still up for grabs.  Each candidate needs to win enough of
them to reach 270 Electoral Votes.  These are the states that you will be fighting for all the
way to Election Day.
_______________
*Except for Washington, DC, which has no voting members of Congress, but does have 3
Electoral Votes.
7/10/2019

Game Setup:
Teams first put up a partition, or privacy screen, between the maps.
This lets them place their game pieces on their map in secret.
There are 12 white states in the 2020 election, but only 24 game pieces.
Place your pieces on the white states only. You can’t win states that the other side already
has.
These pieces represent campaign resources (time, people, money) that are limited in every
campaign.
Pay attention to the Electoral Votes for each state: some states are worth a lot more than
others.
The more pieces you put down, the more chances you have to win that state.
You can place up to 5 pieces maximum on a single state.
But you don’t have to place any!
You may want to skip some states to save your pieces for states that you really need to win.
7/10/2019

In this case, the board game, and the game that campaign strategists play for real, are in
fact the same game.  Both students and campaigns work with the same maps, have the
same goals, and follow the same rules for victory.  In the board game, dice rolls represent
uncertainty in voter intent, because pre‐election polling is imperfect, and adding resources
to a particular state will increase one’s chances of winning it, but will not guarantee it.
44
7/10/2019

The game play data generated can be used in the classroom for group analysis and
discussion.  It turns out, to understand the nuts and bolts of the Electoral College, you need
to start with middle‐school math standards of statistics and probability. So here is a math
standard: representation of data, that is also the user interface.  For the completely online
version of the game, we are building a handy interface to add and remove campaign
resources for each state.  Migrating the player from placing pieces onto the map to using
the UI dot plot, is not only a convenient mechanism for playing the game on smart phones,
it is also using math standards of dot plots and bar charts.
45
7/10/2019

46
7/10/2019

Here the user adds and removes game from each state by using this interface: but the
interface itself is a dot plot—a middle‐school math standard.
47
7/10/2019

Here the dots have disappeared, and what remains is a bar graph. Perhaps its not properly
a histogram, but its close: a representation of the distribution of resources across one
dimension: the number of Electoral Votes for each state.
48
7/10/2019

The game play data, for games won and lost in played in the same classroom, can be
displayed and discussed.  The first‐order strategy appears to be: place more resources on
states with more Electoral Votes.  But look closer:
Each election year is different, as these two histograms show for 2016 and 1960.  It takes
two very different strategies to win them.  In 2016, there were only 12 swing states, which
made it easier to place heavier bets on the big value states. In 1960, however, there were
20 states up for grabs, which forced players into a much flatter strategy to cover all states in
play, or into tough choices of which states to abandon.  The stars mark two states that do
not follow the trend line. In both cases, players are likely to overlook these states because
they are smaller than their neighbors.
49
7/10/2019

Moreover, how can you be sure that you won because you had a superior strategy, or just
because you were lucky?  It starts with being able to recognize the strategy your opponent
used against you.  Displays like these of resource placement are an important abstraction.
When players learn to recognize resource placement in this way, their thinking becomes
more efficient.
50
7/10/2019

To really get at the answer: “Was I lucky, or was I good?” Players can set their strategy
against an opponent played by the computer, and then run a million simulated games.  The
display then calculates Expected Values for each state battle, and then sums up the entire
strategy.  This is a powerful way of answering whether luck or smartness had anything to
do with it.
51
7/10/2019

In summary, I wanted you to be aware that good learning games may sometimes come
about by accident, but more often they are the result of relentless focus on the content,
with a careful marriage of good game mechanics to that content.
Being less helpful as a teacher can also be another way of saying, to make a good learning
game, or a good game in general, you should strive to make a game that is easy to start, but
hard to master. Invite your players with a perplexing problem, to paraphrase Dan Meyer, so
that they jump right in with both feet.
And resist the temptation to do the easy thing: don’t build a crossword puzzle or a word
search. If you do, you will find you’ve landed in Alphabet Soup.
52
7/10/2019

Election Lab, the learning game about the Electoral College, can be found at electionlab.org
End of Imperial is the working title for our NIGMS‐sponsored sleep mystery game; more
information will be coming soon at endofimperial.com
Finally, indeliblelearning.com is our company website.
53
7/10/2019

Alphabet Soup Cans: Avoiding Bad Tropes of Educational Games

Recommended

Recommended

More Related Content

More from SeriousGamesAssoc

More from SeriousGamesAssoc (20)

Recently uploaded

Recently uploaded (20)

Alphabet Soup Cans: Avoiding Bad Tropes of Educational Games