1. Handheld Augmented
Board Games
Game Interaction Storyboards

2. Settlers of Catan

3. Task Overview
• T1: Board setup at start of game
• During every player turn:
• T2: Roll dice and collect resources
• T3: Trade resources among players
• T4: Take actions and spend resources
• T5: Special case of T2 in case a 7 is rolled

4. TI - Setup
• Board setup is performed once at start of
game
• Two alternatives:
• Player-chosen lay-out (needs to be input
into the device)
• Auto-generated lay-out (needs to be
rebuilt using the physical tiles)

5. T1 - Setup (a)
First, Freddy comes up with a board lay-
out and constructs the isle of Catan using
the hexagonal tiles.

6. T1 - Setup (a)
He then places the numbered tokens on
each land tile, within the hollow frame
marker.

7. T1 - Setup (a)
Finally, he scans the marker on each tile
as well as the numbered token, to input
the board lay-out into the game engine.

8. T1 - Setup (a)
He scans the tiles according to a fixed
pattern, to map the physical board lay-out
to the digital one.

9. T1 - Setup (b)
Alternatively, Freddy can have the
handheld generate a board lay-out
(=> no scanning required).

10. T1 - Setup (b)
He then lays out the board according to
the lay-out shown on the handheld

11. T2 - Collect resources
• Performed by each player at the start of his
turn
• Depending on the result of the dice roll,
resources are distributed among all players
automatically

12. T2 - Collect resources
It’s Freddy’s turn now. To begin, he rolls
the dice.

13. T2 - Collect resources
He rolls a 3. To inform the game engine
hereof, he scans the result of his roll.

14. T2 - Collect resources
Freddy has a settlement adjacent to a
‘hills’ tile with a number 3 token on it, so
he receives 1 ‘bricks’ resource card.

15. T3 - Trade resources
• During his turn, a player may trade
resources with other players
• Another option is to trade resources with
the bank (not shown here, requires only
interaction with handheld)

16. T3 - Trade resources
Freddy is in need of ‘wool’ and has got
some ‘ore’ to spare, so he verbally
proposes a trade to the other players.

17. T3 - Trade resources
Jeffrey has agreed to trade wool for ore
with Freddy. They use their handhelds to
perform the transaction.

18. T4 - Perform actions
• Now, the player can perform actions with
his resource and development cards:
• Build a road or settlement
• Upgrade a settlement to a village
• Buy a development card
• Play a development card bought in an
earlier turn

19. T4 - Perform actions
Freddy has some resources to spend! He
starts by building a settlement.

20. T4 - Perform actions
Settlements are built at intersections of 3 hexes. He
scans the desired position to inform the handheld
(which can also tell him if he is allowed to build
there), by scanning the 3 hexes at the same time.

21. T4 - Perform actions
He then places one of his settlement playing
pieces on the chosen position. The required
resources have been paid automatically.

22. T4 - Perform actions
He remembers he bought a development
card earlier, that allows him to build a
road for free, so he plays this card.

23. T4 - Perform actions
As roads are built on edges between
hexes, he scans the two hexes adjacent to
the road position.

24. T4 - Perform actions
He then updates the game board by
placing one of his road pieces on the
chosen position.

25. T4 - Perform actions
Finally, he decides to buy another
development card.
After this, he ends his turn.

26. T5 - Move robber
• If a player rolls 7, no resources are collected,
but the robber piece is moved:
• The tile upon which the robber is placed
does not provide any resources
• Players with more than 7 cards are forced
to discard half of them
• The player that moved the robber steals 1
reosource from a player with a settlement
adjacent to the new robber position

27. T5 - Move robber
On his next turn, Freddy rolled a 7. He
moves the robber piece to a tile next to
Jeffrey.

28. T5 - Move robber
He informs the game engine of the new
robber position by scanning the robber
and its tile.

29. T5 - Move robber
Finally, he receives one of Jeffrey’s
resource cards.

30. Power Grid

31. Task Overview
• Power Grid is played in rounds, each of which
consists of 5 phases:
• Phase 1: Determine player order => automated
• Phase 2: Auction power plants => T1
• Phase 3: Buying resources => T2
• Phase 4: Building => T3
• Phase 5: Bureaucracy => T4
• Ongoing: Check player info => T5

32. T1 - Auctioning Plants
• Players buy power plants through auctions
• One can have up to 3 plants (new plants replace
older ones)
• One does not have to buy a plant each turn
• Each player can start an auction or pass
• During an auction, each player that did not
already buy a plant or pass up on the chance to
select one, can bid or pass
• Auctioned plants are automatically replaced
with new ones

33. T1 - Auctioning Plants
Freddy selects a power plant to auction,
using his handheld.

34. T1 - Auctioning Plants
He enters his starting bid.

35. T1 - Auctioning Plants
Now the other players get a chance to
bid on the plant, or pass. Bidding
continues until one player remains.

36. T2 - Buying Resources
• Players then buy resources, starting with
the weakest player
• As resources are bought, prices go up
• Players then assign the resources bought to
their power plants
• A plant can store up the twice the amount
of resources required for power
production

37. T2 - Buying Resources
Freddy buys the resources for the new
power plant he won during one of the
auctions in the previous phase.

38. T2 - Buying Resources
And he assigns them to the new power
plant.

39. T3 - Building
• Players need to connect cities to their grid
to power them
• They place buildings on the map, this
represents their grid
• The cost of a building is the base cost to
build in a city plus the cost to connect it to
the player’s grid
• The map is graph with nodes and weighted
edges

40. T3 - Building
Freddy scans the city he wants to
connect to his grid. His handheld
indicates the building costs.

41. T3 - Building
Freddy confirms the build. The costs are
paid automatically. He then places one of
his house pieces on the chosen city.

42. T4 - Bureaucracy
• Players declare how much cities they will
power and which power plants they will
use
• Income is automatically distributed among
the players
• Resources are automatically removed from
the plants and the resource market is
automatically resupplied

43. T4 - Bureaucracy
Freddy selects the plants he is going to
use to power the cities connected to his
grid.

44. T4 - Bureaucracy
Freddy automatically receives the income
generated by the powering of the cities.

45. T5 - Player info
• All information is public in Power Grid
• Players may inspect eachothers power
plants, resources and cash at any time
• This functionality can be proved as info
screens in the app
• Or one could try to show it using
augmented reality, using the handheld as
‘magic lens’

46. T5 - Player info
A possible game setup for the augmented
reality case: every player has a handheld,
player marker and house pieces.

47. T5 - Player info
By scanning another player’s marker,
Freddy can get AR information on that
player’s plants, cities, money, ...

48. Settlers of Catan: misc
• Empires of Catan:
• Settlers of Catan combining several
boardgame play instances
• Foreign trade between board gaming
groups over Internet
• Alliances of playing sides over internet
competing locally on boardgame islands
Idea source: http://www.cse.chalmers.se/research/group/
idc/ituniv/kurser/06/uc/ppt/pervasive_games.pdf

49. Settlers of Catan: Misc
• Thesis “Joining Bits and Pieces”: proposes
computer enhanced interactions in games
• Some that could be applied in Catan:
• Espionage: get (some) info on player’s
hand
• Anonymous trading: players indicate what
they want and what they’ll give =>
computer tries to match up these
offerings

50. Power Grid: Playtest
• First time is hard, since you don’t know any
tactics/strategy yet
• Bidding is an important part
• Quite some information available and
required when devising strategy => way
this will be presented to the player is a vital
aspect

51. Power Grid: Playtest
• One player needs to play bank:
• Return change to players
• Make sure the correct amount is
received
• Tedious => automation desirable

52. Power Grid: Playtest
• When a power plant is bought, a new one
has to be added to the game
• At various points, power plant cards need
to be put back in the deck; thrown out of
the game; reordered according to price
• Automating this would relieve the players
of this task

53. Power Grid: Playtest
• Players tend to connect multiple cities in
one go:
• Keeping track of current cost while
adding cities makes things easier
• Can prevent not having enough money
• Cities that can still be connected could
be shown to players

54. Power Grid: Playtest
• Players tend to plan ahead which cities they
are going to connect:
• Actions of other players can interfere
with this
• Planning function: already enter which
cities you would like to connect => if this
is still possible when it is your turn, this
plan could be executed immediately;
otherwise, the player needs to adapt it