1. Lecture 8
Hybrid Architecture
for Games
15 March 2010
1

2. single
completely
centralized
decentralized
server
2

3. P2P with
Central Arbiter
3

4. A trusted central arbiter receives all game
traffic and resolves conflicts if necessary.
4

5. Benefits of
P2P + Arbiter
5

6. low latency among players
6

7. have trusted authority now
7

8. Drawbacks of
P2P + Arbiter
8

9. limited scalability
arbiter becomes bottleneck
9

10. Mirrored Servers
Architecture
10

11. Multiple servers, each replicates the complete game
states and serves clients from different geographical
regions. Each client connects to one server.
11

12. Servers may be connected with high speed,
provisioned networked, reducing synchronization
latency among the servers.
12

13. Updates from a client are sent to its server, which then
forwards it to other servers. The servers then forward
the updates to all relevant clients.
13

14. P2P-style state synchronization
among servers is easier, since
1. servers can be trusted
2. clocks can be synchronized
3. IP multicast may be used
4. higher availability
5. shorter delay
14

15. Benefits of
Mirrored Servers
15

16. no single point of failures
A client can use another replicated server
if their nearest server fails.
16

17. increase total network capacity
good if networking bandwidth is the
bottleneck
17

18. lower latency between
client and server
18

19. Drawbacks of
Mirrored Servers
19

20. latency between players increases
due to additional forwarding
20

21. cannot scale to large world
each server still keeps the complete states
21

22. Q: How to assign
players to servers?
22

23. Assigning players to closest servers may lead to
overloaded servers.
23

24. Assigning players to remote servers may increase
end-to-end latency.
24

25. But we only care about latency between players
that are relevant to each other.
25

26. Zoned Servers
Architecture
26

27. Divide the game world into independent regions.
One server is in-charged of one region.
27

28. A client connects to the server serving its current
region, and is handed-off to another server when it
moves to another region.
28

29. Servers seldom communicate with each other.
29

30. Updates from a client are sent to its server, which then
forwards it to all relevant clients in the same region.
30

31. 31

32. Advantages of
Zoned Servers
32

33. Can scale to large world
Just add more servers
33

34. Drawbacks of
Zoned Servers
34

35. Single point of failures
(But can deploy mirrors for each region)
35

36. Constrained Game Designs
36

37. Supporting Seamless
Game Worlds
37

38. Divide the game world into regions.
One server is in-charged of one region.
38

39. Player may move around the world seamlessly
39

40. A client may see events and objects
from neighboring regions.
40

41. Events occurring in one region may affect
objects in another region.
41

42. Advantage: Partitioning is transparent
to the players. No artificial constraints in
game design.
42

43. Advantage: We can now resize the
regions to load-balance among the servers,
improving server utilization.
43

44. A client connects to the server serving its current
region, and is handed-off to another server when it
moves to another region.
44

45. Server communicates with each other to transfer
states, update states etc.
45

46. Updates from a client are sent to its server, which then
forwards it to all relevant clients in the same region or
neighboring regions.
46

47. or
47

48. A client connects to the servers serving its current
region and regions it subscribes to. Updates from
neighboring server are sent directly.
48

49. Q: How to partition
the game world?
49

50. Ideally: each server
should handle the same
number of players
50

51. Ideally: minimize
communications
between servers
51

52. Ideally: minimize the
number of hand offs
52

53. Generic Gaming
Proxies
53

54. Proxy: trusted host
providing specialized
services for games
54

55. Proxy: typically close
to the players
55

56. Examples of Proxy Services:
Time-stamping
Message Ordering
Interest Management
56

57. Example 1:
Time-stamping Services
57

58. Timestamp Cheat in
P2P Games
58

59. Player generates fake, earlier timestamp to gain
advantage after knowing the opponent's move.
(m1,t1)
(m2,t0)
59

60. But such cheat is impossible if a trusted proxy
timestamp the messages instead of the players.
m1
Proxy
(m1,t1)
(m2,t2)
Proxy
m2
60

61. The proxy can digitally signed the message to proof
that he is the one that added the timestamp.
m1
Proxy
(m1,t1,s)
(m2,t2,s)
Proxy
m2
61

62. Timestamp Servers is
generic and is game
independent
62

63. Example 2:
Message Ordering
63

64. Client/Server
Architecture
64

65. Game is divided into rounds based on updates
from the server.
65

66. Messages (action) from players are grouped into
rounds. Reaction time is the time between receiving
message from server and sending an action.
Reaction Time
66

67. We want to deliver messages
to the server in the following
order:
67

68. 1. for a player, messages are
delivered in the order they
are generated.
68

69. 69

70. 2. messages from the same
round from different players
are delivered in increasing
order of “reaction time”.
70

71. Messages are not delivered according to reaction time.
71

72. 3. messages in a round is
delivered to the server
before messages from the
subsequence rounds.
72

73. The second green message from the top player
violates this condition.
73

74. Server
Server Proxy
Player Proxies
Players
74

75. The server proxy tags each message (“updates”)
from the server with the round number.
Ui
(Ui, i) (Ui, i)
75

76. The player proxy remembers Ri, the time
message Ui is received, and forwards it to players.
(Ui, i) (Ui, i)
Ri
76

77. Action messages from the players are tagged by the
player proxies with (i) round number i (ii) reaction time tij
and (iii) order of action messages from that player.
Aij, i, tij, k
Aij
Pj
77

78. Example of tags inserted by the proxy.
Proxy
k=1 k=2 k=1 k=1
Proxy t12 t22 t32
P2
78

79. The server proxy now determines two things:
(i) in what order to deliver the messages, and
(ii) when to delivery the messages
Aij’, i’, tij’, k’ Aij, i, tij, k
Pj
79

80. Delivery order: sort the
queue according to round
number, then by reaction
time. Deliver messages in
the order inside the queue.
80

81. Delivery time: can
deliver the head of the
queue as long as no other
messages from the same
round is still in transit.
81

82. Receive Time = RTTj + tij + Si
Player tij
Proxy j
Server Si RTTj + tij + Si
Proxy
Player
Proxy
82

83. Message from k within the same
round i should be sent first if tik < tij
RTTj + tij + Si
83

84. If message from k is still in transit when
message from j is received, then
RTTj + tij + Si < RTTk + tik + Si
< RTTk + tij + Si
84

85. If message from k is still in transit when
message from j is received, then
RTTj + tij + Si < RTTk + tik + Si
< RTTk + tij + Si
If message from j is delivered
at this time, then message
from k should have arrived
85

86. It is safe to set the delivery
time of a message from j to be
max {RTTk} + tij + Si
k in the set of hosts
whose messages
have not arrived
86

87. Example: Consider a given round i. Si = 0
ti1 = 1 ti2 = 2 ti3 = 3
RTT1 = 10 RTT2 = 20 RTT3 = 10
At time 11, message from Player 1 is received.
Delivery time for message 1 = 21
1
87

88. Example: Consider a given round i. Si = 0
ti1 = 1 ti2 = 2 ti3 = 3
RTT1 = 10 RTT2 = 20 RTT3 = 10
At time 13, message from Player 3 is received.
Delivery time for message 3 = 23
3 1
88

89. Example: Consider a given round i. Si = 0
ti1 = 1 ti2 = 2 ti3 = 3
RTT1 = 10 RTT2 = 20 RTT3 = 10
At time 21, message from Player 1 is delivered
3
89

90. Example: Consider a given round i. Si = 0
ti1 = 1 ti2 = 2 ti3 = 3
RTT1 = 10 RTT2 = 20 RTT3 = 10
At time 22, message from Player 2 is received.
Delivery time for message 2 = 12. The message
is sent right away. At time 23, message from
Player 3 is delivered.
3 2
(We can actually delivery message 3 earlier as an
optimization)
90

91. When a message is inserted into the queue, the delivery
time of messages ahead in the queue may shorten, while
the delivery time messages behind remains unchanged.
max {RTTk} + tij + Si >k inmax {RTTk} + tij + Si
set of hosts
k in set of hosts
whose messages
whose messages
have not arrived
have not arrived
now
91

92. Consider the case where messages
from previous round is still in transit,
the delivery time should be set as
max {Ti-1, max {RTTk} + tij + Si}
k = set of hosts
whose messages
have not arrived
where Ti-1 is the time where maximum
delivery time from previous round
92

93. What if a message is late?
Then it is delivered
immediately to the server
93

94. Advantages: No need to
synchronize clock or
estimate one way delay.
Estimating RTT is easier.
94

95. Server
Server Proxy
Player Proxies
Players
95

96. Examples of Proxy Services:
Time-stamping
Message Ordering
Interest Management
96

97. Example 3:
Interest Management
97

98. Regions 1 2 3 4
Servers
Proxies
Players
Regions 1 1 2 3 2 4 3
98

99. Regions 1 2 3 4
Servers
Proxies
Players
Regions 1 1 2 3 2 4 3
99

100. Regions 1 2 3 4
Servers
Proxies
Players
Regions 1 1 23 3 2 4 3
100

101. Advantages:
Reduced network cost at server
No connect/disconnect at clients
when player moves/region migrates
101

102. P2P + arbiter
mirrored servers
zoned servers
servers + proxies
102