The document discusses opinionated containers and containerizing game server applications. Containerizing simple applications like TeamSpeak is straightforward, but complex games like Team Fortress 2 pose unique challenges due to requirements for specific configuration files, port mappings, and handling of persisted state. While containerizing opinionated applications is becoming more common, the game server ecosystem presents difficulties around proprietary assets, obscured documentation, and hacker behavior that promote inflexible environments. Standardizing configuration methods and allowing external persisted state could help containerized game servers become more compatible and deployable at scale.
3. 3
Overview
Opinionated Applications
• Background
• Containerizing Applications
• Containerizing a Simple Application
• Containerizing a Complex Application
• The Future
4. 4
• IPad POS Software and Hardware
• Web based reporting and customization
• 24/7 Support
Background
5. Background
5
• Founded in 2008
• 180 employees
• 4 Development teams + Devops
• Rails, Objective C, Golang
• 11k active registers
• 100k transaction per hour at peak
• 700k transactions per day
6. Background
6
Redesign
Microservices
• Transaction Service (Riak)
• Reporting Service
• Authentication Service
• API Service
• Etc..
Docker Stack
• Apps
• Docker CI Pipeline
• Routing
• Monitoring
PB over RPC
8. Containerizing Applications
8
Best Practices
Based on Dockerfile best practices
Applicable for in-house/black box applications
Assumes working Dockerfile
Mostly common sense
9. Containerizing Applications
9
Best Practices
1. File-less configuration
• Load configuration from environment
• Use Fracker to write environment file
• Pass environment file to container
• Otherwise use confd
• Avoid chef
• Disconnects container and application configuration
10. Containerizing Applications
10
Best Practices
1. File-less configuration
2. Separate tasks
• Container = 1 instance of 1 application task
• Separate web heads, crons, job runners
• Makes resource allocation simpler
11. Containerizing Applications
3. Log to sink
11
Best Practices
1. File-less configuration
2. Separate tasks
• Assume container is ephemeral
• Log to stdout
• Log to mounted volume
12. Containerizing Applications
12
Best Practices
1. File-less configuration
2. Separate tasks
• Assume container is disposable
• Externalize persisted state
• External db or mounted volume
3. Log to sink
4. Separate datastores
13. Containerizing Applications
1. File-less configuration
13
Best Practices
4. Separate datastores
5. Externalize data extraction
• Use application methods if possible
• Dump to simple sink
• Don’t juggle multiple tasks
2. Separate tasks
3. Log to sink
14. Containerizing Applications
1. File-less configuration
14
Best Practices
4. Separate datastores
5. Externalize data extraction
6. Don’t daemonize
• Control process via container
• Send signals with docker
• Control through custom ports
2. Separate tasks
3. Log to sink
15. Containerizing Applications
15
Why Game Servers?
• Each server is unique
• Poor existing automation
• Poor open source tooling
• Daniel Gibbs
• Poor isolation
• Why not?
17. Simple Application
17
Simple Application
Teamspeak
Gaming Infrastructure
Black box, client-server
Clients negotiate state changes
Server dispatches voice streams
Simple setup - download and run
18. Simple Application
18
Best Practices
1. File-less configuration
• Requires configuration files
• Use confd
19. Simple Application
19
Best Practices
1. File-less configuration
2. Separate tasks
• Single Task
• No background Jobs
20. Simple Application
3. Log to sink
20
Best Practices
1. File-less configuration
2. Separate tasks
• Configure log sink
• Mount volume to host
21. 21
Best Practices
4. Separate datastores
• Mount volume for sqlite DB
• Use external DB
• Write db config file
Simple Application
1. File-less configuration
2. Separate tasks
3. Log to sink
22. 22
Best Practices
4. Separate datastores
5. Externalize data extraction
• Can parse docker logs
• Or mount logging volume
• Use secondary container to parse token
• Use MachineOf
Simple Application
1. File-less configuration
2. Separate tasks
3. Log to sink
23. 23
Best Practices
4. Separate datastores
5. Externalize data extraction
6. Don’t daemonize
• Run in foreground
Simple Application
1. File-less configuration
2. Separate tasks
3. Log to sink
24. 24
Results
Simple Application
• No state within container
• If host dies logs are lost
• Single DB per container
• Connect using mapped port
• Runs on fleet
26. Complex Application
26
Complex Application
Team Fortress 2
Game Server
Black box, client-server
Clients negotiate state changes
Server dispatches voice streams
Server dispatches state changes
29. Complex Application
29
Best Practices
1. File-less configuration
• Requires configuration files
• Use confd
30. Complex Application
30
Best Practices
1. File-less configuration
2. Separate tasks
• Single Task
• No background Jobs
31. Complex Application
3. Log to sink
31
Best Practices
1. File-less configuration
2. Separate tasks
• Configure log sink
• Mount volume to host
32. 32
Best Practices
4. Separate datastores
• No persisted state
• External DB where mods need it
Complex Application
1. File-less configuration
2. Separate tasks
3. Log to sink
33. 33
Best Practices
4. Separate datastores
5. Externalize data extraction
• Don’t need any data
• Monitoring
Complex Application
1. File-less configuration
2. Separate tasks
3. Log to sink
34. 34
Best Practices
4. Separate datastores
5. Externalize data extraction
6. Don’t daemonize
• Run in foreground
Complex Application
1. File-less configuration
2. Separate tasks
3. Log to sink
35. 35
Results
Complex Application
No state within container.
If host dies logs are lost.
Connect using mapped port.
8Gb is large for an image.
36. 36
Port Mapping
Complex Application
Source servers report ip and port.
Steam master list references static ports.
Favouriting references static port.
Direct connection works.
37. Complex Application
37
Port Remapping
Remap the traffic to the correct port
• External remapper (inbound proxy)
• Cluster/Host-level dynamic remapper
• Container-level dynamic remapper
• Static port bindings - requires port allocator
We can’t control the return journey
38. 38
Port Remapping
Host
49168 49170
Container
49168
Complex Application
Tf2
49168
Client/Master
Server
HostIP:49168
Container
27015
Proxy
27015 -> HostIP:49170
HostIP:49168
Tf2 becomes dependent upon Proxy
39. 39
Host
49168
Container
27015
Complex Application
Tf2
49168
Client/Master
Server
HostIP:49168
Port Mapping
HostIP:49168 Remapper
27015 -> 49168
Fewer containers, but multipurpose
40. Complex Application
40
Tcp/Udp Mapping
Source Engine Server binds to both tcp and udp.
Can’t separate port bindings.
Docker may map to different port numbers.
Open PR to allow EXPOSE 12345/tcpudp
41. Complex Application
41
Favouriting
A favourite is an IP + Port.
Invalidated by container changing Host/Port.
Remapping containers allow port changes.
No simple solution for host changes.
43. Complex Application
43
Game Server Ecosystem
Persisted state is critical.
Many servers are windows only.
Some assets/code are proprietary.
Limited configuration options.
Hackers promote constrictive environment.
Obscured docs, varied user base.
44. The Future
44
The future of game servers
Apply lessons from services to increase compatibility
• Game Server Containers are becoming
commonplace
• Standardize configuration methods
• Allow persisted state to be externalized
• Dedicated linux
• Support clustering
45. Summary
45
Summary
• Containerizing opinionated apps is easy
• Deploying an opinionated container may not be
• Container deployment will become increasingly relevant
• Container compatibility will become increasingly defining
• Game servers have a lot of work to do
how I got into what I do
talk about game servers & personal projects
Pause between sections
customer outcomes focused
tight coupling
Container crazy
mention health checks
Reduced docker footprint
Prepare for next section (best practices)
Starting point of working dockerfile
None of this should be surprising
Container level, application generic configuration interface
Specify over defaults
Single purpose containers
* Will give example of this later
Monitoring through container
I made all these mistakes
Dont trust anything I say
Beautiful snowflakes
* contention around filesystem and network resources
* Application generic configuration interface
Not very opinionated
* extraction keeps game server specific docker file trivial
Wine
First talk about commonplace
Being a services engineer….
The opinionated nature of some applications acts as a barrier to containerization
Hopefully this talk has provided some insight into some mitigation methods