2. Denis Izmaylov
1. Over 17 years of Software and Web
development experience
2. The last 7 years focused exclusively on
Front-end and Web: 15+ projects,
including React.js, SPA and high-load
3. Open Source contributor
4. Talks: HighLoad++ 2015, AgileDays 2016,
DevConf 2016, React Amsterdam, etc
5. Regular speaker on MoscowJS
6. Few popular articles and interviews
on Medium and Habrahabr
7. Events: Moscow Node.js Meetup, React
Moscow Meetup and this one
4. 1. We develop Web, Mobile
and DevOps projects for
our customers
2. Global international
market (EU, APAC, US, HK)
3. Most efficient and
performant technologies
4. More than 20 talents
engineers and designers
5. Remote-friendly
11. 11
Customer Goals
1. Production cluster on 3-5 machines
2. Be ready for fast growing (up to 10x)
3. Fault tolerance
4. Faster changes delivery
5. Full control for each service (in-house,
independent, no-SaaS)
6. Blue-Green (Zero Downtime) Deployment
7. Good DX (Developer Experience)
25. 25
“Microservices allow engineering teams
to move quickly to grow a product…
assuming they don’t get bogged down
by the complexity of operating a
distributed system”
33. 33
1. First Container-optimized OS: 3+ years, since July 2013
2. Mini OS: based on ChromeOS, small and fast
3. Security-focused: by using auto-updates (like
browsers)
4. No package manager: each application is running as
a Container on top of OS
5. Built-in support: Containers, Container networking,
Service discovery
6. Services: runs through systemd
7. Configuration: one Cloud Config file
42. 42
Summary
1. CoreOS and Docker: you don’t think
about OS anymore
2. Containers space
3. Now a server is just a core of your“CPU”
4. Auto-schedule: you don’t have to know
where is your Container is running
5. Configuration-less
6. Amazing expandable
52. 52
Customer Goals
Production cluster on 3-5 machines
Be ready for fast growing (up to 10x)
Fault tolerance
Faster changes delivery
Full control for each service (in-house,
independent, no-SaaS)
Blue-Green (Zero Downtime) Deployment
Good DX (Developer Experience)
55. 55
Real Life
1. Launch a new containers
2. Integrate containers between each
other
3. Connect containers with Load Balancers
4. Rolling updates for the launched
containers
5. Use Blue-Green Deployment strategy
56. 56
Simple Steps
1. Create Fleet unit file for each service
2. Each service requires Service Discovery
3. Sidekick model: create a Discovery Unit
4. Rolling updates: small set of commands
5. BGD deployment: small script
58. 58
Project Profile
1. Isomorphic React.js Application
2. RESTful API on Node.js
3. Serve Static service (nginx)
4. Golang service
5. C++ service
6. PHP service
7. JSON Configuration service
59. 59
Go Ahead
• Fleet unit for each service
• Discovery Unit for each service
• X-scaling: multiple instances
• Environments (dev, stage, prod)
60. 60
Go Ahead
• Fleet unit for each service
• Discovery Unit for each service
• X-scaling: multiple instances
• Environments (dev, stage, prod)
6
2
6
3
66. 66
Real Summary
1. CoreOS is OS only
2. Not enough for microservices
management
3. Good enough as a platform of other
tools (PaaS, SaaS, etc)
4. Configure CoreOS Cluster Step-by-Step:
https://github.com/axept/devops-open
71. 71
Kubernetes
1. Open-Source Cluster Management
2. Originally developed by Google
3. Open-sourced in June 2014
4. Deploy and manage applications
declaratively
5. Declarative: state your desired results, let
the system actuate
6. Control loops: observe, rectify, repeat
7. No groups: labels is more powerful
73. – How can we spend more time
building useful services
and less time maintaining the platform?
74. 74
Some Concepts
1. Controller: drives current state to target
state
2. Pod: a small group of tightly-integrated
Containers
3. Label: identifying metadata attached to
other objects (e.g. to Pods)
4. Service: exposes a name, port and IP for a
groups of Pods
5. Ingress: connects Load Balancers with
Services by using URL rules
75. 75
Controller
1. Run on Master
continuously
2. Each K8s object gets
its own Controller
3. Control Loop:
• get state
• calculate diff
• apply changes
4. Pluggable and lightweight
Get state
from etcd
Compare
with
worker’s
state
Actuate it
76. 76
Pods
1. Group of Containers
2. One or multiply containers
3. Has its own IP and localhost
4. Scale together
5. Shared secrets
6. Shared volumes
7. Shared resources (CPU and RAM)
8. Can be tagged with labels
77. 77
Label
1. Identifying metadata
2. Can be attached to any API
object
3. Generally represents identity
4. It could be application name,
role, phase, environment,
version, etc
5. Using by Selectors
6. The grouping mechanism only
78. 78
Example for Labels
app: demo
phase: dev
role: FE
app: demo
phase: dev
role: BE
app: demo
phase: test
role: FE
app: demo
phase: test
role: BE
79. 79
Selectors
Think about it like SQL:“select … where …”
app: demo
phase: dev
role: FE
app: demo
phase: dev
role: BE
app: demo
phase: test
role: FE
app: demo
phase: test
role: BE
app = demo
80. 80
Selectors
Think about it like SQL:“select … where …”
app: demo
phase: dev
role: FE
app: demo
phase: dev
role: BE
app: demo
phase: test
role: FE
app: demo
phase: test
role: BE
app=demo
role=FE
81. 81
Selectors
Think about it like SQL:“select … where …”
app: demo
phase: dev
role: FE
app: demo
phase: dev
role: BE
app: demo
phase: test
role: FE
app: demo
phase: test
role: BE
app=demo
role=BE
82. 82
Selectors
Think about it like SQL:“select … where …”
app: demo
phase: dev
role: FE
app: demo
phase: dev
role: BE
app: demo
phase: test
role: FE
app: demo
phase: test
role: BE
app = demo
phase = dev
83. 83
Selectors
Think about it like SQL:“select … where …”
app: demo
phase: dev
role: FE
app: demo
phase: dev
role: BE
app: demo
phase: test
role: FE
app: demo
phase: test
role: BE
app = demo
phase = test
84. 84
Services
1. Serve as a kind of
Load Balancers for Pods
2. Expose name, port and
stable IP for a group of Pods
3. IP can be internal or public
4. Can implement Service Discovery
between applications and features
to support Zero-downtime deployments
85. 85
Ingress
1. Works with Load
Balancers, Cloud LB,
nginx, etc
2. Provides a single root
URL to multiple services
3. Publicly expose private
networks
4. Can use HTTP Headers
5. Supports SSL
88. 88
In next talk
1. How to configure Concourse CI
with Kubernetes properly?
2. How to implement TLD (Top-Level Domains,
e.g. google.de, yahoo.eu, yandex.ru)?
3. How to configure persistent SSL certificates
and temporary SSL certificates by
Letsencrypt?
4. How to configure monitoring and alerting
using Grafana?
5. How to install the cluster in a one click?
90. 90
Summary
1. Kubernetes saved us from“1000 Fleet Units”
hell in CoreOS and a decades of custom
scripts
2. K8s saved us a lot of months
3. Some other interesting features:
• Role-based Access Control
• AWS, GCE, Azure support
• Apache License Version 2.0
91. 91
What We Got
1. Micro-services Management
2. Continuous Delivery / Deployment
3. Easy SSL and TLD management
4. Monitoring
5. Notifications
92. 92
Achievements
1. Production cluster on 3-5 machines
2. Be ready for fast growing (up to 10x)
3. Fault tolerance
4. Faster changes delivery
5. Full control for each service (in-house,
independent, no-SaaS)
6. Blue-Green (Zero Downtime) Deployment
7. Great DX (Developer Experience)
95. 95
One-Click Installer
1. Uses Ansible
2. Prepare Configuration directory
with keys, vars, secrets, etc
3. Optional - prepare network interfaces on target
machines
4. Configure servers list and SSH credentials
5. Launch the script
6. Get a new cluster in 3 minutes with
isolated CI and Private Docker Registry
98. 98
Effects
1. CoreOS, Kubernetes and Application became
an isolated node
2. Micro-clusters Architecture
3. Possible to get Cross-cluster Load Balancing and
Service Discovery
4. Automated Installation
5. 24/7 Support friendly
6. No“Black box”inside
7. Efficient Geo-scalability
99. 99
Take aways
1. CoreOS is not about microservices management
2. CoreOS is just Container OS (Container Linux!)
3. CoreOS is good as a tool for SaaS, PaaS, etc
4. Kubernetes is really great for microservices
management and large clusters
5. Kubernetes is the new operation system (OS)
6. Some interesting things will be in next talk
100. “Microservices allow engineering teams
to move quickly to grow a product…
assuming they don’t get bogged down
by the complexity of operating a
distributed system”
7.
101. 101
Read (1 of 2)
1. Kubernetes The Hard Way
https://github.com/kelseyhightower/kubernetes-the-
hard-way
2. Design Patterns for Container-based Distributed
Systems by Brendan Burns and David Oppenheimer,
Google
https://www.usenix.org/node/196347
3. Site Reliability Engineering: How Google Runs
Production Systems
http://shop.oreilly.com/product/0636920041528.do
102. 102
Read (2 of 2)
4. Kubernetes Cluster Federation (previously "Ubernetes")
https://github.com/kubernetes/kubernetes/blob/
release-1.5/docs/design/federated-services.md
5. From Google to the world: the Kubernetes origin story
https://cloudplatform.googleblog.com/2016/07/from-
Google-to-the-world-the-Kubernetes-origin-story.html
6. Minikube is a tool that makes it easy to run Kubernetes
locally. Minikube runs a single-node Kubernetes cluster
inside a VM on your laptop.
https://kubernetes.io/docs/getting-started-guides/
minikube/