Google has pioneered the usage of containers at huge scale. Learn how we designed our systems to handle insane traffic loads, orchestrating complex, globally distributed applications, and how you can leverage this infrastructure and our agile development technologies to embrace the power of DevOps and Cloud on our Google Cloud Platform.

1. Improving utilization and portability with
Containers and CI/CD on Google Cloud
Mattia Gandolfi - Google Cloud Customer Engineer
CODEMOTION MILAN - SPECIAL EDITION
10 – 11 NOVEMBER 2017

4. Confidential + Proprietary
Everything at Google runs in containers:
• Gmail, Web Search, Maps, ...
• MapReduce, batch, ...
• GFS, Colossus, ...
• Even Google’s Cloud Platform: our VMs
run in containers!

5. Confidential + Proprietary
Everything at Google runs in containers:
• Gmail, Web Search, Maps, ...
• MapReduce, batch, ...
• GFS, Colossus, ...
• Even Google’s Cloud Platform: our VMs
run in containers!
We launch over 2 billion containers per
week

6. Proprietary + Confidential
But why containers? What makes them so useful?

7. Confidential + Proprietary
Containers in a nutshell:
• Lightweight - Fewer layers between the
app and hardware. Efficient use of
resources.
• Scale - The lower overhead permits
smarter and faster scheduling.
• Speed - Quickly iterate on a build to
launch or make a change to a product.
Improvements to productivity.
VM
App
1
Bins/
Libs
Guest
OS
App
1.1
Bins/
Libs
Guest
OS
App
2
Bins/
Libs
Guest
OS
Hypervisor (Type 1)
Hardware

8. Confidential + Proprietary
Containers in a nutshell:
Container
• Portability - Run the container almost
anywhere.
• Deployment speed - Sealed deployable
application.
• Eases Development - Don’t need to install
MySQL, MongoDB, RabbitMQ, Hadoop
locally. Pull from a repo and run.
• Predictability - Pre-packaged nature of
the container helps to know what to
expect.
App
1
Bins/
Libs
Docker
HOST OS
Hardware
App
1.1
App
2.0
App
2.1
App
3.0
App
4.0
Bins/
Libs

9. Containers enable scale by allowing:
● Developers to design, build and
package applications as a set of
micro-services that can be combined
together into portable, reusable and
modular architectures.
● Operations to run and support
applications comprised of
self-contained, repeatable artefacts
that deploy and execute predictably
across multiple environments.

10. Confidential + Proprietary
Too many...containers…..
• Deployment
• Monitoring
• Updates
• Discovery
• Scaling, replication sets
• Isolation - Noisy Neighbours
This all needs to be controlled, at scale.
You can manage 1 container easily, but what
about thousands? At that scale you can easily
drown without the correct management.

11. Confidential + Proprietary
We are the Borg
Google’s own container cluster manager.
Manages many thousands of applications and machines.
Whether it's service jobs like
● Web front-ends and stateful servers
● Systems like Bigtable and Spanner,
● Batch frameworks like MapReduce and Millwheel,
We declare to Borg what we want, and the system works out
how to make it happen. At scale step by step instruction
becomes cumbersome.
Abstract that away and just tell the system what you need.

12. Confidential + Proprietary
Kubernetes Greek for “Helmsman”; also the root of the
words “governor” and “cybernetic”
● Manages container clusters
● Inspired and informed by Google’s
experiences and internal systems
● Supports multiple cloud and bare-metal
environments
● Supports multiple container runtimes
● 100% Open Source, written in Go
Manage applications, not machines

13. Confidential + Proprietary
Runs in many environments, including
“bare metal” and “your laptop”
The API and the implementation are
100% open
The whole system is modular and
replaceable
Build your apps on-prem, lift-and-shift
into cloud when you are ready
Kubernetes

14. Kubernetes has great momentum
50,000+ commits
in Kubernetes
1,200+ unique
contributors
Top 0.001% of all
GitHub Projects
5,000+ External
Projects Based
on Kubernetes
Companies
Contributing
Companies
Using

15. Competitive landscape: trends
Source: https://trends.google.it/trends/explore?q=kubernetes,Cloud%20Foundry,Docker%20swarm,Apache%20Mesos,Amazon%20Container%20Service
Kubernetes
Cloud Foundry
Docker Swarm
Apache Mesos
Amazon Container Service

16. Confidential + Proprietary
Architecture fundamentals
The master instance runs a number of services to
manage and coordinate the container cluster nodes
that constitute the cluster.
A node provides dedicated CPU and memory
resources to the container cluster.
Each node belongs to one container cluster.
Each node runs Docker, which is responsible for
downloading images and running containers
Pods act as the basic units of scheduling,
deployment, and horizontal scaling
Network
USER
Kubernatomy
Cntr
Bins/
Libs
Docker
HOST OS
Hardware
Cntr
Cntr
Cntr
Cntr
Cntr
Bins/
Libs
Cntr
Bins/
Libs
Docker
HOST OS
Hardware
Cntr
Cntr
Cntr
Cntr
Cntr
Bins/
Libs
Controller
Manager
Scheduler
MASTER
API SERVER
PODPODPODPOD
SERVICE SERVICE

17. Confidential + Proprietary
Architecture fundamentals
A Pod is the basic building block of
Kubernetes–the smallest and simplest unit
in the Kubernetes object model that you
create or deploy.
A service presents a stable endpoint.
A Kubernetes object is a “record of
intent”–once you create the object, the
Kubernetes system will constantly work to
ensure that object exists.
Kubernatomy

18. ● The easiest way of getting started with Kubernetes
● Fully managed by Google
● Manages operational logistics like logging, monitoring, and health
● Let you focus on the interesting part
● Once connected, it’s just Kubernetes
Google Container Engine
# Create cluster
gcloud container clusters create hello-world --num-nodes 3
# Connect Kubernetes
gcloud container clusters get-credentials hello-world
# Use Kubernetes
kubectl get pods

19. Multi-zone clusters
● Deploy up to 5000 nodes and 60k pods per
cluster
● Multi-zone HA working out of the box (inside
the same region)
● Improves availability in event of Zone failure
● By default, scheduler will spread pods across
all available zones in the cluster
gcloud container clusters create cluster-name
--num-nodes=3 --zone us-central1-a
--additional-zones=us-central1-b, us-central1-f

20. Node and pod autoscaling
● Configure pod autoscaling in your Replica Set
● Configure node autoscaling in your cluster
● Minimum and maximum values per zone
kubectl autoscale rc nginx --min=1 --max=4
--cpu-percent=80
gcloud container clusters update cluster-name
--enable-autoscaling --min-nodes=1 --max-nodes=10

21. Always up-to-date
● One new release every three months
● 1-2 weeks after the release, GKE is updated to the new
release
● Our own Google Container-optimized VM Image based
on Chromium OS helps to make it possible
● One-click update to latest version

22. Seamless upgrades
Get notified and perform the upgrade directly from gcloud or UI:
$ gcloud container clusters list
NAME ZONE MASTER_VERSION ... NODE_VERSION NUM_NODES STATUS
fy-old-cls us-central1-b 1.5.2 ... 1.3.1 ** 1 RUNNING
fy-prod-cls us-central1-f 1.5.3 ... 1.4.9 * 3 RUNNING
* - There is an upgrade available for your cluster(s).
** - The current version of your cluster(s) will soon be out of support, please
upgrade.

23. A single, open, standard API
Container Engine API Kubernetes API
Served by Kubernetes API Server
process running on the
kubernetes-master node
RESTful API with client library written
in and for Go
Also accessible through the kubectl
command-line utility
Standard RESTful Google Cloud
Platform API
Supports the standard Google API
Client Libraries
Used by the Cloud SDK ‘gcloud
container’ command-line utility

25. Demo