Alibaba Cloud Conference 2016 - Docker Enterprise

What’s New in Docker
John Willis
Director of Ecosystem Development

About Me
• One of the founding members of the “Devops” movement.
• Author of the “Devops Handbook”.
• Author of the “Introduction to Devops” on Linux
Foundation edX.
• Podcaster at devopscafe.org
• Devops Enterprise Summit - Cofounder
• Found of Socketplane (Acquired by Docker)
• Formally Director of Devops at Dell
• Formally Director at Chef
• 10 Startups over 25 years
2
Github: botchagalupe/my-presentations
Twitter: botchagalupe
Wechat: botchagalupe

Devops Practices and Patterns
• Continuous Delivery
• Everything in version control
• Small batch principle
• Trunk based deployments
• Manage flow (WIP)
• Automate everything 
• Culture
• Everyone is responsible
• Done means released
• Stop the line when it breaks
• Remove silos
3
itrevolution.com/devops-handbook
http://www.bookdepository.com/

The world needs
tools of mass innovation

A programmable Internet would be the ultimate
tool of mass innovation

Let’s eliminate friction in the development cycle

A commercial product,
built on
a development platform,
built on
infrastructure,
built on
standards.
Docker is building a stack to program the Internet

8
Docker Project Sponsor
•Primary authors, contributor maintainer
•6B+ Downloads, 3000+ Contributors, 500,000+ Applications
•100’s of ecosystem partners
•Millions of developers use Docker. Millions of servers run Docker
Commercial Docker Solutions
•Integrated solutions to build, ship, run Docker at scale
• Orchestration, registry, security, workflow, control plane
• CaaS (containers as a service)
•Official providers of commercial technical support
•10K’s cloud customers, 300+ F500 customers
About Docker, Inc.
Gerber, Anna. “The State of Containers and the Docker
Ecosystem: 2015” O’Reilly, September 2015
Docker users already
running in production
60%

China is part of Worldwide Docker Community
Meetups in Beijing, Changsha, Chengdu, Chongqing, Dalian, Fuzhou,
Guangzhou, Hangzhou, Hong Kong, Nanjing, Qingdao, Shanghai, Shenzen,
Suzhou, Tianjin, Wuhan, Xi’an, Xiamen, and Zhuhai

Cloud and Docker Status Update in China
China cloud relative to the West – DB
! China’s cloud spending to reach 24.5% of the West by 2018 from 5.7% in 2015

! More than 80% respondents are already using or plan to use containers, but only 10% used in production environment.
Survey result from 350+ feedbacks by Alibaba Cloud, June 2016
Docker Adoption in
China

Other
Government
Sample of Docker Customers
Financial Services Healthcare
Consumer Education Tech Services
11

Dockerizing applications
12
Legacy App: One container per app. Microservices: one service per container. App
comprised of many containers linked together

Enterprises Are Containerizing both Legacy and Microservices Applications
13
80% 46% plan to build new microservices in 2016
#4 container workload is traditional databases
1 in 3 have already containerized legacy apps
Will leverage Docker to
enable hybrid cloud
initiatives
Docker platform standardizes environment AND enables workload portability

3 Paths to Containers Adoption
1
Containerize Monoliths
Build-Test for CI; Migrate to the Cloud;
Get Better CapEx/OpEx than VM
2
3
Containerize Monolith; Transform to Microservices
Look for Shared Services to Transform
Enable New Microservices and Apps
Greenfield CaaS

Migrate any workload anywhere
Infrastructure agnostic platform
• Standardize: Docker abstracts away
the infrastructure and virtualization
away from the standardized app
containers
• Portability: Containers move
without friction from one environment
to another – no recoding needed
• Lift and Shift: Containerize legacy
and microservices to gain portability
15
Bins/Libs
App
OS
Bins/Libs
App
OS
Bins/Libs
App
OS
Before: VM formats are proprietary
Bins/Libs
App
Bins/Libs
App
Bins/Libs
App
OS
After: Docker abstracts above VMs for portability
Docker Engine
OS
Docker Engine

Infrastructure optimization with Docker
Swisscom
• 20:1 VM consolidation ratio
• Running the same 400 MongoDB
instances in 400 containers in 20
VMs
• Reduce CapEx and OpEx costs
Leading Energy Company
• Containerize legacy apps for
portability
• Entire cloud to datacenter site
migration in 5 months
• Dramatically accelerated release
process
16

17
Enable Modern App Initiatives with CaaS
Cloud Migration
Hybrid Cloud
Multi Cloud
Modernizing Apps
DevOps
CI/CD
DevOps
Cloud DevOpsApps

18
Enterprises Can Decide How To Adopt
Enable CI and DevOps
Better Resource Usage
Enable Cloud Initiatives
Transform iteratively
Ship faster, with better
reliability
Enable developers to self service
Enable business transformations
Containerize Build New AppsContainerize &
Transform

The Docker ecosystem
Dev Tools
Official Repositories
Operating Systems
Big Data
Service Discovery
Build / Continuous Integration
Configuration Management
Consulting &Training
Management
Storage
Clustering & Scheduling
Networking
Infrastructure & Service Providers
Storage
Security
Monitoring & Logging
19

Docker and Alibaba Announce  
Commercial Agreement
• Localized Docker image store and distribution for Docker Hub on Alibaba
Cloud
• Alibaba will resell Commercially Supported (CS) Docker Engine and Docker
Datacenter, enabling enterprises to manage their production workloads
across the entire application lifecycle.
• Alibaba will provide enterprise support options for CS Docker Engine and
Docker Datacenter, backed by Docker, Inc.
20

COMPARING CONTAINERS AND VIRTUAL MACHINES

Isolation using Linux kernel features
namespaces
●
pid
●
mnt
●
net
●
uts
●
ipc
●
user
cgroups
●
memory
●
cpu
●
blkio
●
devices

Docker Engine extensibility and plugins
• Built in orchestration expands the opportunity for
the plugin to manage swarm wide vs a single
Engine
• Updated architecture standardizes plugin process
for ecosystem partners
– Benefits users and vendors
– Standardized process of granting plugin permissions
– Containerized plugins on roadmap
Docker Engine
Networking
Swarm Mode
Volumes
Plugins

1. Get out of the way
The best tools…
2. Adapt to you
3. Make the
powerful simple

Docker for Mac Docker for Windows

Introducing the best way to
orchestrate Docker: Docker.
Docker 1.12: now with orchestration built-in.

Swarm mode
Service API
Node identity
Built-in routing mesh
Docker 1.12: now with orchestration built-in.

Combine your engines in swarms of any scale
Self-organizing, self-healing
No external data store required 
No single points of failure
Infrastructure-agnostic topology
Swarm mode

Desired state reconciliation
Scaling 
Rolling updates
Advanced scheduling
Application-specified health checks
Rescheduling on node failure
Docker Service API

How service deployment works
Declare
ScheduleReconcile
$ docker service create declares
the service name, network, port, image:tag
and scale
Managers break down service into
tasks, schedules them and workers
execute tasks
Engines check to see what is running
and compared to what was declared to
“true up” the environment

API
Allocator
Orchestrator
Scheduler
Dispatcher
R
A
F
T
Manager Node
Worker
Executor
Worker Node
Accepts command from client and creates service object
Reconciliation loop for service objects and creates tasks
Allocates IP addresses to tasks
Assigns tasks to nodes
Checks in on workers
docker service create
Connects to dispatcher to check on assigned
tasks
How service deployment works
Executes the tasks assigned to worker
node

Manager ManagerManager
Worker WorkerWorkerWorker Worker
Example service on a swarm
The declarative command
describes a new service:
• Named Frontend
• Made of 5 containers based
on the latest
my_frontend_image
• Connected on an overlay
network called “my overlay”
• Assigned to port 80
$ docker service create --replicas 5 --name frontend
--network myoverlay
--publish 80/tcp mikegoelzer/
my_frontend_image:latest

This state is what swarm
mode and the service
deployment API will
maintain.
Check to ensure 5
containers are always
running for the frontend
service$ docker service create --replicas 5 --name frontend
--network myoverlay
--publish 80/tcp mikegoelzer/
my_frontend_image:latest

This command creates a
new service to join an
existing overlay network to
communicate with frontend
$ docker service create --name redis --network
myoverlay redis:latest

Built-in Routing Mesh
Swarm-wide overlay networking
Container-native load-balancing
DNS-based service discovery
No separate cluster to setup
Works with your existing load-balancers
Rock-solid kernel-only data path with IPVS

Routing mesh for application services
• Container-aware dynamic
load balancing
• Assign ports to service that
do not change
• Built in load balancing into
the Engine
• Automatic service discovery
Worker
Load Balancing
Worker
Load Balancing
Worker
Load Balancing
Port 1000Port 1000 Port 1000
Docker user assigns a
global port for a service
End user accessing
www.website.com

Docker 1.12 is Simple
> docker swarm init
> docker service create

How to get Docker 1.12
Mac OS X
Developer Workstations Cloud Providers
Windows PC
AWS
Azure
Commercially Supported
Docker Datacenter
CS Docker Engine
Docker Cloud
Open Source Docker
Engine installers for the
following Linux distros
Optimized for and integrated directly into the
underlying platform with custom plugins and drivers

Deep integration with native load-balancers, templates,
SSH keys, ACLs, scaling groups, firewall rules…
beta.docker.com

Distributed Application Bundle
experimental
www.docker.com/dab
A portable format for multi-container applications

5
Goldilocks and the 3 XaaS
Just rightToo highToo low
IaaS PaaS CaaS

5
Goldilocks and the 3 XaaS
Platform As A Service
Infrastructure As A Service
Software
As A Service
Too high
Too low
Just right
Container As A Service

Docker Datacenter core values
48
Management at scale
Integrated Content Trust
Secure Access (RBAC)
Integrates with existing
systems
Full support of Docker API
Seamless dev to prod
workflow
Infrastructure, network and
storage portability
Easy to setup and use
Native Docker solution
Extend existing Docker
developer experience
+ +Agility Portability Control

Agility, portability and control for developers and IT
Developers IT Operations
• Freedom to create and
deploy apps fast
• Define and package
application needs
• Quickly and flexibly respond
to changing needs
• Standardize, secure, and
manage
Frictionless portability across teams, environments, infrastructure
49

Docker Datacenter platform
Management
Universal Control Plane
Security
Content Trust, RBAC, LDAP/AD
Orchestration
Swarm
Container
Runtime
Engine
Registry Service
Trusted Registry

BUILD
Development Environments
SHIP
Registry: Secure Content &
Collaboration
RUN
Control Plane: Deploy,
Orchestrate, Manage, Scale
Networking Volumes MonitoringLoggingConfig MgtCI/CD
IT Operations
Developers IT Operations
Docker CaaS Workflow

Docker Containers as a Service platform
52
BUILD
Developer Workflows
SHIP
Registry Services
RUN
Management
Docker for Mac and Windows Docker Trusted Registry Docker Universal Control Plane
Docker Cloud
Docker Container Engine
Ecosystem Plugins and Integrations

Docker UCP 1.1 - DTR 2.0
• HA
• Unified Auth
• Compose deployment
• UI to add nodes

Security scanning in Docker Cloud

2013-05
2013-06
2013-07
2013-08
2013-09
2013-10
2013-11
2013-12
2014-01
1,000,000
0
2014-02
2014-03
2014-04
2014-05
2014-06
2014-07
2014-08
2014-09
2014-10
2014-11
2014-12
2015-01
2015-02
2015-03
2015-04
2015-05
2015-06
2015-07
2015-08
2015-09
2015-10
2015-11
2015-12
2016-01
1,000,000,000
~
10,000,000
9,000,000
8,000,000
7,000,000
6,000,000
5,000,000
4,000,000
3,000,000
2,000,000
6,000,000,000
5,750,000,000
5,500,000,000
5,250,000,000
5,000,000,000
4,750,000,000
4.500,000,000
4,250,000,000
4,000,000,000
3,750,000,000
3,500,000,000
3,250,000,000
3,000,000,000
2,750,000,000
2,500,000,000
2,250,000,000
2,000,000,000
1,750,000,000
1,500,000,000
1,250,000,000

2013-05
2013-06
2013-07
2013-08
2013-09
2013-10
2013-11
2013-12
2014-01
1,000,000
0
2014-02
2014-03
2014-04
2014-05
2014-06
2014-07
2014-08
2014-09
2014-10
2014-11
2014-12
2015-01
2015-02
2015-03
2015-04
2015-05
2015-06
2015-07
2015-08
2015-09
2015-10
2015-11
2015-12
2016-01
~
2016-09
1,000,000,000
~
10,000,000
9,000,000
8,000,000
7,000,000
6,000,000
5,000,000
4,000,000
3,000,000
2,000,000
6,000,000,000
5,750,000,000
5,500,000,000
5,250,000,000
5,000,000,000
4,750,000,000
4.500,000,000
4,250,000,000
4,000,000,000
3,750,000,000
3,500,000,000
3,250,000,000
3,000,000,000
2,750,000,000
2,500,000,000
2,250,000,000
2,000,000,000
1,750,000,000
1,500,000,000
1,250,000,000
Notary
runC •
containerd •
HyperKit , VPNKit, DataKit •
SwarmKit •
libcontainer •
libnetwork • • Docker 1.8 : Docker Content Trust
• Docker for Mac
Docker for Windows
• Docker 1.12
with built-in
orchestration
• Docker 0.9 : Pluggable execution
• Docker 1.7 : Multi-Host Networking
• Docker 1.11:
OCI support

InfraKit
A toolkit for building declarative, self-
healing infrastructure.

Declarative
• JSON configuration for desired infrastructure state:
• Specification of instances — vm image, instance type, etc.
• Group properties — size, logical identifiers, etc. 
• Design patterns encourage
• encapsulation
• composition 
• Config is input to all operations — system figures out what to do 
66

Self-healing
• Composed of a set of active components / processes that
• monitor infrastructure state
• detect state divergence
• take actions 
• Continuous monitoring and reconciliation — always on 
• No downtime — rolling update
67

Toolkit
• Primitives for managing collections of resources
• create, scale, destroy
• rolling update
• Abstractions & Developer SPI
• Group - manages collection of resources
• Instance - describes the physical resource
• Flavor - extra semantics for handling instances
• A collection of executable, active components — plugins
• Initially, Go daemons in the toolkit
• Soon, easy management via Docker Plugins (runc)

Instance Plugin
• Spec: specification / model of an instance (e.g. vagrant, EC2):
• Logical ID, Init, Tags, and attachment
• Platform-specific properties
• Methods:
• /Instance.Validate
• /Instance.Provision
• /Instance.Destroy
• /Instance.DescribeInstances
• Examples: instance plugins for EC2, Azure VM, Vagrant, …
70

Flavor Plugin
• Gives more context about the group members:
• Size, or list of Logical ID’s (e.g. IP addresses for ‘pets’)
• Application-specific notions of ‘health’  
Is the node not only present but also joined a swarm?
• Methods:
• /Flavor.Validate
• /Flavor.Prepare
• /Flavor.Healthy
• Examples: flavor for Zookeeper members, Docker swarm nodes
71

Group Plugin
• Main entry point for user interaction:
• Create, describe update, update, destroy
• Config JSON is always the input
• Composed of Instance and Flavor — mix and match to
manage cattle (fungible) or pets (special)
• Methods:
• /Group.Watch
• /Group.Unwatch
• /Group.Inspect
72
• /Group.DescribeUpdate
• /Group.Update
• /Group.StopUpdate
• /Group.Destroy

Configuration
Example config file (zk.conf): Group configuration = Instance + Flavor
{
"Properties": {
/* raw configuration
*/
}
}
{
"groups" : {
"my_zookeeper_nodes" : {
"Properties" : {
"Instance" : {
"Plugin": "instance-vagrant",
"Properties": {
"Box": "bento/ubuntu-16.04"
}
},
"Flavor" : {
"Plugin": "flavor-zookeeper",
"Properties": {
"type": "member",
"IPs": ["192.168.1.200", "192.168.1.201",
"192.168.1.202"]
}
}
}
}
}

Operations
• Make sure the plugins are running:
• infrakit/group &; infrakit/zookeeper &; infrakit/vagrant &; 
• “Watch” the group starts management:
• infrakit/cli group watch zk.conf 
• Update the config, e.g. change size or add IP address
• Describe changes before committing —  
infrakit/cli group describe zk.conf
• Begin update — 
infrakit/cli group update zk.conf
74

Today
75
• InfraKit is just getting started… only
primitives for working with groups like
clusters of hosts 
• But we have big plans
• Improve group management strategies
• More resource types — networking, load
balancers, storage… 
• A cohesive framework for active
management of infrastructure — physical,
virtual, or containers

Get Involved
• Help define and implement new and interesting plugins
• Instance plugins for different infrastructure providers
• Flavor plugins for systems like etcd or mysql clusters
• Group controller plugins — metrics-driven auto scaling
and more 
• Help define interfaces and implement new infrastructure
resource types — load balancers, networks and storage
volume provisioners
76

More Info
• Github:  
https://github.com/docker/infrakit
• A quick tutorial:  
https://github.com/docker/infrakit/blob/master/docs/
tutorial.md
77

Alibaba Cloud Conference 2016 - Docker Enterprise

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Alibaba Cloud Conference 2016 - Docker Enterprise

Similar to Alibaba Cloud Conference 2016 - Docker Enterprise (20)

More from John Willis

More from John Willis (20)

Recently uploaded

Recently uploaded (20)

Alibaba Cloud Conference 2016 - Docker Enterprise