The Docker Ecosystem under the hood: Docker’s Key Use Cases, Docker Toolbox, Docker Machine, Docker Compose, Docker Swarm, Multi-Host Docker Networking

1. 1CONFIDENTIAL
The Docker Ecosystem
DZMITRY SKAREDAU, SOLUTION ARCHITECT
NOVEMBER 5, 2015

2. 2CONFIDENTIAL
2
AGENDA
• Introduction to Docker
• Docker’s Key Use Cases
• Docker Toolbox
• Docker Machine
• Docker Compose
• Docker Swarm
• Multi-Host Docker Networking

3. 3CONFIDENTIAL
WHAT IS DOCKER?

4. 4CONFIDENTIAL
4
WHAT IS DOCKER?
Open source engine that leverage LXC and AUFS to package
an application and its dependencies in a virtual container
that can run on any Linux server.
WHAT!?
We are using Windows!
most of us

5. 5CONFIDENTIAL
5
WHAT IS DOCKER?

6. 6CONFIDENTIAL
6
LXC
Wikipedia
https://en.wikipedia.org/wiki/LXC
“
„
Linux Containers (LXC) provide a means to isolate individual services or applications as well as of
a complete Linux operating system from other services running on the same host. To accomplish
this, each container gets its own directory structure, network devices, IP addresses and process
table. The processes running in other containers or the host system are not visible from inside a
container. Additionally, Linux Containers allow for fine granular control of resources like RAM, CPU
or disk I/O.
LXC combines kernel's cgroups and support for isolated namespaces to provide an isolated
environment for applications.

7. 7CONFIDENTIAL
7
CGROUPS
“ „cgroups (abbreviated from control groups) is a Linux kernel feature that limits, accounts for, and
isolates the resource usage (CPU, memory, disk I/O, network, etc.) of a collection of processes.
Wikipedia
https://en.wikipedia.org/wiki/Cgroups

8. 8CONFIDENTIAL
8
NAMESPACE ISOLATION
“ „
namespace isolation, where groups of processes are separated such that they cannot "see"
resources in other groups. For example, a PID namespace provides a separate enumeration
of process identifiers within each namespace. Also available are mount, UTS, network and SysV IPC
namespaces.
Wikipedia
https://en.wikipedia.org/wiki/Cgroups

9. 9CONFIDENTIAL
9
AUFS
“ „aufs (short for advanced multi layered unification filesystem) implements a union
mount for Linux file systems.
Wikipedia
https://en.wikipedia.org/wiki/Aufs

10. 10CONFIDENTIAL
10
AUFS
A typical Linux start to run to two FS:
• bootfs (boot file system) Including bootloader
and kernel, bootloader is the main kernel boot
loader, when after the success of the boot
kernel is loaded into memory after the bootfs
was umount
• rootfs (root file system) Is a typical Linux
system consists of /dev, /proc, /bin, /etc and
other standard directory and file.

11. 11CONFIDENTIAL
11
AUFS
Thus for different Linux distributions, the bootfs is
basically the same, the rootfs will be different, so
different distributions can be public bootfs as
shown below:
• Debian is a Unix-like computer operating
system and a Linux distribution
Size: 136.1 MB
• BusyBox is software that provides several
stripped-down Unix tools in a single executable
file. It was specifically created for embedded
operating systems with very limited resources.
Size: 1.109 MB

12. 12CONFIDENTIAL
12
AUFS
2 custom images:
1. With Apache/Emacs over Debian
2. Over BusyBox

13. 13CONFIDENTIAL
13
CONTAINERS VS VMS

14. 14CONFIDENTIAL
14
DOCKER UNDER THE HOOD

15. 15CONFIDENTIAL
15
DOCKER UNDER THE HOOD

16. 16CONFIDENTIAL
16
DOCKER AT LINUX AND MACOS/WINDOWS

17. 17CONFIDENTIAL
17
DOCKER CONTAINERS IN PRODUCTION
There is currently a pervasive (and faulty)
perception that Docker containers are only
being utilized in dev-test and proof-of-
concept projects. In fact, the question I am
most often asked by IT colleagues and
customers goes like this: “Is anyone using
Docker containers for critical workloads, or
even in production?” The answer is an
unequivocal “Yes” – critical workloads are
being run in Docker containers, and much
more pervasively than is commonly
understood.
Here are a few examples:
• Global financial services corporation ING is
using Docker containers to help accelerate
its continuous delivery process and drive
500 deployments/week, meeting speed to
market goals
• Global investment bank Goldman Sachs uses
Docker containers to centralize application
builds and deployments
• Streaming music leader Spotify uses Docker
containers to make software
deploymentsrepeatable, straightforward,
and fault-tolerant
• Application performance management
leader New Relic is using Docker containers
to solve its most challenging deployment
issues

18. 18CONFIDENTIAL
DOCKER’S KEY USE CASES

19. 19CONFIDENTIAL
19
SIMPLIFYING CONFIGURATION
Cloud Services with built-in Docker support

20. 20CONFIDENTIAL
20
CODE PIPELINE MANAGEMENT
The immutable nature of Docker images, and the ease
with which they can be spun up, help you achieve zero
change in application runtime environments across dev
through production.
ENV DEV
Private Docker Hub
ENV INT ENV QA ENV PRE PROD
ENV PROD

21. 21CONFIDENTIAL
21
DEVELOPER PRODUCTIVITY
In a developer environment, we have two goals that are at
odds with each other:
1. We want it be as close as possible to production; and
2. We want the development environment to be as fast as
possible for interactive use.

22. 22CONFIDENTIAL
22
APP ISOLATION
A couple of such cases to consider are server consolidation
for decreasing cost or a gradual plan to separate a
monolithic application into decoupled pieces.

23. 23CONFIDENTIAL
23
SERVER CONSOLIDATION
Just like using VMs for consolidating multiple applications,
the application isolation abilities of Docker allows
consolidating multiple servers to save on cost. However,
without the memory footprint of multiple OSes and the
ability to share unused memory across the instances,
Docker provides far denser server consolidation than you
can get with VMs.

24. 24CONFIDENTIAL
24
MULTI-TENANCY
Using Docker, it was easy and inexpensive to create
isolated environments for running multiple instances of
app tiers for each tenant.

25. 25CONFIDENTIAL
25
RAPID DEPLOYMENT
Docker creating a container for the process and not
booting up an OS, brings it down to seconds.

26. 26CONFIDENTIAL
DOCKER TOOLBOX

27. 27CONFIDENTIAL
27
• Docker Machine for running the docker-machine binary
• Docker Engine for running the docker binary
• Kitematic, the Docker GUI
• a shell preconfigured for a Docker command-line
environment
• Oracle VM VirtualBox
DOCKER TOOLBOX

28. 28CONFIDENTIAL
28
The Docker VM is lightweight Linux virtual machine made
specifically to run the Docker daemon on Windows. The
VirtualBox VM runs completely from RAM, is a small ~29MB
download, and boots in approximately 5s.
DOCKER MACHINE
docker-machine create --driver virtualbox my-default
Creating VirtualBox VM...
Creating SSH key...
Starting VirtualBox VM...
Starting VM...
To see how to connect Docker to this machine, run:
docker-machine env my-default
docker-machine --native-ssh create -d virtualbox dev

29. 29CONFIDENTIAL
29
DOCKER MACHINE
docker-machine --native-ssh create -d virtualbox dev
docker-machine ls
NAME ACTIVE DRIVER STATE URL SWARM
dev virtualbox Running tcp://192.168.99.100:2376
List your available machines
Create a new Docker VM
Get the environment commands for your new VM
docker-machine env dev --shell cmd
set DOCKER_TLS_VERIFY=1
set DOCKER_HOST=tcp://192.168.99.100:2376
set DOCKER_CERT_PATH=C:UsersDzmitry_Skaredau.dockermachinemachinesdev
set DOCKER_MACHINE_NAME=dev
# Run this command to configure your shell:
# copy and paste the above values into your command prompt

30. 30CONFIDENTIAL
30
DOCKER MACHINE
docker-machine ls
NAME ACTIVE DRIVER STATE URL SWARM
dev * virtualbox Running tcp://192.168.99.100:2376
List your available machines
docker run ^
-d ^
-p 80:80 ^
-v $(pwd)/src/vhost.conf:/etc/nginx/sites-enabled/vhost.conf ^
-v $(pwd)/src:/var/www ^
nginx
Run container
pwd: The pwd command will allow you to know in which directory you're located (pwd stands for "print working directory")

31. 31CONFIDENTIAL
31
DOCKER MACHINE
docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
ef9b3f99a05f nginx "nginx -g 'daemon off" 13 seconds ago Up 9 seconds 0.0.0.0:80->80/tcp, 443/tcp sad_elion
Show containers
docker-machine ip
192.168.99.100
Find machine IP

32. 32CONFIDENTIAL
docker images
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
rnd-saas/service-discovery latest 9f7499191ada 10 seconds ago 722.7 MB
java 8 bdd93cb6443c 4 days ago 641.9 MB
busybox latest c51f86c28340 4 days ago 1.109 MB
nginx latest 914c82c5a678 7 days ago 132.8 MB
ubuntu precise 38f2c35e1b51 13 days ago 136.1 MB
32
DOCKERFILE
FROM java:8
EXPOSE 8761
VOLUME /tmp
ADD service-discovery-0.1.0.jar app.jar
RUN bash -c 'touch /app.jar'
ENTRYPOINT ["java","-Djava.security.egd=file:/dev/./urandom","-jar","/app.jar"]
Dockerfile content
docker build -t rnd-saas/service-discovery .
Build new image
Show images

33. 33CONFIDENTIAL
DOCKER COMPOSE

34. 34CONFIDENTIAL
34
Running multiple containers
• Run your stack with one command: docker-compose up
• Describe your stack with one file: docker-compose.yml
DOCKER COMPOSE

35. 35CONFIDENTIAL
35
HOW TO RUN WORDPRESS
FROM orchardup/php5
ADD . /code
Dockerfile
web:
build: .
command: php -S 0.0.0.0:8000 -t /code
ports:
- "8000:8000"
links:
- db
volumes:
- .:/code
db:
image: orchardup/mysql
environment:
MYSQL_DATABASE: wordpress
docker-compose.yml

36. 36CONFIDENTIAL
36
The features of Compose that make it effective are:
• Multiple isolated environments on a single host
DOCKER COMPOSE

37. 37CONFIDENTIAL
37
Compose uses a project name to isolate environments from each other.
You can use this project name to:
• on a dev host
to create multiple copies of a single environment (ex: you want to run a stable copy for each feature branch of a project)
• on a CI server
to keep builds from interfering with each other, you can set the project name to a unique build number
• on a shared host or dev host
to prevent different projects which may use the same service names, from interfering with each other
ISOLATED ENVIRONMENTS

38. 38CONFIDENTIAL
38
The features of Compose that make it effective are:
• Multiple isolated environments on a single host
• Preserve volume data when containers are created
DOCKER COMPOSE

39. 39CONFIDENTIAL
39
Compose preserves all volumes used by your services. When docker-
compose up runs, if it finds any containers from previous runs, it copies
the volumes from the old container to the new container. This process
ensures that any data you’ve created in volumes isn’t lost.
PRESERVE VOLUME DATA

40. 40CONFIDENTIAL
40
The features of Compose that make it effective are:
• Multiple isolated environments on a single host
• Preserve volume data when containers are created
• Only recreate containers that have changed
DOCKER COMPOSE

41. 41CONFIDENTIAL
41
Compose caches the configuration used to create a container. When you
restart a service that has not changed, Compose re-uses the existing
containers. Re-using containers means that you can make changes to your
environment very quickly.
RECREATES ONLY CHANGED CONTAINERS

42. 42CONFIDENTIAL
42
The features of Compose that make it effective are:
• Multiple isolated environments on a single host
• Preserve volume data when containers are created
• Only recreate containers that have changed
• Variables and moving a composition between environments
DOCKER COMPOSE

43. 43CONFIDENTIAL
43
Your configuration options can contain environment variables. Compose
uses the variable values from the shell environment in which docker-
compose is run. For example, suppose the shell contains
POSTGRES_VERSION=9.3 and you supply this configuration:
VARIABLE SUBSTITUTION
db:
image: "postgres:${POSTGRES_VERSION}"

44. 44CONFIDENTIAL
44
Common use case is multiple compose files: changing a Compose app for
different environments
MOVING A COMPOSITION BETWEEN ENVIRONMENTS
docker-compose -f docker-compose.yml -f docker-compose.prod.yml up -d

45. 45CONFIDENTIAL
DOCKER SWARM

46. 46CONFIDENTIAL
46
Docker Swarm is used to host and schedule a cluster of
Docker containers.
DOCKER SWARM

47. 47CONFIDENTIAL
47
Since Swarm ships as a standard Docker image with no
external infrastructure dependencies, getting started is a
simple, three-step process:
1. Run one command to create a cluster.
2. Run another command to start Swarm.
3. On each host where the Docker Engine is running, run a
command to join said cluster.
SETUP

48. 48CONFIDENTIAL
48
Swarm is aware of the resources available in the cluster and will place
containers accordingly.
To choose a ranking strategy, pass the --strategy flag and a strategy value
to the swarm manage command. Swarm currently supports these values:
• spread
• binpack
• random
RESOURCE MANAGEMENT
docker run -d -m 1g redis

49. 49CONFIDENTIAL
49
In order to meet the specific requirements of each container, their
placement can be fine-tuned using constraints.
CONSTRAINTS
docker run -d -e constraint:storage==ssd mysql
Constraints operate on Docker daemon labels. To make the previous
example work, Docker must be started with the–label storage=ssd option.
More advanced expressions are also supported:
docker run --rm -d -e constraint:node!=fed*
docker run --rm -d -e constraint:node==/ubuntu-[0-9]+/

50. 50CONFIDENTIAL
50
In some cases, the placement of a container must be relative to other containers. Swarm lets you define
those relationships through affinities.
The following will run two Redis servers, while guaranteeing they don’t get scheduled on the same
machine:
AFFINITY
docker run -d --name redis_1 -e ‘affinity:container!=redis_*’ redis
docker run -d --name redis_2 -e ‘affinity:container!=redis_*’ redis

51. 51CONFIDENTIAL
51
At some point, Swarm will be able to reschedule containers upon host failure.
Let’s say you schedule a frontend container with some constraints:
FAULT-TOLERANT SCHEDULING
docker run -d -e constraint:storage==ssd nginx
If the host of this container goes down, Swarm will be able to detect the outage
and reschedule this container on another host that can respect the
constraint storage==ssd

52. 52CONFIDENTIAL
MULTI-HOST DOCKER NETWORKING

53. 53CONFIDENTIAL
53
Networking is a feature of Docker Engine that allows you to
create virtual networks and attach containers to them so you
can create the network topology that is right for your
application.
NETWORKING

54. 54CONFIDENTIAL
54
1. Connect containers to each other across different physical or virtual hosts
2. Containers using Networking can be easily stopped, started and restarted
without disrupting the connections to other containers
3. You don’t need to create a container before you can link to it. With
Networking containers be created in any order and discover each other using
their container names
NETWORKING

55. 55CONFIDENTIAL
55
You can create a new network with docker network create. In this example,
we’ll create a network called “frontend” and run an nginx container inside it:
NETWORKING
docker network create frontend
docker run -d --net=frontend --name web nginx
Then we could run a web application in a network called “app” and then use the
docker network connect command so our Nginx container can forward
connections to it.
docker network create app
docker run -d --name myapp --net=app <my application container>
docker network connect app web
Now Nginx should be able to connect to your application using the hostname “myapp.app”