CONTAINERS WORKSHOP DURING SAUDI HPC 2016 : DOCKER 101, DOCKER, AND ITS ECO SYSTEM FOR DISTRIBUTED SYSTEMS by Walid Shaari
This workshop will cover the Theory and hands-on of Docker containers, and Its eco system. The foundations of the Docker platform, including an overview of the platform system components, images, containers and repositories, installation , using Docker containers from repositories e.g. dockerhub, how to create a container using Dockerfile, containers development life cycle. The strategy is to demonstrate through "live demo, and shared exercise" the reuse and customization of components to build a distributed system case service gradually
http://www.hpcsaudi.com/
2. § KACST to giving us this opportunity
§ Docker Inc. For allowing us use some of their graphics from docker slideshare
§ Christian Qnib @CQnib for the valuable HPC insights, and slide ideas for
CNAME
§ All for attending
4. 0.0.3 March
2013
1.0 June
2014
1.1 July
2014
1.2 August
2014
1.3 October
2014
1.4
December
2014
1.5
February
2015
1.6 April
2015
1.7 June
2015
1.8 August
2015
1.9
November
2015
1.10
Feburary
2016
1.11 April
2016
9. § There are other containers
§ Shifter
§ Didn’t like it,it made wrong assumptions about containers, and meant for CRAY, it
exploits the current Docker registry system to build its own chrooted squashefs
containers in loopback devices, does not have the full Docker advantages.
§ Singularity
§ Rocket
§ Back to the Linux/Unix philosophy,developed by CoreOS team
§ Intel clear containers
§ One possible solution to Container security
10.
11. Container encapsulate an application completely with all of its
software dependencies into a standardized unit for software
portable across different platforms*.
https://www.docker.com/what-docker
14. • Open Source Devops platform
• empowers developer to build and package services and
applications
• ship containerized applications as independent building unit
“Lego”
• Enables continuous delivery, and deployment
• Portability, reproducibility
• Behaves the same way in production, or other env.
• Run standalone or distributed
15. § Docker Core Components:
§ Engine : Portable (light/heavy) weight run time, and packaging tool
§ Registry : Central repos to manage sharing of packaged Docker images, and
enabled (build, ship, release, run) workflows ( the default registry Docker
HUB)
§ Containers Eco System
§ Modular
§ Check Schedulers
§ Compliant
§ OCI
§ Holistic
§ End to end workflows
16.
17. § cgroups by Google
§ namespace by IBM
§ selinux by NSA
§ iptables
§ Docker in March 2013 was a lightweight runtime and robust tooling to build and
ship the encapsulated operating environment to run virtual containers on Linux for
your distributed applications.
18.
19. § Image:
The basis of a Docker container
§ Docker Container:
The standard unit in which the application service resides
"running image instance"
§ Docker Engine:
Runs on Linux, Creates,ships and runs Docker containers deployable on physical or virtual host
locally,in a datacenter or cloud service provider
§ Docker Machine:
Docker Machine is a tool that lets you install Docker Engine on virtual hosts,and manage the
hosts with docker- machine commands.You can use Machine to create Docker hosts on your
local Mac or Windows box, on your company network,in your data center,or on cloud providers
like AWS or Digital Ocean.
§ Docker Registry:
On-premises registry for image storing and collaboration
20.
21.
22.
23. Containers
• Packages up software binaries and dependencies "minimal"
• Isolates software from each other
• Container is a standard format
• Easily portable across environment
• Allows ecosystem to develop around its standard Container
24.
25. § Deploy faster, and more often
§ Shorter time between deploys
§ Resource usage improved too (2x-10x)
§ Portability Freedom:
Move containers around
(laptop to workstation, laptop to servers, colo-to-cloud, cloud-to-cloud, cloud-to-colo, ...)
28. Host
Isolation is achieved via Linux kernel namespaces
Container 3
pbs_serve
r
pbs_sched
sshd
Container 2
mpi_task(s)
pbs_mom
sshd
Container 1
mpi_task(s)
pbs_mom
sshd
Mount IPC UTSNetworkPID USER
29. Container have several name spaces
Host
Container 2
mpi_task(s)
pbs_mom
sshd
Container 1
mpi_task(s)
pbs_mom
sshd
Mount IPC UTSNetworkPID
Container 3
pbs_serve
r
pbs_sched
sshd
USER
30. Network namespace can be shared between one or more container
Host
Container 2
mpi_task(s)
pbs_mom
sshd
Container 1
mpi_task(s)
pbs_mom
sshd
Mount IPC UTSNetworkPID
Container 3
pbs_serve
r
pbs_sched
sshd
USER
34. 1- Go to https://docs.docker.com/windows/
2- then start installation https://docs.docker.com/windows/step_one/
3- Ensure git bash terminal is installed
4- invoke a git bash terminal
5- verify the docker-machine
35. 1- Initialize the Docker machine environment
2-Verify the Docker-machine
docker-machine ls
docker-machine env default
3-You might need to create a virtual default machine
36. You might need to create a virtual default machine
docker-machine create --driver virtual box default
48. for i in {1..10}; do docker run alpine uptime;done
docker ps -a
man docker-rm
docker ps –a
docker rm container [container...]
Or:
docker rm `docker ps --all -q`
What if you want to remove:
volumes associated with the container?
Links associated with the container?
Running container(s)?
57. § https://github.com/docker/docker-birthday-3
§ In your homedir
§ git clone https://github.com/docker/docker-birthday-3
§ cd docker-birthday-3/example-voting-app/
§ Follow https://github.com/docker/docker-birthday-3 instruction for compose, and swarm
§ Check example enteries
§ https://blog.docker.com/2016/04/docker-birthday-app-challenge-winners/
58. https://github.com/yadutaf/infilter
To share the host network, PID, and IPC namespaces with the
container,
run the following command:
$ docker run -it --net=host --pid=host --ipc=host alpine sh
59. • There is more
• Schedulers
• Nomad
• Kubernetes
• Mesos
• Storage engines
• ZFS
• BTRFS
• CEPH
• Network overlays
• And definitely there is more to be awesome!