Containers - Portable, repeatable user-oriented application delivery. Build, ship, run any app anywhere!

Containers: Portable, repeatable user-oriented application delivery
HPC Saudi 2017 - KAUST
15 th March 2017
#dockerbday
@walidshaari
walid.shaari@gmail.com
https://www.linkedin.com/in/walidshaari/

$whoami
● Passionate about openness, open source, devops, Infosec
● Member of the Saudi Aramco Expec Computer Center/HPC team
● Red Hat Certified Architect RHCA
● SANS GIAC Incident handler, Forensics and Web security certified
● Dhahran Docker & Ansible meetup organizer/mentor
@walidshaari
Walid.shaari@gmail.com
https://www.linkedin.com/in/walidshaari/

AGENDA : Good Morning Containers
8:30 - 8:35 Introduction, Networking, Socializing
8:37 - 9:38 Interactive theory session "Presentation with Q&A"
9:40 - 10:15 Play with Docker Birthday 4 Labs
10:15 - 10:30 Coffee break
10:30 - 11:55 Singularity, rkt, lxd

© 2013-2016 Docker, Inc. All rights reserved
Docker 4th Birthday
#dockerbday

Docker Bday #4 celebrations worldwide!
• 150+ Bday meetups!
• 6000+ RSVPs
• 700+ mentors
#dockerbday

Join the Docker Student
Community! Sign up here:
http://dockr.ly/students (with your school email) for
access to our free Docker Student Developer Kit and
more!
Become a Docker
Campus Ambassador!
For leaders on campus who want to help their
peers learn Docker! Learn more and apply here:
http://dockr.ly/campus-ambassador
Are you a student?

Surveys and expectations
Assuming everyone knows a bit of Linux/Unix/Mac OSX CLI ?
Development, Operations, Security, Business, Others?
Devops
Configuration management
Containers
Schedulers
Containers eco system
Clusters, Load balancers, Orchestration

HPC

What is HPC?
▪ HPC workloads mostly
▪ Runs on Linux
▪ Runs on bare-metal for maximum performance, lower overhead
▪ HPC Application
▪ Broken into smaller parallel distributed problems across cluster
nodes.
▪ Utilizes inter-process communications heavily, shared memory, or
across network.
▪ Scientific computing

HPC
▪ HPC dominated by Academics research and discovery
▪ Industry in the last 5-10 years seen an increase in HPC interest (Car , O&E)
▪ Possible constraints:
▪ Snowflake deployments, each HPC cluster/supercomputer is build in mind with
specific use cases
▪ Long lived nodes.
▪ Bloated/drift/unclean maybe diskless reboots
▪ Reboot time, or launching app could be long due to system/memory checks, bootstrapping
▪ Traditional Data Center Linux distribution
▪ Fixed installation based on single enterprise distro (Scientific, RHEL, SLES)
▪ Old kernel features
https://arxiv.org/pdf/1702.05513.pdf #cHPC

Containers

First Step, Definition?
• The Application matters
• The application can be a process or a set of processes
• The use case might be not a running app
• Set of tools to develop an app
• Set of scripts "apps" that are part of a pipeline
• Isolated contained environment "Encapsulation"
• Synonyms
• chroot
• jail
• partition
• namespace
• zone

chroot/jail
A chroot on Unix operating systems is an operation that
changes the apparent root directory for the current running
process and its children. A program that is run in such a
modified environment cannot name (and therefore normally
cannot access) files outside the designated directory tree.
The term "chroot" may refer to the chroot(2) system call or
the chroot(8) wrapper program. The modified environment
is called a chroot jail.
https://en.wikipedia.org/wiki/Chroot

CONTAINERS?
WHAT ARE THEY REALLY?
Linux features?
Namespace
cgroupsLXC
Union file systems
Configuration management?
Virtualization technology?
npm
jar
Packaging ?
rpm
deb
tar.gz
Virtual/environment management ?
Sandboxing?
chroo
t
BSD jail Solaris zones
IBM VM/370 (1972)
seccomp

IT DEPENDS
Manual
Configuration
Traditional VMs
Less Portable
Minimal overhead
Most Portable
Lots of overhead
Configuration
Management tools
Containers
Docker
Intel Clear Containers
Singularity
LXC/LXD
Non-Repeatable Repeatable
rkt

DEVELOPERS LOVE DOCKER
17https://www.slideshare.net/dberkholz/cloud-native-in-the-enterprise-realworld-data-on-container-and-microservice-
adoption

KUBERNETES SEEING THE MOST DEVELOPER
TRACTION
18https://www.slideshare.net/dberkholz/cloud-native-in-the-enterprise-realworld-data-on-container-and-microservice-
adoption

Container
Containment, isolation or encapsulation of an environment.
Machine container:
Encapsulates a complete system image. e.g. Ubuntu, RHEL, Scientific Linux.
Process container:
Encapsulates a service/process(es) . e.g. Django, ROR, Gitlab, redis, Openfoam, kafka, spark.
What is the smallest application container?

Container Runtime
docker < 1.11.0
└── systemd
└── docker run OpenFoam
└── Docker Engine
└── OpenFoam
docker > 1.11.0
└── systemd
└── docker run OpenFoam
└── Docker Engine
└── containerd
└── runc
└── OpenFoam
rkt > 1.0
└── systemd
└── rkt run OpenFoam
└── OpenFoam
singularity (2.2.x)
└── systemd/(init)
└── bash
└── OpenFoam
https://medium.com/@adriaandejonge/moving-from-docker-to-rkt-310dc9aec938#.1glm3o1t3

Image formats
Layered
Overlay filesystems/Graph drivers
chrootDirectory
Archive
#OCI
#ACI

Use Cases: Packaging
Agnostic packaging
Captures
○ Dependencies
○ Environment
○ Configurations
○ Executables
○ How about data?
○ What Else?
■ hint: m*
Pack once, Run everywhere
http://hpcbios.readthedocs.io/en/latest/HPCBIOS_2012-92.html
#EasyBuild #lmod #GUIX #NYU-Environment-POSTER

Use Case: Portability
Portable/Scalable across
● platforms
● Distributions
● Environments
Separation of concerns, e.g. development pack and ship, operations scale and deploy.
development ensures app is resilient, operations enure infra is HA resilient and scalable

Use Case: Portability
Portable/Scalable across
● systems
● subsystems
● Anywhere
#BYOE

Use Case: Reproducible
Paolo Di Tommaso from the Center for Genomic Regulation presented : Manage Reproducibility of Computational Workflows with Docker Containers and
Nextflow.
https://www.slideshare.net/insideHPC/reproducible-computational-pipelines-with-docker-and-nextflow
https://youtu.be/Doo9H2-gBAk

27
Data Center current silo inefficient state
SchedulerScheduler
Jobs
Jobs
Jobs
Jobs
Jobs
Jobs
Scheduler
Jobs
Jobs
Jobs
Cluster Management A
Cluster Management B
Cluster Management C
Node as a work unit,
traditiontial single level
(silo) schedulers. No
holistic awareness of other
workloads

28
Data Center
Efficient Secure Allocation of Resources
VC3
BigData
VC1
Infra
VC2
HPC
Schedu
ler
Schedu
ler
Schedu
ler
DataCenter
Scheduler
jobs
Jobs
Jobs
Jobs
Jobs
Jobs
Jobs
Jobs
2nd Generation Cluster Management
Containers as a work unit,
container aware workload
schedulers integrated with
cluster management
software

29
Mesos DC/OS:
Example of Data Center/Container aware scheduler
▪ Mature, Open Source Apache Project
▪ Cluster Resource Manager
▪ Scalable to 10,000s of nodes
▪ Fault tolerant, no single point of failure
▪ Multi-tenancy with strong resource isolation
▪ Improved resource utilization
▪ Can schedule batch and interactive workloads for HPC and Big data.
https://people.eecs.berkeley.edu/~alig/papers/mesos.pdf
https://katacoda.com/courses/mesos/playground

30
HPC workload
runs on the cloud
25%

31
Which workloads and frameworks are running on
OpenStack?
Source : https://www.openstack.org/assets/survey/Public-User-Survey-Report.pdf
> 38%
scientific/technical
computing already
happening on
Openstack

EXAMPLE HPC Data Center Use Case
https://fosdem.org/2017/schedule/event/magnumcern/

33
NVIDIA Example use case
https://github.com/NVIDIA/nvidia-docker
http://www.nvidia.com/object/docker-container.html

Possible HPC Caveats/Constraints
1. Memory/storage deduplication
2. Code Optimization for specific architecture
3. Hardware environment Optimizations
4. Limited take on HPC specific orchestration and scheduling
5. Hardware topology assumptions (e.g. GPU brand, interconect)
6. Chroot based containers have limited tooling (e.g. introspection,
history, search)
7. chroot based containers might be hard to scan for security
vulnerabilities, hardening, and composition.

Container image security
Black listed artifacts
e.g. passwords, keys
3rd party software
e.g. libraries/packages
compiled from sourceSecurity Permissions
Configuration
Packages
License
Network
Metadata
Environment Variables
Context

36
MPI batch jobs
● use ssh inside container
● dssh http://www.qnib.org/2016/03/31/dssh/
● Capitalize on openmpi
○ Openmpi/pbs/TORQUE ( mpiexed does’t use ssh)
● Singularity examples uses Openmpi/Slurm
● Mesos mpi frameworks
● Commercial Univa/LSF/ support
● Research, and contribute ideas, pull requests to swarm,
kubernetes, slurm, mesos, and the alike.
● https://github.com/ambu50/wrapper-sq

37
Docker performance benchmarks
http://www.theregister.co.uk/2014/08/18/docker_kicks_kvms_butt_in_ibm_tests

DISCLAIMER
@kelseyhightower :
The problem with most blog posts attempting to compare two different systems is
the author not having the sufficient experience to do so.
https://twitter.com/kelseyhightower/status/826974374536187905

1. Introduction to Docker
#dockerbday

#dockerbday
Interesting Numbers
17k+
pull requests
40k+
stars
800k+
repos
10B+
downloads
2000+
contributors
280+
meetups
220k+
members
80+
countries

What is Docker?
The leading open source platform to pack, ship and run apps
as lightweight containers.
Developers: use Docker to eliminate “works on my machine” problems when
collaborating on code with co-workers.
Operators: use Docker to run and manage apps side-by-side in isolated
containers to get better compute density.
Enterprises: use Docker to build agile software delivery pipelines to ship new
features faster, more securely and with confidence for both
Linux and Windows Server apps.
#dockerbday

• Standardized packaging for
software and dependencies
• Isolate apps from each other
• Share the same OS kernel
• Works for all major Linux
distributions
• Containers native to Windows
Server 2016
What are Docker containers?

Comparing Containers and VMs
Containers are an app
level construct
VMs are an infrastructure level
construct to turn one machine
into many servers

Containers and VMs together
Containers and VMs together provide a tremendous amount of
flexibility for IT to optimally deploy and manage apps.

Evolution of the Docker Platform
Beginning
• Single purpose
• Linux developer community
#dockerbday

Evolution of the Docker Platform
Many purposes, users and infrastructure
Today
Developer
Community
Need to experiment
and innovate with
leading edge tech
Ops
Community
Enterprise
Partner
Ecosystem
Run business
critical apps at
scale anywhere
Extend and add
value to a platform
with a shared path
to monetization
Need a predictable
system to deploy
and run apps
#dockerbday

The Docker Platform
Developers Ops Enterprise Ecosystem
ONE PLATFORM
For Developers and IT
For Linux and Windows
On Premises and in the Cloud
Traditional Homegrown, Commercial ISV, Microservices
Docker Community Edition
Docker Enterprise Edition
Docker Certified
Docker Store
#dockerbday

What is a Docker Edition?
Making things simple for a great user experience
#dockerbday
NEW! Certification program for
Infrastructure, Plugins and Containers
Community EditionEnterprise Edition

Docker Community Edition (CE) & Enterprise Edition (EE)
Enterprise Edition (EE)
• CaaS enabled platform subscription
(integrated container orchestration,
management and security)
• Enterprise class support
• Quarterly releases, supported for
one year each with backported
patches and hotfixes.
• Certified Technology: Infrastructure,
Plugins, Containers
• Free Docker platform for “do it
yourself” dev and ops
• Monthly Edge release with latest
features for developers
• Quarterly release with maintenance
for ops
Community Edition (CE)
#dockerbday

Docker old versioning scheme
0.0.3 March
2013
1.0 June
2014
1.1 July
2014
1.2 August
2014
1.3 October
2014
1.4
December
2014
1.5
February
2015
1.6 April
2015
1.7 June
2015
1.8 August
2015
1.9
November
2015
1.10
Feburary
2016
1.11 April
2016
1.12.0 July
2016
1.12.1
August 2016
1.12.2
October
2016
1.12.3
October
2016

Product Versioning & Support
DockerCE
Edge
Stable
● NEW! Product Versioning follows a Year.Month model
● `docker-engine` package no longer exists. There’s only `docker-ce` and `docker-ee`.
● The binary formerly known as the engine is versioned YY.MM
DockerEE
EE
Released quarterly
Each version
supported for 1 year
v17.03 v17.04 v17.07v17.06v17.05 v17.08
v17.03
v17.06
v17.03
v17.06
v17.09 v17.10
v17.09
v17.09
#dockerbday

Where do you
download Docker
Community Edition?
#dockerbday

Docker Store!
• A marketplace for you to get the
latest trusted containers, plugins,
and Docker editions!
• You can search, browse, purchase
and manage from one location.
• Community Edition for:
− Mac
− AWS
− Fedora
− CentOS
−Windows
−Azure
−Ubuntu
−Debian
#dockerbday

Want to build and publish a container in Docker
Store?
Visit store.docker.com and click apply to publish through the Store
Publisher Program!

2. Learn Docker with Bday #4 Labs!
#dockerbday

Lab Instructions
STEP 1: Visit
http://birthday.play-with-docker.com/
Join the slack channel - #docker-bday-4
Join the Docker Community - dockr.ly/community
#dockerbday

STEP 2: Select the lab
you’d like to take.
http://birthday.play-with-
docker.com/
Lab Instructions
#dockerbday

As a special thank you for attending, use this code for a 30%
discount to attend DockerCon in Austin!
Register: http://2017.dockercon.com/
Code: BDAY4

Take a
#dockerselfie
#dockerbday

http://birthday.play-with-docker.com/
Join the slack channel: #docker-bday-4
Join the Docker Community:
https://community.docker.com/registrations/
groups/4316
#ISC2017 Docker Workshop
#dockerbday
#dockerselfie

Singularity

Scientific computing container

Singularity Container Selection Criteria
http://hpcugent.github.io/easybuild/files/EUM17/20170208-1_Singularity.pdf

Singularity speculations against Docker

Docker use in scientific computing
http://geekyap.blogspot.ch/2016/11/docker-vs-singularity-vs-shifter-in-hpc.html

Counter arguments I
Docker Singularity
privilege model namespace since 1.10
Feb 2016
suid, namespace added sep
2016
support current Linux
distro
kernel 3.10+ 2.6 kernel
Image build Dockerfile based build, some
configuration management tools
are trying to automate it, or
abstract it even more.
most of the time
bootstrapping from Docker is
the only working method out
of 4.
No additional network
configuration
configurable, one can use none,
host, or whatever network
plugin
None, which is fine for a
minimal HPC binary
No additional hardware shares kernel, view limited by
pid,user,ipc,mnt,network
except of network
namespace, chrooted
process shares host kernel

Counter arguments II
Docker Singularity
development maturity 5 years internal, 4 years
Open Source, 2000+
contributors
4 core developers, 1 year
old, limited community
security audited, scrutinized, running
in internet facing production
sites
- no key signing
- no introspection
- no vulnerability
scanner
- history, layer tracing
capabilities
….
eco system Huge eco system, vendor
support, and ISVs
small few companies
production usage Ubercloud, CERN, several
use cases presented in ISC
workshop
None, which is fine for a
minimal HPC binary

Counter arguments III
Docker Singularity
rdma Mellnox have provided
RDMA name space for multi
tenant hosts
None
Image caching works, options to inspect,
clean/prune it when needed
did not work for me on 2.2.0
rich API yes minimal functions, no restful
API to integrate with others,
other than SHUB
inspection, accounting yes None

https://singularity-hub.org/tools/compare

Play With Singularity
Demos
• https://asciinema.org/~bauerm
• https://asciinema.org/~vs
Vagrant Environment
• https://github.com/singularityware/singularity-vagrant
Workshop for last month Intel HPC devcon:
• https://github.com/singularityware/intel-hpc-devcon

Regardless of Singularity claims against Docker
Singularity benefits from Docker ecosystem
Given the context of internal HPC clusters not facing public internet and using in-house images.
- Singularity is minimalistic, simpler architecture, user interface and integration with existing HPC infrastructure.
- Doesn't require operations to install root Daemons.
- Enables separation of duties between Dev and Ops, allowing end users to bring their own packaged apps #BYOE
- Needs the support and contribution of the HPC and scientific community
Features wish list:
- Follow current standards, such as the OCI.
- Provide introspection and traceability
- Metadata
- Private SHUB
Scientific computing loves Singularity

rkt

What is rkt?
From the rkt GitHub page, "rkt (pronounced "rock-it") is a CLI for
running app containers on Linux. rkt is designed to be secure,
compassable and standards-based.
#ACI

Why rkt not Docker?
§ Don’t want to run Docker’s daemon.
§ Don’t require the Docker’s rich feature set/eco system. #KISS
§ Can’t trust Docker security yet, even though it is no longer an issue.
§ Have a modern Linux distro :
kernel > 4.3 and systemd version > 222
Similar reasons to why Singularity not Docker apart from the last

rkt playground
https://www.katacoda.com/courses/coreos/rkt-hello-world
https://www.katacoda.com/courses/coreos/playground

lxd

The Canonical Solution
https://www.slideshare.net/dustinkirkland/container-world-2017-the-questions-youre-afraid-to-ask-about-containers

§ Front end for LXC
§ Complete Linux environment
§ Enables simple restful management API to LXC
§ Secure by default
§ Simpler and less confusing tools
§ Checkpoint, restore, snapshot support
§ No drastic change in Infrastructure
§ Controls multi local and remote containers
§ OpenStack Nova plug-in for managing virtual LXD hosts in the cloud
LXD

§ https://linuxcontainers.org/lxd/try-it
§ https://stgraber.org/2017/03/05/run-your-own-lxd-demo-server/
Play with LXD

Container Distributions

Minimalist Container Distributions
q Atomic http://www.projectatomic.io/
q Container OS ( previously called CoreOS) https://coreos.com/os/docs/latest
q Rancher http://rancher.com/rancher-os/
q VMware Photon https://vmware.github.io/photon/
q SUSE MicroOs https://www.suse.com/communities/blog/rise-caas-platform/

What is Next in application management?
Not yet viable for HPC, however, have brilliant ideas. claims to be for modern
and legacy app. Still less than a year old
When you create a container image with Habitat, You know exactly what
went into the container and what is configurable about the application
Build immutable infrastructure but allow
last mile Application config changes
Build containers with a
Minimum Viable OS
Decouple the application build from the final
production ready container
Orchestrate the application
launch order and topology
required
https://www.habitat.sh/

References
q https://www.nextplatform.com/2017/03/02/solving-hpc-conflicts-containers/
q http://geekyap.blogspot.co.za/2016/11/docker-vs-singularity-vs-shifter-in-hpc.html
q https://www.enterprisetech.com/2017/03/02/docker-platform-fills-gaps-container-ecosystem/
q https://arxiv.org/pdf/1702.05513.pdf #cHPC, the HPC container prototype
q https://www.fosdem.org/2017/schedule/event/singularity/ https://www.nextflow.io/blog/2016/more-fun-
containers-hpc.html
q http://jvns.ca/blog/2016/10/02/i-just-want-to-run-a-container/
q Videos from 2nd EasyBuild User Meeting : Singualirty, Lmod, XALT and EasyBuild
https://www.youtube.com/playlist?list=PLVA9BuLC1j-yfxp2w-wraAGDCmhjb3o5Y
q http://www.vanessasaur.us/

Containers - Portable, repeatable user-oriented application delivery. Build, ship, run any app anywhere!

Recomendados

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente (20)

Similar a Containers - Portable, repeatable user-oriented application delivery. Build, ship, run any app anywhere!

Similar a Containers - Portable, repeatable user-oriented application delivery. Build, ship, run any app anywhere! (20)

Más de Walid Shaari

Más de Walid Shaari (12)

Último

Último (20)

Containers - Portable, repeatable user-oriented application delivery. Build, ship, run any app anywhere!