Exploring Docker in CI/CD.

1. CI/CD with Docker
Henry Huang

2. Agengda
• Docker brings benefits on CI/CD process
• Coding: IDE with Docker
• Build with Docker
• Jenkins with Docker
– 1 Master x Slaves
– Masters x Slaves

3. Docker Provides
• Easy to prepare dev build environments
• No languages/libraries dependencies
• Build starts in seconds - more faster
• Increased robustness
• Simplified deployment on CI nodes
• Faster Rollback & Canary Release
• Easy to scale on CI nodes
• Well compatible with existing CI tools

4. CI/CD Process

5. Coding
• IDE with Docker
– Can run your code any where
If it runs on your laptop it will run on the integration
server too
– Don't have to bother about libraries, conflicts or
installing GCC, etc.
– Easy to build with different language packs or
libraries, etc.

6. Coding
• Requirements for IDE with Docker
– Dockerfile
• Syntax highlighting
• Autocomplete
• Syntax validation
– Compose yml file
• Define inter container relation as links and volumes
• Run multiple containers with one click
– IDE build system
• Run containers from the IDE with different language packs
or library requirement, etc.
– IDE runtime system
• Launch runtime environment inside the container for
further troubleshooting and testing

10. Docker Plugin for IDE
• Doclipser
• Sublime Docker
• IntelliJ IDEA 14.1
• Eclipse JBossTools
• Visual Studio 2015 RCTools for Docker -
Preview extension
Summary Page: http://domeide.github.io/

11. Build
• Currently, we cannot meet the CI principles:
– Every commit (to baseline) should be built
– Keep the build fast
– Make it easy to get the latest deliverables
• Problems:
– [Dragged] Update dev build environment
– [Dragged/Fragile] Build the source files
– [Complicated] Get the latest/specified
deliverables

12. Build with Docker
• Easy to prepare dev build environments
• No languages/libraries dependencies
• Build starts in seconds - more faster
• Increased robustness
• Simplified deployment on CI nodes
• Faster Rollback & Canary Release
• Easy to scale on CI nodes
• Well compatible with existing CI tools

13. Example: Way of Building Docker
• Chicken and Egg
– build the latest docker binary in old version of docker container
• Dockerfile: provides the necessary dependencies and
environment to build
– COPY source files into Docker build image
• hack/make.sh: provides the build script for
– binary, cross, unit test, integration-cli-test, etc.
• Verify the new version via dind (docker-in-docker)
• Deliverables
– Docker Images
– Binary
• mounting host folder to the dev container
• Via `docker cp`

14. dind - build docker
# git clone https://github.com/docker/docker.git
# cd docker; docker build -t docker-dev .
# docker run --privileged --rm -ti docker-dev /bin/bash
root@383dd9ee7613:/go/src/github.com/docker/docker# hack/make.sh binary
---> Making bundle: binary (in bundles/1.8.0-dev/binary)
Building: bundles/1.8.0-dev/binary/docker-1.8.0-dev
Created binary: bundles/1.8.0-dev/binary/docker-1.8.0-dev
root@383dd9ee7613:/go/src/github.com/docker/docker# docker
bash: docker: command not found
… …
root@383dd9ee7613:/go/src/github.com/docker/docker# docker -dD
root@383dd9ee7613:/go/src/github.com/docker/docker# docker run hello-world

15. dind - hello world

16. Jenkins to build docker
• https://jenkins.dockerproject.org/job/Docker%20Master/

17. Docker Registry
• Docker Registry
– V1: named as registry (python, boto)
• https://github.com/docker/docker-registry
– V2: named as distribution (re-write from scratch, golang)
• https://github.com/docker/distribution
• Storage model
– inmemory
– Filesystem
– S3
– azure: Microsoft Azure Blob Storage
– rados: Ceph Object Storage
• Frontend UI
– https://github.com/kwk/docker-registry-frontend
– No supportV2

18. Docker Registry
# http://docs.docker.com/registry/deploying/
• Take Dockerfile
– HTTP
• Add “--insecure-registry” into docker daemon options
# docker pull private-registry.iwsaas:5000/centos:latest
– HTTPS: SSL private key and certificate
• Take Compose
– Extra data volumes, etc.

19. Multiple Registries
• Mirroring (still in proposal)
• Cross repository (included in the v2.1 milestone)

20. Jenkins with Docker
• Build/Push docker images
– Execute docker CLI in jobs: case #1
– Docker build step plugin: case #2
• https://wiki.jenkins-ci.org/display/JENKINS/Docker+build+step+plugin
• Execute tasks inside docker container
– Docker plugin (dind): case #3
• https://wiki.jenkins-ci.org/display/JENKINS/Docker+Plugin
– Patched docker plugin (master and slave in the same
container level): case #4
• https://github.com/henrysher/docker-plugin

21. Demo for case 4
• Add new options into docker daemon marked in red:
OPTIONS='-H tcp://0.0.0.0:4243 -H unix:///var/run/docker.sock --selinux-
enabled'
• Run the docker image “iwsaas/jenkins” as a Jenkins container
# docker run -d -p 8080:8080 --name jenkins iwsaas/jenkins
b675af71332782a9bbfe6d92469d9a280ae751593b460520ac6bd6ab047b8b70
• Configure Docker Plugins
– Docker URL(docker0 virtual bridge address)
• http://172.17.42.1:4243
– DockerTemplate (iwsaas/jenkins-slave)
• Default username/password: jenkins/jenkins
• Create a “test” job and execute some basic commands

24. Case 4 - Console Output
• Execute the command inside the container
– Container ID: e380cfb050a1

25. Jenkins Problems
• Multiple Masters
– Sync & Standby
• Resource isolation & allocation (for slaves)
– Multiple tasks running on one slave
– CPU, Memory, Disk, etc. per task?

26. Mesos + Marathon +Docker+ Jenkins
• Mesos: kernel of distribute systems
• Marathon: init/upstart for mesos

27. Mesos + Marathon + Docker + Jenkins
• Mesos: kernel of distribute systems
• Marathon: init/upstart for mesos