1. Continuous Delivery Pipeline with Jenkins, Puppet and Mcollective
From an Engineer’s perspective
@diptanu

2. DevOps ?

3. Continuous Delivery

4. Why Continuous Deployment
• Get measurable changes fast
• Reduce the risk of releases
• Loosely coupled but coherent teams
• Get rid of the “works-on-my-box”
phenomenon
• Forces you to think where does state full
things live in your system

5. First Principles
• CI is just the orchestrator
• Package and version everything
• Don’t use the build tool after tests have been
run and packages have been made
• Divide and Conquer
– Use tools that solve one particular problem
• Think hard what the pillars of infrastructure are

6. Jenkins
• Distributed Builds
• Shard responsibility of build infrastructure
– For example, don’t treat the CI server as your
artifact repo
• Think about what happens when Jenkins goes
down
• Discard old builds

7. Jenkins
• Use the Jenkins Job Builder to configure jobs
• DynaSlave plugin if you need a lot of scale
• Build Flow Plugin/Build Pipeline Plugin for
visualization
• Job DSL Plugin
• Configuration Slicing Plugin

8. Jenkins
• Disable Jobs that haven’t built successfully
• Set number of builds
• Mark build permanent
<3 Jenkins everyday

9. Puppet
• Infrastructure as code
• Node-less and master less puppet
• Make your vizualization tool consume the
reports
• Fact Driven puppet
• Masterless puppet
• Decouple data from manifests

10. NodeLess Puppet
• No definitions like
node “webserver01.foobar.com”
• There are only “types” of instances
node default {
include truth::enforcer
}

11. NodeLess Puppet
class truth::enforcer {
if has_role("ci"){
notice("I am a CI Server")
include tools::ci
}
if has_role(”webserver"){
notice(“I am a webserver”)
include tools::frontend
}
}

12. MCollective
Web Servers
Application
Servers
Cacheing
Servers
Metrics/Loggin
g

13. MCollective
• Create Collectives of resources on the network
• And operate on collectives rather than single
hosts
• Understands facts that are exposed by facter
mco package –W /dev_server/ install foo-puppet-86
mco service –W /application_servers/ restart iptables

14. AWS
• Ruby/Python SDK
• Immutable compute instances
• Make instances join the collective
Load Compute Nodeless Puppet
Balancer Instance
Monitori
ng Jenkins
System

15. A Pipeline in the wild
Team City
Build
Code Daily Manual
Commit Deployment
TFS Build DB
Manager Quality
Assurance
Fortnightly
Manual
Deployment

16. Software Delivery Pipeline
Publish
Automated Deploy to Smoke Tests
Commit Package
Tests Dev
Deploy to QA Deploy to Integration
Environments Environments

17. Deployment Pipeline
Test and
Developers SCM
Package
Ops
Package
Repository
curl -i -F package=@foo-puppet-86-1.i386.rpm http://selihost/<repository_name>/upload

18. Deployment Pipeline
Run your
functional
Deploy to Dev
tests from a
different host
mco
Seli/Jenkins
Xtrigger
a. yum install –y foobar-puppet-86-1
Dev Instances b. sudo puppet apply …
Package
Repository

19. Package Repositories
• Upload a package whenever a build happens
• Serve Packages
• Archive Strategy
• Trigger the deployment pipeline whenever
• Statistics
• API to get status of the metadata

20. Introducing Seli
• A REST API to upload packages
• Comes with an embedded HTTP Server
• Shows Stats
• And specify rules for archiving packages
• Tag Packages
• Triggers URL when a package is uploaded

21. In closing
• Make your CD Pipeline reflect your release
workflow
• Automate everything
• Measure everything that moves
• Think about your infrastructure as collectives

22. Questions?
@diptanu