1.
CONTINUOUS
DEPLOYMENT OF PUPPET
MODULES
HOW WE DO IT AT MAILCHIMP

2.
BILL O'NEILL
@WONEILL

3.
EMAIL SERVICE PROVIDER
Deliver ~500 million emails daily
723 million emails delivered on Cyber Monday
of 97Sender Score
http://delivery.mailchimp.com/

4.
HISTORY

5.
Image source: https://blog.engineyard.com/2014/con gure-
before-you-boot

6.
CONFIGURATION MANAGEMENT IS HARD
"With Chef, Puppet, and CFEngine we found a
not-insigni cant learning curve on setting up
the different server daemons and learning the
DSL. This was particularly challenging when
we were con guring unique software not yet
given recipes by the existing community.
Given our cluster sizes, we also didn't really
need any of the advanced features those
systems provided."
- README from internally built tool

7.
MOVE TO COLOCATION
Buy vs. Lease analogy
Grow our Operations team
Needed a tool with dry-run mode

8.
PEOPLE MAKE MISTAKES

9.
HOW DO WE CATCH THESE MISTAKES AS EARLY
AS POSSIBLE?
AUTONOMATION
"automation with a human touch"
1. Detect the abnormality
2. Stop
3. Fix or correct the immediate condition

10.
HOW DO WE CATCH THESE MISTAKES AS EARLY
AS POSSIBLE?
DSL tools
Editor Support
Source Code Management
Continuous Integration

11.
DSL TOOLS
Puppet
ERB
YAML
Puppet Style Guide

12.
PUPPET
puppet parser validate mymanifest.pp

13.
package { 'openssh-server':
ensure => installed,
}
file { '/etc/ssh/sshd_config':
source => 'puppet:///modules/sshd/sshd_config',
owner => 'root',
group => 'root',
mode => '640',
notify => Service['sshd'] /* sshd will restart whenever you
edit this file. */
require => Package['openssh-server'],
}
service { 'sshd':
ensure => running,
enable => 'true',
hasstatus => 'true',
hasrestart => 'true',
}

14.
$ puppet parser validate validate_1.pp
Error: Could not parse for environment production:
Syntax error at 'require'; expected '}' at validate_1.pp:12

15.
ERB
erb -P -x -T '-' mytemplate.erb | ruby -c

16.
restrict default kod nomodify notrap nopeer<% unless @service %> noquery
restrict 127.0.0.1
restrict -6 ::1
driftfile /var/lib/ntp/drift
<% @serverlist.sort.each do |server| -%>
server <%= server %> iburst maxpoll 6
restrict <%= server %> mask 255.255.255.255 nomodify notrap noquery
<% end -%>

17.
$ erb -P -x -T '-' broken-ntp.conf.erb | ruby -c
-:11: syntax error, unexpected $end, expecting kEND

18.
YAML
npm install -g js-yaml; js-yaml hiera.yaml
ruby -e "require 'yaml'; YAML.load_file('hiera.yaml')"

19.
---
ntp::servers:
- 0.us.pool.ntp.org
- 1.us.pool.ntp.org
- 2.us.pool.ntp.org
- 3.us.pool.ntp.org
hp::ilo::settings:
ssh_status
type: global
value: true
ssh_port
type: global
value: '22'
http_port
type: global
value: '80'
https_port
type: global
value: '443'

20.
$ js-yaml hiera.yaml
JS-YAML: bad indentation of a mapping entry at line 9, column 13:
type: global
^
$ ruby -e "require 'yaml'; YAML.load_file('hiera.yaml')"
yaml.rb:133:in `load':
syntax error on line 9, col 14: ` value: true' (ArgumentError)
from yaml.rb:133:in `load'
from yaml.rb:144:in `load_file'
from yaml.rb:143:in `open'
from yaml.rb:143:in `load_file'
from -e:1

21.
PUPPET STYLE GUIDE
https://docs.puppetlabs.com/guides/style_guide.html
gem install puppet-lint
puppet-lint --fix /my/puppet/code

22.
package { 'openssh-server':
ensure => installed,
}
file { '/etc/ssh/sshd_config':
source => 'puppet:///modules/sshd/sshd_config',
owner => 'root',
group => 'root',
mode => '640',
notify => Service['sshd'], /* sshd will restart whenever you
edit this file. */
require => Package['openssh-server'],
}
service { 'sshd':
ensure => running,
enable => 'true',
hasstatus => 'true',
hasrestart => 'true',
}

23.
$ puppet-lint validate_2.pp
WARNING: quoted boolean value found on line 16
WARNING: quoted boolean value found on line 17
WARNING: quoted boolean value found on line 18
WARNING: indentation of => is not properly aligned on line 6
WARNING: indentation of => is not properly aligned on line 7
WARNING: indentation of => is not properly aligned on line 8
WARNING: indentation of => is not properly aligned on line 9
WARNING: indentation of => is not properly aligned on line 10
WARNING: mode should be represented as a 4 digit octal value
or symbolic mode on line 9
WARNING: /* */ comment found on line 10

24.
$ puppet-lint --fix validate_2.pp
FIXED: quoted boolean value found on line 16
FIXED: quoted boolean value found on line 17
FIXED: quoted boolean value found on line 18
FIXED: indentation of => is not properly aligned on line 6
FIXED: indentation of => is not properly aligned on line 7
FIXED: indentation of => is not properly aligned on line 8
FIXED: indentation of => is not properly aligned on line 9
FIXED: indentation of => is not properly aligned on line 10
FIXED: mode should be represented as a 4 digit octal value
or symbolic mode on line 9
FIXED: /* */ comment found on line 10

25.
package { 'openssh-server':
ensure => installed,
}
file { '/etc/ssh/sshd_config':
source => 'puppet:///modules/sshd/sshd_config',
owner => 'root',
group => 'root',
mode => '0640',
notify => Service['sshd'], # sshd will restart whenever you
# edit this file.
require => Package['openssh-server'],
}
service { 'sshd':
ensure => running,
enable => true,
hasstatus => true,
hasrestart => true,
}

26.
--- validate_2.pp 2014-12-08 09:43:38.000000000 -0500
+++ validate_2.pp-fixed 2014-12-08 09:50:51.000000000 -0500
@@ -3,18 +3,18 @@
}
file { '/etc/ssh/sshd_config':
- source => 'puppet:///modules/sshd/sshd_config',
- owner => 'root',
- group => 'root',
- mode => '640',
- notify => Service['sshd'], /* sshd will restart whenever you
- edit this file. */
+ source => 'puppet:///modules/sshd/sshd_config',
+ owner => 'root',
+ group => 'root',
+ mode => '0640',
+ notify => Service['sshd'], # sshd will restart whenever you
+ # edit this file.
require => Package['openssh-server'],
}
service { 'sshd':
ensure => running,
- enable => 'true',
- hasstatus => 'true',
- hasrestart => 'true',
+ enable => true,
+ hasstatus => true,
+ hasrestart => true,
}

27.
EDITOR SUPPORT

28.
VIM
PLUGINS FTW
with
Syntastic
vim-puppet
UltiSnips vim-snippets

29.
EMACS
http://www.emacswiki.org/emacs/PuppetProgramming

30.
GEPPETTO
http://puppetlabs.github.io/geppetto/index.html

31.
SOURCE CODE
MANAGEMENT

32.
COMMIT HOOKS
SCRIPT RUNNING THE DSL TOOLS AGAINST NEW FILES

33.
PEER REVIEW

34.
TRUNK BASED
DEPLOYMENT

35.
CONTINUOUS
INTEGRATION

36.
JENKINS
HTTPS://GITHUB.COM/VSTONE/JENKINS-
PUPPET-SCRIPTS

37.
WHY NOT RSPEC OR
BEAKER?

38.
CONTINUOUS
DEPLOYMENT

39.
REMEMBER TRUNK BASED
DEPLOYMENT?
# Keep environment up-to-date
vcsrepo { '/etc/puppet/environments/production':
ensure => latest,
provider => hg,
source => 'https://localhost/mercurial/puppet-modules',
}

40.
REVIEW TIME!
Catch mistakes early
Automation with a human touch
Trunk Based Deployments

41.
QUESTIONS?

42.
Slide sources at
THANKS!
BILL O'NEILL
WONEILL@POBOX.COM
@WONEILL
http://github.com/woneill/puppetcamp_atlanta_2014