7. Why invest in your workflow?
• Productivity!
• Work smarter, not harder
• You spend a lot of time writing/testing/debugging code
• Optimizing that is worthwhile
• Faster development cycle is more productive
• 5% faster cycle
• 5% more time for testing
• Less bugs
7
8. Revision control
• You must have your code in revision control
• git is preferred
• fast + cheap branches
• everyone else uses it
• github
• gitolite
!
• svn is also ‘workable’ 8
9. How do you run puppet?
• I like cron (daemon also fine)
• Two possible approaches
• —noop mode automatically + manual apply
• Automatic apply
9
10. How do you run puppet?
• I like cron (daemon also fine)
• Two possible approaches
• —noop mode automatically + manual apply
• Automatic apply
10
11. How do you run puppet?
• I like cron (daemon also fine)
• Two possible approaches
• —noop mode automatically + manual apply
• Automatic apply
• I recommend automatic apply
• Scary (don’t push to master unless you’re
confident!)
• puppet agent —disable (monitor this!)
• Testing workflow
• Eventual consistency
11
12. Move just fast enough to not break everything
• Test so that you’re confident
• Branch for every significant change
• Reduce batch size
• Small scary change easier to test
• Easier to roll back
• Otherwise - applying months of changes at once
• Really scary!
• Don’t even know desired effects!
• Communicate!
12
14. —noop
• Use —noop mode for testing!
ssh -A "$HOST" -- "sh -c 'cd $DESTDIR/
$PUPPET_DIRNAME; ./tools/puppet-standalone
--verbose --show_diff —noop'"
• tools/what-would-happen-on
14
15. Dynamic environments
git branch => puppet environment
puppet agent -t —environment my_test_branch
!
• puppet >= 2.7 has environment support
• Use puppetupdate or r10k to push branches
(Links at the end!)
15
17. Reporting
• Need to know what puppet did
• Puppet has logs + reporting functionality
• Push reports to:
• irc
• email (eww!)
• elasticsearch
• mysql
• puppetdb
• Saves compiled catalogs to disk
tools/what-just-happened-on
17
21. puppet-syntax
• Ruby gem
• Trivial to add to your project
• Checks .pp, .erb, .yaml
• Fast enough to run pre-commit
echo ‘bundle exec rake syntax’
>.git/hooks/pre-commit
chmod 755 .git/hooks/pre-commit
21
22. r10k/librarian - Puppetfile
• Awesome module deployment - with robots!
• Easy vendor/modules directory for modules from the forge
• Makes module = git repository pattern easier
• Not every module from the forge is useable immediately
• Fork on github (and make your changes open source)?
• Pull request and get them back upstream!
• Fork into internal git and modify.
• gitolite mirrors
• Improve performance
• No external dependencies
• Private forge (puppet-library)
22
23. Module template
• ‘puppet module generate’ uses a template
• Start from the GDS example one:
github.com/gds-operations/puppet-module-skeleton
• Modify to your taste!
23
24. ‘Real’ testing
• No hard rules.
• Invest to the level that’s right for you!
• Dev heavy teams
• Know about unit testing!
• Sysadmin heavy teams
• Less enthusiastic
• Do what provides value!
24
25. Feedback!
• Tighten your OODA loop!
• Don’t care how!
• N.B. Automated tests don’t work unless they’re
automated.
• I.E. MUST run on commit
• Whatever’s effective for your org
25
26. rspec-puppet
• Unit testing
• At least write a compile test for your code!
• Put it in your module template.
• Explicit dependencies FTW
!
• Use puppetlabs-spec-helper
• Inject mocks into spec/fixtures/manifests/site.pp
echo ‘define my::complex::dependency ($foo,
$bar) {}’ >> spec/fixtures/manifests/site.pp
26
27. serverspec
• Spin up Vagrant VM and apply your code
• Check properties of:
• files
• ports
• services
• Acceptance testing
• Slower and heavier weight than unit tests
• Can be highly valuable!
27
28. Jenkins - simple
• Put your tests together so that they can be run as one job
!
rake test
task :test =>
[:syntax, :spec, :integration]
• Get Jenkins to run it on commit to master
• git polling
• + add a post-receive hook to curl Jenkins
• Shout in email + irc!
28
29. Jenkins - less simple
• If branches are cheap (i.e. git!)
• Encourages people to push branches
• Code review++
• Adhoc
• Or pick your poison
• Run syntax checks and unit and/or integration tests on
every branch.
• Report back to committer
• irc notification
• write in code review
29
30. Jenkins integration branches
• Jenkins can merge branches!
• Push a branch
• Jenkins picks it up
• Merges with master
• Runs tests
• If they pass, pushes results
30
31. Jenkins integration branches
• Jenkins can merge branches!
• Push a branch
• Jenkins picks it up
• Merges with master
• Runs tests
• If they pass, pushes results
31
35. Policy vs Automation
• Testing is awesome
• Code review is awesome
• ‘Process is the scar tissue from previous problems’
• Empower people to change the process!
• If you make the tools simple to use…
• People will use them!
• Make doing the right thing a no-brainer
35