Configuration Management in the Cloud - Cloud Phoenix Meetup Feb 2014

1. Configuration
Management in
the Cloud
Puppet and Chef

2. Let’s talk
•
•
•
•
•
•
•
What is Configuration Management
diff cloud.txt physical.txt > painful.out
Why is it painful?
Infrastructure as Code
Puppet
Chef
Examples

3. What is Configuration
Management
My own definition:
“The art of keeping everything under control”
Wikipedia:
“Configuration management (CM) is a systems
engineering process for establishing and maintaining
consistency of a product‟s performance, functional
and physical attributes with its requirements, design
and operational information throughout its life.”

4. diff cloud.txt physical.txt
> painful.out
• Physical:
o Resources stay there “forever”
o Attributes / properties are static (ips / hostnames / macaddress)
o Some cases is possible to recover the same system
• Cloud:
o Resources are dynamic and in constant change
“Some times they just disappear, WTF is the cloud it should be always
there”
o Attributes / properties change without notice
o Once a system is done, its done

5. Why is it painful?
• Config management systems where design for
static/physical environments.
• Most of them use certs/keys based on hostnames.
• With things as “bursting into the cloud” the config
management server that supported 100 servers now
it has to support 1K, 2K 15K servers.
• Most cloud environments cloud instances come
and go.
• In physical environments you don‟t need
completely automation from 0 to app
• Most CMS‟s don‟t have rollbacks.

6. Infrastructure as Code in
the Cloud
• Keep your CM code in repositories (git/svn)
• Replicate… replicate… replicate…
• The CM system wont do everything by itself
• Have your Dev, Test and Prod environments
• If something fails… destroy and rebuild
• Go Masterless whenever possible

7. Puppet
• Pros
o Ruby based
o Easy to read and learn
o You can do pretty much anything
• Cons
o Custom changes require you to build specific prividers, resources and the
DSL is not as good as you would like
o Based on certs using hostnames to generate them
o Master/Client communication
o Does not scale very well

8. Chef
• Pros
o
o
o
o
o
Ruby based
You literally can code in it
You can apply order to the things he will execute
Provides an encrypted way to pass sensitive data
Provides more utilities (knife and search)
• Chef
o
o
o
o
Master server requires more components
Syntax a little bit more complex
You need to learn ruby to get the good out of it
Master/Client communication

9. Puppet Arch
• Semi Masterless
• Architecture:

10. Chef Arch
• Master/Client
• Architecture

11. Puppet Module
• Apache
o Files
• Cert.key
• Ca.key
o Templates
• Vhost.erb
o Manifests
• Init.pp
• Redhat
o Install.pp
o Config.pp
o Postconfig.pp
o Service.pp

12. Puppet Code – init.pp
• Init.pp
Class apache (
$servername = “myserver”,
$port
= 80,
$serveradmin = “admin@email.com”
){
case @::operatingsystem {
“redhat”, “centos”:
{ require apache::redhat::service }
“ubuntu”:
{ require apache::ubuntu::service }
default:
{ require apache::redhat::service }
}
}

13. Puppet Code – install.pp
Class apache::redhat::install (
){
package {
“httpd”:
ensure => “latest”;
}
}

14. Puppet Code – config.pp
Class apache::redhat::config (
$servername
$serveradmin
$serverport
= $apache::servername,
= $apache::serveradmin,
= $apache::serverport
){
require apache::redhat::install
}
file {
“/etc/httpd/conf.d/myvhost.conf”:
owner
=> “apache”,
group
=> “apache”,
content => template(„apache/vhost.erb‟);
}

15. Puppet Code – service.pp
Class apache::redhat::service (
){
require apache::redhat::config
service {
“httpd”:
ensure => “running”;
}
}

16. Puppet Masterless
• Create bootstrap script that:
• Download Repository into the Cloud instance
• Create a manifest.pp with the contents of the node
definition
• Call puppet apply -vd -modulepath=/location/modules/ manifest.pp
• Example manifest.pp
import “whatever”
class { “apache”:
servername
=> “myserver.com”,
serveradmin
=> “myemail@gmail.com”,
port
=> 8080
}

17. Chef Code
• Roles
o Webserver.json
• Cookbooks
o Attributes
• Default.rb
o Files
• Cert.key
• Ca.key
o Templates
• Vhost.erb
o Libraries
o Providers
o Resources
o Recipes
• Default.rb
• install.rb
• Config.rb
• Vhost.rb

18. {
Chef Roles
"name": ”webserver",
"default_attributes": {
"service": ”httpd”,
“port”: “80”,
"packages": {
"extras": [
”httpd"
]
}
},
"chef_type": "role",
"env_run_lists": {
},
"run_list": [
"recipe[minitest-handler@1.0.6]",
"recipe[basenode@1.0.50]",
"recipe[chef-client@1.1.26]",
"recipe[release_version@8.0.19]",
"recipe[ops@1.0.16]",
"recipe[chef-workstation@1.0.5]”,
“recipe[apache@1.0.1]”
],
"override_attributes": {
},
"description": ”webserver",
"json_class": "Chef::Role"
}

19. Chef Cookbook Attributes
default['dns']['subdomains']
= ['production', 'test', 'development']
default['dns']['basedomain']
= 'demiops.com.'
default['dns']['route53']['register'] = true
default['dns']['route53']['default_ttl'] = '300'
default['resolver']['options']
= ['rotate', 'attempts:5']
default['resolver']['nameservers']
= ['127.0.0.1']
default[„web‟][„port‟]
= „80‟
default[„web‟][„servername‟] = „myserver.com‟
default[„web‟][„serveradmin‟] = „myemail@gmail.com‟

20. Chef Cookbook - Recipes
Default.rb
include_recipe “apache::install"
include_recipe ”apache::config"
include_recipe “apache::vhost"
include_recipe ”apache::authorized_keys”
Authorized_keys.rb
cookbook_file "/root/.ssh/authorized_keys" do
group "root"
owner "root"
mode 0600
source "authorized_keys"
end

21. Chef in the Cloud
• Create a bootstrap script that:
• Download the chef repository into the cloud
instance
• Use minitests to check everything worked
• Install chef-client and knife in the instance
• Use knife to search chef-client inventory and
update dynamically config files
• Use ohai

22. Questions ?