Deploying to the cloud has made it easy to run large numbers of servers, but users may become dissatisfied with their particular cloud platform for reasons such as price, support and performance. There are a number of vendor lock-ins to avoid, this talk discusses how to do so with the open source configuration management and infrastructure automation platform Chef. Chef makes it easy to deploy to nearly every public and private cloud platform as well as virtualized and physical servers. Chef may also be used to deploy cloud infrastructures such as OpenStack, Eucalyptus or CloudStack. By abstracting away the platform, infrastructure becomes portable and you are free to deploy wherever necessary.

1. Achieving Infrastructure Portability
with Chef
CloudOpen 2012
Matt Ray
matt@opscode.com
IRC/Twitter/GitHub: mattray
www.opscode.com

2. Data Centers...

3. To the Cloud!

4. Why the Cloud?
• Instant infrastructure
• Unlimited capacity
• Autoscaling
• No commitment
• Immediate replacement

5. Cloud Differentiation
• Each cloud defines themselves
against Amazon
• Entry into the market is easier
• Not a lot of price competition
• Feature parity is growing

6. The Dark Side
of the Cloud

7. Why not the Cloud?
• Reliability
• Performance
• Security
• Price

8. Data Gravity

9. Know our escape plan
for every infrastructure
provider

10. Why Chef?

11. See Node
Application Server

12. See Nodes
Application Server
Application Database

13. See Nodes Grow
Application Server
Application Databases

14. See Nodes Grow
Application Servers
Application Databases

15. See Nodes Grow
Load Balancer
Application Servers
Application Databases

16. See Nodes Grow
Load Balancers
Application Servers
Application Databases

17. See Nodes Grow
Load Balancers
Application Servers
Application Database Cache
Application Databases

18. Tied together with Config
Load Balancers
Application Servers
Application Database Cache
Application Databases

19. Infrastructure is a Snowflake
Load Balancers
Application Servers
Application Database Cache
Floating IP?
Application Databases

20. Evolving Complexity
Load Balancers
ApplicationCache
Application Servers
NoSQL Database Cache
Database Slaves
Database

21. Complexity Grows Quickly
DC2
DC1
DC3

22. And it Continues to Evolve
That's great and all,
but tell me about
Chef!
http://www.flickr.com/photos/16339684@N00/2681435235/

23. Chef is Infrastructure as Code
• Programmatically
provision and configure
• Treat like any other code
base
• Reconstruct business from
code repository, data
backup, and bare metal
resources.
http://www.flickr.com/photos/louisb/4555295187/

24. Nodes
• Chef-Client generates
configurations directly
on nodes from their
run list
• Reduce management
complexity through
abstraction
• Store the configuration
of your programs in
version control
http://www.flickr.com/photos/ssoosay/5126146763/

25. Collections of Resources
• Networking • Routes
• Users
• Files • Groups
• Directories • Tasks
• Symlinks • Packages
• Mounts • Software
• Services
• Configurations
• Other Stuff
http://www.flickr.com/photos/stevekeys/3123167585/

26. Declarative Interface to Resources
• Define policy
• Say what, not how
• Pull not Push
http://www.flickr.com/photos/bixentro/2591838509/

27. Ruby!
extra_packages = case node['platform']
when "ubuntu","debian"
%w{
ruby1.8
ruby1.8-dev
rdoc1.8
ri1.8
libopenssl-ruby
}
end
extra_packages.each do |pkg|
package pkg do
action :install
end
end

28. Recipes and Cookbooks
• Recipes are collections of
Resources
• Cookbooks contain
recipes, templates, files,
custom resources, etc
• Code re-use and
modularity
• Hundreds already on
Community.opscode.com
http://www.flickr.com/photos/shutterhacks/4474421855/

29. Search
• Search for nodes
with Roles
• Find configuration
data
• IP addresses
• Hostnames
• FQDNs
http://www.flickr.com/photos/kathycsus/2686772625

30. Pass Results to Templates
pool_members = search("node","role:webserver”)
template "/etc/haproxy/haproxy.cfg" do
source "haproxy-app_lb.cfg.erb"
owner "root"
group "root"
mode 0644
variables :pool_members => pool_members.uniq
notifies :restart, "service[haproxy]"
end

31. Pass Results to Templates
# Set up application listeners here.
listen application 0.0.0.0:80
balance roundrobin
<% @pool_members.each do |member| -%>
server <%= member[:hostname] %> <%= member[:ipaddress] %>:> weight 1 maxconn 1 check
<% end -%>
<% if node["haproxy"]["enable_admin"] -%>
listen admin 0.0.0.0:22002
mode http
stats uri /
<% end -%>

32. So when this
Graphite Nagios
Jboss App
Memcache
Postgres Slaves
Postgres Master

33. Becomes this
Graphite Nagios
Jboss App
Memcache
Postgres Slaves
Postgres Master

34. Updates can be automatic
Graphite Nagios
Jboss App
Memcache
Postgres Slaves
Postgres Master

35. Count the resources
• Load balancer config
Graphite Nagios • Nagios host ping
• Nagios host ssh
Jboss App • Nagios host HTTP
• Nagios host app health
Memcache • Graphite CPU
• Graphite Memory
Postgres Slaves • Graphite Disk
• Graphite SNMP
• Memcache firewall
• 12+ resource changes for 1 node addition • Postgres firewall
Postgres authZ config

36. Build anything
• Simple internal applications
• Complex external applications
• Workstations
• Hadoop clusters
• IaaS infrastructure
• PaaS infrastructure
• SaaS applications
• Storage systems
• You name it
http://www.flickr.com/photos/hyku/245010680/

37. And manage it simply
• Automatically
reconfigure
everything
• Linux, Windows,
Unixes, BSDs
• Load balancers
• Metrics collection
systems
• Monitoring systems
• Cloud migrations
become trivial
http://www.flickr.com/photos/helico/404640681/

38. knife

39. knife with the Chef Server
• knife node
• create/delete/edit
• list
• knife cookbook ...
• knife role ...
• knife environment ...

40. knife bootstrap
knife bootstrap SERVER -r 'role[webserver]' -i ~/.ssh/id_rsa
• SSH to the machine given existing
credentials
• Install the Chef Client
• Register with the Chef Server
• Run the initial Run List
• Now managed with Chef!

41. knife ec2
$ knife ec2
Available ec2 subcommands: (for details, knife SUB-COMMAND --
help)
** EC2 COMMANDS **
knife ec2 flavor list (options)
knife ec2 instance data (options)
knife ec2 server create (options)
knife ec2 server delete SERVER [SERVER] (options)
knife ec2 server list (options)
$ knife ec2 server create -S keypair -i ~/.ssh/id_rsa -x ubuntu
-I ami-4721882e -f m1.small -r 'role[webserver]'

42. knife openstack
$ knife openstack
Available openstack subcommands: (for details, knife SUB-
COMMAND --help)
** OPENSTACK COMMANDS **
knife openstack flavor list (options)
knife openstack image list (options)
knife openstack server create (options)
knife openstack server delete SERVER [SERVER] (options)
knife openstack server list (options)
$ knife openstack server create -S keypair -i ~/.ssh/id_rsa
-x ubuntu -I 1231 -f standard.small -r 'role[webserver]'

43. Chef for Infrastructure Portability
• knife ec2 • knife cloudstack
• knife rackspace • knife openstack
• knife hp • knife vsphere
• knife google • ... and many
• knife azure others

44. The Chef Community
• Apache License, Version 2.0
• 850+ Individual contributors
• 150+ Corporate contributors
• HP, Dell, Rackspace, VMware, Joyent,
Calxeda, Heroku, SUSE and many more
• 550+ cookbooks
• http://community.opscode.com

45. Desktop, Virtualization, Private & Public Clouds
• Vagrant • AWS
• VMware • Rackspace
• CloudStack • HP
• Eucalyptus • Google
• OpenStack • Azure
• bare metal • many others

46. Desktop, Virtualization, Private & Public Clouds
• Vagrant • AWS
• VMware • Rackspace
• CloudStack • HP
• Eucalyptus • Google
• OpenStack • Azure
• bare metal • many others

47. Tale of the Tape
• Artur Bergman, CEO at Fastly
• It's All About Speed
• http://youtu.be/qRnTejOMbZU
• Jason Stowe, CEO at Cycle Computing
• CycleCloud + Chef = 50,000-core Utility
Supercomputer for Science
• http://youtu.be/cEaQB6e7G0Q

48. Chef and Abstractions
• Resources and Providers
• Cookbooks (may) normalize deployment
• Knife treats APIs the same
• Chef strives to not be opinionated

49. Environments
• Lock down versions of cookbooks
• stable releases vs. development
• Enforce attributes for deployment
• Ports, addresses, etc.
• Different run lists based on environment
• Debugging enabled?

50. Environments
• Use the same infrastructure code for
wherever you deploy
• Development, QA, Pre-Production, Prod
• Role-based Access Controls to restrict the
promotion of deployment code

51. TL;DL
• Every infrastructure is a unique snowflake
• Understand the costs associated with the
features of your platform(s) of choice.
• Chef enables Infrastructure Portability
• "Data Gravity" is the primary concern

52. Thanks!
Matt Ray
matt@opscode.com
IRC/Twitter/GitHub: mattray
www.opscode.com