1. Automating Your Infrastructure
Antons Kranga, OUG Harmony 2012, 1, June 2012

2. Name: Antons Kranga
Role: Technology Architect, Technology Trainer
Works: Accenture, Member of JUG Latvia
Experience: 10+ in IT, Java
Current assignment: Cloud Computing.

3. Problem

4. Trends in IT
Copyright © 2012 Accenture All rights reserved. © behindthecloud.net 4

5. IT complexity and the timeline
Virtual Nodes
Physical Hardware
1980 1990 2000 2010+
Mainframe Client/Server Datacenter Cloud
1980 1981 1982 1983 1984 1985 1986 1987 1988 1989
1990
1991
1992
1993
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
Copyright © 2012 Accenture All rights reserved. © Mitchell Hill, Opscode, 5

6. Managing IT Complexity
Web Servers
Application
Servers
Database
Copyright © 2012 Accenture All rights reserved. 6

7. Managing IT Complexity
+1 server Web Servers
Application
Servers
Database
Copyright © 2012 Accenture All rights reserved. 7

8. Managing IT Complexity
20+ changes Web Servers
Application
Servers
Database
Copyright © 2012 Accenture All rights reserved. 8

9. Take IT Into the Cloud
Copyright © 2012 Accenture All rights reserved. 9

10. Take IT Into the Cloud
Cloud gives huge
Virtualization
Power
Copyright © 2012 Accenture All rights reserved. 10

11. Take IT Into the Cloud
Cloud gives huge
Virtualization
Power at low cost
Power
Copyright © 2012 Accenture All rights reserved. 11

12. Take IT Into the Cloud
…and Cloud
Infrastructure
Increases
Copyright © 2012 Accenture All rights reserved. 12

13. Take IT Into the Cloud
Cost of change
increases
exponentially
Copyright © 2012 Accenture All rights reserved. 13

14. Take IT Into the Cloud
As well as
infrastructure
Changes complexity
increases
exponentially
Copyright © 2012 Accenture All rights reserved. 14

15. Take IT Into the Cloud
How will we
manage our
infrastructure?
Copyright © 2012 Accenture All rights reserved. 15

16. Challenges of Cloud Adoption
• Virtualization Efficiency
• Infrastructure and Configuration
Complexity
• Security, Availability etc
• Cloud Provider Lock-in
• Critical Skills Shortage
Copyright © 2012 Accenture All rights reserved. 16

17. Example: Log Collection Becomes Complex
Example of Facebook Scribe
Copyright © 2012 Accenture All rights reserved. 17

18. How to survive in the Clouds?

19. Why Automate?
• Scalable
Infrastructure
• Accelerate Changes
• Cloud Provider
Abstraction
• Leverage Skills
Demand
• Manage
Configuration
Copyright © 2012 Accenture All rights reserved. 19

20. Configuration Management Tools
• Tight Integration with Infrastructure
• Enterprise Ready
• Write DSL (not shell commands)
• Promote Reuse
• Active Community
Copyright © 2012 Accenture All rights reserved. 20

21. Chef Overview

22. What is Chef?
Infrastructure automation platform
Copyright © 2012 Accenture All rights reserved. 22

23. What is Chef?
Chef Users write Cookbooks and
Recipes to describe infrastructure in
code
Copyright © 2012 Accenture All rights reserved. 23

24. Anatomy of the Cookbook
Cookbook
Configuration
Attributes
Recipe Artifacts
Copyright © 2012 Accenture All rights reserved. 24

25. Anatomy of the Cookbook
Cookbook
Written in
Ruby DSL
Configuration
Attributes
Recipe Artifacts
Copyright © 2012 Accenture All rights reserved. 25

26. Anatomy of the Cookbook
Cookbook
Configuration
Attributes
Recipe Artifacts
Resource
Recipe operates
with platform
agnostic resources
Copyright © 2012 Accenture All rights reserved. 26

27. Anatomy of the Cookbook
Cookbook
Configuration
Attributes
Recipe Artifacts
Resource
Ensures that
configuration
matches to the
Providers platform
Providers
Providers
Copyright © 2012 Accenture All rights reserved. 27

28. Anatomy of the Cookbook
Cookbook
Configuration
Attributes
Recipe Artifacts
Node and
Resource cookbook
configuration
properties
Providers
Providers
Providers
Copyright © 2012 Accenture All rights reserved. 28

29. Anatomy of the Cookbook
Cookbook
Configuration
Attributes
Recipe Artifacts
Cookbook
Resource contain other
configuration
information
Providers
Providers
Providers
Copyright © 2012 Accenture All rights reserved. 29

30. Anatomy of the Cookbook
Cookbook
Configuration
Attributes
Recipe Artifacts
Resource
Templates Files Libs
Providers
Providers
Providers
Copyright © 2012 Accenture All rights reserved. 30

31. Chef Architecture
Chef Server
(private or
hosted)
Chef Server
(Rest API)
Remote Node
Remote Node
Remote Node
Copyright © 2012 Accenture All rights reserved. 31

32. Chef Architecture
Chef Server
(private or
hosted)
Chef Server
(Rest API)
May have
WebUI
Remote Node
Remote Node
Remote Node
Copyright © 2012 Accenture All rights reserved. 32

33. Chef Architecture
Chef Server
(Rest API)
Nodes in your
infrastructure,
managed by
chef server
Remote Node
Remote Node
Remote Node
Copyright © 2012 Accenture All rights reserved. 33

34. Chef Architecture
Stores all nodes
configuration
attributes, cookbooks
cookbooks etc
Chef Server
CouchDB attributes
(Rest API)
…
databags
Remote Node
Remote Node
Remote Node
Copyright © 2012 Accenture All rights reserved. 34

35. Chef Architecture
All data
stored in
JSON cookbooks
Chef Server
CouchDB attributes
(Rest API)
…
databags
Remote Node
Remote Node
Remote Node
Copyright © 2012 Accenture All rights reserved. 35

36. Chef Architecture
cookbooks
Chef Server
CouchDB attributes
(Rest API)
…
search solr databags
Can be used by
user or
Remote Node cookbook
Remote Node
Remote Node
Copyright © 2012 Accenture All rights reserved. 36

37. Chef Architecture
cookbooks
Chef Server
CouchDB attributes
(Rest API)
…
search solr databags
Can be used by Provides powerful
user or search capabilities
Remote Node cookbook for CoachDB
Remote Node
Remote Node
Copyright © 2012 Accenture All rights reserved. 37

38. Chef Architecture
cookbooks
Chef Server
CouchDB attributes
(Rest API)
…
search solr databags
Remote Node RabbitMQ
Remote Node
Remote Node
Stores and
forwards data from
Chef to solr indexer
Copyright © 2012 Accenture All rights reserved. 38

39. Chef Architecture
cookbooks
Chef Server
CouchDB attributes
(Rest API)
…
search solr databags
Remote Node RabbitMQ
Remote Node Updates solr
Remote Node indexer
Stores and
forwards data from
Chef to solr indexer
Copyright © 2012 Accenture All rights reserved. 39

40. Chef Architecture
cookbooks
Chef Server
CouchDB attributes
(Rest API)
…
search solr databags
Remote Node has
All recipes
RabbitMQ
been combined in
Remote Node
Remote list and
run Node
executed by chef
client
Chef Client
Copyright © 2012 Accenture All rights reserved. 40

41. Chef Architecture
Chef client uses cookbooks
for execution “pull”
strategy which is
Chef Servermore flexible in
CouchDB attributes
(Rest API) real life
…
search solr databags
Remote Node RabbitMQ
Remote Node
Remote Node
Chef Client
Copyright © 2012 Accenture All rights reserved. 41

42. Chef Architecture
cookbooks
Chef Server
CouchDB attributes
(Rest API)
…
search solr databags
Remote Node RabbitMQ
Remote Node
Remote Node
Ohai is the tool
to discover
platform
attributes
Chef Client Ohai
Copyright © 2012 Accenture All rights reserved. 42

43. Chef Architecture
cookbooks
Chef Server
CouchDB attributes
(Rest API)
…
search solr databags
Remote Node RabbitMQ
Remote Node
Remote Node
User Workstation knife
knife
knife
Chef Client Ohai
Copyright © 2012 Accenture All rights reserved. 43

44. Chef Architecture
cookbooks
Chef Server
CouchDB attributes
(Rest API)
…
search solr databags
Remote Node RabbitMQ
Remote Node User executes
Remote Node
knife CLI to
operate with Chef
Server
User Workstation knife
knife
knife
Chef Client Ohai
Copyright © 2012 Accenture All rights reserved. 44

45. Demo!

46. Reading…
(video) Chef in 5 minutes Cookbook Community Cookbooks from 37 Signals
http://goo.gl/Of1p5 http://goo.gl/yAlPA http://goo.gl/uHC7A
Questions?

47. Thank You!