The DevOps Toolkit session by Jonas Rosland, presented at EMC World 2015

1. 1© Copyright 2015 EMC Corporation. All rights reserved. 1© Copyright 2015 EMC Corporation. All rights reserved.

2. 2© Copyright 2015 EMC Corporation. All rights reserved.
THE DEVOPS TOOLKIT
GET IT, USE IT, MAKE IT YOURS
2© Copyright 2015 EMC Corporation. All rights reserved.

3. 3© Copyright 2015 EMC Corporation. All rights reserved. 3© Copyright 2015 EMC Corporation. All rights reserved.
Jonas Rosland
Developer Advocate
EMC {code}
@jonasrosland
Loves retro gaming
Talks a lot

4. 4© Copyright 2015 EMC Corporation. All rights reserved.

5. 5© Copyright 2015 EMC Corporation. All rights reserved.
WHAT IS DEVOPS?

6. 6© Copyright 2015 EMC Corporation. All rights reserved.
AN IDEA OR COLLABORATIVE
CULTURE/PHILOSOPHY BETWEEN
TECHNICAL TEAMS

7. 7© Copyright 2015 EMC Corporation. All rights reserved.
UNIFIED PROCESSES AND TOOLS
FOR FASTER END-TO-END DELIVERY
OF QUALITY SOFTWARE

8. 8© Copyright 2015 EMC Corporation. All rights reserved.
NOT A JOB DESCRIPTION!

9. 9© Copyright 2015 EMC Corporation. All rights reserved.
SO WHY SHOULD YOU
CARE?

10. 10© Copyright 2015 EMC Corporation. All rights reserved.
YOU CAN'T WAIT 6 MONTHS FOR A
FEATURE UPGRADE/BUG FIX/ETC

11. 11© Copyright 2015 EMC Corporation. All rights reserved.
SOLVE CONFIGURATION DRIFT
ISSUES QUICKLY

12. 12© Copyright 2015 EMC Corporation. All rights reserved.
KNOW EXACTLY WHAT'S WRONG,
WHERE AND WHY

13. 13© Copyright 2015 EMC Corporation. All rights reserved.
http://46zwyrvli634e39iq2l9mv8g.wpengine.netdna-cdn.com/wp-content/uploads/2013/06/DevOps-infinity-loop2.png

14. 14© Copyright 2015 EMC Corporation. All rights reserved.
LAST BUT NOT LEAST, IMPROVE
TEAM COMMUNICATION

15. 15© Copyright 2015 EMC Corporation. All rights reserved.
SO WHAT'S THE RESULT OF
DEVOPS?

16. 16© Copyright 2015 EMC Corporation. All rights reserved.
IT'S REALLY BIG
ACTUALLY...

17. 17© Copyright 2015 EMC Corporation. All rights reserved.
DEVOPS PRACTICES AND IT
PERFORMANCE IMPACT
ORGANIZATIONAL PERFORMANCE
http://www.slideshare.net/realgenekim/2014-state-of-devops-findings-velocity-conference

18. 18© Copyright 2015 EMC Corporation. All rights reserved.
LET'S START WITH
INFRASTRUCTURE AS
CODE

19. 19© Copyright 2015 EMC Corporation. All rights reserved.
WHAT DO WE DEFINE AS
INFRASTRUCTURE?

20. 20© Copyright 2015 EMC Corporation. All rights reserved.

21. 21© Copyright 2015 EMC Corporation. All rights reserved.

22. 22© Copyright 2015 EMC Corporation. All rights reserved.

23. 23© Copyright 2015 EMC Corporation. All rights reserved.
• Deploy, monitor and connect together all the pieces
needed to run services for the organization
• Processes and tools for faster end-to-end delivery of
quality services
• Automation comes built-in
WHY DEFINE INFRASTRUCTURE AS CODE?

24. 24© Copyright 2015 EMC Corporation. All rights reserved.
• Desired state specified in text files
• Autonomic (self-corrects to desired state)
• State should be known through monitoring
• Remove snowflake servers
HOW DO WE DO THIS?

25. 25© Copyright 2015 EMC Corporation. All rights reserved.
• Easy to read and edit
• Shareable
• Can use standard version control like Git or SVN
• Becomes executable documentation
WHY STORE THEM IN TEXT FILES?

26. 26© Copyright 2015 EMC Corporation. All rights reserved.
node 'www2' {
class { 'apache': } # use apache module
apache::vhost { 'awesomewebsite.com': # define vhost
port => '80',
docroot => '/var/www/html'
}
}
SIMPLE EXAMPLE

27. 27© Copyright 2015 EMC Corporation. All rights reserved.
user { root:
ensure => present,
password => '$ecretP@ssw0rd',
}
CHANGE ROOT PASSWORD

28. 28© Copyright 2015 EMC Corporation. All rights reserved.
MANUALLY CONFIGURED
ENVIRONMENTS ARE LIKE A HOUSE
OF CARDS IN A CHINA SHOP
- NEAL FORD

29. 29© Copyright 2015 EMC Corporation. All rights reserved.
• Deploying, provisioning and scaling automatically is
virtually impossible if every server is unique
• Adds friction between the requestor and the deployer
• Mistakes happen
• We're all human
SNOWFLAKE SERVERS

30. 30© Copyright 2015 EMC Corporation. All rights reserved.
WHAT CAN HAPPEN IF YOU
DON'T TREAT
INFRASTRUCTURE AS CODE?

31. 31© Copyright 2015 EMC Corporation. All rights reserved.
UP TO 60% OF FAILURES ARE
CAUSED BY HUMAN ERROR, NOT
HARDWARE FAILURE

32. 32© Copyright 2015 EMC Corporation. All rights reserved.
EXAMPLE - KNIGHT CAPITAL GROUP
https://infocus.emc.com/wp-content/uploads/2012/08/KCG.jpg

33. 33© Copyright 2015 EMC Corporation. All rights reserved.
• Manual deployment of new trading software
• 7 of 8 servers correctly updated
• Old function still alive on the 8th server led to…
KNIGHT CAPITAL GROUP - WHAT HAPPENED?

34. 34© Copyright 2015 EMC Corporation. All rights reserved.
$440 MILLION LOSS
IN 45 MINUTES

35. 35© Copyright 2015 EMC Corporation. All rights reserved.
SO

36. 36© Copyright 2015 EMC Corporation. All rights reserved.
TREAT YOUR
INFRASTRUCTURE AS
CODE

37. 37© Copyright 2015 EMC Corporation. All rights reserved.
• Self documenting infrastructure
• You now have source code for how anything in your
datacenter is setup
• Executable documentation
RECAP OF THE BENEFITS

38. 38© Copyright 2015 EMC Corporation. All rights reserved.
ALRIGHT

39. 39© Copyright 2015 EMC Corporation. All rights reserved.
THEN WHAT?

40. 40© Copyright 2015 EMC Corporation. All rights reserved.
HOW ABOUT
AUTOMATING SOME
MORE?

41. 41© Copyright 2015 EMC Corporation. All rights reserved.
• Jenkins CI
• Travis CI
• Codeship
• CloudFoundry
• OpenShift
DEPLOYMENT AUTOMATION

42. 42© Copyright 2015 EMC Corporation. All rights reserved.
• Docker
• Framework for application container management
• HUGE community
CONTAINERS

43. 43© Copyright 2015 EMC Corporation. All rights reserved.
• Splunk
• Loggly
• LogInsight
• ELK Stack
– ElasticSearch
– LogStash
– Kibana
LOG MANAGEMENT

44. 44© Copyright 2015 EMC Corporation. All rights reserved.
• vCenter Operations
• New Relic
• AppDynamics
• DataDog
PERFORMANCE MANAGEMENT

45. 45© Copyright 2015 EMC Corporation. All rights reserved.
• Nagios
• Sensu
• VictorOps
• PagerDuty
MONITORING

46. 46© Copyright 2015 EMC Corporation. All rights reserved.
• Slack
• HipChat
• Trello
• Asana
COMMUNICATION

47. 47© Copyright 2015 EMC Corporation. All rights reserved.
• Github
• Quip
COLLABORATION

48. 48© Copyright 2015 EMC Corporation. All rights reserved.
• And by smartly we mean cooperatively between
Devs and Ops
• Will help operators understand what's running and
happening in the infrastructure
• Will give developers insight into the underlying
infrastructure and the way it behaves under load
USE THESE TOOLS SMARTLY

49. 49© Copyright 2015 EMC Corporation. All rights reserved.
• Start talking about it
• Choose low-hanging fruit
• Learn, make wrong decisions, learn again
• Involve the broader team
– Give books, presentations, blog posts etc.
SO HOW DO WE GET STARTED?

50. 50© Copyright 2015 EMC Corporation. All rights reserved.
QUESTIONS?
Jonas Rosland
Developer Advocate @ EMC {code}
@jonasrosland