vodQA Pune, 2012

1. Automated Infrastructure Testing
Ranjib Dey
System Administrator, ThoughtWorks

2. Part 1
What we are building?

3. What you already know?

4. To Code (develop functionalities)
To Test (ensure if its working fine)

5. To Code (develop functionalities)
To Test (ensure if its working fine)
Automate

6. Enter DevOps/CD

7. To Code (develop functionalities)
To Test (ensure if its working fine)
To Deploy (ensure if it can be released)

8. To Code (develop functionalities)
To Test (ensure if its working fine)
To Deploy (ensure if it can be released)
Automate

9. To Code (develop functionalities)
To Test (ensure if its working fine)
To Deploy (ensure if it can be released)
Automate
Automate

10. To Code
To Test
To Deploy
Automate
Automate
CI

11. To Code
To Test
To Deploy
Automate
Automate
CI
CD

12. Part 2
How it is different?

13. Tested code
Deployed code
Automate

14. Tested code
Deployed code
Automate
Infrastructure

15. Tested code
Deployed code
Automate
Infrastructure
Operations

16. Tested code
Deployed code
Automate
Infrastructure
Operations
Data

17. Tested code
Deployed code
Automate
Infrastructure
Operations
Data
Compliance

18. Tested code
Deployed code
Automate
Infrastructure
Operations
Data
Compliance
End users

19. Tested code
Deployed code
Automate
Infrastructure
Operations
Data
Compliance
End users
Integrations

20. On infrastructure

21. To Code
To Test
To Deploy
Automate
Automate

22. To Code
To Test
To Deploy
Automate
Automate
Build infrastructure

23. To Code
To Test
To Deploy
Automate
Automate
Build infrastructure
Production
Infrastructure

24. Part 3
On testing

25. Tested code
Deployed code
Automate

26. Tested code
Deployed code
Automate
Infrastructure as Code

27. Tested code
Deployed code
Automate
Infrastructure as Code
Code

29. Tested code
Deployed code
Automate
Infrastructure as Code
Code
Test

30. Tested code
Deployed code
Automate
Infrastructure as Code
Code Test
Unit Testing

32. Tested code
Deployed code
Automate
Infrastructure as Code
Code Test
Unit Testing

33. Tested code
Deployed code
Automate
Infrastructure as Code
Code Test
Unit Testing
Functional Testing

34. Scenario: Nginx is running as a daemon
When I run "ps aux | grep nginx"
Then I should see “nginx:master" in the output

35. Tested code
Deployed code
Automate
Infrastructure as Code
Code Test
Unit Testing
Functional Testing

36. Tested code
Deployed code
Automate
Infrastructure as Code
Code Test
Unit Testing
Functional Testing
Integration Testing

37. describe_recipe 'nginx:configuration' do
it 'installs version 1.0.15' do
node[:nginx][:version].should == '1.0.15'
file("/etc/nginx/nginx.conf").must_have(:mode, "644")
package("nginx").must_be_installed
service("nginx").must_be_running
end
end

38. Tested code
Deployed code
Automate
Infrastructure as Code
Code Test
Unit Testing
Functional Testing
Integration Testing
Monitoring

39. • Monitoring

40. Tested code
Deployed code
Automate
Infrastructure as Code
Monitoring
Tests Mocks

41. Tested code
Deployed code
Automate
Infrastructure as Code
Monitoring
Tests Mocks
Analytics

42. Part 4
Cloud cometh

43. Cloud

45. I see more code

46. I see more code
I need more tests

47. Cloud
AWS

48. Cloud
AWSRackspace
GCE
HP
Joyent

49. Oh yeah
We love cross browser testing?

50. Part 5
Lessons learned

51. Health of infrastructure

52. Health of infrastructure
Battle tested code

53. Health of infrastructure
Battle tested code
Thorough Monitoring

54. • Remember to build feedback system .
– Test harness should publish metrics in the same
system where ops and project metrics are kept.
– Have analytics or trending system on top of the
metrics.

55. Collaborate more with
Ops.

56. Some advice
• Remember MIME types?
• World wont be fully automatic ever. Always
automate the most time consuming task first.
• No easy way out. The function domain
requires you to understand man pages, rfps
etc.
• Choose a text editor rather than an Window
manager based IDE.

57. References
• Infra-automation: http://www.opscode.com/chef/
• Monitoring: http://www.nagios.org/
• Lint tool: http://acrmp.github.com/foodcritic/
• Unit testing: https://github.com/acrmp/chefspec
• Functional testing: http://www.cucumber-chef.org/
• Integration testing:
https://github.com/calavera/minitest-chef-handler

58. @RanjibDey
http://ranjib.posterous.com
http://www.dzone.com/users/ranjibd
dey.ranjib@gmail.com