Recomendados
Recomendados
Más contenido relacionado
Similar a scale12x
Similar a scale12x (20)
Último
Último (20)
scale12x
- 1. Greater Configuration Re-Use Service Oriented Design & TDD
- 2. Overview • • • • • Background & Motivations Why greater configuration re-use? Service Oriented Design Change Mgmt & TDD Futures Socal Linux Conference 2014 2/21/2014 2
- 3. Background & Motivations • • • • • Developer, PM Became interested in ops Focused on service architecture & delivery Started Consulting Looking how to leverage work across projects Socal Linux Conference 2014 2/21/2014 3
- 4. 10K Ft. View Socal Linux Conference 2014 2/21/2014 4
- 5. Socal Linux Conference 2014 2/21/2014 5
- 6. Working on Distributed Melting Pots Socal Linux Conference 2014 2/21/2014 6
- 7. BIG DATA Socal Linux Conference 2014 2/21/2014 7
- 8. Why greater configuration Re-Use? Socal Linux Conference 2014 2/21/2014 8
- 9. Unique snowflakes Socal Linux Conference 2014 2/21/2014 9
- 10. Greater Re-Use • Leverage work/assets across projects & teams – No matter how different, they are pretty similar • Quicker development and release cycles • Be more nimble/agile from infrastructure perspective • Power in #’s (leverage work of experts) – Both external/internal to organizations Socal Linux Conference 2014 2/21/2014 10
- 11. Desired State • Completed Infrastructure • Fully operating service Socal Linux Conference 2014 2/21/2014 11
- 12. Getting there is complex… Socal Linux Conference 2014 2/21/2014 12
- 13. Everyone wants blocks Socal Linux Conference 2014 2/21/2014 13
- 14. Piece Together Socal Linux Conference 2014 2/21/2014 14
- 15. Developer Blocks Frameworks Infrastructure Languages Socal Linux Conference 2014 2/21/2014 15
- 16. Dev needs to solve a Problem? Socal Linux Conference 2014 2/21/2014 16
- 17. Ops Blocks Infrastrcuture Languages Socal Linux Conference 2014 2/21/2014 17
- 18. Ops needs to solve a Problem? Socal Linux Conference 2014 2/21/2014 18
- 19. Lets download my infrastructure… • puppet module install apache • knife cookbook site download apache Socal Linux Conference 2014 2/21/2014 19
- 20. Doh! Socal Linux Conference 2014 2/21/2014 20
- 21. Debug Time Socal Linux Conference 2014 2/21/2014 21
- 22. Debug Time Socal Linux Conference 2014 2/21/2014 22
- 23. Why me… Socal Linux Conference 2014 2/21/2014 23
- 24. Service Oriented Design/Delivery Socal Linux Conference 2014 2/21/2014 24
- 25. Service Oriented Design/Deliv. • Module/Component • Service Topology/Assembly • Benefits Socal Linux Conference 2014 2/21/2014 25
- 26. Object Oriented Analogy • Model driven approach lays a foundation for OO for infrastructure dev – Objects mapping to (?Whats best terms?): • Puppet classes/defs/resources • Chef recipe/resources – Nginx, Linux User, MySQL, ES, etc, etc – Focus on objects not “actions on objects” • ie: mysql not install_mysql Socal Linux Conference 2014 2/21/2014 26
- 27. Components == Objects • Objects as components that expose an interface • Modules/Cookbooks containing 1+ components • Components expose available attributes • Dependencies/Constraints – “locality” (more later) Socal Linux Conference 2014 2/21/2014 27
- 28. Model Example Socal Linux Conference 2014 2/21/2014 28
- 29. Model Example Socal Linux Conference 2014 2/21/2014 29
- 30. Benefits Socal Linux Conference 2014 2/21/2014 30
- 31. Benefits • Externalized, easy human/machine readable (yaml/json) • Introduce typed attributes (ie: port, log, etc) • Clear dependencies/constraints for usage • Interface compatibility • Foundation for Composition Socal Linux Conference 2014 2/21/2014 31
- 32. Assembly == Service Topology Socal Linux Conference 2014 2/21/2014 32
- 33. Assembly == Service Toplogy Socal Linux Conference 2014 2/21/2014 33
- 34. Assembly • Model definition describing a service instance • Includes – Attributes – Nodes and components that go on them – x-Component Dependencies • “App foo needs a database” – Constraints • “locality” • version Socal Linux Conference 2014 2/21/2014 34
- 35. 1 vs. Many Topologies • 1 Nailed Topology – Static dev/test environments – Prod that doesn’t change much/at all – Not leveraging/sharing x-org/externally • Many Topologies – You are vendor or OSS who’s software can be run in many ways – Fast moving, rapid iteration on architectures – Multivariate testing important (more later) Socal Linux Conference 2014 2/21/2014 35
- 36. Assembly Model Socal Linux Conference 2014 2/21/2014 36
- 37. Assembly Model Socal Linux Conference 2014 2/21/2014 37
- 38. Benefits Socal Linux Conference 2014 2/21/2014 38
- 39. Benefits • Externalized, easy human/machine readable (yaml/json) • Captures cross node dependencies • Basis for network analysis and connectivity • Testing testing testing…. Socal Linux Conference 2014 2/21/2014 39
- 40. Change Management Socal Linux Conference 2014 2/21/2014 40
- 41. Change Management • Upgrades to implementation • Interface changes – Attribute Defaults – Attributes Added/removed • Depedencies • New topology variants Socal Linux Conference 2014 2/21/2014 41
- 42. Test Driven Development Socal Linux Conference 2014 2/21/2014 42
- 43. Service Minded Testing Lint Unit Functional/Behavioral/Integation Socal Linux Conference 2014 2/21/2014 43
- 44. Service Minded Testing Lint Unit Functional/Behavioral/Integation Socal Linux Conference 2014 2/21/2014 44
- 45. Lint/Grammar • Puppet Lint – http://puppet-lint.com • Foodcritic – http://acrmp.github.io/foodcritic/ • Study perspective language style guides • Git post commit hooks Socal Linux Conference 2014 2/21/2014 45
- 46. Unit Testing • Rspec Puppet – http://rspec-puppet.com/ • Chefspec – http://https://github.com/sethvargo/chefspec Socal Linux Conference 2014 2/21/2014 46
- 47. That’s great, but is my SERVICE running!?!?! Socal Linux Conference 2014 2/21/2014 47
- 48. Enter Serverspec… • http://serverspec.org/ Socal Linux Conference 2014 2/21/2014 48
- 49. Serverspec Socal Linux Conference 2014 2/21/2014 49
- 50. Serverspec Socal Linux Conference 2014 2/21/2014 50
- 51. Serverspec Socal Linux Conference 2014 2/21/2014 51
- 52. Futures • Auto-generation – Serverspec verifications – Monitoring – Log Mgmt/Aggregration • Deeper integration with load tests/verifications • Containers to provide isolation and portability • Deep networking integration in component & service model Socal Linux Conference 2014 2/21/2014 52
- 53. Recap • • • • We want blocks to pull off the shelf and Formalizing notion of “component interface” Formalizing notion of service topology Managing changes at different levels – Implementation – Interface – toplogy design Socal Linux Conference 2014 2/21/2014 53
- 54. Thank you • Nate D’Amico • @kaiyzen • nate@reactor8.com • http://slideshare.com/kaiyzen/scale12x • http://github.com/rnp/scale12x Socal Linux Conference 2014 2/21/2014 54
Notas del editor
- Might have slide that says or notion of an interface capturesThe attributes that define the component in the same way that when you create objects in programming languages you provide the creation parameters. Now a pragmatic difference is that configuration objects can take many possible parameters (essentially any setting in a config file). So we view an interface more abstractly than you do in programming languages in away for exampe that is compatible with the use of hiera. The key is to have the user of a component know exactly how you can impact itI would not put internal vs external dependencies as ‘part of an interface’. This is getting low level but you could say that the inetrface definition can capture dependencies. SO in summary you might define an interafec as capturing- In a flexible way (that allows use of Hirea and similar) the attributes that can impact the behavior of the componentThe set of actions that can be performed, two special ones being for creation of the object and deletion
- Note sure main point you are trying to get across here
- Note sure main point you are trying to get across here
- Might put this in more pragmatic terms. There are many use cases where the customer has a nailed topology and can hard code topology assumptions into the components. On the other side there are many cases where topological flexibility is important, such as the vendor of a cluster topology that must support a wide range of topological variations. One of the challenges to achieving topogicalindepedence is wwhat we view as an inherit probem, that in the Puppet world manifests with dealing with the ‘multiple defs’problem. So for example may have many components that need java; if they are on separate nodes then don’t have a multipeldefs problem. On other hand if the correside on same node then the issue arises. Best way to handle this is with consumer and producer model where consumers expose tehir requirements/constraints (e.g., what version of java they need, how many bist) and goal is to determine if they can have tehir own separte resources or if not intersecting the constraints