So I Wrote a Manifest
•Download as PPTX, PDF•
1 like•2,058 views
Puppet Camp Melbourne 2015 by Ryan Armstrong of Kinetic IT
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to So I Wrote a Manifest
Similar to So I Wrote a Manifest (20)
More from Puppet
More from Puppet (20)
Recently uploaded
Recently uploaded (20)
So I Wrote a Manifest
- 1. SO I WROTE A MANIFEST… What next? Ryan Armstrong @cavaliercoder cavaliercoder cavaliercoder.com
- 2. WHAT TO AUTOMATE FIRST?
- 3. We shouldn’t be looking at each local area and trying to trim it. We should be trying to optimize the whole system - Eliyahu M. Goldratt Ryan Armstrong @cavaliercoder cavaliercoder cavaliercoder.com
- 4. Project Start Server build request Change Mgmt Server build QA Firewallrequest Change Mgmt Database request Storage assessment Security assessment Firewall implementation Server build Presentation layer request App install QA Cap/Av assessment Change Mgmt Storage provisioning Storage request Change mgmt Database provisioning Load Balancer request Change mgmt LB Config provisioning Config mgmt Rev. Proxy config provisioing Ryan Armstrong @cavaliercoder cavaliercoder cavaliercoder.com
- 5. Man Machine Method Measure Ryan Armstrong @cavaliercoder cavaliercoder cavaliercoder.com
- 6. # install IIS core package { ‘IIS-CommonHttpFeatures’ : ensure => present, } THE METHOD Install the IIS Web server feature by navigating to Control Panel, Programs, Turn Windows Features on or off. Drill down to Internet Information Services, World Wide Web Services and tick Common HTTP Features. Click OK, OK. Validate the install by navigating to Administrative Tools, Services and ensure the World Wide Web service is started. Secondly open a web browser and navigate to http://localhost and ensure the IIS Welcome page is displayed. Ryan Armstrong @cavaliercoder cavaliercoder cavaliercoder.com
- 7. THE MEASURE
- 9. THE MAN? OR WOMAN #WWCODE Ryan Armstrong @cavaliercoder cavaliercoder cavaliercoder.com
- 10. Any improvements made anywhere besides the bottleneck are an illusion. - Gene Kim Ryan Armstrong @cavaliercoder cavaliercoder cavaliercoder.com
- 11. Project Start Server build request Change Mgmt Server build QA Firewallrequest Change Mgmt Database request Storage assessment Security assessment Firewall implementation Server build Presentation layer request App install QA Cap/Av assessment Change Mgmt Storage provisioning Storage request Change mgmt Database provisioning Load Balancer request Change mgmt LB Config provisioning Config mgmt Rev. Proxy config provisioing Ryan Armstrong @cavaliercoder cavaliercoder cavaliercoder.com
- 13. Ryan Armstrong @cavaliercoder cavaliercoder cavaliercoder.com Bug: Doc shot Branch: Fix Doc shooting Tag: Doc alive Bug: McFly Jr Jailed Tag: I’m OUT Griff Branch: Where we’re going… Branch: Almanac Bug: Biff rich! Branch: Took that guys wallet Tag: Almanac burned Release v1.0.0
- 16. TYING MODULES TOGETHER Ryan Armstrong @cavaliercoder cavaliercoder cavaliercoder.com IIS 8.5 MVC .Net Choco Backup agent AV Agent SQL Server Log stash Zabbix agent WSUS httpd MySQL Bamboo agent Vmware tools Active MQ IIS 8.5 Java RE HA Proxy Tomcat Elastic Search PHP Redis
- 17. ROLES AND PROFILES Ryan Armstrong @cavaliercoder cavaliercoder cavaliercoder.com - org/ # ‘org’ module - .git/ # git database (hidden) - manifests/ - roles/ - my_app.pp # class org::role::my_app - profiles/ - iis85.pp # class org::profile::iis85 - mvc_net.pp # class org::profile::mvcnet - my_app.pp # class org::profile::my_app - resources/ - service_account.pp # define org::resources::service_account - ...
- 18. ROLES (BUSINESS LAYER) • Apply directly to a host (one per host) • Assigned in site.pp or ENC • May only contain profiles • Accepts no parameters # standard configuration for custom # MVC.Net application class org::roles::my_app { include ::org::profiles::iis85 include ::org::profiles::dotnet45 include ::org::profiles::mvc_net include ::org::profiles::my_app }
- 19. PROFILES (IMPLEMENTATION LAYER) • Only applied via Roles • Contains resources • May accept parameters from hiera • Parameters are defined with site specific defaults # standard IIS 8.5 configuration class org::profiles::iis85 ( $log_mount = ‘log_serverlogs’, ) { package { ‘IIS-CommonHttpFeatures’ : ensure => present, } ...
- 20. RESOURCES (COMPONENT LAYER) • Declared with define for repeating an implementation on a node E.g. User accounts • Only applied via Profiles • Complex resources should become their own module • Rarely used Ryan Armstrong @cavaliercoder cavaliercoder cavaliercoder.com # standard local service account define org::resources::svc_account ( $username = undef, $password = undef, ) { user { $username : ensure => present, password => $password, } ...
- 21. DATA • Applied via hiera • Only used when default parameter values are invalid • Target parameters in Profiles and external modules org::profiles::iis85::log_path: logs01logs org::profiles::my_app::database_server: myapp-db01 org::profiles::my_app::database_username: myapp org::profiles::my_app::database_password: P@ssw0Rd321
- 22. ROLES AND PROFILES • https://puppetlabs.com/presentations/designing-puppet-rolesprofiles-pattern • http://www.craigdunn.org/2012/05/239/ • http://garylarizza.com/blog/2014/02/17/puppet-workflow-part-2/ • http://sysadvent.blogspot.co.uk/2012/12/day-13-configuration-management-as- legos.html
- 23. MOVING FROM DEV TO PROD Needs: • Isolate landscapes using Puppet “Environments” • Definitive state for each environment • Module versions • Hiera data • Node classification (site.pp) • Import modules from multiple sources • Automation + audit trail please
- 24. R10K DYNAMIC ENVIRONMENTS • Puppet Environments reduced to one Git repo: r10k-control r10k-control GitLab
- 25. R10K DYNAMIC ENVIRONMENTS • Puppet Environments reduced to one Git repo: r10k-control • Each Environment becomes one git branch Dev Test Prod r10k-control GitLab
- 26. R10K DYNAMIC ENVIRONMENTS • Puppet Environments reduced to one Git repo: r10k-control • Each Environment becomes one git branch • Definitive list of module versions captured in a Puppetfile Dev Test Prod r10k-control Puppetfile Puppetfile Puppetfile GitLab
- 27. R10K DYNAMIC ENVIRONMENTS • Puppet Environments reduced to one Git repo: r10k-control • Each Environment becomes one git branch • Definitive list of module versions captured in a Puppetfile • Run r10k deploy environment -p to sync environments Dev Test Prod r10k-control Puppetfile Puppetfile Puppetfile GitLab
- 28. Dev Maste r Dev Test Prod r10k GitLab Prod Maste r Dev Test Prod r10k Dev Test Prod r10k-control Puppetfile Puppetfile Puppetfile Module s Prod Se rvers Te st Serve rsDev Serve rs
Editor's Notes
- Ryan Armstrong Senior Engineer for Kinetic IT – An Australian owned IT outsourcing company Enterprise Management Systems team at the Department of Education Puppet Camp Melbourne 2014: I was excited but lost in a sea of options My questions: How do I choose what to automate? Should I use source control? How I make sense of all these modules and tie them together? How do I progress my modules from dev to production?
- How do we choose what to automate first?
- Get a view of the whole system… We need to zoom out from our immediate area of concern What are the businesses goals? Where do they rely on us to achieve their goals? Where are the bottlenecks in delivering? That’s where we should be looking
- A naïve example: delivering a new application Each step/workcentre is made up of…
- The Four Ms of Lean manufacturing People say… “There’s no value in automation for us” - Don’t just think in terms of automation…
- Think about your methods…. Usual method: follow documentation: Missing details Never updated Never followed correctly Puppet improves Self documented Same method on multiple platforms Change Implementation planning simplified So ask yourself Where is documentation letting us down? Are our change implementation plans too difficult to follow? Where are we creating quality problems through failure to follow procedures?
- The problem: Testing is manual and expensive Testing is skipped, or rushed Defects are never caught early The answer ALM = defects identified early Testing is enforced at every cycle Testing is cheap, automated and frequent Ask yourself: Where are defects being passed through the pipeline? Where is testing most critical and under performing? Where is configuration drift causing problems? Is compliance auditing expensive and slow?
- The problem: We build things by hand – impacts time, consistency and quality When we automate: Implement: automation, error checking, central reporting, idempotency (sometimes) Not cross platform Rarely reusable on the same platform The answer Puppet code is: Reproducible from code – like a 3d printer for infrastructure Automation implementation - trialed by fire Error checking built in Central reporting built in Idempotency built in Ask yourself: Where are we wasting time, building things by hand? Where are we reimplementing automation from scratch? Where are we missing error checking, reporting, idempotency, etc.?
- The problem: Undocumented knowledge (hand crafted servers) Trust issues = bottlenecks Silo focus: Boost local, break system The answer: Knowledge is captured in code ALM enhances accountability and quality gates to build trust change logging and auditing Automated approvals via Pull Requests Continuous Integration Everyone gets skilled with the same Method, Machine and Measure Work more collaboratively and boost the whole system
- The problem: Our team improved internally with PE but… No body really cared The business never benefitted We were seen to be wasting time chasing “new shiny toys” The truth: They were right!
- The answer: Bolster the business bottlenecks first Evaluate the four Ms Now we focus on: Windows server build automation in the DataCentre which helps with… Package deployment which fixes… Summary: Zoom out, Identify business priorities, Target the bottlenecks Now you’re doing DevOps
- Survey: Who uses source control for work, every day Should we use source control? You must! Infrastructure-as-code: It’s not “Infrastructure-as-Copy-and-Paste-Right-Click-Rename-.bak” When we use code, we unlock the benefits of ALM practices: Enable definitive state of infrastructure Ease deployment of code Ease collaboration Enable access control via pull requests Enable change auditing via commit logs Developers had to learn it too once upon a time. Guys on my team would never look back!
- If you’re new to VCS think of: Commits as ‘events’ committed to the timeline with unique IDs Checkouts as time travel Branches as new timelines Highlights: Perfect recall of any state from any branch Make use of multiple branches Lightweight: Only stores diffs
- Great central location for on-premise Support available
- See the slide deck for Git tutorials
- Lots of moving parts… How do we tie them all together? To: Keep it manageable Reduce complexity Reduce rework The answer: We borrowed from Craig Dunn’s Roles and Profiles idea Created a new module to sit over the top of these components
- Created a new module which looks like this. We create profile classes which group components together into reusable, site specific implementations We create role classes which group profiles together into business applications Break some rules: Contains site specific implementation (and data) Is not portable Stored in GitLab, protected by RBAC This is just what worked for us, not necessarily everybody
- Now we have a great module… how do we deploy it?
- E.g. Updating a module first in dev through to prod
- Thanks to PuppetLabs and my team Videos on my blog