Se ha denunciado esta presentación.
Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.

Using Ansible at Scale to Manage a Public Cloud

12.880 visualizaciones

Publicado el

An overview of three scale challenges at Rackspace and how Ansible key features helps us solve those three challenges.

Publicado en: Tecnología
  • Inicia sesión para ver los comentarios

Using Ansible at Scale to Manage a Public Cloud

  1. 1. Jesse Keating – Linux Systems Engineer IV – Cloud Servers@iamjkeatingUsing Ansible at Scale to Managea Public Cloud06/13/2013 – AnsibleFest
  2. 2. RACKSPACE® HOSTING | WWW.RACKSPACE.COMRackspace cares about scale● Scale of server systems● Scale of environments● Scale of engineers
  3. 3. Scale of Server Systems
  4. 4. Rackspace Public Cloud● 4 “Production” regions– 1 to 8 cells per region– 250 to 500 nodes per cell● Nearly 15K “systems” in production● Another 500~ in CI/pre-production● Mixed use of copy-pasta pssh scripts, pre-configuredagent actions, jenkins automation, and host-basedconfig management● Managed by admins, engineers, developers
  5. 5. RACKSPACE® HOSTING | WWW.RACKSPACE.COMCase study: Hotpatch One ProductionEnvironment● 3900~ compute-nodes– Spread across 8 cells– Out of 6000~ total hosts● Alerting will flood admins● Output is hard to parse
  6. 6. RACKSPACE® HOSTING | WWW.RACKSPACE.COMAnsible Key Features● Inventory plugin● Simple process flow● Reusable playbooks with variable adjustments● Avoids repeated actions on downed hosts● Cleaner output
  7. 7. Need to change
  8. 8. .. and
  9. 9. to...
  10. 10. So we can do...
  11. 11. Or this
  12. 12. RACKSPACE® HOSTING | WWW.RACKSPACE.COMAnsible Use● Replacing use of pssh for Random Tasks● Replacing use of pssh for Expected Tasks (outsideconfig management)● Reuse existing inventory content● Easily bolt together processes such as disabling nagiosalerts prior to execution
  13. 13. Scale of Environments
  14. 14. Rackspace OpenStack Development● At least 7 major software projects– Different feature schedules within each● One Continuous Integration environment● One Pre-production environment● One branch of code that can easily be deployed● New code deploys every two weeks
  15. 15. RACKSPACE® HOSTING | WWW.RACKSPACE.COMCase Study: Create production likeenvironment to test disruptive product codechange● 30~ virtual instances– DB servers– Rabbit servers– Service providers● 40~ capacity nodes– Hypervisor + nova-compute VM● Mixed use of fabric, shell scripts, copy-pasta● No self service
  16. 16. RACKSPACE® HOSTING | WWW.RACKSPACE.COMAnsible Key Features● Intermix local actions and remote actions● External inventory plugin● Start from nothing● API to use directly within another application
  17. 17. Start with localhost prep
  18. 18. Local actions to boot instances
  19. 19. Remote actions on hosts
  20. 20. Existing yaml for host vars
  21. 21. RACKSPACE® HOSTING | WWW.RACKSPACE.COMAnsible Use● Replacing use of fabric, pssh, copy-pasta● Boot strapping environment to the point where existingconfig management can take over● Freeing up Engineer time by making it self-service● Freeing up resources by tearing down environmentsafter use● Working toward using same process to build outproduction environments
  22. 22. Scale of Engineers
  23. 23. Rackspace Engineering● Between 4K and 6K employees/contractors● Between 500 and 1K Engineer/Developer types● Many dozens of summer interns● Countless groups● Countless projects● Rapid team creation / shifting of resources● Mixed use of Mac OSX and Linux● Mixed use of automation, configuration, et al tools● Disjoint ownership of engineering onboarding
  24. 24. RACKSPACE® HOSTING | WWW.RACKSPACE.COMCase study: Ozone Onboard● 30+ git repos● 5+ utilities w/ configuration● Permissions to a plethora of services● Configuration for CI/preprod/prod environments● Details scattered throughout wiki pages and tribalknowledge
  25. 25. RACKSPACE® HOSTING | WWW.RACKSPACE.COMAnsible Key Features● Modular Roles● Minimal dependencies● OS agnostic● Idempotent● Fast● Easy to use and extend
  26. 26. Overview of Ansibox
  27. 27. User edited file
  28. 28. Top level playbook
  29. 29. Generated Playbook
  30. 30. Making it go
  31. 31. Ozone Tasks
  32. 32. RACKSPACE® HOSTING | WWW.RACKSPACE.COMAnsible Use● Developer bootstraps their own system by selectingroles and providing details● Teams own role definitions within a shared framework● Repeatable process– Ansible playbook to clone/update roles– Second playbook to process roles
  33. 33. Conclusion● Ansible solves many problems Rackspace faces● Chip away at edges with Ansible, perhaps one dayreplace existing config management systems withAnsible● Continue to assist in development of Ansiblemodules, plugins, and scale testing● Launch Ansibox soon!
  34. 34. 36RACKSPACE® HOSTING | 5000 WALZEM ROAD | SAN ANTONIO, TX 78218US SALES: 1-800-961-2888 | US SUPPORT: 1-800-961-4454 | WWW.RACKSPACE.COMRACKSPACE® HOSTING | © RACKSPACE US, INC. | RACKSPACE® AND FANATICAL SUPPORT® ARE SERVICE MARKS OF RACKSPACE US, INC. REGISTERED IN THE UNITED STATES AND OTHER COUNTRIES. |WWW.RACKSPACE.COM

×