Se ha denunciado esta presentación.
Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.
Deploying OpenStack Marconi
Creating Parallel Universes with SaltStack
Oz Akan, Cloud Engineering Manager, Rackspace
Outline
• Marconi
• Why SaltStack?
• Universe | Environment
• Salt Concepts
• Framework
• Summary
Marconi
Marconi
Marconi
Message Queue
Marconi
6
data centers
Marconi
360servers
Marconi
5
Billion
transactions per day
Marconi
…but
something more remarkable
Marconi
load balancers
web servers
catalog databases
queues databases
zenoss master
zenoss collectors
graylog servers
elas...
Marconi
from
nothing
Marconi
Marconi
45 minutes
from nothing to web scale
Why Salt?
Challenges
human mitsakes
Challenges
scaleof web
Challenges
sc li g
dynamism
a n
Challenges
environments
multiple
environments
Universe | Environment
Universe | Environment
laws | rules
defined by
Salt Concepts
Salt Concepts
grainon minions
Salt Concepts
pillaron master
Salt Concepts
environmentmaps to a folder
Salt Concepts
directory overlay
for states and pillar
Directory Overlay Example
file_roots:
prod:
- /srv/salt/prod
- /srv/salt/base
Salt Concepts
minequery minions
Salt Concepts
map
for salt-cloud
Salt Concepts
overstate
more orchestration
Salt Concepts / overstate example
set-mongodb_server:
match: 'G@environment_id:marconi-prod-ord and G@roles:mongodb_server...
Framework
Framework
role
is many things
Framework / role
grain
role: web_server
Framework / role
formulas
if..else in db_server
mongodb
Framework / role / formulas example
# queues_server/init.sls
{% if 'roles' in grains and 'queues_server' in grains['roles'...
Framework / role
minions
pillar
Framework / role / minions example
# pillar/minions.sls
minions:
cdb1a-cqp-ord:
roles:
- mongodb_server
attributes:
mongod...
Framework / role
devices
pillar
Framework / role / devices example
# pillar/devices.sls
devices:
load_balancers:
text: 'cloud load balancers'
addresses:
-...
Framework / role
networks
pillar
Framework / role / networks example
# pillar/networks.sls
networks:
vpn-all:
text: ’vpn networks'
addresses:
- '10.1.2.3/2...
Framework / role
roles pillar
sections per formula
Framework / role / pillar example
# pillar/roles.sls
roles:
role:
text:
attributes:
flags:
clients:
minions:
networks:
dev...
Framework / role / pillar example
# pillar/roles.sls
roles:
mongodb_server:
text: 'marconi mongodb database server’
attrib...
Framework / role / pillar example
# pillar/roles.sls
roles:
mongodb_server:
clients:
minions:
-
roles: ['bastion_server']
...
Framework / role / pillar example
# pillar/roles.sls
roles:
mongodb_server:
clients:
minions:
-
roles: ['queues_server','m...
Framework / role / pillar example
# pillar/roles.sls
roles:
mongodb_server:
clients:
networks:
-
name: vpn-all
protocols:
...
Framework / role / pillar example
# pillar/roles.sls
roles:
web_server:
clients:
devices
-
name: load_balancers
protocols:...
Framework
environment
is many things
Framework / environment
• project
• purpose
• location
set of grains
Framework / environment
environment_id
project-purpose-location
Framework / environment
/etc/salt/master
file_roots, pillar_roots
Framework / environment example
file_roots:
base:
- /srv/salt/marconi/base
marconi-prod-lon:
- /srv/salt/marconi/prod-lon
...
Framework / environment example
# folder layout
root@salt1a:/srv/salt/marconi# ls -1
base
prev-ord
prod-dfw
prod-hkg
prod-...
Framework
mine in formulas
Framework / mine
firewall
jinja template
Framework / mine / firewall
{%- if 'scope' in minion %}
{%- if minion.scope == 'project' %}
{%- for key, value in salt['mi...
Framework / mine
hosts
jinja template
Framework / mine / hosts
{%- for key, value in salt['mine.get']('environment_id:' + grains['environment_id'], 'grains.item...
Framework / mine / zenoss hosts
{%- if 'roles' in grains and 'zenoss_server' in grains['roles'] %}
{%- for key, value in s...
Summary
• grains
• mark minions (project, purpose, location, role)
• pillar
• to define global rules per role
• salt-mine
...
Multiple Environments
Q&A
Próxima SlideShare
Cargando en…5
×

SaltConf14 - Oz Akan, Rackspace - Deploying OpenStack Marconi with SaltStack

1.165 visualizaciones

Publicado el

This talk will demonstrate how to use Salt Mine leveraging Salt grains to create several environments (parallel universes) that decide how to run the same Salt formulas with different outcomes. "Roles” will be defined in an OpenStack Marconi (queuing as a service) deployment and a few formulas will be shared to demonstrate the concept.

Publicado en: Tecnología, Educación
  • Inicia sesión para ver los comentarios

SaltConf14 - Oz Akan, Rackspace - Deploying OpenStack Marconi with SaltStack

  1. 1. Deploying OpenStack Marconi Creating Parallel Universes with SaltStack Oz Akan, Cloud Engineering Manager, Rackspace
  2. 2. Outline • Marconi • Why SaltStack? • Universe | Environment • Salt Concepts • Framework • Summary
  3. 3. Marconi
  4. 4. Marconi Marconi Message Queue
  5. 5. Marconi 6 data centers
  6. 6. Marconi 360servers
  7. 7. Marconi 5 Billion transactions per day
  8. 8. Marconi …but something more remarkable
  9. 9. Marconi load balancers web servers catalog databases queues databases zenoss master zenoss collectors graylog servers elastic search servers bastions usage tracking workers usage tracking databases
  10. 10. Marconi from nothing
  11. 11. Marconi
  12. 12. Marconi 45 minutes from nothing to web scale
  13. 13. Why Salt?
  14. 14. Challenges human mitsakes
  15. 15. Challenges scaleof web
  16. 16. Challenges sc li g dynamism a n
  17. 17. Challenges environments multiple environments
  18. 18. Universe | Environment
  19. 19. Universe | Environment laws | rules defined by
  20. 20. Salt Concepts
  21. 21. Salt Concepts grainon minions
  22. 22. Salt Concepts pillaron master
  23. 23. Salt Concepts environmentmaps to a folder
  24. 24. Salt Concepts directory overlay for states and pillar
  25. 25. Directory Overlay Example file_roots: prod: - /srv/salt/prod - /srv/salt/base
  26. 26. Salt Concepts minequery minions
  27. 27. Salt Concepts map for salt-cloud
  28. 28. Salt Concepts overstate more orchestration
  29. 29. Salt Concepts / overstate example set-mongodb_server: match: 'G@environment_id:marconi-prod-ord and G@roles:mongodb_server' sls: - mongodb_server require: - set-firewall set-mongodb_replica: match: 'G@environment_id:marconi-prod-ord and G@roles:mongodb_server and G@mongodb_role:primary' sls: - mongodb_server.replica require: - set-mongodb_server
  30. 30. Framework
  31. 31. Framework role is many things
  32. 32. Framework / role grain role: web_server
  33. 33. Framework / role formulas if..else in db_server mongodb
  34. 34. Framework / role / formulas example # queues_server/init.sls {% if 'roles' in grains and 'queues_server' in grains['roles'] %} include: - marconi - memcached - queues_server.kernel - queues_server.install {% endif %}
  35. 35. Framework / role minions pillar
  36. 36. Framework / role / minions example # pillar/minions.sls minions: cdb1a-cqp-ord: roles: - mongodb_server attributes: mongodb_replica_set: catalog-rs1 mongodb_role: primary db_type: catalog … web4a-cqp-ord: roles: - queues_server attributes: mongodb_replica_set: catalog-rs1 queues_api: queue
  37. 37. Framework / role devices pillar
  38. 38. Framework / role / devices example # pillar/devices.sls devices: load_balancers: text: 'cloud load balancers' addresses: - 10.183.250.0/23 marconi-endpoint: text: 'marconi ORD endpoint' fqdn: ord.queues.api.rackspacecloud.com protocol: https address: 192.237.142.76 … graylog_lb: text: 'graylog load balancer' fqdn: log.marconi-graylog.com
  39. 39. Framework / role networks pillar
  40. 40. Framework / role / networks example # pillar/networks.sls networks: vpn-all: text: ’vpn networks' addresses: - '10.1.2.3/22' - '10.2.3.4/24’ … salt-master: text: 'salt master servers' addresses: - '10.178.129.47/32' - '162.200.150.120/32'
  41. 41. Framework / role roles pillar sections per formula
  42. 42. Framework / role / pillar example # pillar/roles.sls roles: role: text: attributes: flags: clients: minions: networks: devices
  43. 43. Framework / role / pillar example # pillar/roles.sls roles: mongodb_server: text: 'marconi mongodb database server’ attributes: - mongodb_replica_set - mongodb_role - db_type flags: - mongodb_replica_set_configured
  44. 44. Framework / role / pillar example # pillar/roles.sls roles: mongodb_server: clients: minions: - roles: ['bastion_server'] protocols: - name: tcp ports: ['22'] states: ['NEW','ESTABLISHED'] text: 'ssh access'
  45. 45. Framework / role / pillar example # pillar/roles.sls roles: mongodb_server: clients: minions: - roles: ['queues_server','mongodb_server','memcached_server','bastion_server'] protocols: - name: icmp types: ['0','8'] text: 'ping access'
  46. 46. Framework / role / pillar example # pillar/roles.sls roles: mongodb_server: clients: networks: - name: vpn-all protocols: - name: icmp types: ['0','8'] text: 'ping access from zenoss server'
  47. 47. Framework / role / pillar example # pillar/roles.sls roles: web_server: clients: devices - name: load_balancers protocols: - name: tcp ports: ['443'] text: 'http access from lb to server' states: ['NEW','ESTABLISHED','RELATED']
  48. 48. Framework environment is many things
  49. 49. Framework / environment • project • purpose • location set of grains
  50. 50. Framework / environment environment_id project-purpose-location
  51. 51. Framework / environment /etc/salt/master file_roots, pillar_roots
  52. 52. Framework / environment example file_roots: base: - /srv/salt/marconi/base marconi-prod-lon: - /srv/salt/marconi/prod-lon - /srv/salt/marconi/base marconi-test-lon: - /srv/salt/marconi/test-lon - /srv/salt/marconi/base pillar_roots: base: - /srv/salt/marconi/base/pillar marconi-prod-lon: - /srv/salt/marconi/prod-lon/pillar - /srv/salt/marconi/base/pillar marconi-test-lon: - /srv/salt/marconi/test-lon/pillar - /srv/salt/marconi/base/pillar
  53. 53. Framework / environment example # folder layout root@salt1a:/srv/salt/marconi# ls -1 base prev-ord prod-dfw prod-hkg prod-iad prod-lon prod-ord prod-syd test-ord …
  54. 54. Framework mine in formulas
  55. 55. Framework / mine firewall jinja template
  56. 56. Framework / mine / firewall {%- if 'scope' in minion %} {%- if minion.scope == 'project' %} {%- for key, value in salt['mine.get']('project:' + grains['project'], 'grains.items', expr_form='grain').items() %} {%- if role in value['roles'] %} -A INPUT -s {{ key }} -j {{ role|upper }} … {%- elif minion.scope == 'environment_id' %} {%- for key, value in salt['mine.get']('environment_id:' + grains['environment_id'], 'grains.items', expr_form='grain').items() %} {%- if role in value['roles'] %} -A INPUT -s {{ key }} -j {{ role|upper }} …
  57. 57. Framework / mine hosts jinja template
  58. 58. Framework / mine / hosts {%- for key, value in salt['mine.get']('environment_id:' + grains['environment_id'], 'grains.items', expr_form='grain').items() %} {{ value['id'] }}: host: - present - ip: {{ salt['mine.get'](value['id'], 'network.ip_addrs').values()[0][0] }} {%- endfor %}
  59. 59. Framework / mine / zenoss hosts {%- if 'roles' in grains and 'zenoss_server' in grains['roles'] %} {%- for key, value in salt['mine.get']('roles:zenoss_server', 'grains.items', expr_form='grain').items() %} {%- if value['project'] == pillar['project'] %} host_{{ value['id'] }}: host: - present - name: {{ value['id'] }} - ip: {{ salt['mine.get'](value['id'], 'network.ip_addrs').values()[0][1] }} {%- endif %} {%- endfor %} {%- endif %}
  60. 60. Summary • grains • mark minions (project, purpose, location, role) • pillar • to define global rules per role • salt-mine • to be able to query minions in the environment • environments and directory overlay
  61. 61. Multiple Environments Q&A

×