This talk will demonstrate how to use Salt Mine leveraging Salt grains to create several environments (parallel universes) that decide how to run the same Salt formulas with different outcomes. "Roles” will be defined in an OpenStack Marconi (queuing as a service) deployment and a few formulas will be shared to demonstrate the concept.

1. Deploying OpenStack Marconi
Creating Parallel Universes with SaltStack
Oz Akan, Cloud Engineering Manager, Rackspace

2. Outline
• Marconi
• Why SaltStack?
• Universe | Environment
• Salt Concepts
• Framework
• Summary

3. Marconi

4. Marconi
Marconi
Message Queue

5. Marconi
6
data centers

6. Marconi
360servers

7. Marconi
5
Billion
transactions per day

8. Marconi
…but
something more remarkable

9. Marconi
load balancers
web servers
catalog databases
queues databases
zenoss master
zenoss collectors
graylog servers
elastic search servers
bastions
usage tracking workers
usage tracking databases

10. Marconi
from
nothing

11. Marconi

12. Marconi
45 minutes
from nothing to web scale

13. Why Salt?

14. Challenges
human mitsakes

15. Challenges
scaleof web

16. Challenges
sc li g
dynamism
a n

17. Challenges
environments
multiple
environments

18. Universe | Environment

19. Universe | Environment
laws | rules
defined by

20. Salt Concepts

21. Salt Concepts
grainon minions

22. Salt Concepts
pillaron master

23. Salt Concepts
environmentmaps to a folder

24. Salt Concepts
directory overlay
for states and pillar

25. Directory Overlay Example
file_roots:
prod:
- /srv/salt/prod
- /srv/salt/base

26. Salt Concepts
minequery minions

27. Salt Concepts
map
for salt-cloud

28. Salt Concepts
overstate
more orchestration

29. Salt Concepts / overstate example
set-mongodb_server:
match: 'G@environment_id:marconi-prod-ord and G@roles:mongodb_server'
sls:
- mongodb_server
require:
- set-firewall
set-mongodb_replica:
match: 'G@environment_id:marconi-prod-ord and G@roles:mongodb_server and
G@mongodb_role:primary'
sls:
- mongodb_server.replica
require:
- set-mongodb_server

30. Framework

31. Framework
role
is many things

32. Framework / role
grain
role: web_server

33. Framework / role
formulas
if..else in db_server
mongodb

34. Framework / role / formulas example
# queues_server/init.sls
{% if 'roles' in grains and 'queues_server' in grains['roles'] %}
include:
- marconi
- memcached
- queues_server.kernel
- queues_server.install
{% endif %}

35. Framework / role
minions
pillar

36. Framework / role / minions example
# pillar/minions.sls
minions:
cdb1a-cqp-ord:
roles:
- mongodb_server
attributes:
mongodb_replica_set: catalog-rs1
mongodb_role: primary
db_type: catalog
…
web4a-cqp-ord:
roles:
- queues_server
attributes:
mongodb_replica_set: catalog-rs1
queues_api: queue

37. Framework / role
devices
pillar

38. Framework / role / devices example
# pillar/devices.sls
devices:
load_balancers:
text: 'cloud load balancers'
addresses:
- 10.183.250.0/23
marconi-endpoint:
text: 'marconi ORD endpoint'
fqdn: ord.queues.api.rackspacecloud.com
protocol: https
address: 192.237.142.76
…
graylog_lb:
text: 'graylog load balancer'
fqdn: log.marconi-graylog.com

39. Framework / role
networks
pillar

40. Framework / role / networks example
# pillar/networks.sls
networks:
vpn-all:
text: ’vpn networks'
addresses:
- '10.1.2.3/22'
- '10.2.3.4/24’
…
salt-master:
text: 'salt master servers'
addresses:
- '10.178.129.47/32'
- '162.200.150.120/32'

41. Framework / role
roles pillar
sections per formula

42. Framework / role / pillar example
# pillar/roles.sls
roles:
role:
text:
attributes:
flags:
clients:
minions:
networks:
devices

43. Framework / role / pillar example
# pillar/roles.sls
roles:
mongodb_server:
text: 'marconi mongodb database server’
attributes:
- mongodb_replica_set
- mongodb_role
- db_type
flags:
- mongodb_replica_set_configured

44. Framework / role / pillar example
# pillar/roles.sls
roles:
mongodb_server:
clients:
minions:
-
roles: ['bastion_server']
protocols:
-
name: tcp
ports: ['22']
states: ['NEW','ESTABLISHED']
text: 'ssh access'

45. Framework / role / pillar example
# pillar/roles.sls
roles:
mongodb_server:
clients:
minions:
-
roles: ['queues_server','mongodb_server','memcached_server','bastion_server']
protocols:
-
name: icmp
types: ['0','8']
text: 'ping access'

46. Framework / role / pillar example
# pillar/roles.sls
roles:
mongodb_server:
clients:
networks:
-
name: vpn-all
protocols:
-
name: icmp
types: ['0','8']
text: 'ping access from zenoss server'

47. Framework / role / pillar example
# pillar/roles.sls
roles:
web_server:
clients:
devices
-
name: load_balancers
protocols:
-
name: tcp
ports: ['443']
text: 'http access from lb to server'
states: ['NEW','ESTABLISHED','RELATED']

48. Framework
environment
is many things

49. Framework / environment
• project
• purpose
• location
set of grains

50. Framework / environment
environment_id
project-purpose-location

51. Framework / environment
/etc/salt/master
file_roots, pillar_roots

52. Framework / environment example
file_roots:
base:
- /srv/salt/marconi/base
marconi-prod-lon:
- /srv/salt/marconi/prod-lon
- /srv/salt/marconi/base
marconi-test-lon:
- /srv/salt/marconi/test-lon
- /srv/salt/marconi/base
pillar_roots:
base:
- /srv/salt/marconi/base/pillar
marconi-prod-lon:
- /srv/salt/marconi/prod-lon/pillar
- /srv/salt/marconi/base/pillar
marconi-test-lon:
- /srv/salt/marconi/test-lon/pillar
- /srv/salt/marconi/base/pillar

53. Framework / environment example
# folder layout
root@salt1a:/srv/salt/marconi# ls -1
base
prev-ord
prod-dfw
prod-hkg
prod-iad
prod-lon
prod-ord
prod-syd
test-ord
…

54. Framework
mine in formulas

55. Framework / mine
firewall
jinja template

56. Framework / mine / firewall
{%- if 'scope' in minion %}
{%- if minion.scope == 'project' %}
{%- for key, value in salt['mine.get']('project:' +
grains['project'], 'grains.items', expr_form='grain').items() %}
{%- if role in value['roles'] %}
-A INPUT -s {{ key }} -j {{ role|upper }}
…
{%- elif minion.scope == 'environment_id' %}
{%- for key, value in salt['mine.get']('environment_id:' +
grains['environment_id'], 'grains.items', expr_form='grain').items() %}
{%- if role in value['roles'] %}
-A INPUT -s {{ key }} -j {{ role|upper }}
…

57. Framework / mine
hosts
jinja template

58. Framework / mine / hosts
{%- for key, value in salt['mine.get']('environment_id:' + grains['environment_id'], 'grains.items',
expr_form='grain').items() %}
{{ value['id'] }}:
host:
- present
- ip: {{ salt['mine.get'](value['id'], 'network.ip_addrs').values()[0][0] }}
{%- endfor %}

59. Framework / mine / zenoss hosts
{%- if 'roles' in grains and 'zenoss_server' in grains['roles'] %}
{%- for key, value in salt['mine.get']('roles:zenoss_server', 'grains.items', expr_form='grain').items()
%}
{%- if value['project'] == pillar['project'] %}
host_{{ value['id'] }}:
host:
- present
- name: {{ value['id'] }}
- ip: {{ salt['mine.get'](value['id'], 'network.ip_addrs').values()[0][1] }}
{%- endif %}
{%- endfor %}
{%- endif %}

60. Summary
• grains
• mark minions (project, purpose, location, role)
• pillar
• to define global rules per role
• salt-mine
• to be able to query minions in the environment
• environments and directory overlay

61. Multiple Environments
Q&A