My talk @blndevops Meetup 12Q3

1. Managing large systems with
Marionette Collective
andreas.schmidt@cassini.de
twitter @aschmidt75
© 2012 Cassini Consulting GmbH

2. About myself
Software Development
C++, Java (EE)
Web Operations
Solaris, Linux, Shell
Infrastructure Development
Puppet, Ruby
About Cassini
Systems Management
IT Security
Configuration Mgt & Deployment Automation
2

3. What is MCollective?
What is needed to set it up?
How does it work?
Small demo
3

4. 1 year later
Test / Production 200 hosts
stage
20 hosts
Development
Stage
5 hosts
4

5. 5

6. „ “
ssh in a for loop is not a solution
- Luke Kanies
6

7. „ If you ever have to log on to a
single box, you‘re doing it wrong. “
Kristian Köhntopp, OSDC2012
7

8. „ mcollective is ssh on steroids “
@kbuytaert
8

9. SSH … mcollective …
is a protocol connecting is a broadcast-oriented
two endpoints framework
is used primarily to transports structured
transport text data within an API
9 19.07.2012

10. Marionette Collective Overview
10

11. Architecture | What does it do? Me
Messaging
Middleware
My
Hosts
11

12. Ruby 1.8.7+
Architecture | Components mcollective client
stomp
ActiveMQ | RabbitMQ
Ruby 1.8.7+
mcollective
stomp
facter
agent code
12

13. Architecture | Collectives
Collective
13

14. Architecture | Collectives
Collective
Subcollective
14

15. facter: Adressing by Facts
operatingsystem=centos
ipaddress=~^10.1.30
15

16. Walk-Through client sends request
to middleware topic
2
All hosts watch 1
topic and, reading
request as it becomes 5
available
3 Client reads 6
all results from topic
Results are delivered
4 Host w/ matching
facts execute
agent call
16

17. DEMO
17

18. Backup // DEMO
18

19. Backup // DEMO
19

20. Backup // DEMO
20

21. Backup // DEMO
21

22. Backup // DEMO
22

23. Backup // DEMO
23

24. Backup // DEMO
24

25. Backup // DEMO
25

26. Backup // DEMO
26

27. Backup // DEMO
27

28. Backup // DEMO
28

29. Backup // DEMO
29

30. 30

31. What about security?
AES Security Plugin
Client Messaging Node
Middleware
Stomp Stomp
Agent
mcollective Message mcollectived server
Payload
User SK AES User PK
Server PK Server SK
+ Authorization Plugin
+ Auditing Plugin
31

32. MCollective 2.0
• Direct Addressing Mode
• Message TTLs
• Complex Discovery Statements
• Pluggable discovery
32

33. True Parallelism
API w/ data validation
Deterministic timing
Added Velocity
Scales really well
33

34. Q& A
Cassini Consulting
andreas.schmidt@cassini.de
twitter @aschmidt75
34