Puppet Camp DC 2014: "Puppet Type and Provider Execution Presentation" by Jon Mosco, Corporation Services Company

1. Managing Puppet with
MCollective
Jon Mosco
@jpmosco

2. Who am I?
Infrastructure Engineer
Puppet User for 2+ years
Puppet Certified Professional
Linux user for 10+ years (Slackware!)
Metal Head m/

3. What is MCollective?

4. The Marionette Collective (MCollective)
is a framework to build server
orchestration or parallel job execution
Works perfectly along side Puppet
providing real-time command execution
across your infrastructure
This is what it really is…

5. Configuration
Management
Focuses on single node management
Great at managing one server at a
time

6. Orchestration
Coordination of management of multiple
systems

7. MCollective Architecture
Servers == Puppet Agent
Clients
Middleware == Puppet Master
Client Middleware
Server
Server
Server
Server

8. Servers
Systems that can be controlled via
MCollective
Run the daemon mcollectived
puppet agent == mcollectived
Multiple Agent Plugins
small Ruby libraries that enable custom
command execution

9. Clients
Clients send and receive requests to servers
Admin server for managing MCollective
Security and connector plugin
mco command-line client
interactive usage or in scripts

10. Middleware
!
Clients and servers communicate through the
middleware system (ActiveMA, RabbitMQ)
MCollective uses a publish/subscribe system
for all communication
Sends and receives messages using STOMP
Used for transporting messages between
clients and servers

11. Message Flow
Broadcast paradigm
Messages are sent from the administrator
account on the client server to the
middleware server and broadcast to all
nodes
Each agent is both a publisher and
subscriber

12. MW
MW =
Middleware
S = Server
s
s
s
Client
mco

13. Plugins
Enable MCollective to execute commands on
the entire collective
Provide the functionality to use with
MCollective
puppet agent, service, process, package, etc.

14. Integration with Puppet
Pre-installed and configured with Puppet
Enterprise
Puppet Labs modules for opensource
https://github.com/puppetlabs/puppetlabs-
mcollective

15. PuppetLabs MCollective Module

16. Orchestrating Puppet
MCollective can be used to control the
Puppet Daemon
The performance of a Puppet Master is
related to the max concurrent Puppet nodes
that are applying a catalog
MCollective can determine how many servers
are currently enabled and applying catalogs

17. mco puppet
$ mco help puppet
[14/180]
!
Schedule runs, enable, disable and interrogate the Puppet Agent
!
Usage: mco puppet [OPTIONS] [FILTERS] <ACTION> [CONCURRENCY|MESSAGE]
Usage: mco puppet <count|enable|status|summary>
Usage: mco puppet disable [message]
Usage: mco puppet runonce [PUPPET OPTIONS]
Usage: mco puppet resource type name property1=value property2=value
Usage: mco puppet runall [--rerun SECONDS] [PUPPET OPTIONS]
!
The ACTION can be one of the following:
!
count - return a total count of running, enabled, and disabled nodes
enable - enable the Puppet Agent if it was previously disabled
disable - disable the Puppet Agent preventing catalog from being applied
resource - manage individual resources using the Puppet Type (RAL) system
runall - invoke a puppet run on matching nodes, making sure to only run
CONCURRENCY nodes at a time
runonce - invoke a Puppet run on matching nodes
status - shows a short summary about each Puppet Agent status
summary - shows resource and run time summaries

18. Controlled Nodes
Query and run Puppet from any node that
has the client installed:
$ mco puppet count
Total Puppet nodes: 9
!
Nodes currently enabled: 9
Nodes currently disabled: 0
!
Nodes currently doing puppet runs: 0
Nodes currently stopped: 9
!
Nodes with daemons started: 9
Nodes without daemons started: 0
Daemons started but idling: 9

19. Puppet Status
$ mco puppet status
!
* [ ============================================================> ] 7 / 7
!
db02.example.com: Currently idling; last completed run 4 minutes 50 seconds ago
fpvcs01.example.com: Currently idling; last completed run 24 minutes 44 seconds ago
puppet.example.com: Currently idling; last completed run 6 minutes 48 seconds ago
mco01.example.com: Currently idling; last completed run 24 minutes 35 seconds ago
fp01.example.com: Currently idling; last completed run 4 minutes 55 seconds ago
web01.example.net: Currently idling; last completed run 24 minutes 16 seconds ago
mail02.example.com: Currently idling; last completed run 3 minutes 17 seconds ago
!
Summary of Applying:
false = 7
Summary of Daemon Running:
running = 7
Summary of Enabled:
enabled = 7
Summary of Idling:
true = 7
Summary of Status:
idling = 7
!
Finished processing 7 / 7 hosts in 96.09 ms

20. Running Puppet
Basic Puppet run:
$ mco puppet runonce
!
A noop run:
$ mco puppet runonce —noop
!
A run when noop is set:
$ mco puppet runonce —no-noop
!
A splay run:
$ mco puppet runonce —splay —
splaylimit 120

21. Filters
Host Filters
-W, --with FILTER Combined classes and facts filter
-S, --select FILTER Compound filter combining facts and classes
-F, --wf, --with-fact fact=val Match hosts with a certain fact
-C, --wc, --with-class CLASS Match hosts with a certain config management class
-A, --wa, --with-agent AGENT Match hosts with a certain agent
-I, --wi, --with-identity IDENT Match hosts with a certain configured identity
$ mco puppet runonce web01.example.com
* [ ============================================================> ] 9 / 9

22. Don't forget filters….

23. Invoking Actions
with filters
Facts
$ mco find -F osfamily=RedHat
$ mco find —with-fact operatingsystemmajrelease=5
Classes
Puppet provides a list of classes applied to
a node:
$ mco find —with-class /apache/

24. Agent Runs
$ mco puppet runall 1
2013-01-16 16:14:26: Running all nodes with a concurrency of 1
2013-01-16 16:14:26: Discovering enabled Puppet nodes to manage
2013-01-16 16:14:29: Found 2 enabled nodes
2013-01-16 16:14:32: Currently 1 node applying the catalog; waiting for less than 1
2013-01-16 16:14:37: dev1.example.net schedule status: Started a background Puppet run using the 'puppet agent
--onetime --daemonize --color=false' command
2013-01-16 16:14:38: 1 out of 2 hosts left to run in this iteration
2013-01-16 16:14:40: Currently 1 node applying the catalog; waiting for less than 1
2013-01-16 16:14:44: Currently 1 node applying the catalog; waiting for less than 1
2013-01-16 16:14:48: Currently 1 node applying the catalog; waiting for less than 1
2013-01-16 16:14:52: Currently 1 node applying the catalog; waiting for less than 1
2013-01-16 16:14:56: Currently 1 node applying the catalog; waiting for less than 1
2013-01-16 16:15:00: Currently 1 node applying the catalog; waiting for less than 1
2013-01-16 16:15:04: Currently 1 node applying the catalog; waiting for less than 1
2013-01-16 16:15:08: Currently 1 node applying the catalog; waiting for less than 1

25. Last run summary

26. Plotting Performance

27. MCollective Puppet agent can make changes
based on the Puppet RAL
Puppet Resources

29. Allowing resource control

30. Enable and Disable
Puppet
$ mco puppet disable message=“turn off all the things”
$ mco puppet enable

31. Misc Plugin Examples
$ mco process list java -I dradis.example.com
!
* [ ============================================================> ] 1 / 1
!
dradis.example.com
!
PID USER VSZ COMMAND
1380 logstash 1.794 GB /usr/bin/java -Djava.io.tmpdir=/var/lib/logstash -Xmx500m -X
1436 elasticsea 2.298 GB /usr/bin/java -Xms256m -Xmx1g -Xss256k -Djava.awt.headless=t
!
!
Summary of The Process List:
!
Matched hosts: 1
Matched Processes: 2
Resident Size: 146.453 MB
Virtual Size: 4.092 GB
!
!
Finished processing 1 / 1 hosts in 113.93 ms

32. $ mco package bash status -I db02.example.com
!
* [ ============================================================> ] 1 / 1
!
db02.example.com: bash-4.1.2-15.el6_4.x86_64
!
Summary of Arch:
!
x86_64 = 1
!
Summary of Ensure:
!
4.1.2-15.el6_4 = 1
!
!
Finished processing 1 / 1 hosts in 128.67 ms

33. InventoryRetrieve all facts, classes, and agent plugins:
!
$mco inventory web01.example.com
Agents:
discovery filemgr package
puppet rpcutil service
thanks
!
Data Plugins:
agent fstat puppet
resource service
!
Configuration Management Classes:
agent epel
epel::params mcollective
mcollective::common mcollective::common::config
mcollective::common::config::connector::activemq mcollective::defaults
mcollective::server mcollective::server::config
mcollective::server::config::connector::activemq mcollective::server::config::factsource::yaml
mcollective::server::config::rpcauditprovider::logfile
mcollective::server::config::rpcauthprovider::action_policy
mcollective::server::config::securityprovider::ssl mcollective::server::install
mcollective::server::service ntp
…..
!
Facts:
architecture => x86_64
augeasversion => 1.0.0
bios_release_date => 04/17/2013
bios_vendor => Xen
bios_version => 4.0.1
blockdevice_xvda_size => 21559771136
blockdevice_xvdb_size => 17179869184
blockdevices => xvda,xvdb

34. inventory do
format "%40s:tt%20stt%s"
!
fields { [ identity, facts["operatingsystemrelease"], facts["ipaddress"] ] }
end
$ mco inventory -C /epel/ —script inventory.rb
mco01.example.com: 6.5 192.168.2.25
web02.example.com: 6.1 192.168.2.30
mon01.example.com: 6.5 192.168.2.28
db01.example.com: 6.5 192.168.2.40
db02.example.com: 6.3 192.168.2.60
mail02.example.com: 6.5 192.168.2.50
Inventory Reports

35. Puppet Enterprise Live
Management

36. Demo time!
Stay away demo daemon!

37. Questions?
Thank you!