This document discusses using Puppet in non-traditional ways for deploying distributed applications across heterogeneous servers. It outlines pushing configuration from a central node using MCollective instead of relying on Puppet's master-agent architecture. Issues with the traditional approach are noted along with experiments trying different methods to dynamically generate configurations, including passing parameters via facts instead of using site.pp. The goal is to eliminate single points of failure and scalability bottlenecks. Future steps proposed include deploying in stages based on dependency graphs and allowing other configuration management tools.

1. Non-traditional
use of Puppet
Mike Scherbakov
Moscow, September 2013

2. 1. Initial conditions
2. Push approach with MCollective
3. Data Flow
4. Next steps
Agenda

3. ● Heterogenous servers
● User-defined roles for distributed application
● Operations
○ Remove / add nodes
○ Add roles and deploy
● Deployment is activated by user
● Multiple envs, different versions of code to be
deployed
● Simultaneous deployments and other actions
1.1 Use case

4. 1. Puppet Master serving the configuration code
2. Puppet agents are ran by cron every 30min
3. Text file site.pp with definition what to deploy
and where
4. PuppetDB to share data between nodes
5. ENC to get variables from external source
6. Hiera: Key-value store
1.2 Traditional Puppet Usage

5. 1. Use Puppet
2. It should run by click on Deploy, not every 30min
3. Multiple environments
4. Simultaneous deployment where possible
5. Operations (redeployment, adding roles, etc.)
1.3 Our Initial Requirements

6. 1.4 Our use case & Puppet

7. ● Publish-subscribe middleware over AMQP
(STOMP)
● Simple agents, written in Ruby
● Agents on nodes can be run by simple command
from master node
● Agent to run puppet already existed
● MCollective can be used as a library
2.1 Marionette Collective

8. 2.2 We thought

9. ● Multithreading issues
● Existing agent for puppet had multiple issues
● Time synchronization sensitive
● No reconnect after network failure
2.3 And we tried...

10. 2.4 Finally it worked out

11. 3.1 Data Flow
User
Input
Hardcoded
Hardware
Discovered
Data
Data
Calculation
Puppet

12. Expected:
require ‘puppet’
Puppet::Resource::Catalog.compile(all-data)
Puppet.apply
Actually got:
….monkey-patching of Puppet code, which worked
only for single Puppet release.
POC at http://goo.gl/xdcjlL
3.2 Not so ideal with Puppet...

13. Pros: unknown for our purposes
Cons:
● site.pp is code, and logic to write it properly is
required
● concurrency issues if simultaneous deployment
3.3 site.pp
Data
Calculation
Puppet
agent
site.pp
Puppet
master

14. Pros:
● Simple well-known mechanism of passing params
Cons:
● Architecture gets complicated
○ Have to make sure that nothing changes data
for ENC before puppet reads
○ Hard to develop: data must be in storage
3.4 ENC
Data
Calculation
Puppet
agent
Puppet
master
Storage
ENC

15. Cons:
● Additional point of failure
● Two storages: need to handle consistency
● Additional dependency: Cobbler is not just PXE
anymore, hard to replace
3.5 ENC + Cobbler YAML Storage
Data
Calculation
Puppet
agent
Puppet
master
Cobbler
YAML
Storage
ENC

16. Pros:
● Puppet Master is optional
● Easy to debug and develop: all data in one file
Cons:
● Facts can be strings only
● All variables loaded are global
3.6 Passing parameters via facts
Data
Calculation
Puppet
agent
facts
on
nodes
Puppet
master

17. ● YAML config is generated for every puppet run
● Placed to /etc/ by MCollective
3.6.1 YAML config is created on node
Data
Calculation
---
role: compute
nova:
- rabbitmq: ":5672"
api: ":8774"
ip: 10.20.0.2/24

18. data = YAML.load_file(‘/etc/naily.
facts`)
Facter.add(‘fuel_settings_yaml`) do
set_code { data.to_yaml }
end
3.6.2 Puppet reads YAML into fact

19. $fuel_settings = parseyaml
($fuel_settings_yaml)
case $::fuel_settings[‘role’] {
“compute” : {
include common::compute
…
3.6.3 Parse your fact in site.pp

20. 3.7 And we got it running

21. Main reasons:
1. Additional SPoF
2. Scalability issues
3. Issues with certificates
4. Multiple envs with multiple versions of manifests
5. Temptation to use PuppetDB and other, which
add points of failure
4.1 Next steps: Get rid of Puppet Master

22. ● Controller role
○ nova-api
○ nova-scheduler
○ MySQL <- what if customer wants it on the other
server?
○ glance-api
○ glance-registry
○ keystone
4.2 Next steps: Deployment by stages
DB
Controller
node 1 node 2

23. [{ ‘role’: ‘nova-api’, ‘depends’: [‘MySQL’, ‘keystone’],
{ ‘role’: ‘keystone’, ‘depends’: [‘MySQL’] ]
4.2.1 Dependency graph
MySQL
Keystone
nova-api

24. 4.2.2 Allow other tools for stages
Such as, but not limited to:
● Opscode Chef
● SaltStack
● Just Ruby or Python code

25. ● http://docs.mirantis.com/fuel-dev/
● https://github.com/Mirantis/fuel/pull/572/files
● http://nuknad.com/2011/02/11/self-
classifying-puppet-nodes/
● https://github.com/stackforge/fuel-{main, web,
astute, ostf}
● http://puppetlabs.
com/presentations/continuously-integrated-
puppet-dynamic-environment
Links

26. Mike Scherbakov,
mscherbakov@mirantis.com
Questions?