Puppet Camp Chicago 2014: Running Multiple Puppet Masters (Beginner) presented by Jeffrey Miller, University of Iowa

1. Jeffrey Miller
Sr. Systems Administrator
The University of Iowa: ITS – EI – SST – IT
Email: Jeff-L-Miller@uiowa.edu
Freenode: millerjl1701

2. StartedVAX mainframes, Sun SPARC systems in
college
Chemistry/Physics teacher…WGS Pro Linux
In 2001, sys admin for the UI Department of
Chemistry doing various things
Now, a infrastructure admin for the UI campus IT
group doing cross-platform stuff
Puppet infrastructure
Red Hat Satellite / Spacewalk
SC: OM,VMM
22+Years in the MN ARNG going various places

3. Yes, without a doubt absolutely… maybe

4. No infrastructure required other than content
distribution… which you probably already
have..
puppet apply….
All nodes have access to the modules and
manifests for your environment
Multiple zones? Multiple data centers?
Configuration churn?
Test Driven Development administration…

5. All machines have access to the entire
environment… perhaps your security office
has some concerns?
Reporting is limited
No exported resources
Exported resources allow nodes to share
information with each other
Infrastructure as code

6. Can’t I just get by with a single puppet
master?
Yes, without a doubt absolutely… maybe…

7. http://docs.puppetlabs.com/learning/agent_master_basic.html

8. https://docs.puppetlabs.com/puppet/
Seriously, the docs are great!
Pro Puppet. 2013. Krum, van Hevelingen,
Kero,Turnbull, and McCune

9. This is the holy grail of running puppet open
source….
maybe…

10. Should the servers be physical or virtual?
If you have an virtualization infrastructure
already, use it…
If you don’t have one, build it and then use
it… perhaps puppet can help you with that…

11. Leverage redundancy and resilience of the
VMware infrastructure
Ability to scale as needed quickly as more
systems are deployed with puppet
Flexibility in providing puppet infrastructure
to non-central units
Flexibility to test and roll through upgrades as
they come out from PuppetLabs

12. Java application (ok… it’s clojure inside a JVM…
just ask Deepak) that data generated by Puppet
with a postgresql database backend
Stores the most recent set of facts, most recent
catalog and by default 14 days of reports
Provides a very robust API for access to
information
Great for exported resources! (don’t even think
about using storeconfigs)

13. Really? Hmmmm….
https://docs.puppetlabs.com/guides/scaling_
multiple_masters.html
To scale beyond a certain size
Geographic distribution / disaster recovery
Administrative boundaries (politics)

14. There isn’t a golden “though shall must
always have more than one at this level”
Number of resources declared in manifests
and infrastructure?
Exported resources?
Number of nodes?
Time to compile catalog?
Time between runs?
How are other administrative groups using
puppet?

15. After adding hosts via pdsh After random distribution reestablishes

17. Methods for getting the manifests modules out
to the puppet masters
Software distribution (RPM, Jenkins testing, etc.)
Puppetfile management with R10K, puppet- librarian,
puppet-librarian-simple
Puppet librarian
Run your own forge! (Pulp)
git pull via cron jobs…
NFS share…
Puppet 3.6 with the new path is a potential issue
for migration in this module…

18. LB: 1 core, 2 GB ram
PM-CA: 1 core, 2 GB ram
PM: 4 cores, 4 GB ram
PuppetDB: installed on
each puppet master
PostgreSQL: 2 cores, 8
GB Ram
(VMware Hosts are
ProLiant DL385 G7s w/
AMD 6176 2.3 GHz procs)

21. http://www.reddit.com/r/linux/comments/18weoo/i_manage_a_1600_node_puppet_configuration_ama/

22. theForeman and Dashboard use the same
postgresql DB backend (2 cores, 8 GB Ram)
theForeman web frontend: 2 cores, 4 GB Ram
Dashboard frontend: 2 cores, 8 GB Ram (have
puppet master and puppetdb installed on
same server for reporting access)
YMMV on any of these specs

23. LB (mod_proxy) splits
traffic to appropriate PM
Puppet Master (PM) can
serve out a single or
multiple environments via
apache/mod_passenger
The CA CRL is distributed
to the LB and PMs
All reports are forwarded
to foreman and dashboard
Shared file bucket across
all puppet masters via NFS

24. Leverage redundancy
and resilience of the
VMware infrastructure
Can quickly scale as
needed for systems or
environments
Security and partitioning
of environments
Safely test and roll
through server upgrades
Fast resource additions
to reduce chokepoints

25. A single load balancer
in front (hardware or
software)
A single server running
PuppetDB
A single server running
postgresql
Reporting?

26. Nope…
Statically assign masters to the clients in the
puppet.conf file
Use Round-Robin DNS
DNS SRV Records

27. Running puppet as a service for multiple
groups with different administrative
boundaries
Everything is just peachy for getting
manifests out and reports separated with
RBAC! except…
PuppetDB: All for one and one for all… full
access to any facts, catalogs, and reports in
the database… no RBAC

28. Questions?
Comments?
Divine revelations?