An overview of Puppet principles (from 0) and a presentation of a complete control-repo to use as starting point for a state of the art Puppet setup.

1. Puppet.
From 0 to 100
in 30 minutes
(intermediary and next steps are on you)
Alessandro Franceschi
@alvagante

2. From 0
Puppet essentials

3. What is Puppet?
• Puppet is a Configuration Management Tool
• It manages applications, systems' components,
network and cloud resources.
• It provides a language that describes the
managed IT resources abstracting the
implementation details.
• It implements paradigms like Infrastructure as
Code and Infrastructure as Data

4. Puppet components
• On managed nodes are executed Puppet agent and Facter.
• On the central Master runs the Puppet server, here public
modules and our Puppet code and data are deployed with
r10k.
• Puppet generated data is stored to PuppetDB
(it uses PostgreSQL for persistence ;)
• Our data to configure classes and manage Infrastructure can
be defined using Hiera.
• It's also possible to define what classes to use in which
nodes, with External Node Classifiers (ENC) like Puppet
Enterprise Console or The Foreman.

5. A Puppet run in essence
• Puppet agent use facter to collect facts about the
system and sends them to the Puppet Master
• Puppet Master, using nodes' facts and our Puppet code
and data builds a catalog of resources to manage
• The catalog is sent back to the client. The included
resources are applied locally: packages are installed, files
changed, services started, databases initialised...
• Whatever the number of runs, at the end the system should
be at the desired configuration state (idempotency)
• A report of what happens on the client is sent back to the
Puppet Master and typically stored on PuppetDB

6. Puppet language essentials
• Puppet represents managed resources on the systems
via a declarative Domain Specific Language (DSL)
• Puppet language is written in files called manifests,
which have a .pp extension
• The single unit of configurations are resource types
• Puppet resources are grouped in classes or defines
• Classes, user defined types (defines), along with relevant
configuration files, are shipped in modules
• Public modules are released on Puppet Forge or
GitHub

7. (Resource) types
• Puppet types manage resources abstracting the
underlying OS, elements of the system. Examples:
package { 'postgresql':
ensure => present,
}
service { 'postgresql':
ensure => running,
enable => true,
}
file { '/var/lib/pgsql/data/pg_hba.conf':
ensure => present,
source => puppet:///modules/postgres/pg_hba.conf',
}
• Common native resources: package, service, file,
user, group, host, cron, exec, host, mount...

8. Extra resources from modules
• Additional resources can be provided by dedicated
modules. Examples from puppetlabs-postgresql module:
postgresql::server::db { 'mydatabasename':
user => 'mydatabaseuser',
password => postgresql_password('mydatabaseuser', 'mypassword'),
}
postgresql::server::role { 'marmot':
password_hash => postgresql_password('marmot', 'mypasswd'),
}
postgresql::server::database_grant { 'test1':
privilege => 'ALL',
db => 'test1',
role => 'marmot',
}
postgresql::server::table_grant { 'my_table of test2':
privilege => 'ALL',
table => 'my_table',
db => 'test2',
role => 'marmot',
}
postgresql::server::pg_hba_rule { 'allow application network to access app database':
description => "Open up PostgreSQL for access from 200.1.2.0/24",
type => 'host',
database => 'app',
user => 'app',
address => '200.1.2.0/24',
auth_method => 'md5',
}

9. Extra resources examples
• There are modules to manage almost everything:
- Specific applications (nginx, postgresql...)
- Systems' features (iptables, sysctl, network...)
- Network equipment (cisco, arista, cumulus... )
- Storage equipment (netapp, emc...)
- Cloud resources (aws, azure, digitalocean...)
- Cloud infrastructures (openstack, opennebula...)
- Containers (docker, mesos, rkt...)
• Each module may provide specific classes and resources
to manage with Puppet DSL specific components

10. Classes
• Puppet classes expose parameters which define
what and how are the managed resources
class postgresql::server (
$postgres_password = undef,
$package_ensure = $postgresql::params::package_ensure,
$service_ensure = $postgresql::params::service_ensure,
$service_enable = $postgresql::params::service_enable,
) inherits postgresql::params {
[ ... ]
package { 'postgresql':
ensure => $package_ensure, # [...]
}
service { 'postgresql':
ensure => $service_ensure, # [...]
enable => $service_enable, # [...]
}
}
• Class parameters can be set via Hiera, for example
using the yaml backend
postgresql::server::postgres_password: 'my_cleartext_password'
postgresql::server::package_ensure: absent
# Secrets can be encrypted using the hiera-eyaml backend. They look like:
postgresql::server::postgres_password: 'ENC[PKCS7,MIIISA ...]

11. Modules
• Modules are distributable directories containing manifests,
extensions, templates and files to manage.
• They have a standard structure:
mysql/ # Main module directory
mysql/manifests/ # Manifests directory. Puppet code stays here.
mysql/lib/ # Plugins directory. Ruby code that extends
# Puppet (types, providers, facts...) is here.
mysql/templates/ # ERB and EPP Templates directory
mysql/files/ # Static files directory
include mysql
# Main mysql class is placed in: $modulepath/mysql/manifests/init.pp
include mysql::server
# This class is defined in: $modulepath/mysql/manifests/server.pp
mysql::db { ...}
# This define is defined in: $modulepath/mysql/manifests/db.pp
file { '/etc/motd':
content => template('motd/motd.conf.erb'),
} # Template is in: $modulepath/motd/templates/motd.conf.erb
• This allows some conventions:

12. Hiera
• Hiera is a key-value lookup tool based on a
configurable hierarchy
• It can have different backends to store data in
different places (yaml, eyaml, json, redis, mysql...)
• Hiera is configured in hiera.yaml
---
:backends:
- yaml
:hierarchy:
- "hostname/%{::trusted.certname}"
- "role/%{::role}-%{::env}"
- "role/%{::role}"
- common
:yaml:
:datadir: "/etc/puppetlabs/code/environments/%{environment}/hieradata"

13. Roles and profiles
• A popular pattern to manage what resources
should be manages on which nodes
• A role is assigned to nodes (using different
methods: custom facts, ENC...)
• A role represents what a node does and can
include one or more profiles
• In profiles resources from modules are used to
configure local systems in the desired way

14. Puppet environments
• A Puppet environment is directory under
/etc/puppetlabs/code/environments/
where are is placed our Puppet manifests, modules
and Hiera data
• Puppet environments can can be distributed as git
control-repos. They contain:
manifest directory for common manifests
modules directory for modules
hieradata directory for Hiera data, if used
Puppetfile to configure r10k for public modules

15. to 100
Introducing example42 Puppet control repo

16. example42 control-repo
• The result of several years of Puppet experience.
A git repository with the content of a Puppet 4
environment featuring:
• Data and code for a state of the art modern Puppet setup
• Fabric integration for common operations
• Puppet 4 optimised code following updated design principles
• Customisable Vagrant environments to locally test code
• Docker integrations to test code or build images
• Classification based on Roles and Profiles pattern
• In green fields: whatever is needed to start a new Puppet project from scratch
• In brown fields: source of inspiration
• ... and, yes, a lot more.

17. Setup
• Download the repo from GitHub
• Setup environment via a shell script ...
• Or via Fabric:
• Install git hooks for pre-commit syntax checks
git clone https://github.com/example42/control-repo
cd control-repo
bin/setup.sh
fab puppet.setup
fab git.install_hooks

18. Explore
• Give a look to the following files and directories:
# The first manifest parsed by Puppet server
manifests/site.pp
# r10k Puppetfile and directory for public modules
Puppetfile
modules/
# Directory containing local site modules
site/
# Sample Hiera configuration file and data directory
hiera.yaml
hieradata/
# Directories for Vagrant and Docker operations
vagrant/
docker/
# Blueprint directory for new Puppet 4 modules
skeleton/

19. Test local code with Vagrant
Different Vagrant environments available:
fab vagrant.all_status
Single roles can be tested in relevant VMs:
fab vagrant.up:vm=dev-local-log-01
fab vagrant.provision:vm=dev-local-log-01
# All Linux servers use this class of common resources:
# site/profile/manifests/base/linux.pp
# Common settings are in:
# hieradata/common.yaml
# For role "log" specific Hiera data is in
# hieradata/role/log.yaml
Vagrant environments can be customised:
# vi vagrant/environments/$vagrant_environment/config.yaml
vi vagrant/environments/puppetinfra/config.yaml

20. Test local code with Docker
Build multiOS Docker images:
fab docker.build_multios
# Images are buid based on the data in:
# hieradata/role/docker_multios_build.yaml
Test a role on a given OS:
fab docker.test_role:log,ubuntu-14.04
# Available images: ubuntu-12.04, ubuntu-14.04, ubuntu-14.06
# centos-7, debian-7, debian-8, alpine-3.3
Build an image based on a role (WIP):
fab docker.build_role:log,ubuntu-14.04
Build multiple OS images based on a role (WIP):
fab docker.build_role_multios:log

21. next steps
learn Puppet and customise your control-repo

22. Learning Puppet
Check the Official docs
http://docs.puppet.com
Ask and get involved in the Community
https://puppet.com/community
Look for modules of Puppet Forge
http://forge.puppet.com
Give a look to Tiny Puppet
(used in example42/control-repo)
http://www.tiny-puppet.com

23. Customise
• The control-repo is just the starting point for a
greenfield modern Puppet setup
• Define a way to set your nodes' roles
• Select the public modules to use and add them to
Puppetfile
• Write local profiles in site/profile/manifests
• Review hiera.yaml logic and customise data in
hieradata/
• Customise your Vagrant environment
• Customise the skeleton to use for custom modules

24. Contribute
• example42's control repo has a lot of room for
enhancements
• Provide profiles to manage applications
• Send bug and features reports
• Use and improve our code
• Spread the word!
#example42

25. From 0 to 100
in 30 minutes?
Thank You
(feel free to ask me anything)
Alessandro Franceschi
@alvagante