PuppetConf. 2016: External Data in Puppet 4 – R.I. Pienaar

R.I.Pienaar
PuppetConf 2016
External Data in Puppet 4

R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
Who am I?
• Puppet User since 0.22
• Really into external data
• Author facts.d, extlookup, Hiera, MCollective
• Consultant - DevOps, Automation,
Architect, Development
• Open Source @ github.com/ripienaar
• Volcane on IRC

External Data
Is Conﬁguration For
Your Code

Hierarchical Data
class monitor (
Array[Monitor::Plugin] $plugins = []
) {
…
}

Hierarchical Data
class{“monitor”:
plugins => [
“cpu",
“write_graphite”,
“df”, …
]
}

Automatic Param Lookup
include monitor
# hieradb/nodes/host1.yaml
monitor::plugins:
- my_app
class{“monitor”:
plugins => [
“my_app”, “cpu”, “df”, …
]
}
+
=

Hierarchical Data
host1 host2 host3 host4 host5 host6
prod stage dev
dc1 dc2
Site wide common data

Hierarchical Data
host1 host2 host3 host4 host5 host6
prod stage dev
dc1 dc2
Site wide common data
$trusted[“certname”]
$environment
$facts[“dc”]

Lookup
• lookup() function replaces hiera*()
• Automatic Param Lookup more useful
• environment and module aware
• Great CLI
• YAML/JSON out of the box, more to come
• Data compatible with Hiera, cohabits with
Hiera
• Pluggable

Lookup - CLI
$ puppet lookup monitor::plugins —merge unique
——
- my_app
- write_graphite
- cpu

$ puppet lookup monitor::plugins —merge unique —explain
Merge strategy unique
…
Data Provider "Nodes"
Path “/etc/…/production/data/nodes/dev3.devco.net.yaml”
Original path: "nodes/%{trusted.certname}"
Found key: "monitor::plugins" value: [
"my_app"
]
…
Data Provider "common"
Path “/etc/…/data/common.yaml”
Original path: "common"
"cpu",
"write_graphite"
]
…
Merged result: [
"my_app",
"cpu",
"write_graphite"
]

$ puppet lookup monitor::plugins —merge unique —explain
Merge strategy unique
…
Data Provider "Nodes"
Path “/etc/…/production/data/nodes/dev3.devco.net.yaml”
Original path: "nodes/%{trusted.certname}"
"my_app"
]
…
Data Provider "common"
Path “/etc/…/data/common.yaml”
Original path: "common"
"cpu",
"write_graphite"
]
…
Merged result: [
"my_app",
"cpu",
"write_graphite"
]
Every tier
Final result

Lookup - Conﬁg
# environments/production/hiera.yaml
---
version: 4
datadir: data
hierarchy:
- name: "Nodes"
backend: yaml
path: "nodes/%{trusted.certname}"
- name: "common"
backend: yaml

Lookup - function
$plugins = lookup(“monitor::plugins”)

Lookup - function
$plugins = lookup(“monitor::plugins”, {
“value_type” => Array[Monitor::Plugin],
“default_value” => [“cpu”, …]
})

Lookup - function
function monitor::params() {
$result = {
“monitor::plugins” => [
“cpu”, …
]
}
}
$plugins = lookup(“monitor::plugins”, {
“default_value_hash” => monitor::params()
})

Lookup - function
$servers = lookup(“monitor::plugins”, {
“merge” => “first”
})
Default behaviour

Lookup - function
“merge” => “unique”
})
Array Merge (hiera_array())

Lookup - function
“merge” => “hash”
})
Shallow Hash Merge (hiera_hash())

Lookup - function
“merge” => “deep”,
})
Deep Hash Merge

Lookup - function
# common.yaml
users::local:
rip:
email: “rip@devco.net"
comment: “R.I.Pienaar”
sudoer: true
shell: “/bin/zsh”

Lookup - function
# nodes/dev3.devco.net
users::local:
rip:
email: “arri.pienaar@client.com”

Lookup - function
$ puppet lookup users::local —merge deep
---
rip:
email: arri.pienaar@client.com
comment: R.I.Pienaar
sudoer: true
shell: "/bin/zsh"

Lookup - function
$lookup_options = {
“merge” => {
“strategy” => “unique”
}
}
$plugins = lookup(“monitor::plugins”, $lookup_options)

Lookup - function
$plugins = lookup(“monitor::plugins”)
# hieradata/common.yaml
lookup_options:
monitor::plugins:
merge:
strategy: unique

include monitor
# …/production/hieradata/common.yaml
lookup_options:
monitor::plugins:
strategy: unique
monitor::plugins:
- cpu
- …
class monitor (
Array[Monitor::Plugin] $plugins = []
) {}
Lookup - function

Lookup - Data In Modules
# modules/monitor/hiera.yaml
---
version: 4
datadir: "data"
hierarchy:
- name: "common"
backend: "yaml"
# modules/monitor/data/common.yaml
lookup_options:
monitor::plugins:
strategy: unique

include monitor
# hieradata/common.yaml
monitor::plugins:
- cpu
- write_graphite
Lookup - Data In Modules
# hieradata/nodes/node1.yaml
monitor::plugins:
- my_app

Tips and Tricks

lookup(“classes”, {“merge” => “unique”}).include
hiera_include()

hiera_include()
# common.yaml
lookup_options:
classes:
merge:
strategy: “deep”
knockout_prefix: “-“
classes:
- nagios
# nodes/node1.yaml
classes:
- -nagios
- sensu

lookup(“classes”).include
hiera_include()
# common.yaml
lookup_options:
classes:
merge:
strategy: “deep”
knockout_prefix: “-“

params.pp
# modules/ntp/hiera.yaml
---
version: 4
datadir: data
hierarchy:
- name: "Operating System Family"
backend: yaml
path: "%{facts.os.family}"
- name: "common"
backend: yaml

params.pp
# modules/ntp/data/AIX.yaml
---
ntp::driftfile: '/etc/ntp.drift'
ntp::package_name:
- 'bos.net.tcp.client'
ntp::restrict:
- 'default nomodify notrap nopeer noquery'
- '127.0.0.1'
ntp::service_name: 'xntpd'

params.pp
# modules/ntp/data/Debian.yaml
ntp::restrict:
- '-4 default kod nomodify notrap nopeer noquery'
- '-6 default kod nomodify notrap nopeer noquery'
- '127.0.0.1'
- '::1'
ntp::service_name: 'ntp'
ntp::servers:
- '0.debian.pool.ntp.org'
- '1.debian.pool.ntp.org'

Future
• Hiera deprecated
• No more site wide hiera, only environment
and modules
• Pluggable via functions in modules - native or
ruby, much much simpler than old backends
• lookup.yaml or lookup.conf

Questions?
twitter: @ripienaar
email: rip@devco.net
blog: www.devco.net
github: ripienaar
freenode: Volcane
slack.puppet.com: ripienaar
https://www.devco.net/

PuppetConf. 2016: External Data in Puppet 4 – R.I. Pienaar

Recomendados

Recomendados

Más contenido relacionado

Destacado

Destacado (20)

Similar a PuppetConf. 2016: External Data in Puppet 4 – R.I. Pienaar

Similar a PuppetConf. 2016: External Data in Puppet 4 – R.I. Pienaar (20)

Más de Puppet

Más de Puppet (20)

Último

Último (20)

PuppetConf. 2016: External Data in Puppet 4 – R.I. Pienaar