Puppet Camp Paris 2016 Data in Modules

Copyright example42 GmbH - 2016
Puppet 4 - Data in Modules
PuppetCamp Paris 2016
Martin Alfke - ma@example42.com
Image: Tatlin - tatlin.net

Martin Alfke
!
Berlin/Germany
!
CEO example42 GmbH
Freelance Puppet Expert
Network
!
Puppet since 2007
!
Puppet Trainer, Consultant
!
Co-Author of “Puppet 4
Essentials”

Puppet 4
Data in Modules
• Separation of Code and Data
• Data in Modules
• Lookup Priority
• Data in Component Modules
• Data in Environments

Separation of
Code and Data

Separation of
Code and Data
• data in code
class my_ntp {!
!if $::environment == ‘dev’ {!
$ntp_server = [‘pool.ntp.org’]!
} else {!
if $::facts[‘fqdn’] == ‘ntp1.example42.com’
{!
# switch back to ntp1 when issue is solved!
$ntp_server = [‘ntp2.example42.com’]!
} else {!
$ntp_server = [‘127.0.0.1’]!
}!
}!

Separation of
Code and Data
• explicit lookup
• hiera(‘key’[, ‘default’][, ‘override hierarchy’])

Separation of
Code and Data
• implicit lookup
class my_ntp (!
!Array $ntp_server,!
) {!
!# …!
}!
!
contain my_ntp!
!
# hiera data!
my_ntp::ntp_server:!
- ‘pool.ntp.org’!

Separation of
Code and Data
• hiera.yaml
# version 1!
:backends:!
- yaml!
:yaml:!
:datadir: “/etc/puppetlabs/code/
environments/%{environment}/hieradata”!
:hierarchy:!
- “nodes/%{::trusted.certname}”!
- “os/%{::facts[‘os’][‘osfamily’]}”!
- common!

Separation of
Code and Data
• hieradata
os/Debian.yaml
apache::pkgname:!
- ‘apache2’!
- ‘apache2-ssl’!
!
os/RedHat.yaml
apache::pkgname:!
- ‘httpd’!
!
common.yaml
apache::purge_configs: true!

Separation of
Code and Data
• hieradata
os/FreeBSD.yaml
apache::pkgname:!
- ‘apache’!

Separation of
Code and Data
• puppet code
# apache/manifests/params.pp
class apache::params {!
case $::operatingsystem {!
‘Debian’: { # … }!
‘RedHat’: { # … }!
default: {!
fail(‘OS not supported’)!
}!
}!
}!

Data in Modules

Lookup Priority

Lookup Priority
• Hiera -> Global Overrides
!
!
!

Lookup Priority
!
• Environment Data -> Core Puppet Lookup
!

Lookup Priority
!
• Environment Data -> Core Puppet Lookup
!
• Module Data -> Defaults

Lookup Functions
• Explicit lookup: lookup(‘key’)!
!
• CLI lookup: puppet lookup ‘key’
!
• Automatic lookup: ‘<namespace>::<key>’

Lookup Functions
• lookup(‘key’, <Type>, <merge_behavior>, <default>)!
• e.g. lookup(‘ntp_servers’, Array)
• Merge behavior:
• first!
• unique (array merge)!
• hash!
• deep!

Data Provider
Conﬁguration
• Global Environment Provider: puppet.conf
• environment_data_provider = <data provider>!
!
!
!

Data Provider
Conﬁguration
• Environment Provider: environment.conf
!

Data Provider
Conﬁguration
• Environment Provider: environment.conf
• Module Provider: metadata.json
• “data_provider”: “<data provider>”

Data Provider
• none -> standard hiera lookup
!
• hiera -> hiera lookup (hiera v4)
!
• function -> data function lookup

Data Provider
Hiera
• replace hiera, hiera_array, hiera_hash with ‘lookup’
• needs hiera.yaml v4 conﬁguration ﬁle
• set data_provider to ‘hiera’ in puppet.conf,
environment.conf or metadata.json
• modify global hiera.yaml to use datadir outside
environment

Data Provider
Hiera
# /etc/puppetlabs/code/environments/production/hiera.yaml
# /etc/puppetlabs/code/environments/production/modules/<module>/
hiera.yaml
#
- - -!
version: 4!
datadir: hieradata!
hierarchy:!
- name: “Nodes”!
backend: yaml!
path: “nodes/%{trusted.certname}”!
- name: “OS”!
backend: json!
path: “os/%{facts.os.family}”!
- name: “common”!
backend: yaml

Data Provider
Function
• write data function
• Puppet 4 Function
• <module>/functions/<module>/data.pp
• <env>/functions/<env>/data.pp
• set data_provider to function in puppet.conf,

Data Provider
Function - Puppet
# ntp/functions/ntp/data.pp
function ntp::data() {!
$params = {!
‘ntp::ntpservers’ => [‘pool.ntp.org’],!
}!
$os_params = case $facts[‘os’][‘family’] {!
‘Debian’: {!
{ ‘ntp::ntpackage’ => ‘ntpd’, }!
},!
default: {!
{}!
}!
}!
$params + $os_params!
}

Data Provider
Function
• write data function
• Ruby Function (Puppet 4 function API)
• <module>/lib/puppet/functions/<module>/
data.rb
• <env>/lib/puppet/functions/<env>/data.rb
• set data_provider to function in puppet.conf,

Data Provider
Function - Ruby
# ntp/lib/puppet/functions/ntp/data.rb
Puppet::Functions.create_function(:’ntp::data’) do!
def base_data()!
{ ‘ntp::ntpservers’ => [‘pool.ntp.org’], }!
end!
def os_data()!
case Facter.value(:os)[‘family’]!
when ‘Debian’!
{ ‘ntp::pkgname’ => ‘ntpd’, }!
else!
{}!
end!
def data()!
self.base_data.merge!(self.os_data)!
end!
end

Data in Component Modules

Data in Component Modules
• add data provider to metadata.json
• provide OS defaults
• remove params.pp / remove inheritance
• allow users to overwrite any data

Data Provider
Function - Ruby
# my_ntp/manifests/init.pp
class my_ntp (!
$server = $my_ntp::params::server,!
$pkgname = $my_ntp::params::pkgname,!
$secure = $my_ntp::params::secure,!
) inherits my_ntp::params {!
# ...!
}!

Data in Environments

Data in Environments
• old hiera replacement
• add hiera.yaml to environment base path
• overwrite data from modules, roles & proﬁles

Summary

Summary
# /etc/puppetlabs/code/
! ! hiera.yaml!
! ! hieradata/!
! ! environments/production/
! ! ! ! ! ! environment.conf!
! ! ! ! ! ! hiera.yaml!
! ! ! ! ! ! hieradata/!
modules/my_module/
! ! ! ! ! ! ! ! ! ! metadata.json!
! ! ! ! ! ! ! ! ! ! hiera.yaml!
! ! ! ! ! ! ! ! ! ! hieradata/

Summary
# /etc/puppetlabs/code/
! ! hiera.yaml!
! ! hieradata/!
! ! environments/production/
! ! ! ! ! environment.conf!
! ! ! ! ! lib/functions/data.pp!
! ! ! ! ! lib/puppet/functions/ !
! ! ! ! ! ! ! ! production/data.rb!
modules/my_module/
! ! ! ! ! ! !! ! metadata.json!
! ! ! ! ! ! ! ! lib/functions/data.pp!
! ! ! ! ! ! ! ! lib/puppet/functions/ !
! ! ! ! ! ! ! ! ! ! ! my_module/data.rb

Summary - Pro
• Per hierarchy Hiera Data backend possible
• Data Function lookups without need for hiera
backend (e.g. Cloud Management API data)
• No more inheritance required

Summary - Con
• No single Source of Authority?
• Debugging can be complex when iterating over
many data providers and hierarchies

Module Developers
• switch to data in modules
• give users the possibility to provide own data
• allow users to overwrite any data
• allow users to know their data for missing OS
support

Module Users
• switch to hieradata in modules
then
• switch to data in environments
• keep data simple and readable
• don’t overcomplicate !

Module Users
• hieradata
common.yaml
my_ntp: ‘pool.ntp.org’!
apache::default_mods: false!
apache::purge_configs: true!
mysql::remove_default_accounts: true!
mysql::root_password: ‘puppet’!
oradb::database::version: ’12.1’!
oradb::shout: ‘MISSING DATA’

References
• http://docs.puppetlabs.com/puppet/4.3/reference/
lookup_quick.html
• http://docs.puppetlabs.com/puppet/4.3/reference/
lookup_quick_module.html
• http://puppet-on-the-edge.blogspot.de/2015/01/
puppet-40-data-in-modules-and.html

Puppet 4 - Data in modules
PuppetCamp Paris 2016
Martin Alfke - ma@example42.com

Puppet Camp Paris 2016 Data in Modules

Recomendados

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente (20)

Similar a Puppet Camp Paris 2016 Data in Modules

Similar a Puppet Camp Paris 2016 Data in Modules (20)

Más de Martin Alfke

Más de Martin Alfke (17)

Último

Último (20)

Puppet Camp Paris 2016 Data in Modules