Talk given for the Belgian Puppet User Group meetup at Inuits office on the 12th November 2014.

1. .
AAuuggeeaass
SSwwiissss--kknniiffee rreessoouurrcceess ffoorr yyoouurr ppuuppppeett ttrreeee
Julien Pivotto
Belgian Puppet User Group
Holiday is over Meetup!!! - November 12th, 2014

2. .
wwhhooaammii
JJuulliieenn PPiivvoottttoo
• Open-Source consultant at inuits.eu
• FOSS defender since 2004
• DevOps believer and evangelist
• Puppet User since 2011
• @roidelapluie on twitter/github

3. .
.
ınuits.eu

4. .
.
SSyyssaaddmmiinn 110011
CC BY-SA 2.0 https://www.flickr.com/photos/arthur-caranta/2926332140

5. .
SSeettttiinngg uupp aa sseerrvviiccee
• Install the package
• Change the configuration
• Start the daemon

6. .
3 steps.
What can go wrong?

7. .
PPaacckkaaggiinngg
• Where is the package?
• Which version do we need?
• Does it conflict with something else?

8. .
.
DDeeppeennddeenncciieess HHeellll
CC BY-SA 2.0 https://www.flickr.com/photos/coconinonationalforest/4587053982

9. .
CCoonnffiigguurraattiioonn
• Where is the file?
• How many files?
• Configuration is in the database?
• The file is *huge*

10. .
SSttaarrttiinngg tthhee sseerrvviiccee
• Does not start
▶ Bad config file
▶ Stale lock file
▶ Data corruption
• High Availability
• Replication

11. .
LLeett''ss ttaallkk aabboouutt PPuuppppeett aanndd ffiilleess
• Classical approach: File[] resource
• Advanced approach: Concat[] define
• Broken approach: Exec[sed] resource
• Surgical approach: Augeas[] resource

12. .
HHiiddddeenn wwaayyss ttoommaannaaggee ffiilleess
• Ssh_authorized_key[]
• Nagios_*
• To purge or not to purge

13. .
.
TThhee FFiillee[[]] rreessoouurrccee
CC BY 2.0 https://www.flickr.com/photos/80497449@N04/10567875696/

14. .
FFiillee
• Built-in puppet resource
• Most used
• Works with a lot of usecases
• Text files, binary files

15. .
.
file{"${::icinga::confdir_server}/cgi.cfg":
ensure => present,
content => template('icinga/redhat/cgi.cfg.erb'),
owner => $::icinga::server_user,
group => $::icinga::server_group,
require => Class['icinga::config'],
notify => [
Service[$::icinga::service_client],
Service[$::icinga::service_server],
Exec['fix_collected_permissions']
],
}
.

16. .
CCoonntteenntt ooff aa ffiillee
• content => String, template(), file()
• source => puppet:///, /local/file

17. .
FFiillee[[]] bbeehhaavviioouurr
• Array as "source": Puppet will pick the first
available one
• Multiple arguments to template(): Puppet
will concatenate them all
• File[/foo/bar] will autorequire File[/foo]

18. .
DDoowwnnssiiddee ooff FFiillee[[]]
• You can only have at one "content"
• That resource describe the whole file
• Works in almost every situation

19. .
.
concat
Public Domain http://commons.wikimedia.org/wiki/File:Adhesive_tapes_clear.JPG

20. .
CCoonnccaatt
• A "reference" puppet module:
puppetlabs/concat
• https://github.com/puppetlabs/puppetlabs-concat
• Provides definitions to manage file
• Alternative modules:
▶ onyxpoint/pupmod-concat
▶ theforeman/puppet-concat (fork of onyxpoint)

21. .
CCoonnccaatt??
• Concat takes a bunch of snippets
• Assemble them info a file
• Each snippet is a define
• The final file is a define

22. .
.
concat { '/tmp/file':
ensure => present,
}
concat::fragment { 'tmpfile':
target => '/tmp/file',
content => 'test contents',
order => '01'
}
.

23. .
BBaassee aanndd ffrraaggmmeennttss
• Concat[] defines owner, ensure, mode of the
file
• Concat::Fragment[] defines the contents of
the file
• One Concat[] has multiple
Concat::Fragment[]

24. .
AAddvvaannttaaggeess ooff ccoonnccaatt
• More flexibility
▶ if
▶ virtual resource
▶ exported resources
▶ create_resources
• Mix templates and files

25. .
DDiissaaddvvaannttaaggeess ooff ccoonnccaatt
• External Puppet module
• Concat[] is the whole file
• Performances

26. .
.
Exec{sed: onlyif => grep}
CC BY-SA 3.0 http://commons.wikimedia.org/wiki/File:Ca%C3%AFn_par_Henri_Vidal.jpg

27. .
.
https://github.com/search?o=desc&q=exec+sed+onlyif+grep+language%3APuppet

28. .
eexxeecc[[sseedd]] iiss bbrr00kkeenn
• Which options to pass to sed and grep?
• You should use as few Exec[] as possible
• grep ....
• Escape, regexes…

29. .
AAnnootthheerr aalltteerrnnaattiivvee:: ccoonnff..dd
• Some services support conf.d directories
• But it is hard to change existing parameters
• In which order are the files read?
• Don't forget to purge

30. .
.
Augeas
CC BY-SA 3.0 http://commons.wikimedia.org/wiki/File:Students_assisting_surgery.JPG

31. .
AAuuggeeaass
• Configuration editing tool
• First release in 2007
• API coded in C
• Command-line tools
• bindings for different languages

32. .
CCoonnffiigguurraattiioonn eeddiittiinngg ttooooll
• Parsing the configuration files
• Turning them into a tree
• Edit the tree & save the configuration

33. .
.
$ cat /etc/nsswitch.conf
# /etc/nsswitch.conf
##
Example configuration
#
passwd: db files
group: db files
initgroups: db [SUCCESS=continue] files
shadow: db files
gshadow: files
.

34. .
.
augtool> ls /files/etc/nsswitch.conf/
#comment[1] = /etc/nsswitch.conf
#comment[2] = Example configuration
database[1]/ = passwd
database[2]/ = group
database[3]/ = initgroups
database[4]/ = shadow
database[5]/ = gshadow
.

35. .
.
augtool> ls /files/etc/nsswitch.conf/database[1]/
service[1] = db
service[2] = files
.

36. .
NNaattiivvee ffoorrmmaatt -->> ttrreeee
• Augeas understand comments
• Augeas does not care about empty lines
• The cli tool (augtool) has autocomplete
• It recognize a lot of formats

37. .
.
augtool> set /files/etc/nsswitch.conf/database[1]/
service[last()+1] ldap
augtool> save
Saved 1 file(s)
.

38. .
.
$ cat /etc/nsswitch.conf
# /etc/nsswitch.conf
##
Example configuration
#
passwd: db files ldap
group: db files
initgroups: db [SUCCESS=continue] files
shadow: db files
gshadow: files
.

39. .
.
augtool> match /files/etc/nsswitch.conf/*/* ldap
/files/etc/nsswitch.conf/database[1]/service[3]
augtool> print /files/etc/nsswitch.conf/database[1]
/files/etc/nsswitch.conf/database[1] = "passwd"
/files/etc/nsswitch.conf/database[1]/service[1] = "db"
/files/etc/nsswitch.conf/database[1]/service[2] = "files"
/files/etc/nsswitch.conf/database[1]/service[3] = "ldap"
.

40. .
.
augtool> rm /files/etc/nsswitch.conf/database[1]/service[3]
rm : /files/etc/nsswitch.conf/database[1]/service[3] 1
augtool> print /files/etc/nsswitch.conf/database[1]
/files/etc/nsswitch.conf/database[1] = "passwd"
/files/etc/nsswitch.conf/database[1]/service[1] = "db"
/files/etc/nsswitch.conf/database[1]/service[2] = "files"
augtool> save
Saved 1 file(s)
.

41. .
OOnnee AAPPII ttoo eeddiitt tthheemmaallll
• Can talk XML, ini, named, nginx, …
• Only change what is needed
• Ensure the syntax is right

42. .
AAuuggeeaass LLeennsseess
• Lenses are files that explain how to edit files
• It contains paths and syntax
• There are a lot of them available
• You can write your own lenses

43. .
”This brings the total number of lenses to
178. […] It’s depressing to think that
Linux/Unix systems have managed to grow
this many special snowflake formats.”
David Lutterkort, main developer
about Augeas 1.3.0

44. .
117788 lleennsseess
activemq_conf activemq_xml aliases aptconf
apt_update_manager backuppchosts bbhosts bootconf build
carbon cgrules channels cobblermodules cobblersettings collectd
crypttab cyrus_imapd darkice debctrl desktop device_map dhcpd
dnsmasq dovecot dpkg dput ethers exports fai_diskconfig fonts
fuse gdm grub gshadow hostname inetd inputrc interfaces iproute2
iptables jaas jmxaccess keepalived known_hosts koji krb5 ldif limits
login_defs logrotate mcollective memcached mke2fs
mongodbserver mysql nagioscfg nagiosobjects netmasks nginx ntp
ntpd odbc openshift_config openshift_http openvpn pam passwd
pbuilder postfix_main postfix_transport postfix_virtual
puppet_auth qpid rabbitmq resolv rmt securetty sep services shells
shellvars_list sip_conf slapd smbusers squid sshd stunnel
subversion sudoers sysconfig systemd thttpd up2date vfstab

45. .
AA sshhoorrtt lleennssee
.
module Hostname =
autoload xfm
(* View: lns *)
let lns = [ label "hostname" . store Rx.word . Util.eol ]
(* View: filter *)
let filter = incl "/etc/hostname"
. incl "/etc/mailname"
let xfm = transform lns filter
.

46. .
PPuuppppeett <<33 aauuggeeaass
• Native "augeas" resource
• Support for pluginsync
• Helpers available

47. .
PPuuppppeett eexxaammppllee
.
augeas { $name:
context => "/files${fstab::variables::fstab_file}",
changes => [
"rm ${fstab_match_line}",
],
onlyif => "match ${fstab_match_line} size > 0"
}
.

48. .
RReeaall uusseeccaasseess
• Change grub options
• Modify /etc/hosts
• Modify XML's (puppetlabs-tomcat)
• Configure Jenkins

49. .
PPlluuggiinnssyynncc
• Puppet has pluginsync support for Augeas
• Drop your lenses in your modules
• lib/augeas/lenses
• Use the "lens" parameter of the augeas
resource

50. .
PPuuppppeett eexxaammppllee
.
augeas{"jboss_conf":
.
context => "/files/etc/jbossas",
changes => [
"set jbossas.conf/JBOSS_IP $ipaddress",
"set jbossas.conf/JAVA_HOME /usr",
],
lens => "Jboss.aug",
}

51. .
AAuuggeeaass ccoommmmaannddss
set rm mv clear insert …

52. .
AAuuggeeaass ccoommppaarraattoorrss ((oonnllyyiiff))
match get

53. .
AAuuggeeaasspprroovviiddeerrss
• Helpers around augeas
• Puppet modules
• No augeas knowledge needed

54. .
aappaacchhee
.
apache_setenv { "SPECIAL_PATH":
ensure => present,
value => "/foo/bin",
}
.

55. .
kkeerrnneell__ppaarraammeetteerr
.
kernel_parameter { "quiet":
ensure => present,
bootmode => "normal",
}
.

56. .
Conclusion

57. .
DDiissaaddvvaannttaaggeess
• Learning required
• Library to install
• Writing lenses is hard

58. .
AAddvvaannttaaggeess
• Augeas is a mature tool
• Preserves comments in files
• It fails (if needed)
• Only changes what is needed
• A lot of lenses available
• Puppet integration
• Helpers available

59. .
FFiinnaall nnoottee
Most of the time, File[] resources are the way
to go. Augeas can help when you need to
change files generated by an application or
that you can not manage entirely.

60. .
RReeaaddiinnggss
• http://augeas.net/
• http://augeasproviders.com/
• https://docs.puppetlabs.com/

61. .
TThhaannkk yyoouu
Any question?
Thanks to @raphink

62. .
CCoonnttaacctt
Julien Pivotto
julien@inuits.eu
@roidelapluie
INUITS bvba
Belgium
+32 473 441 636
https://inuits.eu