Infrastructure automation for sysadmins

1. https://lynt.cz
Ing. Vladimir Smitka
vladimir.smitka@lynt.cz
@smitka
Lynt services s.r.o.
6. 12. 2017 1

2. https://lynt.cz
Intro
• Manual -> Script -> Infrastructure automatization
• Easy
• Procedural
• Idempotent
• Push
• Agent-less
• Batteries included
6. 12. 2017 2
No OS abstraction (e.g. Packages)
No good noop

3. https://lynt.cz
Platforms
• Linux/Mac/BSD – SSH + Python
• Windows – PowerShell
• SSH raw mode:
– Cisco IOS/ASA
– Juniper Junos
– VyOS
– Any SSH enabled node
• Mikrotik – unofficial/experimental via API
6. 12. 2017 3

4. https://lynt.cz
Content
• http://edu.lynt.cz/course/ansible
6. 12. 2017 4
• Inventory
• Patterns
• Tasks
• Playbooks
• Lookups
• Modules
• Jinja
• Handlers
• Roles
• Variables
• Facts
• Vault
• Galaxy
• Troubleshooting
• Performance
• Strategies

5. https://lynt.cz
Lab
6. 12. 2017 5

6. https://lynt.cz
Versions
• https://github.com/ansible/ansible/blob/devel/CHANGELOG.md
• …
• 1.9
• …
• 2.2 – Python 3 (3.5+), Stretch
• 2.3 – dest->path (backward compatible)
• 2.4 – Centos 7 (Centos 6 - Epel)
• deb http://ppa.launchpad.net/ansible/ansible/ubuntu trusty main
6. 12. 2017 6

7. https://lynt.cz
Installation
• apt install ansible
• alias apl="ansible-playbook"
6. 12. 2017 7

8. https://lynt.cz
Inventory
• /etc/ansible/hosts
• ansible --list-hosts all
• Localhost: ansible ansible_connection=local
• ansible -m ping all
• ansible -m command -a "uptime" all
• ansible -a "uptime" all
6. 12. 2017 8
Default module
INI, YAML

9. https://lynt.cz
Groups
[apps]
192.168.1.115
hostname
[mgmt]
ansible ansible_connection=local
6. 12. 2017 9

10. https://lynt.cz
Local Inventory
• …./work_dir/hosts
• ansible -i hosts --list-hosts all
• (-i <dir> - "inventory" file lookup)
• Config overloading:
– ENV ANSIBLE_CONFIG
– ./ansible.cfg
– ~/.ansible.cfg
6. 12. 2017 10
[defaults]
inventory = ./hosts
group_vars
host_vars

11. https://lynt.cz
Patterns
• http://docs.ansible.com/ansible/latest/intro_
patterns.html
• ansible --list-hosts all
• ansible --list-hosts *
• ansible --list-hosts 192*
• ansible --list-hosts ansible
• ansible --list-hosts 'all,!ansible'
6. 12. 2017 11

12. https://lynt.cz
Dynamic inventory
Script output:
{
"app": {
"hosts": [
"192.168.1.115"
]
},
"mgmt": [
"ansible"
],
"_meta": {
"hostvars": {
"ansible": {
"ansible_connection": "local"
}
}
}
}
6. 12. 2017 12

13. https://lynt.cz
Tasks
• Base building blocks
• ansible –m module –a mod_arguments hosts
• ansible –m command –a uptime all
• Options
-f 5 / --forks=5 - FORKS, parallel runs
-b / --become – privileged mode (su, sudo)
-l / --limit – subset of hosts
6. 12. 2017 13

14. https://lynt.cz
Modules
• http://docs.ansible.com/ansible/latest/modules_
by_category.html
• Commands – command, shell, raw, …
• Packing – yum, apt, pip,…
• System – service, cron, iptables, authorized_keys
…
• Cloud – various cloud providers
• Files – various files operations
• Utilities – playlist logic helpers
6. 12. 2017 14

15. https://lynt.cz
Playbooks
• tasks YAML format
---
- hosts: all
tasks:
- command: uptime
https://gist.github.com/lynt-
smitka/6d915fd1deff917ac2bf2c45ceb39c82
http://docs.ansible.com/ansible/latest/common_return_values.html
6. 12. 2017 15

16. https://lynt.cz
Add SSH key
---
- hosts: new
tasks:
- name: Add SSH key
authorized_key:
user: root
state: present
key: "{{lookup('file', '~/.ssh/id_rsa.pub') }}"
• apl sshkey.yml --ask-pass -u user --become --become-method=su --
ask-become-pass --ssh-common-args='-o
StrictHostKeyChecking=no'
6. 12. 2017 16

17. https://lynt.cz
Lookups
• http://docs.ansible.com/ansible/latest/playbo
oks_lookups.html
• File
• Password
6. 12. 2017 17

18. https://lynt.cz
Inventory arguments
[new]
192.168.1.117
192.168.1.118
192.168.1.119
[new:vars]
ansible_user=user
ansible_become=yes
ansible_become_method=su
ansible_ssh_common_args='-o StrictHostKeyChecking=no'
6. 12. 2017 18

19. https://lynt.cz
APT
---
- hosts: all
tasks:
- name: install package
apt:name=mc state=present
update_cache=yes
6. 12. 2017 19

20. https://lynt.cz
APT multiple packages
---
- hosts: all
tasks:
- name: install packages
apt:name=mc,curl
state=present update_cache=yes
6. 12. 2017 20

21. https://lynt.cz
APT multiple packages – yaml syntax
---
- hosts: all
tasks:
- name: install packages
apt:
name: mc,curl
state: present
update_cache: yes
6. 12. 2017 21

22. https://lynt.cz
APT multiple packages - loop
---
- hosts: all
tasks:
- name: install packages
apt: name={{item}} state=present update_cache=yes
with_items:
- mc
- curl
- vim
- git
http://docs.ansible.com/ansible/latest/playbooks_loops.html
6. 12. 2017 22

23. https://lynt.cz
Files operations
• http://docs.ansible.com/ansible/latest/list_of_fil
es_modules.html
• Copy
• File
• LineInFile/BlockInFile
• Ini_file
• Replace
• Template
6. 12. 2017 23

24. https://lynt.cz
File module
• http://docs.ansible.com/ansible/latest/file_m
odule.html
• Set owner, mode
• Create dir (whole path), symlink, hardlink
• Touch file (state=touch)
• Remove file (state=absent)
6. 12. 2017 24

25. https://lynt.cz
Copy module
• http://docs.ansible.com/ansible/latest/copy_mo
dule.html
- copy:
src: nginx.conf
dest: /etc/nginx/nginx.conf
backup: yes
• content/src
• force
6. 12. 2017 25

26. https://lynt.cz
Copy content
- copy:
dest: /etc/hosts
backup: yes
content: |
127.0.0.1 localhost
192.168.1.115 ansible
6. 12. 2017 26

27. https://lynt.cz
LineInFile module
http://docs.ansible.com/ansible/latest/lineinfile_module.html
- lineinfile:
dest: /etc/hosts
line: '8.8.8.8 dns'
- lineinfile:
dest: /etc/nginx/nginx.conf
regexp: '^user '
line: 'user www-data;'
* last instance
6. 12. 2017 27

28. https://lynt.cz
Replace module
- replace:
dest: /etc/nginx/web.conf
regexp: old.domain'
replace: 'new.domain'
backup: yes
* All instances
6. 12. 2017 28

29. https://lynt.cz
BlockInFile module
http://docs.ansible.com/ansible/latest/blockinfile_
module.html
- blockinfile:
dest: /etc/ssh/sshd_config
block: |
…
# BEGIN ANSIBLE MANAGED BLOCK
…
# END ANSIBLE MANAGED BLOCK
6. 12. 2017 29

30. https://lynt.cz
IniFile module
http://docs.ansible.com/ansible/latest/ini_file_
module.html
- ini_file:
dest="/etc/php/7.0/fpm/php.ini"
section="Date"
option="date.timezone"
value="Europe/Prague"
6. 12. 2017 30

31. https://lynt.cz
Template module
http://docs.ansible.com/ansible/latest/template
_module.html
- template:
src=nginx.conf.j2
dest=/etc/nginx/nginx.conf
6. 12. 2017 31

32. https://lynt.cz
Jinja Templates
• http://jinja.pocoo.org/docs/2.10/templates
• Variables
{{ ansible_distribution }}
• Filters
{{ ansible_distribution|capitalize }}
• Loops
{% for item in groups.app %}
server {{ item }};
{% endfor %}
• Conditionals + tests
6. 12. 2017 32

33. https://lynt.cz
Jinja Conditionals + tests
{% if variable is defined %}
value of variable: {{ variable }}
{% else %}
variable is not defined
{% endif %}
6. 12. 2017 33

34. https://lynt.cz
Handlers
tasks:
- name: config nginx
template: src=nginx.conf.j2
dest=/etc/nginx/nginx.conf
notify:
- restart nginx
handlers:
- name: restart nginx
service: name=nginx state=restarted
6. 12. 2017 34

35. https://lynt.cz
Roles
• ansible-galaxy init <role-name>
test-role/
├── defaults
│ └── main.yml
├── files
│ └── file.txt
├── handlers
│ └── main.yml
├── meta
│ └── main.yml
├── tasks
│ └── main.yml
├── templates
│ └── template.j2
├── tests
│ ├── inventory
│ └── test.yml
└── vars
└── main.yml
6. 12. 2017 35

36. https://lynt.cz
Variables
• http://docs.ansible.com/ansible/latest/playbo
oks_variables.html#variable-precedence-
where-should-i-put-a-variable
• apl x.yml --extra-vars "…"
• roles:
- { role: mysql, db_name: demo }
• group_vars folder
• host_vars folder
6. 12. 2017 36

37. https://lynt.cz
Facts
• ansible –m setup all
• /etc/ansible/facts.d/*.fact
- name: Upload simple fact
copy:
content: "1"
dest: /etc/ansible/facts.d/test.fact
6. 12. 2017 37

38. https://lynt.cz
Facts
• ansible –m setup all
…
"ansible_local": {
"one": 1
}
…
6. 12. 2017 38

39. https://lynt.cz
Dynamic facts
• chmod +x ;-)
• Json output
#!/bin/bash
curl --silent --connect-timeout 1 ifconfig.co/json
6. 12. 2017 39

40. https://lynt.cz
Vault
• ansible-vault create vault.file
• ansible-vault edit vault.file
• apl stack-init.yml --ask-vault-pass
[default]
vault_password:file = ~/.vaultpass
6. 12. 2017 40

41. https://lynt.cz
Vault example
---
- pass: 'secret'
cat vault.file
$ANSIBLE_VAULT;1.1;AES256
326564303339633037383163333836653564633339313130306632323635313336353
36637373162…
---
- hosts: mgmt
vars_files:
- 'vault.file'
tasks:
- debug:
msg: '{{pass}}'
6. 12. 2017 41

42. https://lynt.cz
Galaxy
• https://galaxy.ansible.com/
• ansible-galaxy install username.role_name
• ansible-galaxy install -p roles -r requirements.yml
requirements.yml:
# from galaxy
- src: yatesr.timezone
# from GitHub
- src: https://github.com/bennojoy/nginx
6. 12. 2017 42

43. https://lynt.cz
Performance
• gather_facts: false
- hosts: all
become: true
gather_facts: false
tasks:
- name: update apt cache
apt: update_cache=yes cache_valid_time=86400
[ssh_connection]
Pipelining = True
6. 12. 2017 43

44. https://lynt.cz
Ansible-pull
• https://github.com/ansible/ansible-
examples/blob/master/language_features/ans
ible_pull.yml
6. 12. 2017 44

45. https://lynt.cz
Discussion
• Strategies
• Troubleshooting
6. 12. 2017 45