(Click 2nd slide for video) Deploy PHP apps faster in 2017. This talk focuses on how PHP developers can use simple Ansible scripts to rapidly configure new dev and production servers from scratch, and deploy their apps. No more "snowflake servers"!
This is a general introduction to DevOps essentials and Ansible, with a few extras for PHP developers, including some best practice tips and overview of two major Ansible-based PHP projects, Drupal-VM and Trellis (modern WordPress setup).
7. What Ansible Does
Configuration Management
App Deployment
• Basic, "Atomic", zero downtime, ...
Orchestration
• Sequence operations on servers, APIs, etc.
• Harder with Configuration Management
11. Key Concepts
Playbook = series of tasks
• Targets one server or thousands
• Servers defined by Inventory
Task = "ensure X is done" action
Play = set of tasks in playbook
13. Running a Playbook (2)
Tasks will "skip" if state already OK
(Idempotence)
14. Writing a Playbook
Play – hosts to process, become == sudo
Tasks - descriptive name
- invoke module (apt) with parameters
Play
Task
15. The Secret Life of Tasks
Each task runs SSH commands that
• Upload a Module (e.g. apt)
• Run module with task's parameters
• Return JSON output
16. Inventory and Variables
Group your servers and assign key parameters ("variables")
[web]
10.0.1.51
10.0.1.52
[db]
10.0.1.61
[web:vars]
ansible_port=2222
$ ansible-playbook -i prod apache.yml --limit web
Run different Ansible code per group
Recommended:
• Inventory file per environment (or dynamic inventory)
• Put vars in group_vars/mygroup/vars.yml
17. Apache Playbook (1)
Vars = parameters for this playbook
Can be in separate include files
Or attach to hosts or host-groups in Inventory
- e.g. Listening IP address should be in inventory
18. Apache Playbook (2)
template task runs Jinja2 on local file and copies to
server
notify sends event to Handler
- Each Handler runs just once, at end of whole playbook
- Restart a service, notify Slack, ...
19. Apache Playbook (3)
service task uses systemctl to enable start on boot
- {{ apache_service }} instantiates var with Jinja2
Handler restarts apache at end if any task does a notify
22. Modules (2)
More modules (AWS alone has 87)
- Firewalls, routers, switches, ...
- AWS, Google, Digital Ocean, ...
- Docker, VMware, …
- Fallback to shell, upload script, …
Runs best on Linux/Unix including Mac
Windows as a target only
23. Roles
"Modularised playbooks"
- Split playbook into folder per type of content
- defaults folder for "parameter vars"
- vars folder for "role vars" – hard to override
- meta folder for role dependencies
Vars
Tasks
Handlers
Apache + PHP
playbook
Apache
role
PHP
role
24. Roles
Use Roles for everything!
Skinny playbooks + modular roles
Ideal playbook only calls roles, not tasks
Typical roles:
• mysql
• apache
• php, php-fpm
• deploy-app
"Wrapper roles" to invoke third party roles
25. Ansible Galaxy
Hub for 1000s of roles: galaxy.ansible.com
Discovery: Galaxy, GitHub, blogs, …
Assess quality carefully
Install the roles needed by project:
ansible-galaxy install –r requirements.yml
Pin the role to a version or Git commit
26. Testing Infra Code
Basic testing:
- Separate test playbook using Vagrant VM
- Travis CI popular for open source
- Smoke test at end of playbook:
Test frameworks:
- Test-Kitchen, ServerSpec, InSpec, testinfra
- Run whole series of tests - easier diagnosis
34. Advanced: Write a Module
Much more common to write a role
Required for major new features:
• New API
• New package tool
• New container format
Most modules written in Python
Any language works: PHP, C, Go, Perl, …
Writing a module using PHP
Go Faster without creating new problems...
"Faster" really means more iterations per day / week
Servers created much faster, without manual work
Apps deployed automatically, without mistakes or omissions
Deveopers on-boarded very fast, without manual installs
This means:
Configuration must be correct, even when moving faster
Avoid snowflake servers (every one is different in detailed config)
scale across dev, test, staging, production
Server config problem? Code a fix
Some people like the definition that CM is "Executable Documentation" and it's true that CM replaces server documentation. However, the more complex CM setups are definitely software not just documentation.
Competition:
Your memory!
Shell scripts
…
Capistrano style "atomic symlink" deployment involves having a single symbolic link called "current" pointing to the release that's live. Deployment means pushing the new files into a new release directory, then atomically switching the link.
See Ansistrano, which enables this Capistrano model using Ansible, enabling customisation for more complex models as required. There is also f500.project-deploy, which is less powerful e.g. no before/after hooks.
Rocketeer is a PHP tool that does the same thing.
Fabric is a Python tool that can do anything on servers, often used for simpler deployments – requires Python coding and is lower level than Ansistrano
Although Ansible can do both CM and App deployment, you normally want quite separate server configuration management and app deployment scripts (Playbooks).
- "CM is a process, deployment is an event"
– agentless - only requires SSH & Python on server
simple but powerful
sequential model for tasks
Declarative tasks
easy to learn compared to Puppet/Chef/Salt
- explicit ordering visible in code, unlike Puppet/Salt
easy orchestration
e.g. easy to remove servers from LB during upgrade, and rollback if X% fail:
Remove server from load balancer
Deploy app & reboot
Restore to load balancer
Upgrades are easy with the "Any Linux" method – just do "pip install --upgrade ansible"
For Windows, use a Linux VM on VirtualBox – Ubuntu is probably easiest option. Or Windows 10's "bash shell" option may work fine.
Setup task = get server info (OS, hardware) – built in, can disable with "gather_facts: no" at top of play
Parameters – single line or indented YAML
The single line format (e.g. update_cache=yes) is unique to Ansible, and not part of YAML
The indented format (e.g. "apt:") is YAML
YAML is equivalent to JSON, using indentation.
Use the "-" syntax to create a list – each new item is a list entry (equivalent to a PHP array with integer index values)
Use the "foo: bar" syntax to create a key-value pair within a dictionary (equivalent to a PHP array with string index values)
Originally known as "hosts file" – term still used in Ansible error messages
Default is /etc/ansible/hosts but that's not very useful in most cases where you have different projects, environments, etc.
Getting the right inventory structure, and locating variables in right place, is a key decision in Ansible projects. Generally, try to have variables in as few places as possible, e.g.: role defaults, role vars, host group vars, and use wrapper roles (mentioned later) to encapsulate vars you need for roles.
Some people prefer a "group + environment" naming scheme, or moving each inventory file to a separate directory – this enables vars that are specific to combination of
- This is a simplified extract from a real-world Ansible project, the Drupal-VM role "geerlingguy.apache", combining various elements into a single playbook.
Can also target network switches and routers that don't have Python, using modules that don't require Python on the target device.
If you use Windows, you would need a Linux VM to actually run Ansible, targetting Windows servers. Ansible's Windows support is still evolving and less complete than on Linux/Unix.
Roles are more reusable if they focus on a single logical service, or a related set of config – e.g. Apache, PHP and PHP-FPM are more re-usable if written as separate roles.
Files are searched for within the current role, avoiding complex relative paths in many cases
Look for steady development, several contributors, many commits, discussion on GitHub
Always pin the role – this avoids surprises when your project changes
"Vendoring" the role can help i.e. install into a "vendor/roles" folder in your project – be sure to adjust your ansible.cfg's role_path
Galaxy is not that easy to browse/search, and most of the action is on GitHub
I usually install directly from GitHub using Ansible Galaxy as a discovery and installation tool only.
Goal: Test-driven infrastructure
ServerSpec and InSpec provide English-like BDD testing using a DSL based on Ruby
Testinfra provides simple unit tests using Python
Just as with app testing, infra testing frameworks are generally more powerful than using Ansible to test itself – e.g. you can easily run just a few tests
RoleSpec specifically tests Ansible roles – the others focus more on the outputs of the roles
Try these as well:
ansible web -m setup -a 'filter=ansible_all_ip*
ansible devbox -m setup -a 'filter=ansible_mem*'
Fully open source
Requires node, bower and gulp and some Vagrant plugins
Should work with Ansible 2.0.2.0 as specified in the docs (run "pip install --upgrade ansible==2.0.2.0" as long as you're already using the "pip install ansible" method to install Ansible).
Some fixes needed for Ansible 2.2 (mailhog and logrotate roles, php-fpm pool from wordpress-setup role, and composer role)
Trellis clone gives you trellis tree (ansible), then Bedrock clone gives you WP site tree, and Sage clone populates WP theme inside that
Includes Nginx "micro caching" for 5 sec to Reddit-proof your site
- can't really track "Chef" searches as Google Trends doesn't offer a software classification of term.