ONE COMMIT
ONE RELEASE
CONTINUOUSLY DELIVERING A PROJECT

Javier López @loalf
Senior Platform Engineer @Rightster!
formerly Software Architect @TimeOut
Certified Symfony Developer
Zend Certified PHP
Co-organizer @desymfony

DISCLAIMER

AGENDA
CONTINUOUS DELIVERY IN A NUTSHELL
BUILDING THE PIPELINE
BEFORE AND AFTER
1
2
3

CONTINUOUS DELIVERY …
… IN A NUTSHELL
1

COMMIT BUILD TEST DEPLOY
When and how a change in your code is
going to trigger the pipeline

COMMIT BUILD TEST DEPLOY
Checks the integerity of your code.
Produces an artifact.

COMMIT BUILD TEST DEPLOY
Checks the integerity of your project.
!
After this step you should be 100% confident that
you could deploy your artifact.

COMMIT BUILD TEST DEPLOY
Does what it says in the tin. Triggering this
step should be “a click away”.
Manual

COMMIT BUILD TEST DEPLOY
Does what it says in the tin. Triggering this
step should be “a click away”.
Manual

THE BOOK

THE PIPELINE
BUILDING
2

TOOLS!
OF THE TRADE

BUILDING!
THE PIPELINE

COMMIT BUILD TEST DEPLOY
master
6ebb017

COMMIT BUILD TEST DEPLOY
master
featureA
6ebb017

COMMIT BUILD TEST DEPLOY
master
featureA
6ebb017
PULL REQUEST

COMMIT BUILD TEST DEPLOY
master
featureA
6ebb017 9046c48

COMMIT BUILD TEST DEPLOY
master
featureA
6ebb017 9046c48
BUILD TEST DEPLOY

COMMIT BUILD TEST DEPLOY

1. Checkout out master branch
COMMIT BUILD TEST DEPLOY

1. Checkout out master branch
2. Generate parameters.ini for each environment
COMMIT BUILD TEST DEPLOY

1. Checkout out master branch
2. Generate parameters.ini for each environment
3. Fetch PHP dependencies
COMMIT BUILD TEST DEPLOY

1. Checkout out master branch
2. Generate parameters.ini for each environment
3. Fetch PHP dependencies
COMMIT BUILD TEST DEPLOY
—optimizer-­‐autoload —prefer-­‐dist

1. Checkout out master branch
2. Generate parameters.ini for each environment
3. Fetch PHP dependencies
COMMIT BUILD TEST DEPLOY

1. Checkout out master branch
2. Generate parameters.ini for each environment
3. Fetch PHP dependencies
4. Run PHPUnit tests
COMMIT BUILD TEST DEPLOY

1. Checkout out master branch
2. Generate parameters.ini for each environment
3. Fetch PHP dependencies
4. Run PHPUnit tests
5. Fetch JS dependencies
COMMIT BUILD TEST DEPLOY

1. Checkout out master branch
2. Generate parameters.ini for each environment
3. Fetch PHP dependencies
4. Run PHPUnit tests
5. Fetch JS dependencies
6. Generate assets (JS, CSS)
COMMIT BUILD TEST DEPLOY

1. Checkout out master branch
2. Generate parameters.ini for each environment
3. Fetch PHP dependencies
4. Run PHPUnit tests
5. Fetch JS dependencies
6. Generate assets (JS, CSS)
7. Generate artifact (zip file)
COMMIT BUILD TEST DEPLOY

1. Checkout out master branch
2. Generate parameters.ini for each environment
3. Fetch PHP dependencies
4. Run PHPUnit tests
5. Fetch JS dependencies
6. Generate assets (JS, CSS)
7. Generate artifact (zip file)
COMMIT BUILD TEST DEPLOY
discard unneeded files

1. Checkout out master branch
2. Generate parameters.ini for each environment
3. Fetch PHP dependencies
4. Run PHPUnit tests
5. Fetch JS dependencies
6. Generate assets (JS, CSS)
7. Generate artifact (zip file)
COMMIT BUILD TEST DEPLOY

1. Checkout out master branch
2. Generate parameters.ini for each environment
3. Fetch PHP dependencies
4. Run PHPUnit tests
5. Fetch JS dependencies
6. Generate assets (JS, CSS)
7. Generate artifact (zip file)
8. Upload artifact to S3
COMMIT BUILD TEST DEPLOY

1. Checkout out master branch
2. Generate parameters.ini for each environment
3. Fetch PHP dependencies
4. Run PHPUnit tests
5. Fetch JS dependencies
6. Generate assets (JS, CSS)
7. Generate artifact (zip file)
8. Upload artifact to S3
COMMIT BUILD TEST DEPLOYbuild.xml

<?xml version="1.0" encoding="UTF-­‐8" ?>
<project name="Time Out Miyagi" description="Time Out Website V4" default="build"
!
<!-­‐-­‐ Install PHP dependencies (composer) -­‐-­‐>
<target name="build:php-­‐dependencies" depends="build:params">
<echo msg="Installing PHP dependencies (composer install)" />
<exec command="/opt/composer/composer.phar install -­‐-­‐optimize-­‐autoloader -­‐-­‐prefer-­‐dist"
logoutput="true"
checkreturn=“true"
/>
</target>
!
<!-­‐-­‐ Test JS & Generate JS/CSS assets (grunt) -­‐-­‐>
<target name="build:frontend-­‐dependencies">
<echo msg="Test JS and Generate JS/CSS assets (grunt)" />
<exec command="npm install"
logoutput="true" checkreturn="true" dir="./web-­‐src" />
<exec command="xvfb-­‐run grunt -­‐-­‐env=dist"
logoutput="true" checkreturn="true" dir="./web-­‐src" />
</target>
</project>
build.xml

COMMIT BUILD TEST DEPLOY
Split into two jobs in Jenkins
1. Test deployment script
2. Run automated tests

COMMIT BUILD TEST DEPLOY
TEST DEPLOYMENT SCRIPT

COMMIT BUILD TEST DEPLOY
TEST DEPLOYMENT SCRIPT
1. SSH to QA server

COMMIT BUILD TEST DEPLOY
TEST DEPLOYMENT SCRIPT
1. SSH to QA server
2. Fetch artifact from S3

COMMIT BUILD TEST DEPLOY
TEST DEPLOYMENT SCRIPT
1. SSH to QA server
2. Fetch artifact from S3
3. Unzip artifact

COMMIT BUILD TEST DEPLOY
TEST DEPLOYMENT SCRIPT
1. SSH to QA server
2. Fetch artifact from S3
3. Unzip artifact
4. Set right permissions

COMMIT BUILD TEST DEPLOY
TEST DEPLOYMENT SCRIPT
1. SSH to QA server
2. Fetch artifact from S3
3. Unzip artifact
4. Set right permissions
5. Update virtual host

COMMIT BUILD TEST DEPLOY
TEST DEPLOYMENT SCRIPT
1. SSH to QA server
2. Fetch artifact from S3
3. Unzip artifact
4. Set right permissions
5. Update virtual host
6. Reload PHP-FPM and nginx

-­‐-­‐-­‐
-­‐ hosts: qa:beta:prod
sudo: true
vars:
amazon_bucket : "releases-­‐miyagi"
base_dir : "/var/www/v4.timeout.com"
project_dir : "{{ base_dir }}/{{ git_hash }}"
web_dir : "{{ project_dir }}/web"
tasks:
-­‐ name: Fetch artefact from S3
s3: bucket={{ amazon_bucket }} aws_access_key={{ amazon_key }}
aws_secret_key={{ amazon_secret }} object={{ git_hash }}.gzip dest="{{ project_dir
}}/{{artefact_zip}}" mode="get"
!
-­‐ name: Uncompress zip file
command: chdir="{{ project_dir }}" tar -­‐xf {{ artefact_zip }}
!
-­‐ name: Apply right permissions to project root
file: dest="{{ project_dir }}" state=directory mode=0755 group=nginx
owner=nginx recurse=true
-­‐ name: Restart PHP-­‐FPM
service: name=php-­‐fpm state=reloaded
deploy.yml

COMMIT BUILD TEST DEPLOY
RUN AUTOMATED TESTS

COMMIT BUILD TEST DEPLOY
RUN AUTOMATED TESTS
1. Checkout 9046c48

COMMIT BUILD TEST DEPLOY
RUN AUTOMATED TESTS
1. Checkout 9046c48
2. Fetch PHP dependencies

COMMIT BUILD TEST DEPLOY
RUN AUTOMATED TESTS
1. Checkout 9046c48
2. Fetch PHP dependencies
3. Run behat against QA box

COMMIT BUILD TEST DEPLOY
Split into two jobs in Jenkins
1. Run deployment script (production)
2. Run automated tests agains production

BUILD DEPLOY
AUTOMATED!
TESTS
6ebb017
6ebb017
QA
6ebb017
QA
DEPLOY
AUTOMATED!
TESTS
6ebb017
PROD
6ebb017
PROD
6ebb017
6ebb017.zip

BUILD DEPLOY
AUTOMATED!
TESTS
6ebb017
6ebb017
QA
6ebb017
QA
DEPLOY
AUTOMATED!
TESTS
6ebb017
STAGING
6ebb017
STAGING
DEPLOY
AUTOMATED!
TESTS
6ebb017
PROD
6ebb017
PROD
6ebb017
6ebb017.zip
SAFETY NET

PARAMETERS.INI!
DILEMMA

1. One parameters.ini per environment
2. Inject environment specific parameters on virtual host
http://symfony.com/doc/current/cookbook/configuration/external_parameters.html

TIME

BUILD DEPLOY
AUTOMATED!
TESTS
~3 mins ~30 secs ~5 mins

VISIBILITY
EVERYBODY IN THE TEAM SHOULD BE
AWARE OF THE STATUS OF THE PIPELINE

BUILD DEPLOY
AUTOMATED!
TESTS
9046c48
9046c48
QA
9046c48
QA
9046c48
BUILD DEPLOY
AUTOMATED!
TESTS
6ebb017
6ebb017
QA
6ebb017
QA
6ebb017
BUILD DEPLOY
AUTOMATED!
TESTS
b0b325
b0b325
QA
b0b325
QA
b0b325
BUILD DEPLOY
AUTOMATED!
TESTS
99e6d6
99e6d6
QA
99e6d6
QA
99e6d6

TRACEABILITY
EVERYBODY SHOULD KNOW WHAT VERSION
IS DEPLOYED IN WHICH ENVIRONMENT

curl -­‐I http://www.timeout.com/las-­‐vegas
!
HTTP/1.1 200 OK
Server: nginx/1.4.7
Vary: Accept-­‐Encoding
Cache-­‐Control: no-­‐cache
Content-­‐Type: text/html; charset=UTF-­‐8
Date: Fri, 19 Sep 2014 06:07:29 GMT
Transfer-­‐Encoding: chunked
Access-­‐Control-­‐Allow-­‐Origin: http://media.timeout.com
Connection: Keep-­‐Alive
X-­‐TIMEOUT-­‐V: d645127afb423e543d90ab5a7b8eae94f248b137
X-­‐Powered-­‐By: PHP/5.5.14
VERSION NUMBER

https://github.com/symfony/symfony/commits/a469c56

ROLLING!
BACK

YOU’LL NEED!
ALLIES

YOUR ALLIES

DEV TEAM
YOUR ALLIES

DEV TEAM
QA TEAM
YOUR ALLIES

DEV TEAM
QA TEAM
DEVOPS
YOUR ALLIES

DEV TEAM
QA TEAM
DEVOPS
PRODUCT
YOUR ALLIES

DEV TEAM
QA TEAM
DEVOPS
PRODUCT
THE BRASS
YOUR ALLIES

BEFORE AFTER&
3

ONE QA BOX PER FEATURE
SAME QA BOX FOR EVERYONE
BEFORE
AFTER

~ !WEEKS FROM DEVELOPMENT TO RELEASE
~ !DAYS FROM DEVELOPMENT TO RELEASE
BEFORE
AFTER

5 PEOPLE TO RELEASE TO PRODUCTION
1 PERSON TO RELEASE TO PRODUCTION
BEFORE
AFTER

~30 MINUTES TO RUN DEPLOYMENT SCRIPT
~30 SECONDS TO RUN DEPLOYMENT SCRIPT
BEFORE
AFTER

RELEASING WAS AN EVENT
RELEASING WAS A NO EVENT
BEFORE
AFTER

@loalf