Today’s test organizations often have sizable investments in test automation. Unfortunately, running and maintaining these test suites represents another sizable investment. All too often this hard work is abandoned and teams revert to a more costly, but familiar, manual approach. Jared Richardson says a more practical solution is to integrate test automation suites with continuous integration (CI). A CI system monitors your source code and compiles the system after every change. Once the build is complete, test suites are automatically run. This approach of ongoing test execution provides your developers rapid feedback and keeps your tests in constant use. It also frees up your testers for more involved exploratory testing. Jared shows how to set up an open source continuous integration tool and explains the best way to introduce this technique to your developers and testers. The concepts are simple when presented properly and provide solid benefits to all areas of an organization.

1. T17
Continuous Testing
5/8/2014 1:30:00 PM
Continuous Test Automation
Presented by:
Jared Richardson
Agile Artisans
Brought to you by:
340 Corporate Way, Suite 300, Orange Park, FL 32073
888-268-8770 ∙ 904-278-0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com

2. Jared Richardson
Agile Artisans
Principal consultant and a member of the core team at Agile Artisans, Jared Richardson works
with other software craftsmen to build excellent custom software. He sold his first software
program in 1991 and has been immersed in software ever since. Jared has authored and
coauthored a number of books, including the best selling Ship It! A Practical Guide to
Successful Software Projects and Career 2.0: Take Control of Your Life. He is a frequent
speaker at software conferences and a thought leader in agile. Jared lives with his wife and
children in North Carolina where they recently, quite by accident, became backyard chicken
farmers. He's on the web at AgileArtisans.com.

3. Continuous
Testing
J a r e d R i c h a r d s o n
A g i l e A r t i s a n s
Who are you?
Jared Richardson
CoAuthored Ship It! and Career 2.0
Screencast editor for PragProg.com
Started AgileRTP in 2007
2nd public signatory of the Agile Manifesto
Agile coach
AgileArtisans.com
What Is Agile?

4. AgileManifesto.org
Feedback Loops
Feedback Loops are
Fractal
flickr.com/photos/dancentury/4579906149
flickr.com/photos/bobowen/4501875907
flickr.com/photos/30691679@N07/3808395213
flickr.com/photos/kevino/1438502858

5. Pairing
Test First
Daily Meetings
Demos
???
Feedback Loops
Inside Feedback Loops Different Levels
Good at the bottom
Great at the top
Ignore the middle
Continuous
Feedback
Continuous Integration
Continuous Delivery
Continuous Testing
Continuous Integration
Watch Code
Build product
Run tests
Publish results

6. Continuous Testing
Watch CI
Deploy
product
Integration
tests
Publish results
Continuous Testing
Build in CI (compile & unit tests)
Deploy to dev (staging)
Run integration tests
What does this tell us?
If it works
Continuous deployment is controversial.
Most people ... think I’m advocating
low-quality code
or an undisciplined
cowboy-coding
development process.
ERIC RIES
radar.oreilly.com/2009/03/continuous-deployment-5-eas.html

7. Language is a Tool
Use it
What's this look like?
Java Tooling
Publish results
Example
Publish results
javac (Oracle & IBM)
message broker
etc
ws ejb deploy
message broker tools
sshexec (to scripts)
JUnit
HP's UFT
Selenium
Web page
RSS
Email

8. Continuous Testing
CI CT
Compile &
unit test
Deploy &
integration
test
Multiplatform Deploys
CI
CD Win7
CD Linux
CD AIX
Binaries
Nuke & Pave
Delete
Recompile
Deploy
Use
23
flickr.com/photos/x-ray_delta_one/4152356464
Data
Nuke & Pave
Delete schema
Drop tables
Recreate
Restock
Rails
Liquibase
24
flickr.com/photos/x-ray_delta_one/4152356464

9. Rails ActiveRecord
CREATE&TABLE&products&(
&&&id&int(11)&NOT&NULL&auto_increment,
&&&name&varchar(255),
&&&PRIMARY&KEY&&(id)
);
25
Rails ActiveRecord
change_table :card_deck do |t|
t.remove :description, :name
t.string :first_name
t.index :part_number
t.rename :list_name, :last_name
end
26
Rails ActiveRecord
rake db:migrate
rake db:migrate VERSION=0
rake db:reset
27
Operating System
Nuke & Pave
Virtualize
Recreate
28
flickr.com/photos/x-ray_delta_one/4152356464

10. Vagrant
vagrant init lnx http://lnx/lnx.box
vagrant up
vagrant ssh
vagrant reload --provision
29
Vagrant
Vagrant.configure("2") do |config|
config.vm.box = "lnx"
config.vm.provision :shell, :path => "bootstrap.sh"
end
30
Benefits
Reproducible environment
Solid results
Fast results
Repeatable results
Fast
feedback
leads to
fast fixes

11. Minimize Dark Times
Code Change
Validation
http://flickr.com/photos/bogenfreund/556656621/
When?
Day one
Easiest
Easy?
No
Difficult
Necessary
Code
Alternatives

12. flickr.com/photos/lightsinthedark/5810724330/
Work in a vacuum Ruthlessly
Automate!
Builds
Deploys
Tests
flickr.com/photos/pmillera4/9317924474
Steps
Automate builds
Continuous Integration
Add unit tests
Automate deployments
Continuous Delivery
Add integration tests
flickr.com/photos/untitlism/2547423465
Your move

13. Exercise
Two 3x5 cards
3 reasons for Continuous Testing
3 barriers to adoption
Pass cards to the next table
Brainstorm on solutions
flickr.com/photos/curtisperry/6119618990
Tips
Open standards...
Junit or Junit compatible
Avoid vendor lock in
Glue tools
Time matters....
Fast servers
Incremental change
Resources
JUnit JUnit.org
Jenkins Jenkins-ci.org
BDD Dannorth.net/introducing-bdd
Cucumber Cukes.info
Continuous Deployment en.wikipedia.org/wiki/Continuous_delivery
Devops Resources
VagrantUp.com
GetChef.com/chef
PuppetLabs.com
Ansible.com
VirtualBox.org

14. Data Resources
Guides.RubyOnRails.org/active_record_basics.html
Liquibase.org
FlywayDB.org