Your shrinking project deadlines are increasing the need for automated tests—but, simultaneously, reducing the time available for writing them. The system requirements are continually changing. The implementation is changing. You spend more and more time maintaining old tests, leaving less time to write new ones. The tests take longer and longer to run. And when they fail, the problem is as likely to be in the tests as in the system. What’s a developer to do? Dale Emery and George Dinwiddie share hard-won lessons learned from their decades of software development and test automation. Discover the factors that make automated tests maintainable, expressive, informative, fast, reliable, and repeatable. Practice achieving these qualities in hands-on exercises. Apply new techniques and your existing software development expertise in new ways. Take home powerful principles and practices to meet the unique challenges of test automation and to help your project deliver sooner with greater confidence.

1. ML
PM Half day Tutorial
11/11/2013 1:00 PM
"The Developer's Guide to Test
Automation"
Presented by:
Dale Emery, DHE
George Dinwiddie, iDIA Computing, LLC
Brought to you by:
340 Corporate Way, Suite 300, Orange Park, FL 32073
888 268 8770 904 278 0524 sqeinfo@sqe.com www.sqe.com

2. Dale Emery
DHE
Since 1980, Dale Emery has worked in both IT
organizations and software product development
companies as a developer, manager, process steward, trainer, and
consultant. He helps people apply the agile values of communication,
feedback, simplicity, courage, and respect to software development. Dale's
combination of deep technical expertise and extensive organizational
development experience makes him particularly effective in working with
software teams. In 2007 Dale received the Ward Cunningham Gentle Voice
of Reason Award, which the Agile Alliance created to recognize Dale’s
unique contribution to the agile community. Dale's personal mission is to
help people create joy, value, and meaning in their work. Learn more about
Dale at dhemery.com
George Dinwiddie
iDIA Computing, LLC
George Dinwiddie is an independent software development consultant
who helps organizations, large and small, increase the effectiveness of
their software development efforts. As a coach, George he provides
guidance over a broad range, at the organizational, process, team,
interpersonal, and technical levels. As a trainer, he offers experiential
education in technical practices and agile methods. George is currently
crusading to break down the barriers that hinder effective collaboration
between the business, the programmers, and the testers. George is a
frequent speaker at agile conferences and contributor to online
publications. Learn more about his work at his company
website idiacomputing.com and read his blog.

3. The Developer’s Guide
to
Test Automation
Dale Emery
@dhemery
http://dhemery.com
George Dinwiddie
@gdinwiddie
iDIA Computing, LLC
http://blog.gdinwiddie.com
http://idiacomputing.com
1
HELP!
These tests are too slow!
These tests are flaky!
There are too many changes!
What is this test testing?
What does this failure mean?
These tests don’t prevent bugs!
What tools should I use?
2

4. What Makes Tests
Too Slow?
Avoidable delays
- Avoidable use of slow interface
- Avoidable use of unnecessary technology
- Pessimistic fixed waits
Run-on tests
Large setup
The tests don’t pull their weight
3
“Avoidable” Delays
Test
Browser
Driver
Sauce
Labs
Browser
Web GUI
Web
Server
System
Under Test
3rd Party
Web Service
Legacy
System
Database
4

5. What Makes Tests
Flaky?
Depends on variables not under test control
- Database contents, clock time, calendar, ...
- Interference from other tests
Asynchrony
- Variable response times
- Race conditions, threading
Identifiers overly restrictive or permissive
Untested test helpers
5
What Causes
“Too Many” Changes?
Changes in requirements
Changes in system implementation
Changes in execution environment
Changes in test tools and libraries
Growth of test code
Test code that is hard to change
Test automation as a separate activity
6

6. What Makes Test Code
Hard To Change?
Difficult to understand
- Cryptic code
- Large blocks of code
Details in inappropriate places
Duplication
Coupling
Poorly organized
7
How To Make Test Code
Easy To Change
Write small, understandable blocks of code
Eliminate duplication
Name every important idea
Hide incidental details
- System implementation details
- Test implementation details
- Interface between test and system
Put code where you will look for it
8

7. What Is This Test Testing?
Essential details are hidden
Implicit data and conditions
Essence obscured by incidental details
-
Test does not describe its intentions clearly
Vague terminology
Missing abstractions
Key ideas left unnamed
Inaccurate description
-
Run-on tests
9
What Does This Failure
Mean?
Omits useful information
Swamped with distracting information
Misleading or irrelevant message
Delays between problem and discovery
Implicit or incidental dependencies
Failures from test environment and tools
10

8. What Do Our Tests
Not Do?
Test everything
Test the things we’re changing
Find all unintended effects of our changes
Let us know what’s not tested
Test the things we care most about
Test the things our customers care about
Test the things they’re testing
11
What Tests
Do We Need?
Tests for important system responsibilities
Tests for areas that change frequently
Tests for changes that might be error prone
12

9. What Can Tests
Do For Us?
Describe the system’s responsibilities
Demonstrate what the system does
Detect unintended changes in behavior
Indicate when a feature is done
Expose defects
Home in on the default
Illuminate which parts of the system are tested
and which are not
13
Criteria For Choosing Tools
Supports writing examples before implementation
Supports expressing examples “naturally”
Fits development tools and languages
Can keep examples under version control
Supports traceability
Strong community support
No license fees
No limit of concurrent users
14

10. Common Ways
To Express Examples
Gherkin: Given / When / Then
Tables
- Each row is a scenario
- Each row is a set of data
Keywords
Arbitrary text with no predefined structure
15
Some Tools That Use
Gherkin
• Cucumber (Ruby, Java, JVM languages)
• SpecFlow (.NET)
• Behat (Python)
16

11. Example Toolstack For
Testing Web Applications
Cucumber
Capybara
Page-object
Selenium WebDriver
Browser
17
Example Toolstacks For
Testing Mobile Applications
Cucumber
Appium
iOS / Android
Simulator
iOS / Android
Device
Cucumber
JUnit
Frank
Victor
iOS
Simulator
iOS
Device
iOS
Simulator
iOS
Device
18