Learn how to best integrate automated testing into your existing workflows and get more return with SmartBear's own Jeff Martin.

1. Use Automation to Assist
—Not Replace—
Manual Testing

2. 22
Ideal Automation Return
E
f
f
o
r
t
Return
The Shark Fin
Manual Automated
Time

3. 33
Reality of Most Automation
E
f
f
o
r
t
Return?
Time
Mountains of No Return
Manual Automated

4. 44
Why the Gap?
1. Failure to align different goals across the
team
2. Automated testing viewed as a replacement
for existing processes
3. Automated testing approached like
development
4. Not enough focus on high-return automation

5. 55
Goals - The Automation Dream
• Less Time
• No faulty tests / false
defects
• Less people
• More like manufacturing
The “Tester”The “Boss”
• Less repetition
• No faulty tests / false
defects
• More time to Test
• More like craftsmanship

6. 66
Goals - The Agreement
1. Less Time & Repetition
2. No faulty tests / false defects
Focus on these two goals

7. 77
Replacing Existing Processes
What Testers and the Business roles are familiar with:
Workflows
Business Processes
Test Cases
User Stories
So this is what they automate

8. 88
Example Test 1
Step
1
Step
2
Step
3
Step
4
Step
5
Step
1
Step
2
Step
3
Step
4
Step
5

9. 99
Misleading Early Results
The obvious workflows get automated first
Less Time & Repetition
No (ok, less) faulty tests / false defects
Success breeds repetition
What gets overlooked is efficiency
Break down workflows into component tasks

10. 1010
Task Efficiency over 5 years
Credit: XKCD https://xkcd.com/1205/

11. 1111
Automate like a Dev?
Ok, so all Tasks in a process are not created equal
One popular alternative:
Think like a developer and break processes
down
2 Developer approaches with the same problem
when applied to testing
Object Oriented Functional
Stable operations Stable “things”
New “things” New operations

12. 1212
Why Dev-focused approaches fail
Developers are used to building “the whole thing”
They can’t choose not to code something!
Development and Tests are based on the same
Requirements and therefore do not capture errors in design
Additionally, Developers can only interact and check for
expected results
Interact Check
Click Values
Enter text Presence
Drag Layouts / Coordinates

13. 1313
An aside: Checking vs. Testing
Automation when done right is more valuable than
“Checking”
Many non-checking functions are better done by
computers
Checks are based on design, Tests are based on
the much messier Usage.
Fully Automated Regression “Testing” is a
screening tool that only Checks

14. 1414
Where are we with the Gap?
1. Failure to align different goals across the team
Find the areas of agreement, focus on those
2. Automated testing viewed as a replacement for
existing processes
Break workflows into Tasks
3. Automated testing approached like
development
Recognize that development and testing, even automated
testing, are different activities
4. Not enough focus on high-return automation

15. 1515
Task Automation
1. Testing is hard work, automation should
free up time for that hard work
2. Processes are made of tasks
3. Tasks do not always equal tests
4. Tasks have different returns on
automation investment

16. 1616
Good Task Automation
Candidates1. Environment Setup / Teardown
2. Data Entry
a) Form Filling
b) Varying data inputs in a repetitive process
3. Exposing backend data (APIs, DB tables, etc….)
4. Repetitive / boring tasks that are prone to inattention errors
5. Tasks with high reuse value across many workflows
6. Tests with timing or screen responsiveness as a criteria for
success
7. Many non-functional test types, such as Performance testing
8. Smoke Test Suites – key user workflows as a screening tool
9. Result Capture

17. 1717
Good Task Automation
Candidates
Ask what your Team hates
Doing!

18. 1818
Example Test 1
Step
1
Step
2
Step
3
Step
4
Step
5
Automation
1
Automation
2
Manual 1
Manual 2

19. 1919
Where are we with the Gap?
1. Failure to align different goals across the team
Find the areas of agreement, focus on those
2. Automated testing viewed as a replacement for
existing processes
Break workflows into Tasks
3. Automated testing approached like
development
Recognize that development and testing, even automated
testing, are different activities
4. Not enough focus on high-return automation
Deconstruct Tests in to Tasks
Rate your tasks on Return

20. 2020
The Hard Part – Combining
Execution
Combining Automation and Manual execution of Tasks is
difficult
Many tools do exist to help combine execution and results:
1. Automation tools that pause and wait for manual testing
either in a form or via a button
2. Test Management or Manual Testing tools that run
scripts on demand
Caution: Switching between execution methods has a cost
that should be factored into Task selection.

21. Questions?