Test automation efforts often result in failure or lack of long term commitment due to complexity and costs associated with maintaining test scripts. When testers find it difficult to construct, execute, or update scripts, expectations of management are not realized, resullting in project termination. We avoided test automation failure by utilizing an innovative framework which reduces maintenance burdens associated with test scripts while increases user productivity.

1. High Reuse, Low Maintenance:
A Practical Approach to Test Automation
Alan White
The Nielsen Company
Alan.White@Nielsen.com

2. Learning Objectives; You will…
• understand the concepts behind one of the more
successful test automation frameworks to date,
• find out which widely used product can be used to
quickly produce a visually cohesive, wizard-based
GUI tool for creating repeatable test cases, and
• learn how to quickly and intuitively generate and
store automated test cases with relative ease while
reducing errors – no programming needed!

3. Problems with Traditional Automation
•
•
•
•
•
•
•
Maintenance Intensive
Fragile in Nature
Programming Experience Needed
Static Data
Tightly Coupled to AUT
Minimal Reuse Capability
Unnoticeable Productivity Gains

4. Solution
•
•
•
•
•
•
Intuitive
Structured
Maintainable
Role Based
Dynamic Data
Reusable Components

5. Goals
• Loose Coupling
– Application Under Test
– Test Automation Tools
– Development Platform
• Self Documenting Test Automation Scripts
– Scripts look like manual scripts
– Detailed account of AUT interactions
• Abstract Complexity from User Community
– No programming knowledge required
– Focus on building Test Assets

6. Framework Overview
• GUI Maps
– Screens & Components
– Controls & Actions
• Test Assets
– Test Steps
– Test Cases
– Test Suites
• AUT Invocation Code (Engine)
– Main Loop
– Function Libraries (for AUT platform)

7. Role Based Framework
• Test Engineer (Programmer)
– GUI Maps
– AUT Invocation Code
• User Community (SQA, UAT, BA)
– Test Script Maintenance
– Test Execution and Results Analysis
• Test Asset Librarian
– Naming Conventions
– Other Standards

8. GUI Maps
• Translate the technical names of screens and
components to more meaningful names

9. Test Steps
•
•
•
•
Are the building blocks for Test Cases
Link components to screens and actions to controls
Interact with the AUT to perform specific functions
Should be broken down by their function:
– Navigation
– Changing Data
– Verifying Data
• Are Highly Reusable Test Assets!

10. Test Step Example
•
•
•
Screens and Components are derived from the GUI map
Controls and Actions are mapped to Screens and Components
Parameters store default values that may be overridden

11. Test Cases
• Are sequential lists of Test Steps that perform
specific test operations beginning and ending at the
AUT central point
• Allow for overriding existing values to parameters
declared in Test Steps
• Resemble manual test cases & narratives

12. Test Case Example

13. Test Suites
• Are the highest level of abstraction that specify
which test cases are to be executed in the order
they should be executed
• Different test suites are defined to operate on
different modules or components of an application

14. Test Suite Example

15. Hierarchy of Assets

16. AUT Invocation Code
• Can be developed using commercial test
automation tools such as Rational Robot and
Mercury QTP or open source tools like WATIR
• Uses Simple Decision Structures
• Portable/Maintainable/Reusable
• Versatile – Can interact with the AUT screen and
components in the following ways:
– Input or Select Values
– Verify Values and Properties
– Verify Existence

17. Main Loop
• Loops through each instruction (data row)
– Each instruction has its own control
• Branches each control for processing
• Reports status to the log

18. Function Libraries
• Contain the functions created for each control
• Branch actions in each control function
• Can be reused by other applications

19. Putting the pieces together
Test Steps/Cases
1
Main Loop
Control Library
3
2
AUT
4
Test Execution Log

20. Microsoft Access Works Well for Building
the Front End, Test Asset Creation Tool!
•
•
•
•
•
•
•
VBA and SQL
Tables/Queries
Forms/Reports
Data Pages
Macros
Code Modules
Security
Note: More Advanced Deployments might consider
developing in an enterprise SDK (e.g. VB, C#, Java).

21. Getting Started!

22. How to GUI Map

23. How to Create a Test Step

24. How to Create a Test Case

25. How to Create a Test Case

26. How to Create a Test Suite

28. Costs
•
•
•
•
Build the Test Asset Creation Tool
GUI Map the Applications Under Test
Build the AUT Invocation Code (Engine)
Train the User Community

29. Benefits
•
•
•
•
•
•
Reduced and Known Test Execution Time
Greater Test Coverage
Long Term Cost Savings
Can Run Scheduled and Unattended
Lower Maintenance Effort
Quicker Response to Application Changes

30. Deployments
• Application Under Test Platforms
– PowerBuilder
– Java
– HTML
• User Communities
– Development
– SQA
– UAT
• Types of Testing
–
–
–
–
Unit
Smoke
Regression
User Acceptance

31. Division of Effort
• Dedicated Test Engineer
–
–
–
–
Ideally 100% Effort Initially
Construction of Framework (Engine)
Construction of Test Asset Creation Tool
Build and Maintain the GUI Maps
• User Community
– Create and Maintain Test Scripts
– Analyze Test Execution Results
– Communicate AUT Changes

32. Questions
• Please feel free to ask your questions or make
comments.
• Thank You!

33. Further Reading
•
•
•
•
•
•
•
Nagle, C. “Test Automation Frameworks” White Paper
http://safsdev.sourceforge.net/FRAMESDataDrivenTestAutomationFramework
s.htm
Hayes, L. “Implementing a Test Automation Framework” Tutorial presented at
STAR EAST 2005 Conference, Orlando, FL, May 16, 2005.
Automated Testing Specialists, Methodologies for Automated Testing
http://www.sqa-test.com/method.html
Zambelich, K. Totally Data-Driven Automated Testing 1998
http://www.sqa-test.com/w_paper1.html
LogiGear Whitepaper Series, Achieving the Full Potential of Test Automation
October 25, 2004
Mosley, D. & Posey, B. Just Enough Software Test Automation New Jersey:
Prentice Hall PTR, 2002.
Buwalda, H. & Janssen D. Integrated Test Design and Automation Great
Britain: Addison-Wesley, 2002