The explosion of multi-platform mobile application development has created an extensive testing burden for quality organizations – and a huge opportunity for test automation ROI. Factoring in the idiosyncrasies of individual device models and carriers magnifies both the challenge and opportunity. Attend this session to learn how to extend test automation to the mobile platform and realize significant ROI. You will come away with best practices for implementing test automation effectively and efficiently across multiple mobile platforms and devices utilizing various types of mobile test automation tools. Specifically, tool independent techniques for creating a single test suite capable of executing across multiple platforms and devices will be discussed.

1. © Utopia Solutions
Extending Test Automation To Mobile Applications
Lee Barnes, CTO
Utopia Solutions

2. © Utopia Solutions
Agenda
•What is Successful MTA?
•How Do We Achieve It?
•Getting Started
•Visual Based Tools
•Object Based Tools
•Evaluating Your Apps
•Tool Evaluation
•Summary & Questions
Mobile Test Automation (MTA)

3. © Utopia Solutions
SECTION 1
What Is Successful Mobile Test Automation?

4. © Utopia Solutions4
WIFI / Carrier Networks
Mobile Test Automation – Goal
Automation
Framework
Test Cases
Backend System
Single set of test
cases executing
across a diverse set
of real devices

5. © Utopia Solutions5
Test Automation Success Criteria
• Issue detection and recovery
• Accurate verification
• Unattended execution
Reliable
• Minimum sensitivity to application and test
case changes
• Test case definition separate from
automation code
Maintainable
• Test coverage expanded efficiently
• Automated test cases created by non-
automation resources
Scalable

6. © Utopia Solutions
SECTION 2
How Do We Achieve These Goals?

7. © Utopia Solutions7
Test Automation Approach
Mobile Test Automation Framework
Automated Test Cases Test Results
Test Definition
Test Execution
Non-technical / Test Analysts & Subject Matter Experts
Technical / Automation Engineers

8. © Utopia Solutions
Mobile Test Automation Framework
Business / Application Testing Components
Login
GoTo Stocks
GoTo News
Add Stock
Edit Stock
Remove
Stock
Logout
Get Position
Verify
Position
…
…
Object Interaction Methods
Object Exists
Set Value
Get Value
Select Item
Touch
Touch and
Hold
Get Image
Get Text
Send Key
Test Case
Login GoTo Stocks Add Stock
Verify
Position
Logout
MobileAutomationFramework

9. © Utopia Solutions9
Test Automation Challenges
• Defining appropriate scope
• Integrating with SDLC
• Addressing technical issues
• Reliability, maintainability, scalability
General
• Platform diversity
• Device diversity
• Tool diversity
• Tool immaturity
• Lack of skilled resources
• Rate of technology change
• Rapid development cycles
Mobile
Specific

10. © Utopia Solutions
SECTION 3
Let’s Get Started

11. © Utopia Solutions11
Mobile Test Automation Tool Categories
Native Platform
Frameworks
Multi-Platform Tools
Visual Based Object Based
• Provided by mobile platform
vendor as part of SDK
• Typically interact with the
application at the UI object
level
• Interacts with devices
using visual methods (text
and image recognition)
• Interacts directly with
application UI objects using
native methods and
properties
• Object based interaction
with app UI controls
• Greatest degree of support
for native UI objects
• Control of device settings
• Supports multiple
platforms
• Ability to execute tests
across multiple devices
• Interact with entire device
• Supports multiple platforms
• Ability to execute tests
across multiple platforms
• Interacts with apps at the
object level for greater
reliability
• Limited to a single platform
• Requires code level access
to the app
• May be limited to simulator
(e.g. Blackberry)
• Generally less reliable
• No access to object
properties
• May require jail-breaking /
rooting
• Interaction is limited to app
under test
• Requires app to be
instrumented
• Android UI Automator
• iOS UI Automation
• SeeTest (Experitest)
• eggPlant (TestPlant)
• Perfecto Mobile
• Zap-Fix (Zap Tech.)
• Trust (Mobile Labs)
• TouchTest (SOASTA)
• M-eux Test (Jamo Solutions)
DescriptionProsConsExample

12. © Utopia Solutions12
Start With a Plan
Scope
OS/Device
Selection
Test Case
Selection
Non-Testing
Tasks
Technical
Challenges
Platform/Device
Diversity
Object
Identification
Process
Integration
Test Suite
Maintenance
Continuous
Integration

13. © Utopia Solutions13
Automation Scope Selection
• Frequently tested
• Predictable results
• Easy to automate
• Manually tedious
“Traditional” rules still apply
• Tests executed across all target devices
• Avoid apps and tests that face many mobile
automation challenges (to be discussed)
Mobile specific

14. © Utopia Solutions
SECTION 4
Visual-based Tool Approach

15. © Utopia Solutions15
Visual Based Identification
Label
Label + Boundary
Image Recognition
Label Offset
Multi-Label Offset

16. © Utopia Solutions16
Cross Platform Issues – Inconsistent UI/UX
Button
Image/Position
Screen
Titles
Button
Labels

17. © Utopia Solutions17
Visual Based Techniques – Complex Interaction
iOS On/Off Switch
Function iosSwitchGetState (Label)
Define OCR parameter ranges
Search for Label on device screen
If Label not found
status = switch not found error
Else
Loop through OCR parameter range
Search for switch state(ON/OFF)
If state found
status = switch state
Else
status = state not found error
End If
End Loop
End If
Return status
End Function
Function iosSwitchSetState (Label, State)
CurrentState = iosSwitchGetState (Label)
If CurrentState = error
status = error
Else
If CurrentState = desired state
status = success
Else
If desired state = ON
Touch right side of switch
Else
Touch left side of switch
End if
End if
status = success
End if
Return status
End Function
Hidden
Complexities

18. © Utopia Solutions18
Visual Interaction – Reusability?
Maybe…
• Label / object offset
• On / Off reversed
not
• Label offset right
• No On / Off label

19. © Utopia Solutions19
Object Identification - Defining Master Object Properties
Environment Object Type Attribute Value
iPhone Screen Method Unique Text
iPhone Screen Label Screen name
iPhone Screen Location 20,25:40,200
iPhone Edit Label Object name
iPhone Edit Offset 0,-20
…
• Defines general rules for identifying objects by platform
• Avoids adding all application objects to Object Repository
• One master file per application (typically)
Master Property File

20. © Utopia Solutions20
Object Identification - Define Object Map
ObjectScreenPlatform
iOS
Login
User
Name
Password
Add
Stock
Symbol
• Contains any screens and objects that don’t meet criteria
defined in Master Property File
• Objects organized hierarchically by platform and screen
UI Object Map

21. © Utopia Solutions
SECTION 5
Object-based Tool Approach

22. © Utopia Solutions© Utopia Solutions
Object Based Interaction
Tool Libraries Included
in Project Build
Object Properties
Visible to Tool
Class: UIButton
Label: Save
Class: UITextField
Label: Name

23. © Utopia Solutions23
Object Based Interaction – Cross Platform Differences
Simple Interaction – “Tap” the Save/Done Button
Class: avwButton
Method: Push
Implementation:
avwWindow(“MyStock”).avwButton(“Save”).Push
Class: iosButton
Method: Tap
Implementation:
iosView(“EditStock”).iosButton(“Done”).Tap

24. © Utopia Solutions24
Object Identification – Handling Cross Platform
Properties
Class: Button
Name: Save
Create Mapping of Universal Objects to Platform
Specific Objects
Properties
Class: avwButton
Label: Save
Method Name
Touch: Push
Object Path
avwWindow(“MyStock”)
Properties
Class: iosButton
Label: Done
Method Name
Touch: Tap
Object Path
iosView(“EditStock”)
Step 1

25. © Utopia Solutions25
Function TouchButton (Name)
Get execution platform
Retrieve platform specific object info
from Object Map
Construct statement to perform operation
Execute operation
Return operation status
End Function
Object Identification – Handling Cross Platform
Create Universal Method for All PlatformsStep 2
Properties
Class: Button
Name: Save
Properties
Class: iosButton
Label: Done
Method Name
Touch: Tap
Object Path
iosView(“EditStock”)
iosView(“EditStock”).iosButton(“Done”).Tap

26. © Utopia Solutions
SECTION 6
Evaluating Your Apps for Potential Challenges

27. © Utopia Solutions27
Analyze Business Processes for Differences
Differences
• Menu always visible
on iOS
• Different edit buttons
• Different column
headers

28. © Utopia Solutions28
Analyze Business Processes for Differences
Differences
• Different screen titles
• Different “add”
buttons
• Different “save”
buttons
• Different background
color on list

29. © Utopia Solutions29
Analyze Business Processes for Differences
Differences
• Cancel button
embedded in search
field on iOS
• Different “expand”
icons

30. © Utopia Solutions
SECTION 7
Handling Workflow Differences

31. © Utopia Solutions31
Handling Workflow Differences – Go To Stocks
Android
1. Tap “MENU”
2. Tap “Stocks”
3. Sync on “My
Stocks” Screen
iOS
1. Tap “My Stocks”
2. Sync on “My
Stocks” Screen
Implement at the Business Process
Component Level
Public Function GoToStocks
Platform = GetPlatform()
Select Case Platform
Case “iOS”
Tap (“MENU”)
Tap (“Stocks”)
status = WaitScreen (“My Stocks”)
Case “Android”
Tap (“My Stocks”)
status = WaitScreen (“My Stocks”)
Case Else
status = INVALID_PLATFORM
End Select
GoToStocks = status
End Function

32. © Utopia Solutions32
Mobile Test Automation Tool Evaluation
Top Ten Questions to Ask Tool Vendors
• Is your tool visual or object based?
• Does your tool require jailbreaking or rooting?
• How quickly do you support new mobile OS releases?
• How strong is the company (e.g. sales, customers, years, etc.)?
• How do I execute a single test across multiple platforms with
your tool?
• Can (or must) your tool integrate with other scripting engines?
• Do your support our mobile development platform?
• Can your tool interact with devices remotely?
• Does your tool integrate with our test management solution?
• Will you assist me with a proof of concept?

33. © Utopia Solutions33
Key Summary Points
• Set appropriate automation goals
• Start with a plan
• Scope
• Technical Challenges
• Process Integration
• Understand how visual/object based tools work
• Analyze your app for platform differences
• Address differences at appropriate level in your
framework
• Evaluate tools carefully
• Start small and expand over time

34. © Utopia Solutions34
Questions…
…. and Answers!
Direct future questions to:
Lee Barnes
Founder and CTO
Utopia Solutions, Inc.
Email: lee.barnes@utopiasolutions.com
Twitter: twitter.com/USI_LeeBarnes
LinkedIn: linkedin.com/in/leebarnes
Blog: www.RaiseYourMQ.com