“Mobile Test Coverage: It’s not just about the devices!" With the proliferation of mobile devices, there is a renewed discussion on Test Coverage as it relates to mobile functional testing. Many of our customers have taken a fresh look at their mobile strategy with a renewed focus on their test coverage strategy. What do they discover? That test coverage is not just about devices. Join us for an hour and we will walk you through the key areas that you need to focus on to keep your mobile strategy covered.

1. May 13, 2015
Three Tips to Increase your mobile test coverage
Dan Boutin – Senior Product Evangelist
dboutin@soasta.com
Mobile (404) 304-9529
@DanBoutinSOASTA

2. © 2014 SOASTA. All rights reserved. March 3, 2015 2
Agenda
o What is Test Coverage, anyway?
o Functional Testing Overview
o How did the game, and the focus, change
o So how do I determine the optimal test coverage for my app?
o Step outside the box with me!
o Let’s show you how you can do it!
o Questions & Final Trivia for #swag

3. © 2014 SOASTA. All rights reserved. March 3, 2015 3
So what is test coverage anyway?
o Answers:
• A. The degree in which tests exercise the software features
• B. Code Coverage
• C. Device Coverage
• D. How many tests are automated

4. © 2014 SOASTA. All rights reserved. March 3, 2015 4
Functional Testing Overview

5. What is Functional Testing?
Functional Testing is testing the functionality of an
app to meet a business requirement.
Ex: Sears testing their app that a user can log in their
account, search for shoes, select the size, go to the
checkout and purchase.

6. What is Test Coverage?
A measure of the proportion of an application exercised
by a test suite, usually expressed as a percentage.
Many different ways to measure. Here are some:
Feature/function
Scenario
Transition Path (e.g. “click thru”.)

7. © 2014 SOASTA. All rights reserved. May 13, 2015 7
How did the game, and the focus, change?

8. © 2014 SOASTA. All rights reserved.
Mobile is a customer acquisition strategy
Mobile market is Frantic
○ How many business apps do you have on your phone?
○ How many applications do you use on your desktop?

9. © 2014 SOASTA. All rights reserved. May 13, 2015 9
Complex Ecosystem
o Fragmentation and Plethora of Devices,
OS/Platforms, & Screen Footprint
o New capabilities to test
o Regression Testing Spikes
o More Network Considerations
o More complicated Apps
What was the fallout?

10. © 2014 SOASTA. All rights reserved.
Testing cycles and development methodology
Agile decreases cycle time but increases regression suites
Agile or Waterfall?
Agile regression tests increase as you progress the
project

11. Functional Testing Optimal Test Coverage
When is it “done”?

12. 12
Automating the test fabrication process is Fundamental for functional
aka “The Assembly Line”
I tested 60% (141 out of 236) of the paths (picked by risk, as the most
critical paths) and 80% (4 out of 5) of the Data Sets in the software, for a
total of 53% (624 out of 1180) overall coverage.
“I tested it.”
Program Manager’s Translation:

13. © 2014 SOASTA. All rights reserved.
Mobile has caused an explosion!
Testing matrix has exploded
Desktop App
300 test cases x 2 browsers = 600 test cases
600 tc * 30 mins per test case = 300 hrs
300 hrs / 6 hrs per day = 50 person days
50 pd / 5 people = 10 days of testing
Mobile App
300 test cases x 26 devices = 7800 test cases
7800 tc * 30 mins per test case = 3900 hrs
3900 hrs / 6 hrs per day = 650 person days
650 pd / 65 people = 10 days of testing – 2 weeks
650 pd / 5 people = 130 days of testing – 26 weeks

14. Functional Testing
Background

15. © 2014 SOASTA. All rights reserved.
Functional Testing History is manual
Manual Testing Automation Testing
100%100%

16. Functional Testing
Needs Innovation

17. CONFIDENTIAL – Not for Distribution © 2015 SOASTA. All rights reserved. January 13, 2015
Build functional tests that are based on user
behavior on your application
First you need to determine the coverage model for the application.

18. © 2014 SOASTA. All rights reserved.
Let your customers be your guide…

19. © 2014 SOASTA. All rights reserved.
Mobile Testing
The Road to Faster Testing
Manual
Testing
Mobile Device
Clouds
Automation TouchTest
Analytics mPulse

20. CONFIDENTIAL – Not for Distribution © 2015 SOASTA. All rights reserved. January 13, 2015
Analyze the most common devices of real users
mPulse

21. CONFIDENTIAL – Not for Distribution © 2015 SOASTA. All rights reserved. January 13, 2015
Analyze the most common session paths of real
users : Data Science Workbench

22. TouchTest DEMO – The path to automation
22

23. © 2014 SOASTA. All rights reserved.
We will close with some Mobile Trivia
@DanBoutinSOASTA

24. © 2014 SOASTA. All rights reserved.
Step out of the Box….
o SOASTA TouchTest
•FREE DownLoad: http://goo.gl/6CErMN
o Visit our Blog: http://goo.gl/TA2HKP
o Questions? @DanBoutinSOASTA

25. May 13, 2015
Three Tips to Increase your mobile test coverage
Dan Boutin – Senior Product Evangelist
dboutin@soasta.com
Mobile (404) 304-9529
@DanBoutinSOASTA