Will Hurley
Quality and Security Services
A practice leader of quality management and security for Idea Integration Corporation, Will
Hurley has been at the crest of numerous software development waves during his twenty-four
year career. Will represented the US Air Force in the DoD’s first joint service system
architecture, led CMM assessments, and supported the development of the CMMi. After leaving
the Air Force, he consulted for industry-leading companies, helping them adopt and apply new
development methods. In his current position, Will consults with clients in North America on
achieving desired system and product lifecycle characteristics.

28/09/2012

It’s a Phone First!
How to Test Your Five‐star Mobile Apps
©2012 Idea. All rights reserved

1

Today’s discussion
Idea and I
Acknowledgements
Five Star Mobile Application

improving mobile project outcomes and time to market.


2

1

28/09/2012

Since 1986, we’ve helped some of the world’s
largest and most respected brands thrive in the
digital space.
space

3


The Presenter

Will Hurley
Practice Executive
Quality Mgmt. & Security
Idea
William.Hurley@idea.com
(781) 831-0389


USAF Systems Engineer
TBMCS Core Systems Lead
Joint Service Architect GCCS
USAF Data Administrator
Consultant
Small Business Owner
Practice Leader
4

2

28/09/2012

Acknowledgements
This discussion captures mobile testing insights developed at Idea’s
Global Solution Centers in Nashua, NH and Houston, TX.
Idea’s insights are supported by more than 10 y
years of mobile
g
pp
y
development and testing.
Insights are not SDLC or tool specific.
Data is updated on the quarter through a verity of sources.
ComScore®
Netbiscuits®
Harris Interactive®
OPA
Crittercism

5


5

5 Star Mobile applications

6


6

3

28/09/2012

5 Star Mobile radar chart


7

Ecosystem insights
Understanding the mobile ecosystem and trends increase test effectiveness
and product quality characteristics.

o

Capture more than 80% of smartphone users
with 3 network carriers.
ih
k
i

o

Capture more than 85% of smart phone
users with Android and iOS platforms.

o

Application use is growing faster than
browser use.

o

Android users spend more time on carrier
networks.

o

iOS users spend more time on wireless
networks.


8

4

28/09/2012

Ecosystem carrier
US Carrier Smart Phone Market Share

As of Aug 2012, 114 million Americans
own smart phones.
p

Verizon
31.60%

60 percent of all mobile devices acquired
in the U.S. are smartphones.
Verizon market share has grown fastest
driven by iOS.

Other 15%
AT&T
26.30%

Consider market share when signing
contracts for cloud services.

T‐Mobile
9.50% Sprint
9 50% Sprint
10.50%

Tracfone
7.10%

9


Ecosystem oem
Samsung Galaxy S III leading 2012
sales may be passed by iPhone 5.

US Smartphone OEM

iPhone 4 took the top US device
sales spot in 2011.

Samsung,
25.6%

Other,
15.8%

iPhone 3GS took 2nd place.
LG, 18.4%

iPhone 4S took 3rd place.
BlackBerry Curve 8530 4th place.

RIM, 6.3%

Apple,
16.3%
16 3%

HTC Evo 4G 5th place.

HTC, 6.4%
Motorola,
11.2%

HTC is the most active OEM for
Android smartphones.

10


10

5

28/09/2012

Ecosystem platform
US OS Share

Other
0.5%

Android
52.2%

Symbian
0.8%
Microsoft
3.6%

RIM
9.5%

Apple
33.4%

66 percent of Americans ages 24-35
own a smartphone
smartphone.
Apple and Android market share are
growing.
Market share for all other platforms is
shrinking.
Android devices out number iPhones
by 3-to-2 in the U.S. market.

11


Platform fragmentation

Fragmentation is a real but
manageable issue for all platforms.
A d id f
i
Android faces more i
issues since
it resides in an open ecosystem.
Android OS adoption is slower
than iOS.
Trades reporting >½ of all iPhone
users that can upgrade to iOS5 have.
iOS6 expected to drive largest
upg ade population date
upgrade popu at o to date.

Crash data are normalized as crashes as a percentage of each application launch.


12

6

28/09/2012

Ecosystem tablet

Tablet Share (Platform)
Other
12%

iPads account for more than 70% of
tablet network traffic.
Apple
46%

More than 80% of tablet traffic is wireless,
not cellular.
Tablet adoption is balanced between
males and females, trending older with
households earning $50K or more.

Android
42%

Kindle Fire
28%

An estimated 74.1 million consumers are
using tablets.

iPad 2 31%


13

Customer Experience insights
Mobile users expect more.

o

Good design is not accidental. Get help if
you need i
d it.

o
o

Good design is sticky.

o

Analytics portray the customer voice.

Unstructured or unsolicited feedback has
more action value than a survey.


14

7

28/09/2012

Experience components
How easy is it for your customer to use
your application.

The application does
what your customer
wants it to do.

How does the
application make
li ti
k
your customer feel.

15


Customer experience
Smartphone Usage

Music,
24.50%
Games,
31.80%
Social net
or Blog,
35.70%


Text
Text
message,
74.60%
Used
browser,
48.60%
Used App,
48.60%

Four in ten mobile users are disappointed
with applications from their favorite
brands.
Three-quarters of users surveyed believe
a company's mobile app should be easier
to use than its Web site.
Users spend more time with mobile
applications than their mobile browser.
82% of iOS and Android Apps crash once
out of every 10 loads
Less than 5% of mobile apps are still in
use 3 months after downloading.

16

8

28/09/2012

Experience sticky


17

Experience design


18

9

28/09/2012

Experience themes
Based on conversations, we found several
common themes:
• All purchasing because it’s required by their
apartment complex
complex.
• All participants were referred to site via a
brochure.
• Participants pleased overall with the
simplicity of the online process
• Liked being able to select from presented
options in order to simplify and speed up the
process.
• Appreciated the ability to get additional
details where necessary.
• A majority didn’t use these features.
• All participants pay annually for their
policy.


19

Experience maturity

o Functional requirements are only part of the customer
o
o
o
o

experience.
Key moments in customer interaction demand good
design.
If you don’t have experienced designers on your team
bring them in.
Understand accessible and emotional goals at every key
moment.
Don’t predict how people will respond to a new
experience,
experience test it
it.
o Allot the time to make incremental changes on all key moments.


20

10

28/09/2012

Functional insights
It’s a phone first.

o

Test while preforming common smart phone
tasks.
k

o

It’s cheaper, faster and easier to test in
emulated environments first.

o

Testing types are the same as Web based ntier applications

o

Manage functional and non-functional
insights across p j
g
projects.

o
o

Create a mobile test strategy document.
Reduce tool and process resistance
between groups.


21

Manage scope

Carriers

Operating
Systems

OEM
Devices

Com &
Sensors


22

11

28/09/2012

Manage knowledge


23

Manage modularity
•
•
•
•
•
•
•
•
•

Display and Graphics
Input Devices
Sensors
Permissions
Intents
Phone and messaging
Deploy and Upgrade
Analytics
A l i
Your application’s key moments


24

12

28/09/2012

Emulation(Simulation) testing
Does not require the physical device.
Can still use network services.
Emulation requires experience with specific
platform development tools.
Static security and performance testing can
be performed on source code.
Emulation is best for:
Investigating architectures and
application layers
g
g
y
Agile testing of key features and
interactions
Evaluating code quality characteristics
Research HTML5 characteristics.
Planning for automation
25


Device testing
Does require a physical device.
Testing on the device is best for:
U E
i
User Experience – th l t 100 il
the last 100 miles.
Acceptance testing – working with
gestures, sensors and hardware.
Soak Testing – how will the device
perform over time.
Provisioning – installation and
removal.
Cloud providers and plugins for popular
automation tools support both functional
and performance testing.
No need to run out and buy all platforms.


26

13

28/09/2012

Automated testing
Browser automation
Tools can be used for ‘closest‐match’,
browser testing on or off the device.
‘Closest‐match’ browser automation
possible on some devices ‐ Safari 3
for the iPhone driven on a PC.
User‐Agent header data to drive
compatibility testing logic.
Work closely with development team to
manage and control Web 2.0 features.
Third party libraries can speed
implementations.
Physical devices and Cloud services can be
used to test physical platforms when
defects are detected.

27


Automated testing
Application automation
Tools can be used in emulated
environments and on the device.
Installation of some type of
automation controller is required on
the device.
Controllers consume resources.
Plug‐in and Cloud support is growing
and stable.
Blu
Ne eto
ar oth
Fie /
lld

Cloud services may not provide
object recognition.
Additional fixtures may be required for
sensor (GPS, Accelerometer) and
hardware (camera, battery) automation.
Applications may incorporate browser
engine.


28

14

28/09/2012

Functional details
Create a mobile test strategy document.
Tight control of manifest and OS support.
Identify development and testing tool stack.
Explain how you plan to manage fragmentation – OS, device, sensors, UI, marketplace.

Test teams should be working in emulated, physical and cloud based
environments.
Test teams should manage insights across releases.
HTML5 platform insights
Carrier and Device insights
OS / Platform insights
Security insights
Marketplace


29

Security insights
Protect your brand.

o

Address security issues programmatically
and start now!

o

Static analysis is a highly effect approach to
address coding issues.

o

Dynamic analysis, Security Penetration
testing, is your last line of defense.

o

SD memory should be tested


30

15

28/09/2012

Security testing
Static Application Security Testing (SAST)

Dynamic Application Security Testing

Static analysis tools can address unsafe
Static analysis tools can address unsafe
practices earlier in the SDLC.
Clang; AppValidator; AnalytiX
SAST should be integrated with build and
reporting practices.
Evaluate application permissions.

Mobile devices are susceptible to
Mobile devices are susceptible to
common browser exploits and more.
Emulators or devices can be used to
ensure applications don’t bleed sensitive
data on to the device or to the network.
Keep up to date with known exploits and
add test cases to qualify the impact.

Don’t build on or deliver on compromised
platforms.
platforms


31

Performance insights
Protect your brand.

o

Static analysis is a effect approach to
address performance issues.
p

o

Dynamic analysis, Security Penetration
testing is your last line of defense before
delivery.

o

SD memory should be tested.


32

16

28/09/2012

Performance testing
Static Application Performance Testing (SAPT)

Performance Testing

Static Analysis tools find issues earlier in
Static Analysis tools find issues earlier in
the SDLC and improve coding practices.

Load stress and longevity testing are
Load, stress and longevity testing are
critical to the customer experience.

Tools address code that slow application
performance.

Load and stress testing are focused on
middle and backend service layers.

In a 53K line project we found 328
instances of code that would
contribute to performance issues.
SAPT should be integrated with build and
reporting.

Longevity testing should look at both
backend services and the mobile device.
Performance tools should support
bandwidth throttling and common RESTful
protocols.


33

Final thoughts
o

Cloud based testing services are valuable
but…

o

Will not achieve expected ROI without
proper investigation.
investigation

o
o

Lead to Scrumfall application lifecycles.
Can slow delivery times.

o

Functional, performance, security and test
automation should be considered for each
mobile project.

o

Simplify the testing space through active
p y
g p
g
management.

o
o

Diagram test environments.
Support agile approaches for all team
members.


34

17

28/09/2012

Final thoughts
o

Develop sticky applications with the 5
star insights.

o

Fill out our five start survey and see
how your team stacks up!


35

18

It's a Phone First! How to Test Your Five-star Mobile Apps

Recomendados

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente (19)

Destacado

Destacado (15)

Similar a It's a Phone First! How to Test Your Five-star Mobile Apps

Similar a It's a Phone First! How to Test Your Five-star Mobile Apps (20)

Más de TechWell

Más de TechWell (20)

Último

Último (20)

It's a Phone First! How to Test Your Five-star Mobile Apps