Hp perfecto webinar - UFT Mobile

A

How to Build an Enterprise Grade
Mobile Testing Strategy
Eran Kinsbruner
Product Marketing @Perfecto Mobile
Antoine Aymer
Product Marketing @HP

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

A

How different is Mobile?

2


Tap & Multi tap
Double / long tap
Drag
Swipe
Pinch & zoom

Click
Double click
Right click
Drag

Snapshot
Type

Rotate / Tilt
Locate
Authenticate

Type
3


Speak

Communicate

A

CES 2014, Smart Innovation

BYOD Inflight Contactless
Entertainment payment

4

Smart
e-bike

Smart
gardener

Smart
clothes


Smart
nappy

Smart
appliance

A

Augmented
reality

A

Starbucks’ app
Features:
•
•
•
•
•

•
•

 Improve in-store customer experience
 Generate customers loyalty
 Increase sales
5


Pay for your purchases
Track and redeem rewards
Reload Starbucks Card’s balance
Get directions to the nearest store
Explore coffees, beverages, food and
nutritional information
Send eGifts instantly to your contacts
Stay informed by being the first to
hear of special offers

But this is not enough…

6


A

A

Use case #1
Sport App connected with smartwatch

• Data transfer via Bluetooth
• 100% manual/human testing
• Unit test
• 100% emulation

Unit test

Interoperability

Industrialisation
• Functional testing on major devices
• 80% automated testing

SaaS-based applications are expected to generate a growth rate of 17.4%.
7


Why Mobile Testing
looks so difficult?

8


It is the jungle!
Development platform
versus

versus

Various OS types, versions, device/tablet, OEMs

Evolving

9


Browsers

DEVICE

MARKET
Fragmented landscape

Performance
Network (>400 operators)
Contextual issues
Peripherals
Impact on back-end

10

(OS, version, input, screen, style)

USER

Know the user
Users’ feedback


Time to market

TESTING
Emulator vs real mobile
Automation vs manual
Device management
What Devices to test on?
Bug-fix latency
Unavailable services
Off-shore

A

48%
lack
expertise

38%
lack tools

Why Mobile Testing looks so difficult
11


Mobile testing disciplines
Functional

Non Functional

Interoperability (IOP)

•
•
•
•

• Network Strength / Outage /
Recovery
• Different NW Types
• Peripheral Testing
• Services

• Voice / SMS interrupts
• Notifications
• Battery /Cable Removal

Memory Leak

Installation Testing

Language Testing

• Memory Usage
• Memory Leaks
• Garbage Collection

• New App Install
• Uninstall and Reinstall
• Upgrade testing

•
•
•
•

Validation of Functionality
Smoke / Regressions Testing
Offline access testing
Negative Testing

Validation for Locales
Images and Text
Currencies, time zones etc.
Context

Performance Testing

Security Testing

Usability Testing

• CPU Usage testing
• Network Usage
• Page Render time or activity
Render time

•
•
•
•

• User Experience
• Competitive Analysis
• Expert Review

12

OWASP Vulnerabilities
Dynamic Testing
Static Code Analysis
Data Encryption


Definition of Fully Tested

13


A

The Truth Is Out There!
Which of the following statements reflect your development team's approach to
testing mobile apps?
60%

We test our app across multiple device orientations
We test across multiple OS versions

55%

We test across multiple devices that run the same
version operating system
We do a quick test on an emulator/simulator or on
a single device
We test devices that are deployed on multiple
cellular carriers
We use a cloud-based service to test multiple
devices
We automate our testing process as part of a
continuous integration process
We integrate crash reporting/analytics into our
deployed application to monitor and fix problems
We don't formally test or instrument our apps
before deploying them; we fix changes as needed

55%

55%
34%
31%
27%
21%
4%
Source: Forrester, Q1 2013
Base: 765 Software developers developing mobile applications

14


A

Mobile is Agile on Steroids

15


Cumulative business value

The ideal Mobile testing picture

Sprint 1

Sprint 2

Sprint 3

Sprint 4

CI

CI

CI

CI

Release, Iteration, Nightly, Continuous
End-to-End test cycle
On any device, by any team, from any location
Time

V1.0

V1.1
Longevity testing (independent of sprint cycles)

16


V1.2

Agile and Testing -- 5 o’clock on Friday

the 30th, do you know if your code is “done”?


A

The Truth Is Out There!

49% of
developers spend
more than an hour
a day testing new
code

52% spend

only 15%

more than an hour
debugging/fixing
code

spend that much
time writing test
automation code

Source: Forrester, Q1 2013
Base: 765 Software developers developing mobile applications

18


Agile mandates automation

19


Use case #2
Top US Bank

20


A

Use case #3
Mobile banking, 12 Millions unique users
native application and mobile enabled website

manual

3 weeks
In-house

Type of Testing

Deployment

Impact

21


A

Use case #3
Mobile banking, 12 Millions unique users
native application and mobile enabled website

86%
time savings
60%
cost savings

Automated

3 weeks
Cloud
48 hours

Type of Testing

Deployment

Impact

22


Automation requirements

23


What can get in the way?

store

Silos

24

web

App Style

Cross-Platform
Differences


Use cases

Cross-platform differences
Name position
Menu panel

Labels
Text colours
Images

Button images
25


Building an Enterprise Grade
Mobile Testing Strategy

26


Transform
Predictability, scalability

Grow

Coverage, Reliability & Governance

Start
What fits my workflow?

Sprint 1

Sprint 3

Sprint 4

CI
27

Sprint 2

CI

CI

CI


A

Finding the right balance
Time to
Market

Locations &
Networks

Coverage

Governance

Devices

Existing
Tools
28


Service
Level

HP UFT Mobile

29


#2 Deployment

native

#1 HP UFT Add-in

30


#3 Devices

Real mobile devices

HP UFT Extension

(and emulators)
Perform almost any operations

Integrated with HP ALM

Scalable

Device agnostic

From single, small project to
hundreds of simultaneous users

Develop, run and schedule one
test across multiple platform
Visual & Object analysis

Secured
Full tractability of actions. Device
cleanups and encryption abilities

Continuous testing
Support fast, agile delivery

Global
Accessible from any location
(VPN mandatory with MCE)

Manageable
Full IT control on authentication,
access permission and activities
Predictable availability

31

Complex scenario

#1 HP UFT Add-in


advanced built-in functionalities
such voice injection and
recognition, NFC, location based

#2 Deployment
Public Cloud

Virtual Private Cloud

MobileCloud Express

32


Virgin Media Corporate Network

Use case #4: Virgin Media
Off-shore

Team members
Virgin Media UK Offices

Virtual Private Cloud
Dedicated devices in various geographies

Core team

On-premise lab
33


Public Cloud
Multiple devices in various geographies

#3 Devices
Wide device spectrum
HTC, Apple, Nokia, Motorola,
Samsung, SonyEricsson, LG,
BlackBerry, Palm, Kindle

Scalable
Additional handsets are added ondemand

Powerful replay
all tests are video recorded to allow
easy recreation of defects

List of supported devices: http://www.perfectomobile.com/portal/cms/services/handsets.html

34


Full device control
Fast charging cycle, audio injection,
reboot, clean/wipes

Deploy your app
Testers can deploy their application
on the device, test it as if you were
holding the handset in your hands.

Global
The handsets resides in various
datacenters in the world: US,
Canada, UK, France, Germany,
Spain, South Africa, India.

35

5

Gain mobile objects
experience

2

Be intimate with manual
testing

6

Always sanity test your
previous version

3

Define relevant tests
across all target devices

7

Implement continuous
(before the demo)
integration

4

SCOPE AUTOMATION

1

Traditional non-mobile
automation rules still
apply

Choose the right devices

8

Think about future
maintenance


Hp perfecto webinar - UFT Mobile

Recomendados

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente (20)

Destacado

Destacado (20)

Similar a Hp perfecto webinar - UFT Mobile

Similar a Hp perfecto webinar - UFT Mobile (20)

Más de Perfecto Mobile

Más de Perfecto Mobile (20)

Último

Último (20)

Hp perfecto webinar - UFT Mobile