Experitest & Wipro Co-Webinar

June 2015
Wipro & Experitest Co-webinar:
Perfect Your Mobile App with Load Testing and Test Automation

Guy Arieli
CTO
Experitest
Sudheer Mohan
Director - Mobility
Certification &
Automation
Wipro Technologies
Speakers
June 2015

Agenda
1
2
3
4
Non Functional Testing for Mobile
Demo- See Test Integration with LoadRunner
Case Studies- Major Bank and US Retailer
Challenges in Mobile Load Testing
6
5
Mobility Automation
Wipro’s Automation Framework

© 2015 WIPRO LTD | WWW.WIPRO.COM | CONFIDENTIAL4
Non Functional Testing
Performance issues
( Slow , freeze, crash)
Heavy mobile resource utilization
Uses heavy network bandwidth
Accessibility compliance
Low application ratings
Unsatisfied customers &
Potential loss of Business
Enterprise challenges
Platforms &
platform versions
Device
Manufactures
Real world
conditions
Service
Providers

Wipro Non-Functional test solution
Mobile
Apps
Mobile
Platform
Mobile
network
 App profiling
 Interruptions
 App response time
 Exploratory testing
 Accessibility testing
 Network condition
 Handover situations
 Data transfer
 Sensitive data validation
 Simulate mobile user load*
 Load testing*
 Stress testing*
 Security testing
 Load testing *
 Stress testing *
 Security testing
Enterprise
Services
Focused non-functional testing* Covered as part of SeeTest load testing solutions
3rd Party
Integration
 Social media
 Crash analytic
tools
 Analytic tools

Mobile generate
more load

Mobile users uses your application
more extensively
Mobile generate more load
The number of mobile users
already extended the number of
desktop user.

The Market shifts from
web to Mobile
Mobile generate
more load

Thin-Thick Client
Mobile
applications are
replacing Web
Application
“The mobile
enterprise business
application market is
projected to double
by 2018”
Gartner

Mobile App is relatively
more complex
web to Mobile
Mobile generate
more load

MobileWeb
JSon over http, JSon over TCP,
XML-RPC, Soap, REST, RMI …
Single Protocol- HTTPProtocol
Json, XML, Free Text …HTMLData
Both synchronized and non-
synchronized
Synchronized –
request / response
Communication
Multiple threadsSingle threaded appConcurrency
Unlimited methods
Unified security –
browser based
Security
multiple requests
Single request / single
response -
Submit / next page
Transaction
Thin-Thick Client
Mobile apps are relatively complex

Mobile load create bigger
effect on your servers
more complex
The Market shifts
from web to Mobile
Mobile generate
more load

Mobile load create bigger effect on your servers
One of the main
cause to load on your
server is number of
concurrent session
Mobile session takes
longer time to be
processed due to
mobile network
conditions- result in
over load on your
servers
 Same number of user
mobile compare to
desktop can result in
over 20% more load
on your servers.

Traditional Load
Testing is Synthetic
more complex
The Market shifts
from web to Mobile
Mobile generate
more load

Traditional Load Testing is Synthetic
In traditional load testing the application under test is being removed and a synthetic network traffic is being generated
Typical request can have dozen or even hundreds of fields. One of the big challenges of load engineer is to
understand the nature of all of those fields
How does your application traffic
look like?
Your application open TCP connection to
the server (SYN-> SYN,ACK<- ACK->)
Your application send a request.
The server send a response.
Your application is closing the connection.
What would you find in your
application request?
Key value property data structure include
information like:
Request information
Request parameters
Session ID
Transaction ID

syntactic load is Risky
Traditional Load
more complex
web to Mobile
Mobile generate
more load

2 type
of correlations:
Why
do I need to
modify the
request
fields?
User parameters to request fields
Server response to user request
To simulate different user inputs i.e different user name / password or different UI
form population
To support internal server protocol - a very common server protocol scenario is that after
the login request, the server will return a session ID. The client need to use this session ID
in any request that is been perform.

It will create no effect on the server
The server will ignore this field
The server will ignore the entire request
The request will cause a server internal error
With Synthetic load you don't really know what will happen, it is a trial and error process
Main
Risk
What will
happen if I will
modify the
request field?

Hard to validate
syntactic load
syntactic
load is Risky
Traditional Load
more complex
The Market shifts
from web to Mobile
Mobile generate
more load

validate syntactic load
Performance teams need to adopt methodologies to validate that the generated load is valid.
Few approaches can be:
Database / UI validation to verify that the transaction generated the expected
effect
Looking at the server log files and find transaction completion log messages.
Monitor the server vital to verify the load effect the server CPU, memory, number
of threads…

As you generating the synthetic load you would like to generate few interaction with the servers based on real
devices running your real application.
Monitoring your load using real device transaction
Provide real
transaction
response time.
Provide validation
to the syntactic
load effect.
Benefits
Provide response
time reference
(when executed
without the
syntactic load)

End-to-End Product Suite - Manual, Automation, Monitoring, Load, Network Virtualization, Manager
• Productivity tool for Manual testers
• Report with screenshots of each step
• Used in conjunction with SeeTestCloud
• Allow remote debugging
• Device Lab located onsite (in the customer VPN)
• Devices can be reserved and accessed by any remote user
• Used in conjunction with SeeTestAutomation/Manual for testing of remote devices
SeeTestManual
SeeTestCloud
• Automation tool
• Same test runs on different devices
• Plugs into all testing & CI environments:
SeeTestAutomation
SeeTestNetworkVirtualization
• Device performance under varying network conditions
• Simulating mobile network bandwidth latency and reception while on the move
SeeTestLoad
• Real device generated mobile load
• Full integration to LoadRunner, Jmeter, etc.
Mobile Add-On for UFT
• Full & complete integration into HP UFT:
• Record a test, edit it, run it, and view results
• Integration of object repository and more…
SeeTestManager
Free
Tool!
N
L
G
U
Mobile Add-On for LoadRunner
• Full integration to LoadRunner through
SeeTestAutomation

Mobility Automation

More releases in a year
Parallel Tracks- Phone/ Tablet/ Web
Less time to test
Numerous Devices/ Platforms/ Versions
Not just UI Testing- Services/ Backend
End to End Testing
Why is automation a must?
In Mobility- Automation is the Key!!

1. Early Automation
 Support Agile & BDD projects
 Faster Testing by in- sprint automation
 Optimize testing cost by expanding
automation coverage
2. Platform Agnostic
 Multi platform and device support
 Reusable libraries
 Jump start automated testing in new
projects
 Reduce testing costs
3. End to End Automation
 Services & Back end validation
 Test in various environments by
pointing to different systems
behind the scenes
 High quality apps
4. Unattended Testing
Parallel Execution
 DevOps adaptation
 Fully Automated
 Integration with Test Management
& CI solutions
 Faster test execution
 Shorter test cycles & more release support
Wipro Automation
Framework
Wipro Automation Solutions Overview
5.

5
Early Automation Approach
6
4
2
 Application Flow
 Objects in each screen
 Automatable scenarios
Analyze
Wireframes
 Develop Framework
 Build reusable functions/ pseudo scripts
 Object repository mapping table ( Key->Value pair)
Develop
Automation scripts
 Map actual Objects to
Object repository
Execution Ready
Dry Run
 Ensure modified scripts
are running fine
Re Engineer
 Do modifications to the
scripts to rectify the issues
found in dry run and make it
stable
Execution
 Start test case execution on
test devices
 Dev-ops or CI/CD
 In-sprint automation
 Accelerated Testing
1 3

Automated
Build
Checks
in Code
Automated
Deployment
Automated
Reporting
Automated
Testing
Un-Attended Testing
 DevOps support
 Quick testing
 Integration with existing test management solutions

Demo
 Build on Java, leveraging SeeTest
 Supports keyword driven and data driven approach
 Supports parallel execution for faster testing
 Easy to configure and use even by a manual tester
 Supports early automation approach
 Supports integration with 3rd party test management
tools
 Supports un-attended testing
Framework Highlights
• Configure the environments via environment sheet
• Write the test cases using Excel based template
using keywords
• Demonstrate un-attended test execution using
Jenkins by automatic deployment of build
• Demonstrate parallel execution using SeeTest on
one Android and iOS device
• Integration with HP ALM – Updating results
• Demonstrate standalone report
Demo – In brief

Case in Point: Complete Mobile Automation- Leading Bank in US
 Testing of Bank’s mobile application on
iOS and Android devices
 A flexible and highly reusable
automation framework which supports
unattended testing and integration with
existing test management tool (HP
ALM)
 Backend mainframe validation for
transactions from mobile devices
 Reduce testing cycles and improved
ROI for mobile automation
 Up to 40%* effort savings with automation in
comparison with manual testing
 Continuous integration using Jenkins and
enhanced reporting capability by automated results
being updated in HP ALM
 Reduce testing cycles using early automation and
parallel execution
 Shift-left automation script development by
improving ROI for mobile automation
 Automated backend validation for each transaction
 Easily extendable to other OS platforms if needed
 Leveraged and customized Wipro mobile
automation framework which supports:
 Platform agonistic scripting
 Data driven
 Highly reusable
 Parallel automation execution
 Leveraged SeeTest partnership to build a
Mainframe validation component and
incorporated in to Wipro framework
 Integrated Wipro automation framework with
Jenkins and HP ALM for un attended testing with
SeeTest as the automation tool
 Implemented in-sprint automation using Wipro
automation framework
 Implemented Parallel automation test execution
using SeeTest for faster testing cycles
Problem Statement
Wipro Solution
Benefits!
* Based on 60% automation and early automation approach

Case in Point: Automation Productivity Enhancement- US Retailer
 Testing on iOS and Android devices
 High automation script reusability across
customer mobile applications
 Build an automation framework which is
easy to use and allows non-technical
testers to do automation
 Faster testing cycles and improved ROI
for mobile automation
 Enhanced automated test reports based
on customer requirement
 Reduce overall testing cost by reduce as
much as commercial tools
 Flexibility to run automation across
various test environments
 Jump start automation by enabling non technical
testers to do automation using a simple excel
spreadsheet
 Improved automation development productivity
using excel based framework
 Flexible and highly reusable automation framework
which is completely configurable via excel sheets
 Shift-left automation script development by
improving ROI for mobile automation
 Developed a Java based automation framework
which allows writing of test cases using Excel
sheets which enables even non-technical testers
to do automation
 Core technical team to write new keyword
functions
 Framework supports parallel execution by
configuring the devices in the configuration file
 Custom made reports for holistic view of test
execution across all devices in a single report
 Integrated with Jenkins for Continuous Integration
support
 Implemented Parallel automation test execution
using SeeTest for faster testing cycles
Problem Statement
Wipro Solution
Benefits!

Case in Point: Enhancing Customer Experience for a Leading US
Retailer
 Reduce production defects reported by
end users ( Performance issues, app
freezing , no-product found)
 Improve app rating of existing mobile
applications
 Ensure seamless customer satisfaction
 Ensure latest/upcoming features &
technologies like iBeacon, barcode
scanning , iwatch, etc.
 Improved app ratings from 3 to 4.5 (iOS) & 1
to 4.3 (Android)
 Rated as one of the fastest growing retail
mobile apps
 Stable application under real world scenarios
 Tested app under various network conditions
and identified the real world issues faced by
real user
 Conducted usability testing to analyze the
application from an end user perspective and
identified the usability issues
 Conducted app profiling to find out the
resource utilization of mobile application
 Conducted network monitoring to ensure
optimum network data transfer
 Conducted field/in-store testing to ensure
seamless end user experience
 Simulate network load and backend server
load and identified the issues faced by user
in peak load times
 Conducted app comparisons to identify the
issues and benchmark the app against other
competitors
Problem Statement
Wipro Solution
Benefits!

Thank you
Sudheer Mohan
Sudheer.mohan@wipro.com

Experitest & Wipro Co-Webinar

Recomendados

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente (18)

Destacado

Destacado (10)

Similar a Experitest & Wipro Co-Webinar

Similar a Experitest & Wipro Co-Webinar (20)

Último

Último (20)

Experitest & Wipro Co-Webinar