Oplægget blev holdt ved et seminar i InfinIT-interessegruppen Softwaretest den 11. september 2013. Læs mere om interessegruppen her: http://infinit.dk/dk/interessegrupper/softwaretest/softwaretest.htm

1. Mobile Application Testing
Frederik Carlier
ww Technology Strategist
11th September 2013

2. Agenda
•
The Mobile Market
• Testing for Desktop and Mobile Apps: Now and Then
• Methods for Test Automation
• Functional Testing
• Performance Testing
• Frameworks
• Conclusion

3. Agenda
•
The Mobile Market
• Testing for Desktop and Mobile Apps: Now and Then
• Methods for Test Automation
• Functional Testing
• Performance Testing
• Frameworks
• Conclusion

4. 2013 Gartner CIO Priorities
Ranking
2013
2012
2011
2010
2009
2008
Analytics and Business Intelligence
1
1
5
5
1
1
Mobile Technologies
2
2
3
6
12
12
Cloud Computing
3
3
1
2
16
*
Collaboration Technologies (workflow)
4
4
8
11
5
8
Legacy Modernization
5
6
7
15
4
4
IT Management
6
7
4
10
*
*
Customer Relastionship Management
7
8
18
*
*
*
Virtualization
8
5
2
1
3
3
Security
9
10
12
9
8
8
ERP Applications
10
9
13
14
2
2
* Not an option this year
Source: http://www.gartner.com/imagesrv/cio/pdf/cio_agenda_insights2013.pdf

5. 2013 Gartner Strategic Technology
Trends
Source: http://www.gartner.com/technology/research/top-10-technology-trends/

6. Mobile Market Share
Tablets
Smartphone
Windows
Phone
4%
Windows
Phone
4%
iOS
32%
BlackBerry
3%
Linux
1%
iOS
13%
Android
63%
Android
79%
Android
iOS
Windows Phone
Windows RT
Source:
http://www.idc.com/getdoc.jsp?containerId=prUS24253413
Others
Windows Phone
BlackBerry
Android
iOS
Linux
Symbian
Others
Source:
http://www.idc.com/getdoc.jsp?containerId=prUS24257413

7. Agenda
•
The Mobile Market
• Testing for Desktop and Mobile Apps: Now and Then
• Methods for Test Automation
• Functional Testing
• Performance Testing
• Frameworks
• Conclusion

9. Waterfall approach for testing
•
•
•
•
•
•
Party
Project Definition
Project Realization
Testing (used a project buffer)
Deployment to production
Party
Result: low return on investment on testing effort

10. Test Case Implementation & Execution –
ROI?
•
Reusablity across
•
•
•
•
•
•
Independent of
•
•
•
•
Different mobile OS versions (new)
Different mobile OS vendors (new)
Different hardware (new)
Different versions of the app under test
Different browser
Screen resolution
Keyboards
Localization
Re-use of existing
•
•
•
Testing tools
Automation expertise
Scripting / Programming language

11. Agenda
•
The Mobile Market
• Testing for Desktop and Mobile Apps: Now and Then
• Methods for Test Automation
• Functional Testing
• Performance Testing
• Frameworks
• Conclusion

12. Manual Testing
•
Not reproducible
• Time consuming
• On Premise?
• Test Coverage

13. Jamo Solutions - Mission
The mission of Jamo Solutions is to provide
technology and expertise that enables
successful testing and monitoring solutions
around the mobile end user experience.

14. Object Attributes & Identification
•
Objects are identified by a unique
combination of attributes
• This combination can be set seperately for
every object or globaly for all objects of the
same type
• Access to all attributes of an object
•
Key advantage of object recognition
based approach
•
Depending on the types of application you
are creating — mobile
browser, native, hybrid and the level of
interactivity (use of gesture sets, access to
platform features), this lower-level access
will be the stronger path.
Gartner, Magic Quadrant for Integrated Software Suites, 11 July
2013

15. A typical UI
Consider the differences between testing
this application using OCR (screen
scraping/bitmap recognition) and object
recognition based approach
•
•
All buttons share the same label
All buttons share the same form
All buttons share the same size
•
Object recognition: direct access
•
•
•
•
Reliable
Reusable
OCR: Need for more intelligence
•
•
Error prone
Reduced reusability

16. Portrait view vs. landscape view

17. Native, Web & Hybrid Applications
•
•
•
Native app: packaged application
deployed to the phone
Web app: accessed through the
browser
Hybrid app: native app with
embedded web views
Native App
•
•
•
•
•
Java
C#
C++
Objective C
....
Hybrid
App
Web App
•
•
•
•
HTML
CSS
Java Script
Runs inside a
browser

18. Agenda
•
The Mobile Market
• Testing for Desktop and Mobile Apps: Now and Then
• Methods for Test Automation
• Functional Testing
• Performance Testing
• Frameworks
• Conclusion

19. Functional Testing - Overview
Mobile Test Automation
• M-eux Test allows you to automate your mobile testing
scenarios
• Based on future-proof, object recognition based technology (no
jail breaking / rooting, no OCR, iOS7 compatible)
• Wide range of support across mobile platforms
(iOS, Android, Windows, BlackBerry)

20. Demo

21. Functional Testing - Overview
Integration with exiting tools & processes
• Integration with industry-standard development tools (HP
UTP/QTP, Microsoft,...)
• Integration with application life cycle management tools (HP
ALM/QC, Microsoft, IBM RQM, ...)
• Allows effective re-use of and integration with your existing
testing processes and tools
• Minimizes the total investment, increases the ROI

22. Browser-Based Functional Testing
What?
• Allows you to automate the testing of web applications on
mobile platforms
• Based on our proven object recognition based technology
• Allows for seamless cross-browser, cross-platform, crossdevice scripting
• You test the real app; on real devices or simulators/emulators

23. Demo

24. Browser-Based Functional Testing
Cross Browser Script

25. Agenda
•
The Mobile Market
• Testing for Desktop and Mobile Apps: Now and Then
• Methods for Test Automation
• Functional Testing
• Performance Testing
• Frameworks
• Conclusion

26. Performance/Capacity Management
Aspects of performance/capacity management
• Performance Testing
•
•
•
Load Generation
Response Time Measurement
Performance Monitoring
10/30/2013 © Jamo Solutions
26

27. Performance/Capacity Management
Load Generation
• M-eux Test allows you to create automated test cases
• You can replay these devices on multiple devices
simultaneously (physical or virtual)
• We integrate with load generation tools (LoadRunner, VS,…)
through UFT/VS integration
10/30/2013 © Jamo Solutions
27

28. Performance/Capacity Management
Response Time Measurement
• M-eux Test allows you to create scripts that execute an action
and wait for the operation to complete , and report timing data
• You can integrate this data with ALM tools (HP, Microsoft,…) to
track performance over time
10/30/2013 © Jamo Solutions
28

29. Performance/Capacity Management
Performance Monitoring
• M-eux Test allows you to create scripts that execute an action
and wait for the operation to complete or fail, and report timing
data
• You can integrate this data feed with your NOC operational
tools (HP, Microsoft,…)
10/30/2013 © Jamo Solutions
29

30. Agenda
•
The Mobile Market
• Testing for Desktop and Mobile Apps: Now and Then
• Methods for Test Automation
• Functional Testing
• Performance Testing
• Frameworks
• Conclusion

31. Best Practices Management
•
Re-use scripting best practice from PC world:
•
•
•
•
•
Descriptive programming
Exception handling
Object reconfiguration
Frameworks ( Linear framework, function decomposion
framework, keyword-driven framework, data-driven framework, etc.)
Best pracitices on setup
•
•
Lab setup
Scripting setup
31

32. Agenda
•
The Mobile Market
• Testing for Desktop and Mobile Apps: Now and Then
• Methods for Test Automation
• Functional Testing
• Performance Testing
• Frameworks
• Conclusion

33. Conclusion
•
•
•
•
•
•
Let your test be reproducable
Manage frequent test cycles
Create re-usable tests
Object recognition is the way to go
Change your system and app as less as possbile by
instrumentation
Note that the app is not the only thing to test

34. Questions?
frederik.carlier@jamosolutions.com
+32 485 869 861
www.jamosolutions.com

35. Thank You!
frederik.carlier@jamosolutions.com
+32 485 869 861
www.jamosolutions.com