1. Practical Testing Definitions
For Testing Mobile Devices

2. Purpose
• This document defines general tester roles and different test levels on a
high level which are applicable for any field
• These test levels are then broken down to useful practical definitions for
when testing mobile devices

3. Definition Overview
Test Level
Test Role
Black Box
Gray Box
White Box

4. Level Overview
Validation
IOT
System
Test
Integration
Test
Unit Test

5. Theoretical Definition
Validation Confirmation by examination and through provision of objective evidence that the
requirements for a specific intended use or application have been fulfilled. [ISO 9000]
Verification Confirmation by examination and through provision of objective evidence that specified
requirements have been fulfilled. [ISO 9000]
Interoperability The capability of the software product to interact with one or more specified
components or systems. [ISO 9126]
System Test System testing of software or hardware is testing conducted on a complete, integrated
system to evaluate the system's compliance with its specified requirements. System
testing falls within the scope of black box testing, and as such, should require no
knowledge of the inner design of the code or logic.[IEEE]
Integration Test Integration testing works to expose defects in the interfaces and interaction between
integrated components (modules). Progressively larger groups of tested software
components corresponding to elements of the architectural design are integrated and
tested until the software works as a system. [Beizer, Boris]
Unit Test Testing of individual hardware or software units or groups of related units [IEEE]

6. Practical Definition: Unit Test
• In computer programming, unit testing is a method by which individual units of source code, sets of
one or more computer program modules together with associated control data, usage procedures,
and operating procedures, are tested to determine if they are fit for use. Intuitively, one can view a
unit as the smallest testable part of an application. In procedural programming a unit could be an
entire module but is more commonly an individual function or procedure. In object-oriented
programming a unit is often an entire interface, such as a class, but could be an individual method.
[2]
• API testing is done using a White Box approach
– Functionality
– Reliability
– Efficiency
• Main Challenges of API Testing can be divided into following categories
– Parameter Selection
– Parameter combination
– Call sequencing
– Different system states for the above
• Unit test techniques such as Boundary Value Analysis, State Transition Testing,
Equivalence Partition Testing, Cause-Effect graphing, etc.

7. Practical Definition: Integration Test
• Testing all ISO 9126 software quality attributes [3] during feature development and
when a feature or application is complete and all parts are integrated
– All the different parts have been unit tested but are now integration tested as soon as part of the feature is
testable either through API testing or manual or automatic testing through the UI
– When the feature or application is complete additional testing is done to secure any system impact caused
by the feature or application
• API testing is done using a Gray Box approach
– Functionality
– Reliability
– Efficiency
• Focus on system impact of changes that are being integrated, based on system
dependencies, securing that there are not regressions to other parts of the system
• A feature could be a new integrated phonebook application, a new camera
implementation, a new Bluetooth profile, a new WiFi feature, changes in the file
system, or something similar

8. Practical Definition: System Test
• User Scenarios [1]
• System Capabilities
– System Power Consumption
– System Performance
– System Stability
– System Memory Usage
– System Stress
• User Experience / Usability
• Customer Customizations
– Testing the mobile device with different customer customizations
• Application Composition Testing
– Testing a complete application suite that is going to be released with the mobile device
• System Upgrade Testing
– Securing that it is possible to upgrade the system without any unwanted side effects
• System Age Testing
– Securing that the mobile device works as expected after extended periods of usage
• Localization Testing
– Translation and language testing

9. Practical Definition: IOT
• Device IOT
– Testing the mobile device with different third-party appliances
– TVs
– Laptops
– Cameras
– Speakers
– Etc.
• Accessory IOT
– Testing the mobile device with different mobile device accessories, such as headphones
• SIM IOT
– Testing the mobile device with different SIM cards
• Service IOT
– Testing the mobile device with different services
• External Storage IOT
– Testing the mobile device with different types of external storage devices
• Network Dependant Testing
– Testing the mobile device in different operator networks

10. Practical Definition: Validation
Risk Management
Validation Guidelines Quality Criteria
Verification Results Model
Product Risks Analysis of
Validation Test Analysis of
Verification
Activity Validation Results
Results
Test Gap Validation Quality
Report Report Report
Customer
Requirements
Regulatory
Requirements
IOT, System, or
Integration Tests

11. Not in Scope
• Time aspects have not been discussed in this document – the test levels
defined can be used at any point in the project, even though it may be
difficult to test the system until a certain maturity has been reached
• Automation has not been discussed – all test levels can be automated or
executed manually, even though unit tests are preferably automated, and
validation is often run manually due to the nature of the activity
• Scripted or exploratory testing has not been discussed – both approaches
can be used for any manual testing
• Regression testing can be done on any testing level at any stage during a
project whenever there are possible regressions of the system
11 2012-02-15 179/038 13-LXE 110 0048 Uen PA4 Interview with a Tester Company Internal

12. Summary
• This is just one way to define these different test levels and roles
• Using these definitions to clarify test activities and test roles will facilitate
communication and support testers in understanding their responsibility

13. Reference
[1] An Introduction to Scenario Testing
http://www.kaner.com/pdfs/ScenarioIntroVer4.pdf
[2] Wikipedia – Unit Testing
http://en.wikipedia.org/wiki/Unit_testing
[3] ISO 9126
http://en.wikipedia.org/wiki/ISO_9126