THESSALONIKI SOFTWARE TESTING & QA MEETUP / March 2023
A Journey inside
a React Native App
No pain, no gain

THESSALONIKI SOFTWARE TESTING & QA MEETUP / March 2023
" "
I have changed only 1 line of code. No testing is needed….
Anonymous dev
George Romanas
Software Engineer in Test Lead
@CamelotLS since 2016

THESSALONIKI SOFTWARE TESTING & QA MEETUP / March 2023
" "
Make it work, make it right, make it fast
Kent Beck
Ilias Giannoulis
Software Engineer in Test Lead
@CamelotLS since 2019

THESSALONIKI SOFTWARE TESTING & QA MEETUP / March 2023
No parallelization needed just implementing and running some tests
Only 1 emulator and 1 simulator available
Implementing functional tests and running tests one by one
Only one Mac machine locally used to build the app
How Everything Started

The
Framework
The Journey / The Framework
THESSALONIKI SOFTWARE TESTING & QA MEETUP / March 2023
Mocking
Selectors
Testing
Strategy
Reporting
Visual Testing
im|possible
CI/CD
Final
Thoughts

THESSALONIKI SOFTWARE TESTING & QA MEETUP / March 2023
The Framework

THESSALONIKI SOFTWARE TESTING & QA MEETUP / March 2023
The Framework
Tools:

THESSALONIKI SOFTWARE TESTING & QA MEETUP / March 2023
The Framework
Reqs:
Tool must be cross-platform
1
Ideally developers should be involved on testing (Javascript)
2
Test on emulators / simulators / real devices
3
CI/CD
4
5 Reporting
6 Test BDD style (Given, When, Then)

THESSALONIKI SOFTWARE TESTING & QA MEETUP / March 2023
Appium in order to be able to perform speci
fi
c actions
like swipe, webview context
1
Wdio solid solution, easy to setup, JS based framework, using
several plugins (Appium, image comparison, reporting) in order
to have a robust test suite, built in assertion library, parallel
execution, plays locally/grid/cloud
2
Cucumber for the behavior driven development customer wants
3
The Framework

The
Framework
The Journey / Selectors
THESSALONIKI SOFTWARE TESTING & QA MEETUP / March 2023
Testing
Strategy
Reporting
Visual Testing
im|possible
CI/CD
Final
Thoughts
Selectors
Mocking

Approach
How
THESSALONIKI SOFTWARE TESTING & QA MEETUP / March 2023
Accessibility ID
XPath
iOS class chain
Android UI selector
testProperties
Selectors

THESSALONIKI SOFTWARE TESTING & QA MEETUP / March 2023
Accessibility ID is a cross platform solution
1
XPath selectors only in very rare cases where accessibility
Ids cannot be implemented
2
iOS class chain, a hybrid solution between an XPath and a predicate string,
developed by Appium for hierarchical queries in more performant way
3
Android UISelector speci
fi
es the elements in the layout hierarchy
fi
ltered
by properties (text value, class name etc) usually used for interacting
with Android emulator menu elements
4
Selectors

THESSALONIKI SOFTWARE TESTING & QA MEETUP / March 2023
Selectors
In case of an automation build we are returning
the accessibility labels inside the automation build
1
When test-properties are used into the code of a
view label then it has the accessibility label on it
2In case of an IOS automation build we
introduce testID
3

THESSALONIKI SOFTWARE TESTING & QA MEETUP / March 2023
Selectors

THESSALONIKI SOFTWARE TESTING & QA MEETUP / March 2023
Selectors

The
Framework
The Journey / Mocking
THESSALONIKI SOFTWARE TESTING & QA MEETUP / March 2023
Selectors
Testing
Strategy
Reporting
Visual Testing
im|possible
CI/CD
Final
Thoughts
Mocking

Approach
Mimic the HTTP API’s behaviour
Minimal dependency by 3rd party services
Minimise Flakiness
Testing Edge case/ Error /Delay cases
How
Wiremock
THESSALONIKI SOFTWARE TESTING & QA MEETUP / March 2023
Mocking

THESSALONIKI SOFTWARE TESTING & QA MEETUP / March 2023
Mocking
GET: /api/system/status?op-hours=true
"headers": {
"X-CLIENT-VERSION": {
"matches": ".*"
},
"X-CLIENT-PLATFORM": {
"matches": ".*"
}
}
RESPONSE:
"response": {
"status": 200,
"bodyFileName": "200/system/status/system_status_twenty_four_ophours.json",
"headers": {
"Content-Type": "application/json"
},
}

THESSALONIKI SOFTWARE TESTING & QA MEETUP / March 2023
Mocking
{
"request": {
"method": "GET",
"url": "/api/system/status?op-hours=true",
"headers": {
"X-CLIENT-VERSION": {
"matches": ".*"
},
"X-CLIENT-PLATFORM": {
"matches": ".*"
}
}
},
"response": {
"status": 200,
"bodyFileName": “200/system/status/
system_status_w_all_games_o
ffl
ine.json",
"headers": {
"Content-Type": "application/json"
},
"transformers": ["response-template"]
}
}
POST: /__admin/mappings
GET: /api/system/status?op-hours=true
"headers": {
"X-CLIENT-VERSION": {
"matches": ".*"
},
"X-CLIENT-PLATFORM": {
"matches": ".*"
}
}
RESPONSE:
"response": {
"status": 200,
"bodyFileName": "200/system/status/system_status_w_all_games_o
ffl
ine.json",
"headers": {
"Content-Type": "application/json"
},
}

THESSALONIKI SOFTWARE TESTING & QA MEETUP / March 2023
Mocking
{
"priority": 1,
"request": {
"method": "GET",
"url": "/api/system/status?op-hours=true",
"headers": {
"X-Test-Client-UniqueId":{
"equalTo": "992B86B3-63EC-425C-9053-4EF5AF9649"
},
"X-CLIENT-VERSION": {
"matches": ".*"
},
"X-CLIENT-PLATFORM": {
"matches": ".*"
}
}
},
"response": {
"status": 200,
"bodyFileName": “200/system/status/
system_status_w_all_games_o
ffl
ine.jsonson",
"headers": {
"Content-Type": "application/json"
},
"transformers": ["response-template"]
}
}
POST: /__admin/mappings
RESPONSE:
"response": {
"status": 200,
"bodyFileName": "200/system/status/system_status_w_all_games_o
ffl
ine.json",
"headers": {
"Content-Type": "application/json"
},
}
GET: /api/system/status?op-hours=true
"headers": {
"X-CLIENT-VERSION": {
"matches": ".*"
},
"X-CLIENT-PLATFORM": {
"matches": ".*"
},
"X-Test-Client-UniqueId":{
"equalTo": "992B86B3-63EC-425C-9053-4EF5AF9649"
}
}
UDID: 992B86B3-63EC-425C-9053-4EF5AF9649

The
Framework
The Journey / Testing Strategy
THESSALONIKI SOFTWARE TESTING & QA MEETUP / March 2023
Mocking
Selectors
Testing
Strategy
Reporting
Visual Testing
im|possible
CI/CD
Final
Thoughts

Platforms : iOS && Android
Devices : Based on customer reports
Tests type : Happy Paths / Edge scenarios
Device type : Emulators / Simulators / Real
All device tests : Do not test on all
Tests Execution : Run tests in parallel
Approach
Cucumber JS Tags
Single wdio conf
fi
les using parameters con
fi
gurable during
build
SauceLabs
How
THESSALONIKI SOFTWARE TESTING & QA MEETUP / March 2023
Testing Strategy

THESSALONIKI SOFTWARE TESTING & QA MEETUP / March 2023
Testing Strategy

THESSALONIKI SOFTWARE TESTING & QA MEETUP / March 2023
Testing Strategy

THESSALONIKI SOFTWARE TESTING & QA MEETUP / March 2023
Testing Strategy

THESSALONIKI SOFTWARE TESTING & QA MEETUP / March 2023
Testing Strategy

The
Framework
The Journey / Reporting
THESSALONIKI SOFTWARE TESTING & QA MEETUP / March 2023
Mocking
Selectors
Testing
Strategy
Reporting
Visual Testing
im|possible
CI/CD
Final
Thoughts

Quick overview of all tested features
Thorough analysis of how many scenarios failed / passed
Search for a speci
fi
c scenario
Metadata used
Screenshot of the last screen of a test before a test fails
Share results with client
Approach
multiple-cucumber-html-reporter
How
THESSALONIKI SOFTWARE TESTING & QA MEETUP / March 2023
Reporting

THESSALONIKI SOFTWARE TESTING & QA MEETUP / March 2023
Reporting

THESSALONIKI SOFTWARE TESTING & QA MEETUP / March 2023
Reporting

The
Framework
The Journey / Visual Testing
THESSALONIKI SOFTWARE TESTING & QA MEETUP / March 2023
Mocking
Selectors
Testing
Strategy
Reporting
Visual Testing
im|possible
CI/CD
Final
Thoughts

Screenshots are generated the
fi
rst time a Visual test is triggered
All screens or speci
fi
c elements to be compared against a baseline
screenshot
Speci
fi
c elements to be excluded when capturing a screenshot
Animations to be disabled
Approach
How
Wdio-natine-app-compare-service
THESSALONIKI SOFTWARE TESTING & QA MEETUP / March 2023
Visual Testing

THESSALONIKI SOFTWARE TESTING & QA MEETUP / March 2023
Visual Testing

THESSALONIKI SOFTWARE TESTING & QA MEETUP / March 2023
Visual Testing

THESSALONIKI SOFTWARE TESTING & QA MEETUP / March 2023
Visual Testing

THESSALONIKI SOFTWARE TESTING & QA MEETUP / March 2023
Visual Testing

The
Framework
The Journey / im|possible
THESSALONIKI SOFTWARE TESTING & QA MEETUP / March 2023
Mocking
Selectors
Testing
Strategy
Reporting
Visual Testing
im|possible
CI/CD
Final
Thoughts

THESSALONIKI SOFTWARE TESTING & QA MEETUP / March 2023
im|possible

THESSALONIKI SOFTWARE TESTING & QA MEETUP / March 2023
im|possible - Geolocation

THESSALONIKI SOFTWARE TESTING & QA MEETUP / March 2023
im|possible - Geolocation

THESSALONIKI SOFTWARE TESTING & QA MEETUP / March 2023
im|possible - Biometrics

THESSALONIKI SOFTWARE TESTING & QA MEETUP / March 2023
im|possible - Biometrics

THESSALONIKI SOFTWARE TESTING & QA MEETUP / March 2023
im|possible - Biometrics

THESSALONIKI SOFTWARE TESTING & QA MEETUP / March 2023
im|possible - Ticket Scanning

THESSALONIKI SOFTWARE TESTING & QA MEETUP / March 2023
im|possible - Ticket Scanning

The
Framework
The Journey / CI/CD
THESSALONIKI SOFTWARE TESTING & QA MEETUP / March 2023
Mocking
Selectors
Testing
Strategy
Reporting
Visual Testing
im|possible
CI/CD
Final
Thoughts

CI/CD
THESSALONIKI SOFTWARE TESTING & QA MEETUP / March 2023

Approach
Fast Feedback
Scheduled builds are triggered every night in order to provide extra
stability
No PR is merged unless speci
fi
c builds are triggered and concluded
successfully
Trigger builds manually
How
Bitrise
Saucelabs
THESSALONIKI SOFTWARE TESTING & QA MEETUP / March 2023
CI/CD

THESSALONIKI SOFTWARE TESTING & QA MEETUP / March 2023
CI/CD

THESSALONIKI SOFTWARE TESTING & QA MEETUP / March 2023
CI/CD

THESSALONIKI SOFTWARE TESTING & QA MEETUP / March 2023
CI/CD

THESSALONIKI SOFTWARE TESTING & QA MEETUP / March 2023
CI/CD

THESSALONIKI SOFTWARE TESTING & QA MEETUP / March 2023
CI/CD

The
Framework
The Journey / Final Thoughts
THESSALONIKI SOFTWARE TESTING & QA MEETUP / March 2023
Mocking
Selectors
Testing
Strategy
Reporting
Visual Testing
im|possible
Final
Thoughts
CI/CD

Increased quality
Development process speedup
Test execution speedup
Most of the time one code for both platforms
Con
fi
dence for the team, QAs focusing mainly
on exploratory testing
No 3rd parties services tested
App is not a production app
Animations disabled
Pros Cons
THESSALONIKI SOFTWARE TESTING & QA MEETUP / March 2023
Final Thoughts

App Ratings
THESSALONIKI SOFTWARE TESTING & QA MEETUP / March 2023

Recognition
THESSALONIKI SOFTWARE TESTING & QA MEETUP / March 2023

Thank you.
THESSALONIKI SOFTWARE TESTING & QA MEETUP / March 2023
Ilias Gianoulis George Romanas