Android test automation; overview about current landscape How to test fast, reliable and with scale

1. Danny Preussler | Groupon
Rockstar Android Testing

2. about me
• Android Engineer @ Groupon
• Lead Engineer Android @ ebay Kleinanzeigen
• Team Lead Mobile @ Cortado
• C++ Developer @ PSI AG
• Started career as developer @ Alcatel
• Articles in: android360, mobile Technology, Java Magazin
• Speaker: DroidCon.de/nl, CONFESS, Blackberry DevCon
Believer in Testing

3. Sources: Internal Data; iTunes ranking for US stores available here - https://itunes.apple.com/WebObjects/
MZStore.woa/wa/viewFeature?id=500873243&mt=8&v0=www-itunes25Bcountdown-appstore
Nearly 70 million people worldwide
have downloaded our mobile app to
date; 9 million in Q4 2013.
One of the 25 most downloaded
free apps of all time
Nearly 50% of our global transactions
completed on a mobile device in
December 2013
a mobile company
Our mobile app is available in 43 countries

4. Android Testing?
Android Firewall by Uncalno, CC BY 2.0 flickr.com/photos/uncalno/8538679708
Robotium
Appium
Calabash
Robolectric
Espresso
Cloud
Crowd
Real Device?
uiautomator
monkeyrunner
Emulator
Selenium?
InstrumentationTest

5. Rockstar Testing?
• Fast! Faster!
• Reliable!
• Scalable!
Acrassicau 6 by Bruce Martin, CC BY 2.0, https://www.flickr.com/photos/shotbydarko/4692892946/

6. InstrumentationTests
vs uiautomator
• Base for most frameworks
• Understand the differences!

7. Android Activity Tests
• JUnit3 Tests
• Android Project (android application)
• Tight coupling to app under test (project needed)
• Direct access to activities (find element by id)

8. Android Activity Tests
public class MyActivityTest extends
ActivityInstrumentationTestCase2<MyActivity> {
...
public void testButtons() {
MyActivity activity = getActivity();
Button view = (Button)
activity.findViewById(R.id.button1);
assertEquals("My Button", view.getText());
}

9. Robotium
• Based on Intrumentation Tests
• Removes complexity
• Very popular (used to be the standard)

10. Robotium
public class SimpleActivityTest extends
ActivityInstrumentationTestCase2<MyActivity> {
...
public void setUp() throws Exception {
solo = new Solo(getInstrumentation(), getActivity());
}
public void testPressButton() throws Exception {
solo.clickOnButton("press me");
assertTrue(solo.waitForText("New Window"));

11. UIAutomator
• JUnit3 Tests
• Java Project
• Simple API
• Can access everything on device a user can
• Loose coupling to app under test

12. UIAutomator
public class LaunchSettings extends UiAutomatorTestCase {
public void testPressButton()
throws UiObjectNotFoundException {
getUiDevice().pressHome();
... find and start app
UiObject button = new UiObject(
new UiSelector().text(“clickMe");
button.clickAndWaitForNewWindow();

13. UIAutomator
• Blackbox Testing
• Can only access what is visible
• API 16+

14. InstrumentationTests
vs uiautomator
• Understand the differences
Device
Application
Under Test
uiatuomator
Device
Instrumentaion
Application
Under Test
apps
apps
apps

15. Other Basics...
• Calabash: BDD
Given I am a valid user
And I press "Login"
Then I see "Welcome to coolest app ever”
• Robolectric: unit testing
• Monkey Runner

16. That‘s
not
rock star
testing!
Guitar Hero!!! By JoshBerglund19, CC BY 2.0

17. What about Selenium?
• Any language
• Any test framework
• Selenium Client API (Json Wire Protocol)
• Scale and parallize via Selenium Grid
In Theory:

18. • Used by eBay, Facebook
source: https://github.com/selendroid/selendroid/blob/master/AUTHORS
• Open Source

20. Appium
• Used by
Expedia, LinkedIn, Brands4friends
• Rising Star
• Open Source
• Cross platform
• Based on ui automator

21. Rock
without
Selenium?

22. • Used by: Groupon
• Open Sourced recently
• Remote Control your tests
RoboRemote

23. RoboRemote Architecture
Page 23
Computer/Build Server Device/Emulator
Application
Under Test
RoboRemoteServer
Robotium
UIAutomatorClient
Legend
Groupon Test Engineering Application Developer/Tester Open Source
RoboRemoteClient
UIAutomatorServer
UIAutomator
Tests

24. RoboRemote Sample
• RoboRemote: UIAutomator
UiObject myLabel = new UiObject(
new UiSelector()
.call(“className”, ”android.widget.TextView”)
.call(“text”, “My Text”));
myLabel.call(“click”);
UiObject myLabel = new UiObject(
new UiSelector()
.className(”android.widget.TextView”)
.text(“My Text”));
myLabel.click();
• UIAutomator

25. • Man in the Middle HTTP Proxy
with REST API
• Single war-file
• Open Sourcing planned
RoboRemote + Odo

26. Response Override
• Custom (stub) response
• Modify response contents
• Change HTTP response code
• Add Delay
Odo-ed
(…“soldOut”:“false”,…)(“soldOut”:“true”,…)

27. RoboRemote
• RoboRemote Github:
https://github.com/groupon/robo-remote
• Maven:
http://search.maven.org/ - search|ga|1|
roboremote

28. This was QA view.
What about
developers?

29. • Used by:
Google, XING, mobile.de,
eBay Kleinanzeigen
• Open Sourced October 2013
• Based on Instrumentation
Simple! Reliable! Fast!

30. ViewMatcher
->ViewAction
->ViewAssert
Simple

31. onView(withId(R.id.myview))
.perform(click())
.check(matches(isDisplayed()));
Simple

32. Use Power of Hamcrest Matcher
Simple

33. Waiting is the root of all evil!
Evil face in Ninnaji Temple, Kyoto by MrHicks46, CC BY 2.0, http://www.flickr.com/photos/teosaurio/10138704345/

34. • No „wait()“ in Espresso API!
Speed

35. • Instead:
– Operation on UIThread
– Wait until idle
– Action on UI Thread
– Wait until idle
– Check Result
Speed

36. • What is idle?
– UI Thread
– AsyncTask
– ???? via IdleResource
Speed

37. • Removed flakyness of sleeping
• Improved Testrunner
Reliable

39. • Used by: Google, Paypal, facebook,
Cisco, LinkedIn, eBay, BMW, Intel,
Evernote (source http://testdroid.com/products/success-stories)
• Real devices in the cloud
• Works with:
Espresso, uiautomator, Calabash,
Appium, Robotium + x

40. • Example via Gradle:
...
apply plugin: 'testdroid‘
testdroid {
...
deviceGroup "10_inch_tablets"
deviceLanguageCode "de-DE"
fullRunConfig {
instrumentationRunner =
“... GoogleInstrumentationTestRunner"
}

42. http://testdroid.com/testautomatisierung
Try it and get a free shirt:

43. So what is the
ultimate
rock star test
tool now?

44. Fazit?
Check your needs and your Resources!

45. Let’s rock test!

46. mobile rockstars
• Michael Burton
Creator of Roboguice
Author of „Android Application Development for Dummies“
• Carlos Sessa
Author of „50 Android Hacks“
https://engineering.groupon.com/
https://github.com/groupon

47. More Links
• http://developer.android.com/tools/testing/testing_ui.html
• https://code.google.com/p/robotium/
• https://github.com/calabash/calabash-android
• http://selendroid.io/
• https://github.com/appium/appium
• https://plus.google.com/+StephanLinzner/posts/HhXutBz7tcV-
• https://code.google.com/p/android-test-kit/wiki/Espresso
• https://code.google.com/p/hamcrest/
• https://cloud.testdroid.com