AWS Mobile Week at the San Francisco Loft Automated Device Testing: The Right Way Mobile industry data shows that up to 84% of users may delete an app due to a poor quality or performance experience. Consequently, it’s imperative to deliver high quality builds from day one (even if it’s just an MVP product) in order to not lose the customers that you’ve worked so hard to acquire. A critical component of delivering high-quality apps is testing your app on real devices as they exist in the wild. In this session, you’ll learn about using AWS Device Farm as part of your testing regime and how to incorporate as part of your CI/CD pipeline. Level: Intermediate Speaker: Nikhil Dabhade - Sr. Cloud Solutions Engineer , AWS Mobile Applications

1. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Pop-up Loft
Automated Device Testing: The Right Way
Nikhil Dabhade
Senior Cloud Solutions Engineer | AWS Mobile
Email: nikdabn@amazon.com
Twitter: @in_nikoftime

2. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Wait, what ?

3. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved

4. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
I can fix this!

5. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
But why test on real devices?

6. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Test on actual hardware your users will use
Performance of real devices is faster
Bug replication is easy
Test with real world conditions
Access to sensors, battery data and camera
No false positives

7. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Challenges in mobile testing space
• Fragmentation of devices
• Reliable access to real devices
• Testing: UI vs. functional
• Different test frameworks
• Continuous integration for mobile-automation testing
• Reporting from tests
• Access to test artifacts
• Granular control and maintenance of devices and test setup

8. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
What is AWS Device Farm?
Automated testing Remote Access
Test your app in parallel against a large collection
of physical devices in the AWS Cloud
Gesture, swipe, and interact with devices in real
time, directly from your web browser or local host

9. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
What are we going to do today?
• Discuss ways to make tests a first class citizen.
• Run Tests on Device Farm
• CI/CD integration

10. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
How to make testing on real devices a second nature?
• CI/CD integration.
• Fast on-demand access to real devices.
• Easy IDE and command line access.
• Quick access to test results.

11. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Automation Testing on Device Farm
Your App
UI Automation
UI Automator
XCTest
AWS Device FarmPopular Test Frameworks
Detailed Test
Reports

12. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Demo: Run Tests on Device Farm

13. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Demo: CI/CD integration

14. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
CI/CD integration
• For Jenkins, download the official plugin for AWS Device Farm “aws-device-
farm”
– https://github.com/awslabs/aws-device-farm-jenkins-plugin
• For other CI/CD systems, build a simple one time script using AWS CLI and use
a Post Build plugin
– Reference blog: https://aws.amazon.com/blogs/mobile/get-started-with-the-aws-
device-farm-cli-and-calabash-part-1-creating-a-device-farm-run-for-android-calabash-
test-scripts/

15. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Pop-up Loft
aws.amazom.com/mobile
@AWSForMobile
Everything you need to connect
your mobile app to the cloud