Sikuli is a tool for automating testing on mobile devices through image recognition. It allows testers to write scripts using images of the screen rather than coordinates or IDs. The speaker discusses how Sikuli addresses issues with traditional mobile testing approaches by being neither too visual, new, or old. Sikuli strikes a balance by using image recognition to automate tests across emulators and real devices without relying on fragile elements like coordinates or IDs that can change between builds.

1. SIKULI for Mobile Testing
Tom Yeh
Assistant Professor
Department of Computer Science
University of Colorado Boulder
Mobile Testing Summit 2012

4. Visual Diff

5. Test 1 Test 2 Test 3 Test 4 Test 5
build
012

6. Test 1 Test 2 Test 3 Test 4 Test 5
build
012
build
013

7. Test 1 Test 2 Test 3 Test 4 Test 5
build
012
build
013
Are they different?

8. Test 1 Test 2 Test 3 Test 4 Test 5
build
012
build
013

10. Step 1: Click on

12. Think programming
Which row?
Which column?
What component ID?
What name?
What label?

13. Think Sikuli
Which does it look like?

14. Step 1: Click on

15. Step 1: Click on

16. Goal:
Uncheck Contacts

17. What’s wrong?

21. Harvard Medical School

23. Too Visual

24. 400
manual test cases
5
hours of labour

26. 2x
release rate

27. TELENAV
GPS App Maker

28. Too New

29. Tests are
automated via
emulators

30. NASA

31. Too Old

33. NASA’s part supplier

34. Tests are automated
through a remote
desktop connection

35. Too Visual Too New Too Old

36. What’s coming?

38. http://www.cuberick.com/2010/06/testing-image-generation-with-cucumber.html

44. Mobile Testing
goes Physical

45. Mobile Testing
goes Physical
Visual

46. Baxter by Rethink Robotics

47. Baxter by Rethink Robotics