My talk from Nordic Testing Days.

1. 10 Mobile App Testing
Mistakes to Avoid
Daniel Knott | @dnlkntt

2. Hi, I‘m
Daniel Knott
Lead Software Test Engineer Mobile, XING SE
@dnlkntt
www.adventuresinqa.com
www.handsonmobileapptesting.com https://leanpub.com/smartwatchappt
esting
http://bit.ly/2bUasW8

3. 4
https://stocksnap.io/photo/M5ERMO26E9

4. 5
Mistake
#1

5. 6
Mistake #1: Don’t Listen to Your Customers

6. 7
Mistake #1: Don’t Listen to Your Customers

7. 8
Mistake #1: Don’t Listen to Your Customers

8. 9
Mistake #1: Don’t Listen to Your Customers
• Check the app stores for feedback & reply.
• Check also the error reports from your users.
• Invite customers to your office and ask for feedback.
• Invite them to become beta testers of your product.

9. 10
Mistake
#2

10. Mistake #2: Do not follow the Platform UI/ UX
Guidelines
11

11. Mistake #2: Do not follow the Platform UI/ UX
Guidelines
• Know them!
• Make sure the whole team is aware of them
• Create mind maps with most important elements for your
app
• Check them from time to time
12
https://developer.android.com/guide/practices/ui_guidelines/index.html
https://material.io/

12. 13
Mistake
#3

13. Mistake #3: Don‘t Test Against the Mobile Specific
Uscases
14

14. Mistake #3: Don‘t Test Against the Mobile Specific
Uscases
15

15. Mistake #3: Don‘t Test Against the Mobile Specific
Uscases
16
https://www.flickr.com/photos/37996583811@N01/8033259053/

16. Mistake #3: Don‘t Test Against the Mobile Specific
Uscases
• There are more mobile specific test scenarios like:
• Update testing
• Battery Drainage
• Interruptions
• Log Files
• Performance
• ...
17

17. 18
Mistake
#4

18. Mistake #4: Try to Test Everything
19

19. Mistake #4: Try to Test Everything
• Not special to mobile testing
• Focus is important
• Test the main & critical functions
• Test on most used devices
20

20. 21
Mistake
#5

21. Mistake #5: Not performing in the Wild Testing
22

22. Mistake #5: Not performing in the Wild Testing
23

23. Mistake #5: Not performing in the Wild Testing
• Test the app in the environment the user will use it.
• Test in different mobile networks.
• For example testing a running app, you should go for a
run:
• Sweaty hands
• GPS
• Battery
• Usability while running
• There is more to explore outside of the office!
24

24. 25
Mistake
#6

25. Mistake #6: Not looking behind the UI
26

26. 27

27. Mistake #6: Not looking behind the UI
• Testers must be part of code reviews.
• Pair with developers.
• Use Proxy Tools to check the requests & responses.
• Check log files on the devices.
28

28. 29
Mistake
#7

29. Mistake #7: Automation on Wrong Layer
30
Manual
Testing
Integration Tests
Unit Tests
UI
Tests

30. Mistake #7: Automation on Wrong Layer
• Don‘t write test automation just to have automation
• Use the pyramid as basis to talk to developers
• Not every app can be automated on every layer
• Use multiple tools for the layers
• There is no silver bullet tool
31

31. 32
Mistake
#8

32. Mistake #8: Testing on a Single Device
33

33. Mistake #8: Testing on a Single Device
• Know your customers (Mistake #1)
• Test on devices from customers and nothing else
• Group & Prioritize the test devices from user base
34
Group 1, Prio A
• High End
Devices
• Quad Core CPU
• +3GB RAM
• Retina, Full HD
Display > 5“
Group 2, Prio B
• Mid-range
devices
• Dual Core CPU
• +1GB RAM
• Screen size <= 5“
• Software not
older than 1 year
Group 3, Prio C
• Small devices
• Single Core
• < 512MB RAM
• Low screen size
and resolution
• Older OS, older
browser

34. 35
flickr.com/photos/minxlj/3270996158/
Mistake
#9

35. Mistake #9: No Update/ Installation testing
36

36. Mistake #9: No Update/ Installation testing
• Perform update test before submitting the app.
• Check for data inconsistency.
• User is still logged in to the app.
• No crashes after udapte.
37

37. 38
Mistake
#10

38. Mistake #10: No Security Testing
• Security is key these days.
• Security is a broad and complicated field.
• Different layers of security.
• Too complex for one person.
• Highly suggest to check owasp mobile security project
39
https://www.owasp.org/index.php/OWASP_Mobile_Security_Project

39. Thank You
Questions?
40
Daniel Knott - @dnlkntt
www.adventuresinqa.com/publications

40. Image Resources
• https://upload.wikimedia.org/wikipedia/commons/thumb/0/01/Gestures_Flick.png/512px-Gestures_Flick.png
• https://upload.wikimedia.org/wikipedia/commons/thumb/8/80/Gestures_Two_Finger_Tap.png/512px-Gestures_Two_Finger_Tap.png
• https://upload.wikimedia.org/wikipedia/commons/thumb/c/c2/Gestures_Long_Press.png/512px-Gestures_Long_Press.png
• https://upload.wikimedia.org/wikipedia/commons/thumb/b/bd/Gestures_Unpinch.png/512px-Gestures_Unpinch.png
• https://upload.wikimedia.org/wikipedia/commons/thumb/3/3d/Gestures_Scroll.png/512px-Gestures_Scroll.png
• https://upload.wikimedia.org/wikipedia/commons/thumb/f/f1/Gestures_Two_Finger_Scroll.png/512px-Gestures_Two_Finger_Scroll.png
• https://upload.wikimedia.org/wikipedia/commons/thumb/f/f8/Gestures_Pinch.png/512px-Gestures_Pinch.png
• https://upload.wikimedia.org/wikipedia/commons/thumb/1/16/Gestures_Rotate.png/512px-Gestures_Rotate.png
• https://upload.wikimedia.org/wikipedia/commons/thumb/f/f0/Gestures_Two_Hand_Pinch.png/512px-Gestures_Two_Hand_Pinch.png
• https://stocksnap.io/photo/0ACEPNFQD5
• https://www.flickr.com/photos/37996583811@N01/8033259053/
• https://stocksnap.io/photo/295M7KIZ4P
• https://stocksnap.io/photo/5C6Y92Q2A8
• https://stocksnap.io/photo/XJ699E20JN
• https://stocksnap.io/photo/SHK42FOUFT
41

41. Image Resources
• https://i.imgur.com/1KshsLB.jpg
• https://stocksnap.io/photo/BKWZJHXR57
• http://www.flickr.com/photos/minxlj/3270996158/
• https://upload.wikimedia.org/wikipedia/commons/thumb/7/75/Internet1.jpg/745px-Internet1.jpg
42