This is the presentation Mobtest's CTO and founder Dirk de Kok gave at the Renaissance iOS app makers' conference. He talked about user testing iOS apps, including concept testing, usability testing, beta testing, QA testing. See also https://github.com/mobtest/renaissance.io/
5. Test early, before launch
‣ if high quality, app might get featured
‣ initial ratings drive subsequent downloads
‣ reaching a top 25 normally happens
around launch
‣ critical updates take still 1-2 weeks
because of Apple review process
6. Concept testing
Goal
validate app idea with target market
How
‣ find respondents
‣ do interview with factual questions
‣ show mockup or prototype
10. In person usability testing
‣ meet testers in office or coffee shop
‣ give tasks with only end result
‣ let user think out loud
‣ observe usage, facial expression, posture
11.
12. Record usability test
‣ easier to remember
‣ easier to share with your team
can’t do it on iOS device itself
15. Remote usability testing
‣ recruit larger group of users
‣ distribute ad hoc build
‣ give tasks with only end result
‣ track task completion in app and
ask questions afterwards by survey
16. What to look for
‣ does it work as user expects?
‣ discoverability
‣ areas of confusion
‣ positive experience
17. Beta testing
Goals
‣ final feedback from users
‣ integration test with real data
‣ get early adopters enthusiastic about
app (reviews and ratings!)
18. Beta testing
How
‣ Recruit testers via own network or service
‣ Get their UDID, create ad hoc build and
distribute
‣ Reward feedback
21. QA testing
Goals
‣ make sure app works technically fine
How
‣ get a good test engineer
‣ write a detailed plan
‣ execute plan
22. QA methods
‣ structured testing with predefined test
scripts with expected outcomes
‣ “go break it” exploratory testing
23. What to look for
‣ Hardware models x iOS versions
‣ Connectivity (wifi, cell, intermittent)
‣ Gestures, moves, tilting and multitouch
‣ Location/geofencing
24. Live testing
‣ A/B testing
‣ Monitoring usage patterns
‣ Crash log analysis
‣ Feedback options for user