Usability Testing On A Digital Product
•
9 recomendaciones•3,173 vistas
Usability testing involves planning studies to test a digital product. Key steps in planning include defining goals and participants, designing tasks, scheduling tests, and determining testing methods. Tests can be conducted remotely or in-person. Moderated tests involve a moderator guiding participants through tasks while they think aloud. Unmoderated tests use automated tools to gather metrics from participants remotely. Findings are analyzed to identify usability issues and improve the product's design. Mobile testing requires adaptations for its form factor. Fitting research into agile development requires parallel or staggered sprints.
Recomendados
Recomendados
Más contenido relacionado
La actualidad más candente
La actualidad más candente (20)
Destacado
Destacado (6)
Similar a Usability Testing On A Digital Product
Similar a Usability Testing On A Digital Product (20)
Último
Último (20)
Usability Testing On A Digital Product
- 1. For a digital product usability testing World Usability day colombia 2014 Kyle Soucy @kylesoucy
- 2. Image source: facit digital What is Usability Testing?
- 4. How do you conduct a usability study?
- 5. Lets us begin our Journey...
- 6. Plan Study Define goals and objectives
- 7. What do you want to learn? How do teachers shop for books? Do they use these slider dots? Do they notice these arrows?
- 8. WhatYou Can Learn From A Usability Study
- 9. Do users understand how to complete a desired task?
- 10. Does the design prevent users from completing tasks more effectively?
- 11. Do users notice the buttons or links that you want them to click on?
- 12. Are users engaged in the content and understand it?
- 13. How long does it take for a user to complete a task?
- 14. Does the interface cause users undo frustration or confusion?
- 15. WhatYou Can’t Learn From A Usability Study
- 16. Opinions or other subjective data (We’re not testing hundreds or thousands of people)
- 17. The future (We can only report on what we directly observe)
- 18. Design preferences (You won’t get answers to design questions)
- 19. Why can’t I just use a focus group?
- 20. Qualitative vs. Quantitative Research • Qualitative Research - Data gathered directly by observing the user - Researcher can ask follow-up questions, probe on behavior - Analysis of data is not mathematical (Why? How? Compare...) • Quantitative Research - Data gathered indirectly through surveys, log files, support calls, etc. - Data can be averaged or statistically tested Source: Christian Rohrer Move, Inc. | Realor.com
- 21. Understanding The Domain And The Product
- 22. If you don’t know the product, you can’t test it!
- 23. Plan Study Define what you are testing
- 26. 1. Write a list of the most important tasks that users need to accomplish using the product. 2. Turn each task into a scenario. a) Provide some context. b) Supply needed information. c) Don’t give any clues or hints! 3. Decide on the order of tasks. a) Make the first task easier.
- 27. Bad Task: Buy a book about genealogy online.
- 28. Good Task: You need to get a book about genealogy that covers the reliability of different sources.You want this book in your hands by Thursday. Find and purchase this book.
- 29. What makes a good task: • You’re asking users to do something that they would actually do in real life. • Good scope. Not too broad, not too specific. • Has a clear end point. • Elicits action, not opinion. Don’t start with “how would you” or “where would you”
- 30. Good Task: You need to get a book about genealogy that covers the reliability of different sources.You want this book in your hands by Thursday. Find and purchase this book.Find and purchase this book. ✓ Has a clear end point ✓ Elicits action, not opinion ✓ Good scope
- 31. Your Turn... 1. Take out a sheet of paper (or your computer, phone, or iPad) 2. Write down the most important task your users perform on your product. 3. Turn the task into a scenario. a) Provide some context. b) Supply needed information. c) Don’t give any clues or hints!
- 32. Other things to consider...
- 33. Restrictions: Consider whether or not you want to place restrictions on how participants try and accomplish the tasks (e.g.“don’t use search” or “stay on the website”).
- 34. Printing: If you’re conducting an in-person test, you may want to print out the scenarios so the participant can refer back to it.
- 35. Time-Fillers: Create more tasks than you think you have time to complete just incase someone finishes early. Every minute with a participant is an opportunity to learn something.
- 36. Visual Aids: If you’re worried that revealing the name of something will bias the results, consider showing the participant a picture instead.
- 37. Visual Aid Example: A friend of yours uses on of these to make baby food, so you’d like to buy one...
- 38. Alternating Tasks/Prototypes: Depending on what you’re testing, you may need to alternate tasks. Participants are learning the interface as they go, so the last task may always be the easiest to accomplish unless you alternate them.
- 39. Interview-based vs. Predefined Tasks
- 40. Interview-based Tasks 1. Interview participant about how they use the product. 2. Use the answers given in the interview to create tasks on the fly. When in doubt just ask,“what was the last thing you did when you used this product? Can you show me?”
- 43. Demographics
- 44. Cover all user groups - Prospective as well as existing users
- 45. Be specific and get buy-in from stakeholders!
- 47. Do ItYourself
- 48. Hire a recruiter Warning: It’s never this easy!
- 50. How many to test?
- 52. Budget Allocation What NOT to do... 1 Round of Testing
- 53. Budget Allocation What you SHOULD do... Round 1 Round 2 Round 3 Design Iteration Design Iteration
- 55. Plan Study define testing method
- 57. Remote Testing Pros & Cons Participants stay in their native environment. Accessible to a larger & more diverse pool of participants. Recruiting is easier. Less no shows & easier last minute replacements. Opportunity to gather ethnographic data. Cost & time savings. No travel & lab rental fees. Easier for observers to “attend” a session. Risk of compromising data because the participant is using a foreign PC is removed. Pros: Can’t see the user’s facial expressions. Can’t see if the user is using peripheral devices. Dependent on an Internet connection. Can only recruit participants with a high- speed Internet connection. Cons:
- 58. Remote Testing Tools • GoToMeeting • WebEx • Adobe Connect • Join.me • Many more...
- 59. Remote Testing Tips • If using a cordless phone, have multiple handsets charged and ready • Use a phone recording controller • Use as few tools as possible • Back up, back up, back up! • USB Modem • Screen sharing backup
- 60. • Create a test meeting to make sure the participant can connect beforehand • Instruct participants to log on early and use a landline (noVOIP) • Turn off entry/exit chimes for conference call • Allow only one observer to chat to you • Use a separate chat/IM client
- 62. What is Unmoderated Remote Testing?
- 64. Unmoderated Testing Tools (There are a lot!)
- 65. We Won’t Talk About...
- 66. We Will Talk About...
- 67. Has Anyone Here Used These Tools Before?
- 68. WhatYou Can Learn (Depends on the tool and how you use it!)
- 69. Web Analytics Image source: http://www.hiero.com/web-analytics.html
- 70. Task Completion Rates Image source: Loop11
- 71. Number of Clicks/ Clickstream Image source: Userlytics
- 72. Time on Task/Page Image source: http://blog.usabilla.com/underdogs-beat-expedia-in-usability-test/
- 73. Satisfaction Ratings/ Opinion Rankings Image source: http://wikidoc.org/index.php/Pain It is simple to use: Strongly Disagree ---1---2---3---4---5---6---7 Strongly Agree
- 74. True Intent Image source: http://ethnio.com/images/live_screener_example_page1.jpg
- 75. In-context Feedback Image source: Loop11 “...I’m only offered a search by model number. How ridiculous.” “...I would like to see the products availability and delivery/ shipping information listed.”
- 76. How Actionable is the Data? (Depends on the questions you ask!)
- 78. Process 1. Define the study 2. Recruit participants 3. Launch test and send email invites 4. Analyze results 3. Run a Pilot test! > 4 / 5 /
- 79. Benefits
- 80. Test hundreds of people simultaneously while keeping them in their own natural environment.
- 81. Test multiple websites simultaneously.
- 82. Reduced costs... depending on the tool you use.
- 83. Great way to plant the seed of UCD methodologies and introduce usability testing into a company. UX
- 84. Fewer logistics to manage and no need to worry about no-shows and getting last-minute replacements.
- 85. Fast Results
- 86. Drawbacks
- 87. Nothing beats watching participants in real time and being able to ask probing questions about what they are doing as it’s happening.
- 88. Some participants may only be interested in earning the honorarium you’ve provided as an incentive. $$$
- 89. You cannot conduct interview-based tasks.
- 90. It is simple to use: Strongly Disagree ---1---2---3---4---5---6---7 Strongly Agree Image source: http://www.hiero.com/web-analytics.html What participants report on surveys can be very different in comparison to what they actually do.
- 91. Image source: http://www.hiero.com/web-analytics.html It’s possible for participants to think they’ve successfully completed a task when they haven’t.
- 92. Image source: http://www.hiero.com/web-analytics.html Does it matter if it took someone longer to complete a task?
- 93. When to Conduct Automated Testing?
- 94. Not a replacement for moderated testing or research. Best used to compliment your qualitative research.
- 95. Unmoderated Testing Tools (A closer look...)
- 96. Automated Testing Tools with Recruiting Panels
- 97. The danger... I wonder how a real customer would actually use this website? Panelist
- 100. There are no shortcuts to conducting good research.
- 101. Automated Testing Tools with Other Recruiting Options
- 102. Loop11
- 103. Treejack
- 105. Chalkmark
- 106. Chalkmark Results
- 107. Automated Tools with Audio andVideo
- 111. Plan Study invite participants and observers
- 112. Key to Success… Attendance of product decision makers during testing!
- 113. Example Remote Participant Invite
- 114. Set Up Study
- 115. Usability “Labs”
- 116. Image source: facit digital Source: Kent State University Vs.
- 117. Ideal “Lab-Less” Set Up
- 118. Should observers be in the room?
- 119. Video Recording Options • Techsmith Morae - $1,995 USD • Techsmith Camtasia - $99 USD • Silverback (Mac only) - v2 Free • Online Meeting Built-in Recorders
- 123. Moderate Study
- 124. Greet Participant and Give Introduction
- 130. Think Aloud Techniques Concurrent (CTA) Retrospective (RTA) Vs.
- 132. Variation of RTA... Show the participant their video after they’re done testing.
- 133. Reminder Prompting • So...? • So, what are you thinking? • What are you seeing here? • What are you looking at? • Can you tell me what you think is happening here? Source: Moderating Usability Testing by Joe Dumas and Beth Loring
- 134. Probing Questions • Is that what you expected? • What would you do next? • What did you think about that task? • Help me to understand more about... Source: Moderating Usability Testing by Joe Dumas and Beth Loring
- 135. Encouraging Statements • This feedback is really helpful. • Thank you for thinking out loud. • That’s good to know. • You’re doing fine. Source: Moderating Usability Testing by Joe Dumas and Beth Loring
- 136. Remain Neutral • When in doubt turn the question around: • “I don’t know what do you think?” • “What would you do if I wasn’t here? • Keep acknowledgements limited to:“ah”, “Mm Hmm”,“Ok”, etc. Source: Moderating Usability Testing by Joe Dumas and Beth Loring and Rocket Surgery Made Easy by Steve Krug
- 138. Note-taking
- 141. Formal Usability Testing Report Findings categorized by screens or pages
- 142. When, What, Who, Where, ! and Why Statement! 3-4 ! Positive Findings! 3-4 ! Negative Findings!
- 146. Video Clip Example
- 149. “...the mobile web will be bigger than desktop Internet use by 2015.” - Morgan Stanley Study Source: http://mashable.com/2010/04/13/mobile-web-stats/Source: http://mashable.com/2010/04/13/mobile-web-stats/
- 150. Image source: http://3.bp.blogspot.com/-Oh6rZE2fG1k/Uct4T4KhE0I/AAAAAAAAARo/3c_HFDkMwAE/s1600/girls-on-their-phone.jpg Mobile matters!
- 151. Mobile In-person Usability Testing Image source: http://unmatchedstyle.com/news/usability-testing-on-mobile-devices.php
- 155. Testing on paper prototypes Image source: http://www.90percentofeverything.com/2010/05/07/quick-tip-make-your-own-iphone-usability-testing-sled-for-5/
- 156. What about remote mobile usability testing?
- 159. Current Navigation Tabbed Navigation Hidden Navigation
- 161. What about remote mobile usability testing? unmoderated >
- 169. Fitting Research Into An Agile Process
- 170. Waterfall Analysis Design Coding Testing Analysis Design Coding Testing Agile Analysis Design Coding Testing Analysis Design Coding Testing 1-4 week sprints
- 171. Parallel Sprints #1
- 172. Sprint 1Dev: UX: Sprint 2 Sprint 3 Release Sprint 1 Sprint 2 Sprint 3
- 173. Staggered Sprints #2
- 174. Source: http://www.upassoc.org/upa_publications/jus/2007may/agile-ucd.pdf Tip: Choose a feature to develop first that has a lengthy dev sprint.
- 175. Testing On A Small Budget
- 176. • Conduct your own recruiting • Don’t offer large honorariums • Don’t use a lab • Don’t video record • Test less people • Use surrogate users or just conduct cognitive walkthroughs • Conduct unmoderated studies • Don’t write formal reports
- 177. Learn more... • Book: Interviewing Users by Steve Portigal • Book: Moderating Usability Tests by Joe Dumas & Beth Loring • Book: Paper Prototyping (chapter on task design) by Caroline Snyder • Book: Rocket Surgery Made Easy by Steve Krug • Book:The Moderator’s Survival Guide by Donna Tedesco & Fiona Tranquada
- 178. • Article: Seven Common Usability Testing Mistakes by Jared Spool http://www.uie.com/articles/usability_testing_mistakes/ • Article: Interview-Based Tasks: Learning from Leonardo DiCaprio by Jared Spool http://www.uie.com/articles/interview_based_tasks/ • Article: Mobile Testing Toolbox: Part 1&2 by Amber Derosa http://uxmag.com/articles/mobile-testing-toolbox-part-1 • Article: How To Conduct A Usability Test On A Mobile Device by Jeff Sauro http://www.measuringusability.com/blog/mobile-usability-test.php • Article:Why And How To SegmentYour Customers by Jeff Sauro http://www.measuringu.com/blog/segment-customers.php