The document discusses remote user testing as a method of user-centered design. It summarizes that remote user testing identifies areas of confusion and frustration for users as well as where users are unable to complete tasks, allowing problems to be addressed early when still inexpensive. It provides details on conducting remote user testing sessions, including necessary software, estimated time and costs for preparation, testing and follow up, and both advantages and disadvantages compared to in-person testing.

1. Evaluation for 2008 Remote User Testing Michael Gaigg March, 18th 2009

2. Agenda Show how User-centered Design helps teams to save: Time Money Aggravation from clients Usability testing early in the design process identifies: Areas of confusion and frustration Where the user can’t complete a task While it is still cheap and easy to mitigate these problems 03/18/2009 Remote User Testing 2

3. 03/18/2009 Remote User Testing 3 Agenda User-centered Design User Testing Output (Qualitative & Quantitative) Software & Usage Time Estimates & Cost Advantages & Disadvantages

4. Introduction to User-centered Design UCD is a design philosophy and a process in which the needs, wants, and limitations of the end user of an interface or document are given extensive attention at each stage of the design process. Its measures are: Effectiveness (accuracy and completeness to achieve goals) Efficiency (resources expended in relation to the accuracy and completeness) Satisfaction (comfort and acceptability of the work system to its users) 03/18/2009 Remote User Testing 4

5. User Testing is a UCD Method 03/18/2009 Remote User Testing 5

6. Overview User testing involves: A design A user (participant) An observer (facilitator). Observe user behavior & receive user feedback. Test tasks (not functions). 03/18/2009 Remote User Testing 6 User Design Observer

7. Qualitative Output Understand mental model of the user (navigation). Validate requirements and design assumptions (focus on important functionality early). Discover non-functional design mistakes and areas of improvement (unforeseen user requirements). Evaluate effectiveness of design elements. 03/18/2009 Remote User Testing 7

8. Quantitative Output Success rates / Error rates Task time Satisfaction ratings 03/18/2009 Remote User Testing 8 Task Success Task Time

9. User Testing Software TechSmithMorae Manager: $ 1,295.- Bundle: $ 1,495.- Morae Manager uses the collected data (observation markers and notes, video, keyboard and mouse inputs) to analyze and calculate task times, error quotes and other common measurements. 03/18/2009 Remote User Testing 9

10. Remote User Testing Software TechSmithUserVue $ 149.- / Month or $ 1,495. / Year Web-based app enabling a Facilitator to remotely observe (and record) a Participant using a phone line for communication. Multiple Observers can passively join the Session and share their observations with the Facilitator. 03/18/2009 Remote User Testing 10

11. The Session Setting up a session. Connecting (download of software bundle required). Calling participant. Starting the session. Viewing (and recording) the participant. Removing software bundle from computers of involved parties. 03/18/2009 Remote User Testing 11

12. Time Estimates & Cost Preparation (18 hours) User screening: 8 hours Task creation: 8 hours Environment: 2 hours User testing (10 hours) 2 hours per user 5 users per round Post-test (32 hours) Test report: 16 hours Implementation: 12 hours Communication: 4 hours 03/18/2009 Remote User Testing 12 Our time: 1 Round = 60 hours 2 Rounds = 110 hrs. 3 Rounds = 160 hrs. + User comps + Time additional observers

13. Advantages of remote UT Reduced (or no) travel time and expenses. Higher exposure through easy screen sharing (managers can sneak in easily). Actual user environment, familiar and comfortable. Possibly fewer drop-outs. 03/18/2009 Remote User Testing 13

14. Disadvantages of remote UT Facilitator not physically present (degree of separation can be challenging). Can't see facial expressions or non-verbal cues. Difficult to build rapport and trust. Difficult to control environment. Possibly technical difficulties (firewall, etc.). Setup and use of software or usability lab might be challenging and requires a liaison. 03/18/2009 Remote User Testing 14

15. Summary Increased usability Higher degree of client satisfaction Continued business = Higher revenues Lesser client aggravation (“This is completely unusable”) Assist PM during requirements and design Focus and prioritize on important functionality Identify unforeseen user requirements early! Reduced costs Training and service Viewer Help-Desk calls 03/18/2009 Remote User Testing 15