Given at TechMeetup Edinburgh Aug 2011

1. Usability Testing –
a practical guide
August 2011
Chris Rourke
Managing Director
User Vision
@crourke @uservision
Please note- presentation contents copyright User Vision

2. What is Usability
Effectiveness – Can they reach their goals
 find what they are looking for
 do what they want to do? “The effectiveness, efficiency
and satisfaction with which
Efficiency – How fast
specified users achieve specified
 number of errors goals in particular
environments”
 amount of effort
ISO 13407 – User Centred
 number of steps? Design Process for Interactive Systems
Satisfaction – Was it a good/bad experience?
 Do it again?
 Recommend to others?
| 2

3. Another definition of Usability
Here is a development lifecycle definition, instead of a user outcome
based one:
 “Usability is an approach to product development that incorporates
direct user feedback throughout the development cycle in order to
reduce costs and create products and tools that meet user needs.”
Usability Professional’s Association
|| 3
3 Chris Rourke Managing Director User Vision

4. Usability Activities:
Potential Usability Activities during Development
Planning: Cost-benefit analysis, method selection, goal setting
User / Task / Environment profiles
‘Competitor’ usability benchmark
Scenario creation, Personas, release criteria, usability metrics
Style guide
Information architecture
Prototype design
Usability inspection
Usability testing
User feedback, site
modifications
Requirements Design Build Test Implement
Feedback Design Iteration
| 4

5. Usability Activities: Usability Testing
User Testing
 Involves picking important tasks and watching quietly as
people (usually struggle to) try to do them
 Test plan design, task scenarios and accurate subject
recruitment are all critical
It is NOT: “People will gladly waste a
million dollars on their fancy
 Link, load or code testing design and not spend $4000 to
see if it works”
 Online surveys or focus groups 1997, Jakob Nielsen
 Having friends view your pages on their computers
 As effective when tested by the same people who
|| 5
5
designed the site

6. Usability Activities: Usability Testing Essentials
 Recruit the right users who fit your profile
 Design the test to address typical tasks and those that
are most important for your success
 Decide your metrics to track e.g. success, difficulty
 Use the ‘think out loud’ method
 Observe and listen to subjects, don’t help them, but
probe them where necessary to get them to talk
 Pilot test if possible
|| 6
6

7. Exercise: Creating a Task Scenario
Create a Task Scenario for your site, noting:
 Scenarios should describe the task in a way that takes
some of the artificiality out of the test
A good scenario You and your partner are
interested in applying for a
 Is written in the user’s words, joint current account. You
not the product’s want to transfer the
 Is short and unambiguous balance of your existing
account and you want to
 Gives participants enough pay both your salaries
information to do task directly into this new
account. *
 Is directly linked to your most Apply for this account.
important tasks and concerns
|| 7
7

8. Usability Testing: Measuring Usability
Usability Impact Service Delivery Impact Severity of Change Impact
High (H) Prevents the user from Prevents a service being Changes needed in the
completing their task delivered, or causes Business process, or database
reputational damage, or structure, or application logic, or
excludes major user groups organisation culture or similar
or invalidates campaign major change
messages
Medium (M) Significantly delays or Service or message is only Web page structure, style,
confuses the user or causes partially delivered, or take- graphical, or similar
a significant error up is reduced presentation layer change
Low (L) Makes the user hesitate or Slightly degrades service Editorial content change
think or a minor error quality
Positive (P) Elegant feature that helps Elegant feature that helps No change required
the user users
| 8

9. Usability Activities: When to Usability Test
 Wireframe – Test the information architecture, labelling
and navigation hierarchies
 Early Stage Prototype – Test critical user tasks, identify
where task flows are not aligned with user’s mental map
 Late stage prototype – Test critical user tasks, evaluate
earlier fixes for improved performance and document new
issues
 Launch – Usability testing in production environment
|| 9
9

10. Usability Activities: Time vs. Impact on Design
Usability’s biggest impact is early in the
development process, due to the ability to
make changes at minimal resource cost
Impact
on
Design
Early Development Prototype Production
Time 10
|| 10
10

11. Usability Testing Early
Card sorting
Paper prototyping
|| 11
11

12. Usability Activities: Testing Facilities
We have sophisticated and flexible
testing suites, but simple labs are fine,
basic equipment includes:
Usability Lab
 Computer
 2 chairs
 Camcorder & tripod
Focus Group Room
|| 12
12
Interactive Lounge

13. Usability testing - its not just for the lab anymore
“People will gladly waste a million dollars on their fancy design and not spend $4000 to
see if it works”
Jakob Nielsen
Lab-based testing Paper prototype Eye tracking
testing
Remote moderated testing

14. Being remote
True Intent &
Online Survey Remote
Large
unmoderated
usability testing
Sample Size
Focus Group Usability Lab
Small
Remote card
sorting Market Research Full User Experience
(attitudes) (attitudes & behaviour)
Research type
Remote
unmoderated testing
Remote & True Intent
category
testing

15. Eye tracking testing
An eye tracker is a device for measuring eye
positions and eye movements
Allows testers to determine eye movement and
eye-fixation patterns of a person. Find out:
 which aspects of your product caught attention
 which get ignored.
The heat map is very useful in gathering insight
into a group of users
A gaze plot shows the individual gaze movements
and shows the order and path a user looked.
|| 15
15

16. Good Luck!!
Chris@uservision.co.uk
Twitter @crourke @uservision
London: 020 7917 1840
Edinburgh: 0131 225 0850
www.uservision.co.uk
| 16