Más contenido relacionado Similar a Usability Testing: Targeting Mobile Devices (20) Usability Testing: Targeting Mobile Devices2. Introduction to User Testing
Implementing Quick & Effective User Testing
Wayne Pau, Emerging Technologies SAP
Feb 4, 2013 – MEIC’s Mobile Accelerator Programme – Workshop #6
3. My Apology Slide
“If I'm curt with you it's because time is a
factor. I think fast, I talk fast and I need you
guys to act fast if you wanna get out of this.
So, pretty please... with sugar on top…”
- Winston Wolfe (Pulp Fiction)
© 2012 SAP AG. All rights reserved.
Internal
3
5. Buy this book…
•
Steve Krug
•
2nd Book
•
Cheap & Dirty Usability Testing
•
161 pages
© 2012 SAP AG. All rights reserved.
Internal
5
6. … then buy this next… (and read it first)
•
Steve Krug
•
1st Book
•
Web-Focused
•
216 pages
© 2012 SAP AG. All rights reserved.
Internal
6
8. Steve Krug – Recorded Sample
http://www.rocketsurgerymadeeasy.com/
http://www.youtube.com/watch?feature=player_embedded&v=Q
ckIzHC99Xc
All of Chapter 2 in Rocket Surgery Made Easy
© 2012 SAP AG. All rights reserved.
Internal
8
10. What User Testing is Not…
•
Not statistically relevant Quantitative Research
•
Not a Market Focus Group
•
Not for “Baking-Off” UI Choices
•
Not mining for new UI Designs/Patterns
•
Not Contextual Design or “Voice of God” interviews
© 2012 SAP AG. All rights reserved.
Internal
10
11. Why do User/Usability Testing
1.
No Product is perfect. Everything can be Improved.
2.
Serious problems are easy to Spot. Encountered by
most Users.
3.
User-Centered approach makes for better products.
© 2012 SAP AG. All rights reserved.
Internal
11
12. How to do User Testing
•
One Morning per Month
•
Find 3x “Participants for 1 Hr
session
•
Find 3x Tasks, expand into
Scenarios
•
Record/Broadcast
•
Debrief in Afternoon
© 2012 SAP AG. All rights reserved.
Internal
12
14. # of Subjects
•
3 or 4 for 2+ rounds
•
3 or 4 allows testing
•
AM, debrief in PM
•
2x Rounds w/ 3 > 1x round with 6
•
Recruit loosely, grade on a curve
© 2012 SAP AG. All rights reserved.
Internal
14
15. Recruitment - Exceptions
•
If you site is only used a particular population/relevence
•
If you subject has conflicting or divergent interests
•
If you app is esoteric and needs domain expertise
© 2012 SAP AG. All rights reserved.
Internal
15
16. Recruitment – Good Ides
•
Offer good Incentive (Gift Cert)
•
Keep invitation simple
•
Don’t offer too much info
•
Ask Relatives/Friends/Family
•
Everyone was Newbie once
© 2012 SAP AG. All rights reserved.
Internal
16
20. Welcome – Part 1
Hi,
session.
. My name is Wayne Pau, and I'm going to be walking you through this
You probably already know, but let me explain why we've asked you to come here
today: We're testing a web site that we're working on to see what it's like for actual
people to use it.
I want to make it clear right away that we're testing the site, not you. You can't do
anything wrong here. In fact, this is probably the one place today where you don't
have to worry about making mistakes.
© 2012 SAP AG. All rights reserved.
Internal
20
21. Welcome – Part 2
We want to hear exactly what you think, so please don't worry that you're going to
hurt our feelings. We want to improve it, so we need to know honestly what you think.
As we go along, I'm going to ask you to think out loud, to tell me what's going through
your mind. This will help us.
If you have questions, just ask. I may not be able to answer them right away, since
we're interested in how people do when they don't have someone sitting next to
them, but I will try to answer any questions you still have when we're done.
We have a lot to do, and I'm going to try to keep us moving, but we'll try to make sure
that it's fun, too.
© 2012 SAP AG. All rights reserved.
Internal
21
22. Welcome – Part 3
You may have noticed the camera. With your permission, we're going to videotape
the computer screen and what you have to say. The video will be used only to help us
figure out how to improve the site, and it won't be seen by anyone except the people
working on the project. It also helps me, because I don't have to take as many notes.
There are also some people watching the video in another room.
If you would, I'm going to ask you to sign something for us. It simply says that we
have your permission to tape you, but that it will only be seen by the people working
on the project. It also says that you won't talk to anybody about what we're showing
you today, since it hasn't been made public yet.
Do you have any questions before we begin?
© 2012 SAP AG. All rights reserved.
Internal
22
23. Welcome - Tips
•
Stick to the script, stick to the script, stick to the script…
•
Try to remain neutral & calm
•
Test Participant (Not Subject. They are not being tested)
© 2012 SAP AG. All rights reserved.
Internal
23
25. Questions – Part 1
Before we look at the site, I'd like to ask you just a few quick questions. First, what's
your occupation?
Good. Now, roughly how many hours a week would you say you spend using the
Internet, including email?
How do you spend that time? In a typical day, for instance, tell me what you do, at
work and at home.
Do you have any favorite Web sites?
© 2012 SAP AG. All rights reserved.
Internal
25
26. Questions – Part 2
Now, finally, have you bought anything on the Internet? How do you feel about buying
things on the Internet?
And what have you bought?
OK, great. We're done with the questions, and we can start looking at things.
© 2012 SAP AG. All rights reserved.
Internal
26
27. Questions - Tips
•
Build some rapport with your Participant
•
Assist in your “Grade on a Curve”
•
Get Participant Talking…
© 2012 SAP AG. All rights reserved.
Internal
27
29. Acclimatizing
OK, great. We’re done with the questions, and can start looking at things.
First I’m going to ask you to look at this page and tell me what you make
of it: what strikes you about it, whose site you think it is, what you can do
here, and what you can do here, and what it’s for. Just look around and
do a little narrative.
You can scroll if you want to, but don’t click on anything yet.
© 2012 SAP AG. All rights reserved.
Internal
29
30. Acclimatizing - Tips
•
Get them used to listening to instructions
•
Get them used to verbalizing
•
Get them use to “thinking out loud”
© 2012 SAP AG. All rights reserved.
Internal
30
32. Tasks
Now I’m going to ask you to try doing some specific tasks. I’m going to
read each one out loud and give you a printed copy.
I’m going to ask you to do these task without using Search. We’ll learn a
lot more about how well the site works that way.
And again, as much as possible, it will help us if you can try to think out
loud as you go along.
© 2012 SAP AG. All rights reserved.
Internal
32
34. Tasks #1 – FoodNetwork.ca
Assume you are big fan of Anna Olsen and love all her shows. You are
going to be home tomorrow all day and would like to determine when she
is on first on FoodNetwork tomorrow. If possible, find out:
•
What TV series it is (she has multiple)
•
What is the Episode About
•
What recipes will be shown (in case you want follow along)
© 2012 SAP AG. All rights reserved.
Internal
34
35. Tasks #1 – FoodNetwork.ca
Assume you are big fan of Anna Olsen and love all her shows. You are going to be
home tomorrow all day and would like to determine when she is on first on
FoodNetwork tomorrow. If possible, find out:
•
What TV series it is (she has multiple)
•
What is the Episode About
•
What recipes will be shown (in case you want follow along)
© 2012 SAP AG. All rights reserved.
Internal
35
36. Tasks #2 – FoodNetwork.ca
You’ve recently come back from a short vacation to New Orleans. You had a great
time and especially loved the Cajun food. Check FoodNetwork.ca for possible main
course dinner dishes that you can try at home. Assume you will looking at following
criteria to help you narrow it down:
•
How it was rated
•
You like Fish over Red Meat
•
You really Blacken Anything
© 2012 SAP AG. All rights reserved.
Internal
36
37. Tasks #3 – FoodNetwork.ca
You’ve had a rough week and are looking at cheering yourself up with
some food. However you still want to eat healthy and really don’t like
nuts (actually you could be allergic). Can you find a good, quick recipe
that would entice you?
© 2012 SAP AG. All rights reserved.
Internal
37
38. Tasks - Tips
•
When in doubt *SAY NOTHING*
•
Use every permutation of “uh huh” or “OK” possible
•
Aim for “somewhat” pleased demeanor
•
Deflect ( “What would you do if I was not here?”…)
•
Mine leading non-verbal clues (“Is that what you expected?”)
© 2012 SAP AG. All rights reserved.
Internal
38
40. Probing
Thanks, that was very helpful.
If you’ll excuse me for a minute, I’m just about to see if the people on the
team have any follow-up questions they’d like me to ask you.
© 2012 SAP AG. All rights reserved.
Internal
40
41. Tasks - Tips
•
Relax “hands-off” approach
•
Ask “Why”?
•
Avoid asking for “solutions” or “redesigning on the spot”
© 2012 SAP AG. All rights reserved.
Internal
41
43. Wrap-up & Thank-you
Do you have any questions for me, now that we’re done?
Thanks. That was exactly what we need, it’s been very helpful.
© 2012 SAP AG. All rights reserved.
Internal
43
45. Real Testing World
•
Review results right away
•
Triage the information
•
Find a Fix (or approach)
•
Do the *LEAST* you can do to
fix the problem
© 2012 SAP AG. All rights reserved.
Internal
45
46. 7 Reasons to Tweak (vs. Rebuild)
1.
Cheap & Cost effective
2.
Less work & resource intensive
3.
Less Stressful
4.
Implemented Quicker
5.
Small & more likely to be scheduled
6.
Less Risky than Overhaul
7.
Might know the issue, but not the “perfect” solution
© 2012 SAP AG. All rights reserved.
Internal
46
47. Wayne’s Issue Triage
1.
Is it MVP? Can we “Dump” the culprit?
2.
Is there a Small Tweak we can do?
3.
Was it effective enough? (Do we need to Tweak the Tweak?)
4.
Did we Breaking anything more Important?
5.
Is it a new feature? DON’T ADD ANYTHING!!!
© 2012 SAP AG. All rights reserved.
Internal
47
48. Something is Better than Nothing…
“Perfect is the Enemy of Good”
– Voltaire
“A good plan executed today is better than a perfect plan executed tomorrow.”
-
General George Patton
POGE – Principle of Good Enough
© 2012 SAP AG. All rights reserved.
Internal
48
Notas del editor Steve Krug’s analogy is when giving friends a tour of your hometown, you learn new things too. Steve Krug’s analogy is when giving friends a tour of your hometown, you learn new things too. Steve Krug’s analogy is when giving friends a tour of your hometown, you learn new things too. Steve Krug’s analogy is when giving friends a tour of your hometown, you learn new things too. Steve Krug’s analogy is when giving friends a tour of your hometown, you learn new things too. Steve Krug’s analogy is when giving friends a tour of your hometown, you learn new things too. Steve Krug’s analogy is when giving friends a tour of your hometown, you learn new things too. Jakob Nielsen’s March 2000 Alertbox column “Why You Only Need to Test with 5 Users”at www.useit.com for a good discussion of the topic. Jakob Nielsen’s March 2000 Alertbox column “Why You Only Need to Test with 5 Users”at www.useit.com for a good discussion of the topic. SAP Palo Alto – Building #3 has preferred parking for Usability Testing partipants Steve Krug’s analogy is when giving friends a tour of your hometown, you learn new things too. Steve Krug’s analogy is when giving friends a tour of your hometown, you learn new things too. Steve Krug’s analogy is when giving friends a tour of your hometown, you learn new things too. Steve Krug’s analogy is when giving friends a tour of your hometown, you learn new things too. Steve Krug’s analogy is when giving friends a tour of your hometown, you learn new things too. Steve Krug’s analogy is when giving friends a tour of your hometown, you learn new things too. Steve Krug’s analogy is when giving friends a tour of your hometown, you learn new things too. Steve Krug’s analogy is when giving friends a tour of your hometown, you learn new things too.