Trying to get out of the building but don't know where to go, how to get there, or what to do once you are (and when you get back)? This course is for you.
We'll cover the who, what, when, where, why, and how of field research:
Why field research instead of (or in addition to) other methods
Some of the common techniques and how to perform them
Tips and tricks for your fieldwork
How to analyze all that great data when you get back to the office
The best ways to report your findings to best effect change
This course will include a couple of hours of actual field research!
1. UX Fieldwork
22 June 2015 #UXPA2015 @dgcooley
1
Photo via Flickr user Matthew Fern
2. Agenda
• Introductions
• Why Field Research?
• An Introduction to Field Research Techniques
• Field Research vs. Other Methods
• Conducting Field Research
• Reporting Field Research Findings
• Some Examples from the Real World
• LUNCH!
• Activity: Poolside Field Research
• Activity: Analyze & Present Field Research Findings
• Review, Wrap-up, and Questions
22 June 2015 #UXPA2015 @dgcooley
2
3. Danielle Gobert Cooley
22 June 2015 #UXPA2015 @dgcooley
3
danielle@dgcooley.com
@dgcooley
17 years doing UX work
BE, Biomedical and Electrical Engineering – Vanderbilt University
MS, Human Factors in Information Design – Bentley University
http://linkedin.com/in/dgcooley
Selected Work
33. Photo via Flickr user C.P.Storm
It’s definitely time to get out of the building.
22 June 2015 #UXPA2015 @dgcooley
33
@dgcooley
34. Fieldwork is great for..
• Really understanding your end users.
– Who are they?
– What are their actual goals?
– What keeps them awake at night?
– What’s the best and worst part of their jobs?
• Really understanding what environment your product needs to thrive
in.
– Is it noisy or quiet?
– Very bright or dim?
– Full of distractions, or with very few?
22 June 2015 #UXPA2015 @dgcooley
34
Good PERSONAS
are the result of
good fieldwork.
35. Types of Field Research
22 June 2015 #UXPA2015 @dgcooley
35
Formal
Contextual
Inquiry
Remote
Contextual
Inquiry
On-site
usability
testing
Follow Me
Home
And many
more!
36. Formal Contextual Inquiry
• Part 1 – The conventional interview
• Part 2 – The transition
• Part 3 – The contextual interview proper
• Part 4 – The wrap-up
22 June 2015 #UXPA2015 @dgcooley
36
1998
2005
38. On-Site Usability Testing
• Conducted at
participant’s
workstation
• Provides some
environmental
context
22 June 2015 #UXPA2015 @dgcooley
38
Photo credit unknown
39. 22 June 2015 #UXPA2015 @dgcooley
39
Photo via Flickr user stevenharris
How does fieldwork measure up?
40. Field Research vs. Focus Groups
• Identify “hot buttons & sales points”
• Gathers opinions and self-reported behaviors
• Not contextual or observational
22 June 2015 #UXPA2015 @dgcooley
40
http://www.nngroup.com/articles/first-rule-of-usability-dont-listen-to-users/
41. Field Research vs. Surveys
• Great for quick, quantitative feedback
• Very inexpensive
• Only get opinions and self-reported behaviors
• Tough to focus on details; only effective for broad issues
22 June 2015 #UXPA2015 @dgcooley
41
42. Field Research vs. Formal Lab Testing
• Useful for both quantitative and qualitative results
• Controlled environment
• Easy recording and data collection
• Labs are expensive to build or rent
• Traditionally conducted after-the-fact
22 June 2015 #UXPA2015 @dgcooley
42
http://www.hq.nasa.gov/pao/portal/usability/overview/overview_where.htm
http://www.sea.siemens.com/software/product/uslab.html
43. Field Research vs. Journaling
• Very inexpensive
• Self-reported information, but without time delay
• Considerable after-the-fact data parsing
• Compliance can be difficult
22 June 2015 #UXPA2015 @dgcooley
43
http://www.sftherapy.com/journaling.html
44. Field Research vs. Other Methods
Qualitative Quantitative Inexpensive
Easy to
Implement
Observational Contextual
Focus
Groups
?
Surveys
Lab Testing
Journaling
?
Field
Studies
22 June 2015 #UXPA2015 @dgcooley
44
45. Fieldwork can be helpful
at any stage in the
product lifecycle.
Photo via Flickr user Christopher Sessums 45
22 June 2015 #UXPA2015 @dgcooley
46. 22 June 2015 #UXPA2015 @dgcooley
46
Begin to understand
who your users are
and what will make
their lives better.
Go observe more
specific details
about their
workflows and
environments. Do some or all
of your testing
in the field.
(Still not a bad time
to visit your users in
their workspaces.
You can always
learn something.)
(Yep. Still learning!)
47. How-to
1. Plan your study –
What do you
want to learn?
2. Identify
representative
end users.
3. GO TO WHERE
THEY ARE.
4. Learn all the
things.
47
@dgcooley
Prepare,
prepare,
prepare!
22 June 2015 #UXPA2015 @dgcooley
48. 1. Plan Your Study
• What do you
want to learn?
• Are you defining
requirements?
• Do you need to
learn about a
specific
feature?
• Just trying to
understand your
people?
22 June 2015 #UXPA2015 @dgcooley
48
Photo via Flickr user brand0con
49. 2. Identify representative end users.
• Third-party recruiting firms or market research facilities.
• Customer lists
• Friends & family
• Customer intercepts
• Man-on-the street
22 June 2015 #UXPA2015 @dgcooley
49
53. 53
22 June 2015 #UXPA2015 @dgcooley
Photo via Etsy seller aaasupply
54. Find out what else you’ll need to do there
• Photo ID?
• License plate number?
• Dress codes?
– No shoes allowed? Steel-toed boots? Shoe covers provided?
– Eye or ear protection?
– Will people see your underwear?
• Technical requirements or limitations?
– Laptop virus scanning?
– Phones allowed?
– Can you count on WiFi?
22 June 2015 #UXPA2015 @dgcooley
54
55. Figure out where the place is
• Does Google maps know?
– Do they REALLY know?
• How long will it take to get there?
– Really? Even in traffic?
• Where will you park?
– How far is that from the venue?
– Does that change how you pack all
of your gear?
• Ask for directions anyway.
• Bring a PRINTED backup of your
route, and WRITE DOWN the
participant’s phone number so you
can call if there are issues.
22 June 2015 #UXPA2015 @dgcooley
55
56. 3. Go to where they are.
22 June 2015 #UXPA2015 @dgcooley
56
57. The Usual Rules Apply
• Get written consent for anything you do
• Ask open-ended questions
• Minimize bias
• Provide reassurance to
the participant
22 June 2015 #UXPA2015 @dgcooley
57
61. Homemade Sales Binders
22 June 2015 #UXPA2015 @dgcooley
61
Several FCs
had homemade
sales binders
like this one.
62. Cheat Sheets
22 June 2015 #UXPA2015 @dgcooley
62
“cheat sheet” of BETA codes.
The FAs “rarely use” BETA and
need a reference when dealing
with the home office. (B22)
One FC kept an extensive list of
home office personnel he had
spoken to about various matters.
Each folder includes people’s
names, their extension, and
comments about how helpful or
unhelpful they were when the FC
spoke with them. (B8)
63. Strategic Seating of Clients
22 June 2015 #UXPA2015 @dgcooley
63
The FCs care about projecting an image of absolute excellence. Clients visiting
one office (B25) are seated in a specific location so they can see…
…that, no
matter what,
the market
goes up.
… the FC’s
many
diplomas,
certifications,
and honors.
… an organized set of
files containing the
necessary forms for
various products.
… any hustle and
bustle in the office
to convey that
we’re busily
working on
earning people
money.
64. Workflow Processes
22 June 2015 #UXPA2015 @dgcooley
64
(HO3)The follow-up process for problem
accounts/issues is very cumbersome and
paper-intensive. There isn't an efficient
systematic process for reconciling issues.
The current process relies on the <company>
resource to "remember" to go through these
folders with some frequency, send follow-up
wires (which may or may not be
acknowledged in a timely manner), and try
to reconcile the issue.
Note the names on the folders:
•Pending OK to Trades (Accts in date read
order, need them by Acct # order)
•*IMPORTANT* Daily Immediate Follow Up
•Need Sign-off from Money Manager (Accts in
date read order, need them by Acct # order)
•Problem Follow up
Two items in the far background:
•MGR Index - This hardcopy of about 50+
pages is held together by a binder clip
and contains information/instructions
from the corporate intranet, along with
personal emails and other artifacts on how to
handle certain situations. The emails and
artifacts are peppered with hand-written
notes.
•Select Advisor Account Checklist
65. Checklists & Instructions
22 June 2015 #UXPA2015 @dgcooley
65
3 different teams showed us
their homemade instructions
or checklists for setting up
fee-based accounts.
68. 22 June 2015 #UXPA2015 @dgcooley
68
bit.ly/garconapp
69. Affinity Diagrams
• Not just for
brainstorming
• Useful to organize your
own thoughts or to
aggregate findings from
multiple interviewers
22 June 2015 #UXPA2015 @dgcooley
69
http://www.adamatorres.com/gallery-project/?page_id=106
Usabilitynet.org/tools/affinity.htm
70. Analyze Field Research Findings
22 June 2015 #UXPA2015 @dgcooley
70
Analysis
Photo via Flickr user Jacob Bøtter
71. Other Tools
22 June 2015 #UXPA2015 @dgcooley
71
Microsoft OneNote
mind mapping
72. Reporting Field Research Findings
• The usual rules apply
– Provide a concise executive summary
– Explain what you did
– Share both positive &
negative findings
– Provide visual explanations
whenever possible
– Protect your participants’
anonymity
– Include your artifacts!
22 June 2015 #UXPA2015 @dgcooley
72
73. Reporting Field Research Findings
• Sample Visual Explanations
22 June 2015 #UXPA2015 @dgcooley
73
http://www.xplane.com
74. Recap & Additional Resources
• Field research is an important tool that provides advantages over
other user research techniques.
• Logistical challenges exist (but are worth it).
• It’s FUN!
22 June 2015 #UXPA2015 @dgcooley
74
http://www.portigal.com/series/WarStories/
Notas del editor
The participant room at Mediabarn in Arlington, VA, USA.
Participant room at Mediabarn in Arlington, VA, USA.
Participant room in the recently decommissioned usability lab at Fidelity Investments in Boston, MA, USA.
Participant room in the usability lab at Webcredible, a UX agency in London, UK.
People’s real workspaces don’t look like usability labs.
Ok. *Some* real workspaces kindof look like usability labs.
My home office today, including an IKEA-hacked standing desk.
Multiple monitor setups in varying arrangements are increasingly prevalent in today’s office workspaces.
This person has multiple monitors and multiple computers.
Improvised monitor stands emerged as a theme.
LOTS of improvised monitor stands. (Note 4 screens and 3 machines, here.)
How many of you have feather boas in your labs?
Moreover – what’s *not* in this photo? That’s right. The computer. When I asked people for photos of their “workspace,” this person (an elementary school administrator) didn’t even think to include their computer in the photo – a reflection of just how much their work *isn’t* about its digital components.
The computer is almost entirely out of frame, on the right.
People’s real workspaces don’t look like usability labs.
Humane Society field agent’s workspace.
A Nashville, TN police cruiser.
A New Jersey sportswriter covering the Beijing Olympics.
The same New Jersey sportswriter, his colleagues, and the crew after the 2012 Super Bowl in Indianapolis.
An OR nurse in Florida.
A mountain rescuer. “This is me and Verena, one of our nurses, with a bonked out diabetic hiker waiting for a litter to arrive.”
“Me, shortly after witnessing the guy at my feet pitch 40’ off a cliff sustaining multiple injuries in a remote wilderness climbing area, trying to coordinate what would eventually become a massive rescue effort culminating in a helicopter hoist.”
“Me, shortly after witnessing the guy at my feet pitch 40’ off a cliff sustaining multiple injuries in a remote wilderness climbing area, trying to coordinate what would eventually become a massive rescue effort culminating in a helicopter hoist.”
And now let’s look at some REAL workspaces.
Part 1 – The conventional Interview
* Friendly Q&A, introducing yourself and getting to know the participant
* Explain your role; put participant at ease
* Get signatures on paperwork for recordings/photos
* Get an overview of the problem at hand and the work to be done during the session
* 15 minutes, max
Part 2 – The transition
* Brief, but important
* Basically, explaining that friendly Q&A time is over, and it's time to get to work
* Explain that you'll be observing while the participant works, and that you will be interrupting to ask questions
Part 3 – The contextual interview proper
* Participant works; you observe
* Basically, be a big PITA – interrupt with questions, follow the participant around, ask about any phone calls made or taken, emails sent, printouts, references to job aids, etc.
* Participant may take a break at any time (of course)
* Take copious notes
Part 4 – The wrap-up
* Describe to the participant your understanding of what you saw/learned to ensure you have a proper understanding.
* Customer corrects any misunderstandings you may have/clarifies your findings
* Allow approximately 15 minutes
Advantages:
- See exactly what end users see (screen size, window size, version, customizations, personalizations, actual method of interaction)
- Accurate recording of entire session
- Less expensive and faster than in-person field study
Disadvantages
- Lose environmental context (light, noise, distractions, interruptions, physical layout of workspace)
- May be technical challenges (firewalls, software compatibility)
Like lab-based testing:
Has predefined tasks
Can be recorded (via camtasia or similar) with some extra work
Can be performed at any stage in the design cycle
Unlike lab-based testing:
Is conducted at the participant's workstation
Provides insight into the user's work environment (lighting, sounds, desk space, distractions)
On-site testing does have some logistical challenges. Differences in computing systems may present compatibility issues with your prototype. Site security and intellectual property issues are the same as with any field research technique.
On the other hand, if you don't have a usability lab, this method can provide considerable cost savings over traditional lab testing while gathering contextual information and maintaining the integrity of a qualitative test. (This method is not well-suited for quantitative usability testing.)
The key here is that what people do and what they say they do are two very different things.
While surveys have their place, there's still that (very serious) issue of self-reported behavior. Some, like ForeSee, can provide some accurate reporting of behavior, but the capacity for doing so is highly dependent on the type of site or application for which information is being aggregated.
There's always the option of a freeform response for some survey questions, but that sort of negates the benefits (fast, cheap, quantitative) of a survey.
Traditionally done after-the-fact on completed software, although I'd wager that's not the most typical use anymore.
Has the advantage of a controlled environment – especially useful for before/after testing and when gathering quantitative data like click counts and time on task.
This method can be expensive, particularly if you are renting lab space.
Best used when combined with field methods to ensure the correct features and tasks are being tested.
While journaling also provides self-reported information, that information is (supposedly) recorded in real-time, so the participant's understanding of the situation is clearer, and s/he is not relying on memory.
If the journals will be kept with pen & paper, reading the participants' handwriting can be a real issue.
This is a very inexpensive method of data collection, although it does require a lot of time after the fact to read the journals and organize the information they contain.
Clearly, Field Research isn't appropriate for every situation.
But it's equally clear that it can't be replaced with a simple survey or even by a focus group or formal usability test.
When can or should you do fieldwork? There’s rarely a bad time to do it. As with many User research activities, though, the greatest benefit is realized when performed early in the product lifecycle.
You may have heard there’s no crying in baseball. It’s true.
You may not have heard, though, that there are no copy machines in the field!
Get your paperwork together ahead of time. ALL of it.
There will be no chances to make extra copies! I highly recommend a single manila envelope per participant, plus extra copies of all necessary paperwork.
Get written consent for anything you do, especially for taking any photos or videos where your participant's face is visible.
Ask open-ended questions
-Limiting interaction -
You: Do you like that the button is in the upper right hand corner?
Participant: No.
-Better-
You: Talk to me about that button.
Participant: Well, when I fill out the form, I end up at the bottom of the page, where the “Cancel” button is. A lot of times, I end up accidentally hitting “Cancel,” which means I have to redo the whole form.
Minimize bias
Don't let the participant know what you think about the workflow or the application or tool you're evaluating. You're there to find out what THEY do and think.
Provide reassurance to the participant
Do your best, explicitly and with your body language and tone, to remind them that there are no wrong answers and that you're really interested in what they're doing and their opinions about their work.
Don't be afraid to collect a lot of artifacts. (Always with permission, of course.)
Take photos – of the person's workspace, of important environmental factors, of the distance from the desk to the fax machine or printer, etc.
When feasible, record short videos.
Make copies of job aids people have created to help themselves with their work.
Don't be afraid to ask for screen shots or to take actual physical measurements if necessary.
Scheduling issues
When people come to you, they've set aside time to spend with you, focused on your shared goals. When you go to them, you sometimes need to work around a lot of other people's schedules as well, particularly if you're spending a whole day working with a series of people.
Productivity and morale concerns from management
If you're observing in a call center, they don't stop gathering statistics for that day. In that and other situations, management might be overly concerned that your presence will have a negative effect on their metrics for the day.
Since politics are everywhere, there may be some issues with who does/doesn't get to participate, particularly if participant compensation is involved.
Site security issues
Always find out ahead of time what you need to do to enter the site/parking lot, where to park, what to tell any security personnel when you enter the building, and what documentation (driver's license, plate number, etc.) to have ready. If you need to be preregistered, make sure your on-site contact does so.
Follow ALL applicable site safety rules (e.g., hard hats, eye/ear protection, etc.) - sometimes this can make data collection challenging.
Rules about photos & videos
Not surprisingly, companies are careful about protecting their intellectual property from competitors. You may not be allowed to collect the artifacts you'd like to.
Pre-planning
Organization is critical. Have all questionnaires and any other paperwork/artifacts grouped, packaged, and labeled ahead of time.
For our field research today, we’ll be using a very rough, incomplete prototype of an app called Garcon. It’s intended for use by people enjoying themselves poolside at resorts like this one.
http://usabilitynet.org/tools/affinity.htm
Affinity diagramming is used to sort large amounts of data into logical groups. Existing items and/or new items identified by individuals are written on sticky notes which are sorted into categories as a workshop activity. Affinity diagramming can be used to:
* analyze findings from field studies
* identify and group user functions as part of design
* analyses findings from a usability evaluation
======================
Make a separate note for each item captured during your interpretation of your notes.
Put notes up on the wall one at a time. After each note goes up, add notes that go with it.
When there are too many groups to keep track of, start labeling them, ideally with a different colored note than the one you're using for individual items.
As groups accumulate individual notes, break them down so there are no more than 4 notes in a group.
Add additional notes (again, ideally with different colors) to collect groups.
- Modified from Beyer & Holtzblatt, 1998.
http://office.microsoft.com/en-us/onenote/default.aspx
Office OneNote 2007 is a digital notebook that provides people one place to gather their notes and information, powerful search to find what they are looking for quickly, and easy-to-use shared notebooks so that they can manage information overload and work together more effectively. It delivers the flexibility to gather and organize text, pictures, digital handwriting, audio and video recordings, and more — all in one digital notebook on your computer.
Mind mapping resources:
http://en.wikipedia.org/wiki/Mind_map
http://en.wikipedia.org/wiki/List_of_mind_mapping_software
http://freemind.sourceforge.net/wiki/index.php/Main_Page
Visual descriptions can be extremely polished or as simple as a basic flow chart.