11. Guerilla Usability testing
“
The importance of recruiting
representative users is overrated.
... it’s much more important to test
early and often.
Steve Krug
”
32. When to test in your project
Before
Your project
Usability testing &
undercover testing:
•Learn from current website
•Learn from competitors
Remote testing &
iterative usability testing
on sketches, wireframes
and visual designs
After
Usability testing to verify
A/B & MVT testing to
refine and optimise
33. When to test in your project
Before
Usability testing &
undercover testing:
•Learn from current website
•Learn from competitors
Your project
Remote testing &
iterative usability testing
on sketches, wireframes
and visual designs
After
Usability testing to verify
A/B & MVT testing to
refine and optimise
34. When to test in your project
Before
Usability testing &
undercover testing:
•Learn from current website
•Learn from competitors
Your project
Remote testing &
iterative usability testing
on sketches, wireframes
and visual designs
After
Usability testing to verify
A/B & MVT testing to
refine and optimise
39. Let the stats speak
for themselves
35% dropout at
the last step
626 orders lost
£125,200
every month
40. Bring in an outside consultant
(other agencies are available)
41. Continue to get buy in...
Make it measurable!
Examples of what to measure:
eCommerce
Engagement
Number of transactions
Key goals
Revenue
Average Order Value
Order frequency
Conversion rate
- Registration, log in, newsletter sign up
Micro-conversions
- Download a PDF, check their balance
Churn
- How many users/registrations do you loose a
month/year?
Usability test benchmarks
- How long does it take to complete a task?
43. To sum up today
Test as early and as often as your budget allows
Select your toolkit (or use an agency which has a great
one)
Make it measurable
Shout about it!
Hi i’m Joanne, I Head up the UX team at Orange Bus
I’ve been a UX designer for 7 years experience
- I’ve always been interested in research methods and the surprises that usability testing uncovers.
- In more recent years i’ve been specialising in Conversion Optimisation for big clients such as Philips, ScS, and Sage.
Today I wanted to explain the benefits of testing and showcase some of the Tools we use at Orange Bus throughout our projects and help you take the next steps to reap the benefits.
For the purpose of today we’re going to focus on testing methods which inform or validate our design decisions
So, Why test in the first place?
We obviously have a mix of skills and experience in the room, but we all most likely impact the user experience at some point.
Every day as professionals, we make educated decisions based on years of experience, knowledge and expertise which create the customer experience.
So how can we be confident that our assumptions are correct?
The truth is we can’t unless test with real end users and validate.
I’m sure everyone has come across this. Sometimes we even find this internally that we’ll have conflicting opinions which are equally valid.
So, how can we resolve the issue and know we’re doing the right thing?
And if you’re like me and like delving into the analytics then you can often see where there is a problem, such as people dropping out of a registration process.
But you have no idea why there is a problem unless you test with real end users.
As a UX designer your job is to balance the needs of the audience and the business goals you’ve been tasked with achieving.
Testing provides a measurable, validated way to improve both.
There are many testing tools and methods out there once you start looking around so today i’m going to showcase some of the tools we love to use at Orange Bus throughout our projects.
For anyone not familiar, one-to-one usability testing involves observing representative members of your audience in performing a series of tasks. They are also asked to “think out loud” to gather their thoughts and feedback while you observe their behaviour to understand their actions.
What you can test
Early prototypes, a live website
Tools - Basically screen recorders which track the users mouse movement and and what they’re saying which allows you to watch back the sessions to get accurate feedback, make clips to demonstrate important feedback to your client or watch it back with participants to get their feedback on their actions.
The main benefit of one-to-one testing is that it allows you to ask questions as you go to get to the heart of issues and ask for feedback on areas the participant has not explained.
Negatives
Recruitment can often be an issue and scheduling participants as many cannot make 9-5 and it is difficult to plan tests back to back.
Highly recommend if you run usability tests yourself that you take part in someone elses to help put yourself in participants shoes.
I was lucky enough to take part in Schuh’s usability test a few years ago and they combined other activities with the traditional usability test. One activity was to select from a table full of celebrities which I thought represented Schuh. It most likely didn’t matter which celebrity I picked, it was more the reasons why i thought the represented the brand to get key descriptive words out of me.
Over the years i’ve really learnt from this and we’ve had great success as Orange Bus in letting participants draw the page they’re about to see before the test. This lets them compare their expectation to their first impression and gets them talking and being descriptive straight away.
I mentioned recruitment can be a costly and time intensive tasks, there is an offshoot to this type of testing which goes with the theory that you don’t need to recruit representative users. You will find most usability issues with just about anyone.
It’s worth bearing in mind this is not always the case and you can get much more in depth feedback on your website copy from the target audience who may have previous knowledge or experience which impact their feedback.
Time in turning round results and budget can often leave many clients believing testing is not an option in a project. At Orange Bus we would never recommend running no usability testing at all during your project as you’re leaving the project open to risk and a the end of the day we want the end result to be the best it can be. Online or remote usability testing allows you to easily recruit and gather real user feedback quite quickly to feed into your project.
Using these tools, participants can walk through your tasks and provide verbal feedback whilst their screen is recorded. The only downside to this is that the test are unmoderated and you’re unable to ask questions to the participant to find out more information or details on a issue.
Eye tracking uses sensors to track the participants eye movements during a usability test to see what catches their attention. You can then review these with the participant to get even more feedback on why a particular item drew their attention and how they feel about it..
It’s also great as you can test more than websites and prototypes like the other tools mentioned - you can test videos, tv ads and printed materials.
However it is quite costly to purchase the equipment or rent a lab.
All the tools mentioned work really well for prototypes and live websites, however its important to gather as much feedback as possible and as early as possible. That’s where testing on visuals designs can help validate decisions using these tools.
UsabilityHub is probably the one we use the most to run two key tests.
The left is an example where a visual has been uploaded an users have been asked to click on all the areas they believe are clickable. This helps us understand if your call to action styles are clear.
Another test is 5 second tests where users are shown a page or email design for 5 seconds and then the image is taken away. It’s 5 seconds as this is typically the time it takes to make you’re initial first impression and choose to continue exploring a website or leave. They are then asked a series of questions about what stood out to them to help us understand first impressions and what stood out the most.
We’ll do a quick example of a 5 second test.
So i’ll show you a screen for around 5 seconds and we’ll answer some questions.
ready?
Moving onto what I like to call undercover testing, Clicktale runs on a live website and records every mouse movement a visitor makes allowing you to analyse samples of real suer behaviour. This is great as you can be confident that no-one has changed their behaviour as they may well do when being observed.
The negatives are much like analytics, it won’t tell you why your audience may be performing the action of skipping a step, it can be very addictive to watch and time consuming and does have a cost implication.
Heatmaps track visitors to your websites mouse movements. A lot of users test to read with their mouse pointer to it can be a good way of tracking your copy.
Clicktale also does this, or if you just want heatmaps then crazy egg us a good tool.
A/B testing allows us to put different versions of a page or page elements live to real users and see which performs the best against key goals/metrics.
It’s great as it allows us to experiment and test different assumptions to find the solution which performs the best. Usability testing can often flag the areas we need to focus on and help generate different ideas to test.
Optimizely and Visual website optimizer are paid for tools which provide a drag and drop interface with little CSS knowledge required to achieve simple changes. Which means you don't necessarily need development resource to test quick changes such as button labels and colours.
Google content experiments is a free tool to everyone in Google Analytics, however its functionality is quite limited and you are required to build two versions of the page in order to test - this can often make this a costly and time consuming choice.
All these tools will track your test and help you determine a winner and with after a bit of experience you can be confident in your results and roll out the winning version.
A couple of quick examples. You can vote on this one.
This is a A/B test we conducted for philips on their male grooming products website. we wanted to see if representing the price differently affected visits and sales on the Philips store.
What’s your vote for a winner in the tab placement?
Our assumption was that making the Philips call to action stand out more with different button colours would increase Philips sales and click throughs to the store.
We also thought a more descriptive label “Buy from Philips” would convert much better.
Moving on it can be difficult to know when to use these methods
Before
- Often a step which is missed. There is a lot which can be learnt from the current website which can feed into new requirements to ensure the same mistakes aren't made again and you keep essential features your audience love.
- Usability testing on a competitors websites can equally help test features you do not have and gather feedback
Changes during a project are more expensive as the projetc progresses, therefore its always best to test as early and often as your budget permits.
Testing your website should never stop at the end of the project and if you have a relatively new website now is the time to fully understand your audience, optimise all your processes to they meet audience and business needs.
Hopefully, that all sounds brilliant but why do most people not test?
Number 1 barrier is getting buy in to start testing in the first place
Im sure you can appreciate that the tools and methods mentioned are not free. They take time and resources to set up, manage, analyse and implement results. The tools themselves sometimes also carry a cost. During a project testing can often be seen as a delay and an unnecessary extra step - making it impossible to verify your decisions and know we’re doing a good job.
The most obvious is to get the results and prove with impressive numbers and usability videos that testing really works.
A way to get around this catch 22 situation are to make good use of free trials if you have the in house expertise (and time) to manage the tests, analyse and present their results.
Look into analytics - highlight the problems you can see and showcase how testing can explain why these are an issue to help resolve them
626 x Avg order = £200 = £125,200
If you don’t have the expertise or you don’t have the experience or resources then bring in an agency to showcase their case studies and walk through stakeholders the importance of testing
To continue to get buy in you need to show results.
Before you start make sure you know what success looks like and make sure you’re measuring the impact your changes have made either through analytics or a separate tool.
Sharable reports which sum up the highlights
blog posts
articles
That’ all sounds brilliant but why do most people not test?