Joan Winchester - Usability testing on a dime: What, why, and how?
•Descargar como PPTX, PDF•
1 recomendación•645 vistas
This document provides guidance on conducting usability testing on a limited budget. It discusses what usability testing is, why it's important, and how to effectively conduct tests with few resources. The document recommends testing with 5 users to uncover most usability issues. It provides tips for introducing tests, giving instructions, asking questions, and getting helpful feedback. The goal is to identify problems and improve the design in a cost-effective manner.
Recomendados
Recomendados
Más contenido relacionado
Similar a Joan Winchester - Usability testing on a dime: What, why, and how?
Similar a Joan Winchester - Usability testing on a dime: What, why, and how? (20)
Más de Plain Talk 2015
Más de Plain Talk 2015 (20)
Último
Último (20)
Joan Winchester - Usability testing on a dime: What, why, and how?
- 1. 1 Plain Talk in Complex Times September, 2013 Joan Winchester, Lead Researcher Usability Testing on a Dime What, Why, and How?
- 2. 2 What?
- 4. 4
- 5. 5
- 6. 6
- 7. 7 WhatWhat
- 8. 8 Why?
- 9. 9
- 10. 10
- 11. 11 Please indicate for EACH person listed on this application any health coverage, including Medicare or Medicaid, in effect within 24 months prior to the proposed effective date of this coverage. Each person applying for coverage must be listed below. If no health care coverage was in effect within the past 24 months, please indicate NONE. If coverage is provided for a dependent from a previous marriage or relationship, please attach a copy of the court documentation showing who is responsible for the dependent(s)’ health care coverage so that the insurer can determine whose coverage is primary.
- 12. 12
- 13. 13
- 14. 14 Please answer the questions on the enclosed redetermination form and sign it. You may ask another person to help you complete this form. If you do, make sure both you and the other person sign it. You must also send us certain things to prove your eligibility. Be sure you give us up-to-date information. We need proof of your current income, such as copies of checks, check stubs or a letter from the people who give you the money. If you have assets, we need proof of their current value, such as copies of updated bank books, latest bank statement, copies of bonds, car registration, and life insurance policies.
- 15. 15 1. Answer all questions on the form. 2. Attach proof of your current income, such as copies of pay stubs, pay checks or a letter from the person who gives you money. 3. Attach proof of the value of any assets (money or things you own), such as copies of latest bank statements, car registration, life insurance policies, and bonds. 4. Sign the form. 5. Send the form and your proof to: We Get It 2123 North Market Street City, State, ZIP → You may have someone help you fill out the form. If you do, make sure you and that person both sign it.
- 16. 16
- 17. 17
- 18. 18 If the Department determines that having to pay a premium results in extreme hardship for a member, we may, at our sole discretion, waive payment of the premium or reduce the amount of the premiums assessed for a particular family.
- 19. 19 We may lower or cancel your premium (monthly cost) if it’s too hard for you to pay.
- 20. 20
- 21. 21
- 22. 22
- 23. 23 How?
- 24. 24
- 25. 25
- 26. 26 INTRODUCTION I am __________ from __________… We’re here today to find out… We want to make this better… We’ll use what we find out today to…
- 27. 27 General Instructions I’m going to ask you to… Just give me your honest answers… You can’t do anything wrong… I’d like you to think out loud… We are testing the material not you… You won’t hurt my feelings if you don’t like something…
- 28. 28
- 29. 29
- 30. 30
- 31. 31 Questions Can you tell me at first glance what this is for? Who sent this letter? How would you find…? I’d like you to read this sentence/ paragraph/section. When you’re ready, tell me in your own words what it says.
- 32. 32
- 33. 33 “Checking in questions” What are you thinking? What are you looking at now? What would you do next? Is that what you expected to happen when you clicked that? Can you tell me why you decided to do that/go there?
- 34. 34
- 35. 35 Please indicate for EACH person listed on this application any health coverage, including Medicare or Medicaid, in effect within 24 months prior to the proposed effective date of this coverage. Each person applying for coverage must be listed below. If no health care coverage was in effect within the past 24 months, please indicate NONE. If coverage is provided for a dependent from a previous marriage or relationship, please attach a copy of the court documentation showing who is responsible for the dependent(s)’ health care coverage so that the insurer can determine whose coverage is primary.
- 36. 36 Can you find… Please fill in… Pretend you are applying for… What if you need to… You are trying to…
- 37. 37
- 38. 38 What would you do here? not Can you complete the section below?
- 39. 39 Encouraging Phrases This is exactly what we need... Your comments are very helpful...
- 40. 40 Who?
- 41. 41 DO
- 42. 42 DON’T
- 43. 43 “It takes only five users to uncover 80% of high-level usability problems.” Jakob Nielsen
- 44. 44 TESTING INTERVIEW SCHEDULE TIME ROOM 1 ROOM 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9:00 Sally Peter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10:00 Avis Bill . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11:00 Ralph Ty . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12:00 Lunch Lunch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1:00 Sondra Harry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
- 45. 45
- 46. 46
- 47. 47
- 48. 48
- 49. 49
- 50. 50 Good testers get good results!
- 51. 51
- 52. 52 Where?
- 53. 53
- 54. 54 When?
- 55. 55
- 56. 56 “Perfection is achieved not when there is nothing more to add, but when there is nothing left to take away.” Antoine de Saint Exupery
- 57. 57 Thank you!
Notas del editor
- Thank you, Mercedes.As Mercedes said, I am a writer and researcher at the Center for Health Literacy. At the Center, we specialize in writing in plain language and making print and web materials easy to read and use. In today’s session, I’m going to talk about how we find out if materials – our own and those that others have created – are easy to read and use. I will give you the what, the why, and the how of usability testing – along with who,when and where.Please make sure you have copies of the 4 handouts. If you don’t, then feel free to get up and get them while we do a little networking. I’d like to begin by having you think a minute about how you hope to use what you learn in this session. Then, if you would please, just turn to the person next to you and briefly tell them what you do and how you plan to use this information.
- We’ll begin with what.
- A picture is worth a thousand words, so this short video will tell us what usability testing is. It also speaks to why. Let’s watch. [Show video.]So what is usability testing? [Pause for answers and discussion.]Right. Usability testing is simply watching people use the product. So, if this were a usability test, what did they just learn?[Pause for discussion.]And what might they decide to do?-Nothing – let the parents pick their kids up to reach the fountain-Lower fountain height-Create a different faucet arrangement-Put a child’s fountain next to this one, etc.
- Usability testing is watching someone use the product. For our purposes at the Center, the product is usually a website or a document: a letter, form, notice, application...anything that people will read and use. You simply observe as they read, fill in, navigate, and so on. And you ask questions and take notes as you watch.As you see, there is an observer and a user sitting together with the material. It’s that simple.
- Usability testing is one-on-one. It can be formal, done in a usability lab with cameras and one-way mirrors. Or it can be less formal, like this one, using paper or computer mock-ups or the real thing. One at a time. Sitting at a desk or a table with the document or site and a testing guide.
- Usability testing is not the same thing as a focus group, where a moderator sits with a group of people and leads a discussion about a topic. Usability testing is not asking people to discuss the product or talk about how they would use it or what they think of it. And it’s not just showing people the product and asking, “Do you understand this?”
- It’s sitting with one person as that person reads and uses the material while you observe and ask questions.Are there any questions about the what?
- And now for why.Why? Well, mainly to find out if what you’ve made or written works for people the way it’s supposed to. Like the fountain.
- For example, let’s take a look at a website home page.Why would you want to usability test this? What would you want to learn from usability testing? [Pause for answers:]-Can people tell at a glance what it’s for?-Do people know whose site it is?-Can people find and figure out the menu?-Is it user friendly, easy to navigate?-Do people like the way it looks?-Is the tone friendly or intimidating, welcoming or off-putting?-Is it accessible to people using screen readers and other adaptive devices?
- And for a document? [Elicit from audience:]-What’s the purpose?-Who produced or sent this?-Is the tone welcoming and friendly?-Can people read and understand the content?-Do people understand the messages?-Is the vocabulary too high, too technical or legalistic or bureaucratic?-Is it visually appealing?-Do people “get” the symbols?You can do usability testing to find out the answers to questions you have about a thing already. For example, we struggled with this Medical Home symbol here. We wanted to convey the idea...but we didn’t want to confuse people by implying that there is this one actual building somewhere downtown where everyone can go for all their medical needs. It’s a very abstract concept, and many people are very concrete thinkers. So we asked. And, sure enough, some people thought there was an actual building called “Medical Home.” So we tried to come up with an image that evoked the concept but didn’t look too much like a specific place. Then we tested the image and the explanation to see if it was clear to most people.
- You mentioned doing usability testing to find out if people can read and understand the content. That’s central to our usability testing.Don’t bother to read this now because we’ll come back to it later, but just at glance, you might be thinking this paragraph could really use some testing.Think about the people you’re trying to convince that this thing needs work: The writer, the client, the designers, the lawyers...One of the best, and most persuasive, pieces of evidence you could ever have is actual data from actual readers actually trying to wade through this! So that’s a very important why:If they’re reluctant to take your word for it, field testing leaves little doubt that something is too difficult, too confusing or too wordy.
- And speaking of designers, another good reason to do usability testing is to find out if the design adds to or detracts from the usability. Even, do people like and relate to the pictures, colors and images? Here, for example, we tested this first without the shadings and what do you think we learned? [Pause for answers.]Yes. People needed help seeing and separating the items on the list. So we had the designers add shading, and we learned that the shading helped. How? It broke up the list into manageable chunks for the reader to see what shots went with what age. This kind of thing is very important when you are testing materials that are going to be used by audiences with a wide range of literacy skills.
- Another why, is to find out how long it takes to read or do something. The only way to find out is to have someone read or do the thing while you time it.
- Another importantwhy is to find out if your directions and instructions work. Are they clear and easy to follow? This was part of a notice that I worked on. The client was under court order to make their notices easier to read and understand...and the client thought this was just fine!Usability testing quickly showed that it wasn’t. Usability testing can reveal what – and where – the problems are. And, with test results in hand, you can suggest specific “fixes,” concrete ways to improve the material. Just like with the fountain.
- Like this. With the instructions that people struggled with on the last slide, for example, you might argue for:-Using numbered steps, in the order that they must be completed, so people can follow the steps – and can see at the end if they’ve done everything they need to-Splitting the word “paystubs” in two for easier readability-Defining unfamiliar terms, like “assets” right there in context-Adding what to do with the completed form-Rewording to make the messages easier to understand (“complete” to “fill out,” “another person” to “someone,” moving “both” closer to “sign”)-Taking things out of the instruction steps that aren’t to be done each time, and calling them out with a symbol (You may have...)You’ll notice that this is a little longer than before, because there is more to it, but it’s a lot easier to read and use. To make sure, retest after making the changes.
- Another reasonwhy is to find out if the thing you have created does what it’s supposed to do. Usability testing will show whether this pamphlet does what was intended: explains quickly and convincingly that it’s important to get regular check ups for kids. Testing can reveal whether this is the right vehicle for getting the message out.
- For an application or form, the obvious why is to find out if people can fill it out easily and correctly. This is another way usability testing can save you money, without costing very much. Have someone fill out part of this. Which part? You might not want to have them do the whole application (unless you want to see how long it takes to do that) but rather, choose a section that you anticipate will be difficult, or that’s particularly important for people to get right. And what do you think you’ll probably find out? [Pause for discussion of some of the problems with this form.]
- Of course, the most importantwhy is so you can improve the material. With this paragraph, you’ll find out:If the words are familiar and easy to read [“waive,” “extreme hardship,” “sole discretion,” “premiums,” “assessed”].-If the message is clear. [“What is this thing saying?”]
- Then, if you find out the words are unfamiliar or difficult to read, and the message is lost in the words, you can use the results to make a strong argument for plain language. That’s the best reason to do usability testing.
- You can also get people’s opinions in usability testing. Although I said we usually don’t ask for preferences on things like color, because everyone’s taste is different, we do sometimes want to know:Are the images and colors pleasing?Does the look interfere with or contribute to usability?These kinds of questions can be very empowering for the participant, and they show that the participant’s opinions are valued.Like the time I was testing a pamphlet and one of the participants said, “Hey, there are no pictures of fathers with children. I’m a single father. I want to see that this is for me, too.” We changed one of the pictures to show a man holding the hand of a child.
- Let’s go back for a minute to saving money as a reason to do usability testing.
- Here’s an example of something that happened to us:We were testing this application and, when we got to this part, something happened that we hadn’t thought of and never expected. We asked participants to fill out sections 3 and 4.And what do you think 4 out of 5 people did?Yes. That’s right. They signed next to “here.” And that would have cost the client money. People would have signed in the wrong place, and who knows if the applications would be valid? They might have had to have everyone do it again. Luckily, because it came up in usability testing, we were able to fix it before the application went into production. And how do you think we fixed it? [Pause until correct answer given (change “here” to “below.”)]
- Next we’ll look at how to create and do a usability test – on a dime.
- The first step is to decide what you want to know. What do you want to find out?Think about where you anticipate problems, or where you had questions as the team created the material, and think about what is really important for the user to understand or accomplish with the site or document. Do you want to find out if people understand the title? The purpose? The main messages? The navigation path? How to find things? How to follow the instructions?
- After you’ve figured out what you want to know, the next step is to write a simple usability test.
- Begin with an introduction. Make it brief and friendly, because you are sitting with one person, but do script it so you can deliver it the same way each time. We like to say things like, “We’d like to know...we want to see...we want to find out...”Be sure to state their privacy rights specifically: “We won’t use your name...” and tell them if you are audio- or videotaping the test, having them sign a release form.There’s a sample introduction in your handouts. Notice that the top has space to write:The name of the material being testedThe site location and dateThe participant first name and numberAny classification data you need, such as last grade in school completed
- Also put your general instructions in the introduction – designed to put the person at ease.
- It’s a good idea to state right at the beginning how long the usability test will take, so people don’t get anxious during the test about how long they will have to stay or whether they will be able to pick up their kids on time.“This will take 30 minutes.” Be direct. Of course, you’ll have told them during the recruiting process how long the session will last. We’ll talk more about recruiting later.
- And at the end of the introduction, it’s a good idea to remind the person how much you will pay for their time and travel. “At the end of our session I will give you...” Here’s another way you can save. The fee will depend on your budget. It may be cash, a gift card, food, even a warm thank you and a handshake. [Pause for discussion of fee]
- Next are the questions. Your sample protocol handout has a few sample questions.
- With your questions, you’ll want to start with the big picture and then a few details. Don’t try to cover too much ground. And use a combination of closed and open questions, with more open questions than closed.Closed questions empower participants by asking their opinions or calling for yes or no or short, definite answers (like the first two).Open questions allow participants to branch out (like the last two).
- As participants answer your questions or follow instructions, you’ll want to know what they’re thinking. You’ll want to find out why they do what they do or go where they go or think what they think. We like to check in about that as we are observing.
- It’s a good idea to build these “checking in” questions right in to your questions, so you don’t forget.
- As an even lower cost alternative to writing specific questions, if you can’t afford the time and cost for that, you can simply sit people down and turn ‘em loose. Watch what they do, ask them to think out loud, and don’t tell them how to do it.If the person asks how to do something, say that you want to see how people do without help, but you’ll explain at the end. This is the most valuable part of the testing: seeing whether people can read or navigate the material without help. In a recent website testing, I just asked people to find the application from the home page. Watching their frustration was enough to tell me that the home page needed changes.
- Remember this? We saw it way back in why. With documents, we like to have participants read one particularly sticky section out loud. This makes a very powerful case for plain language. Let’s see what happens. [Have volunteer read aloud and discuss briefly what we learned: the italics, the squished letters, the long paragraph, no time to breathe, the vocabulary, the long bolded sentence.]Of course, you don’t want to force someone to read aloud...and if they do make mistakes, you want to reassure them that the problem is with the material, not them, and that their mistakes will help us know what needs to be done to improve the content. An alternative to reading aloud is to have them read silently and paraphrase: “Can you read this and tell me in your own words what it’s saying?” This will tell you quickly if the material is comprehensible. And it’s handy ammunition for lobbying for plain language.
- With websites and forms, you’ll want to include a few tasks. Be sure to keep them BRIEF, SPECIFIC and REAL.They should be things people will likely do – things that are necessary for successful use of the site or form, like browsing without the search feature or knowing what to do in response to a notice. Here again, you only need a few, focusing on what you want to know and where you think problems might be.
- Although I’ve said that we generally avoid asking questions that involve taste or preference, sometimes it is appropriate to ask this closed question: “Do you like this or that?” Just be sure the focus is on usability, not whether they like something aesthetically. For example, ask about color to find out how easy it is to see, not whether they like blue or green. In this case, we wanted to see whether our streamlined version on the right in plain language, with more white space, less text and easier instructions was, in fact, appealing to readers.So we asked them, “Do you like the one on the left or the one on the right?” And then we asked them to explain why.
- You’ll want to make sure that the wording of your questions doesn’t inadvertently give clues or answers. That way, you can learn whether the directions and vocabulary are clear.
- And you’ll want to include encouraging phrases every now and then. And at the end, “Is there anything else you’d like to tell the writers/designers?”
- So that’s what,why, and how. Here’s the bonus part: who, where and when.
- Who? Who you recruit will depend on your budget, and it can be done at a minimal cost. Do recruit the right participants that fit your budget and do recruit people in the categories of your intended audience: limited literacy, Medicaid recipients, etc.Be sure to recruit men and women of varying ages – unless your targeted audience is one categoryYou can get people just about anywhere. [Pause to tell mammogram notice story and invite comments.]
- Don’t recruit people who have worked on the material or know it well or don’t fit the profile of your intended audience. And don’t think you need large samples. That’s another way to keep costs down if you have a limited budget: Better to test six people than to abandon the idea of testing because you can’t test 26.
- In fact... [Explain who Jakob Nielson is.]Often after the first five or so people, you begin to see people make the same mistakes and you get the same results over and over...and you don’t really get new information.So fewer is quite OK, and keeps costs down. Remember our “Sign here” example? After we saw four people make the same mistake, we didn’t really need to see it any more to know it was a problem.
- It’s very helpful to have a written schedule, and to think ahead of time what to do about now shows and extras. Will you be able to accommodate extras? Pay them?Is there a contingency plan if not enough people show up?To keep costs down, we often choose sites where the people we want to reach are already regular visitors and can get there easily, such as CBOs who have relationships with clients in our target audience and that can do the recruiting and scheduling for us and make confirming calls.That way, if there is an open slot, they may be able to recruit someone right then and there.But often word gets out, especially if you are paying cash, and you must warn the staff or receptionist that you won’t have money to pay or add people. You don’t want to upset folks at the site...or alienate their clients, because they will work with them long after you’ve gone, but you must remain firm about your limits and requirements. [If time, tell story about non-readers.]And notice that, to protect their privacy, the CBOs schedule has only first names of the recruits.
- Who will do the testing? If you are doing this on a dime, it may be you!Yes, this is Mercedes, who conducts usability testing in Spanish for our team.She’s a great tester, and here are some tips for being a great tester:
- A good tester......practices the test beforehand...arrives at the site on time, fully prepared and dressed appropriately – neither intimidating nor too casual...has a method for taking notes – on the protocol or in a spreadsheet, for example...keeps everything together to give to the person who will write the reportYour handouts include a sample checklist for how to be prepared and organized so you aren’t fumbling and searching during the test, especially if you are testing several items.
- A good tester is alsoprofessional, friendly, welcoming and encouraging – greeting participants warmly and putting them at ease during testing.
- A good tester......focuses on exactly what the writers and designers want to know...probes and clarifies...keeps the participant on task, keeps it moving, finishes on time,...and does the test the same way every time.
- A good tester is careful not to......lead participants...supply answers...let personal beliefs or opinions about the material influence resultsIt’s very hard to keep yourself out of the test, but it’s crucial for valid results. So try not to be too helpful! The whole point is to see how it goes without help.
- Focus on the intensity of responses and on what you observe. For example, the participant may say, “This is easy!” But you observed the struggle.
- The most important thing you can do to increase the likelihood that you’ll be a good tester is to PRACTICE.You can practice for free. Corral a colleague, friend, relative or neighbor and run it through once or twice.Practice to get the kinks out of the protocol, to gain confidence, and to be sure you do the test the same way every time.
- Where to test?
- There are professional testing organizations and sites, which you can certainly use if you have the budget. But if you’re doing this on a dime, you’ll want a good testing environment at a low cost. It should be a quiet, private space not too far from a reception area, with comfortable chairs and a table where you can set up note taking materials and laptops, with no outside noise interference.If possible, check the site out in advance and make sure the room is a comfortable temperature.Comfort matters and participant stress can skew results.Of course, you may not have control over this...[Tell lunchroom and hot room stories.]So you can test just about anywhere. [Mammogram testing in lobby.]
- And finally, when?
- Test when you can use the results!You don’t want to test too early, before the thing is far enough along to use, but don’t wait until it is too late to make changes.If you can, test more than once. As to when during the day to test, test when the people are available – morning, afternoon or evening. And know beforehand who will need the results and how you will provide the data. We like to make a quick list of findings and trends right after testing; things like”General impressionsThings that keep coming upSurprising resultsGreat questions and commentsAnd most important of all, direct quotations from participantsThen we do a formal report when all the testing is finished. One of the great benefits and cost savers of usability testing is that sometimes testing yields creative suggestions from participants for fixing problems! You’ll want to fix the most important problems, making small changes rather than redesigns, and doing what is cost effective and fixing what you can within your budget.
- Often, the best solution is to remove something rather than add something.
- So that’s it:What, why, how, who, where and when.I’ll be happy to take questions, as we have some time left.