Ten Steps to Successfully Design and Execute Online Qualitative Research: A Two-Hour “Crash Course”

1. Ten Steps to Successfully Design and Execute Online Qualitative Research: A Two-Hour “Crash Course” Merlien Institute Qualitative Research in Web 2.0 Macau, China 22 February 2011 Jeff Walkowski Minneapolis, Minnesota USA +1.612.377.3439 jeff.walkowski@qualcore.com

2. Ten-Step Process Is online appropriate? Which online method? Which platform to use? How many to recruit? How to adapt screeners and recruit? How much for incentives? How to adapt guides? How to best moderate online. How to keep observers engaged. Data analysis – any different online?

3. STEP 1: Online vs. Offline STEP 1

4. Best Uses of Online Low incidence populations Very busy respondents Widespread geography Need to save on travel

5. When to Stay Away From Online Need to see respondents First time doing research Respondents must interact with physical stimuli Heightened security

6. Important Reminders Not all situations are clear-cut Sometimes both methods will be equally effective Not sure which will work best? Use both!

7. STEP 2: Online Method Selection STEP 2 STEP 1

8. Four Basic Options

9. Text-Based vs. Multi-Media Text-Based Advantages More open, honest Everyone answers every question All have equal voice No waiting for transcripts Multi-Media Advantages Non-verbal cues Less stress for first time users of online qual

10. Synchronous vs. Asynchronous

11. STEP 3: Platform Selection STEP 3 STEP 2 STEP 1

12. Partial List of Platforms Available

13. STEP 4: How Many to Recruit STEP 4 STEP 3 STEP 2 STEP 1

14. The Right Number is Important

15. STEP 5: Screeners and Recruitment STEP 5 STEP 4 STEP 3 STEP 2 STEP 1

16. Recruitment Basics Where to obtain sample? Client-provided sample Online panels Social media Physical focus group facilities Who handles recruitment? Platform providers Online panel companies Physical focus group facilities Email recruitment is least expensive To maximize show rates, include at least one phone contact with respondent before session begins

17. Screener Notes Same screener as for offline projects, plus some potential additional questions: Availability of internet access Dial-up vs. high-speed access Necessary software (e.g. Acrobat) Ability to upload photos, videos Webcam

18. Manage Respondent Expectations Be clear at the invitation stage what is expected For chats, the date and time (in their own time zone) For boards, how much time to spend each day For boards, emphasize the need for interaction Conditions for receiving an incentive (if offered)

19. STEP 6: Decide on Incentives STEP 6 STEP 5 STEP 4 STEP 3 STEP 2 STEP 1

20. Notes About Incentives Offer at least the same amount as for in-person sessions Pay less to consumers than to business-to-business respondents Panel companies have incentive recommendations Payment tips: For chats, pay if show on time, even if not accepted For boards, remind them daily what they can earn Payment can be by check (mail) or electronic gift card (email)

21. STEP 7: Adapt Discussion Guide STEP 7 STEP 6 STEP 5 STEP 4 STEP 3 STEP 2 STEP 1

22. Notes about Moderator’s Guides Expect to spend more time on guide development for text-based online projects than for in-person projects Every word counts! Be precise with instructions/questions Be “chatty” – allow your personality to come through Use words/phrases to “sound” energetic and fun

23. STEP 8a: Moderation – Chats STEP 8 STEP 7 STEP 6 STEP 5 STEP 4 STEP 3 STEP 2 STEP 1

24. Three Challenges in Text-Based Chats Inspiring Participation Managing the Flow Humanizing the Experience

25. Three Tips to Manage the Flow Inspiring Participation Managing the Flow Humanizing the Experience

26. Tip #1 to Manage the Flow:Deliver Guidelines “Listen” and “talk” to each other – not just to me. Stick to the topic. Heed the “STOP!” request if it occurs. Don’t worry about spelling or grammar Don’t worry about how fast (or slow) you type. It’s OK to disagree, but do it in a nice way. If you get “bumped,” just log back in. If I seem to disappear, wait for me.

27. Tip #2 to Manage the Flow:Provide Transitions Please finish your posts so we can move to another topic. I wish we had time to discuss this further, but I need to move us along. Ready to move on? Let’s switch gears and talk about something else.

28. Tip #3 to Manage the Flow:One Question at a Time Casey (Mod): 4. HOW LONG DO YOU TYPICALLY LISTEN TO THIS STATION?DO YOU THEN SWITCH TO ANOTHER STATION OR TURN THE RADIO OFF? Kimberly: 3. I usually don't listen to the radio first thing in the morning. Joan: 4. all day Kimberly: 4. By mid morning I have XXX on. I don’t like a lot of talk radio. Jennifer: 4-mostly all day long Nickie: 4. 3 hours & I turn off the radio Heather: 4 - for about an hour then i shut it off Casey (Mod): KIMBERLY, WHEN DO YOU USUALLY START LISTENING AND HOW LONG HAVE YOU BEEN UP AT THAT POINT? Joan: 4. only chage if they are playong a song I hate Jennifer: 4.I switch to Kson about noon. Karen: 4 on and off all day Kimberly: I usually listen through out the day on the computer.

29. Three Tips to Inspire Participation Inspiring Participation Managing the Flow Humanizing the Experience

31. Read all responses before asking next question

32. Allow respondents to read, too

33. Don’t leave respondents behind

34. Back-track, re-ask as needed

36. Tip #3 to Inspire Participation:Foster Interaction Don’t just answer my questions. Please “talk” to each other. Scroll up and review the ideas mentioned. Which one most appeals to you? Joe said X. Do you agree or disagree with that statement? Be sure explain.

37. Three Tips to Humanize the Experience Inspiring Participation Managing the Flow Humanizing the Experience

38. Tip #1 to Humanize the Experience:Greet Them in the Waiting Room

39. Tip #2 to Humanize the Experience:Use Respondents’ Names Why's it important to ask more questions of people by name? “Shows that you are paying attention” “Keeps us personally involved” “So you feel like it matters that you respond” “So that person can clarify on their answer”

41. Use casual expressions, slang

42. Introduce yourself at beginning

43. Go along with humor

45. STEP 8b: Moderation – Boards STEP 8 STEP 7 STEP 6 STEP 5 STEP 4 STEP 3 STEP 2 STEP 1

46. Three Challenges in Text-Based Boards Inspiring Attendance Fostering Interaction Maximizing Productivity

47. Three Tips to Inspire Attendance Inspiring Attendance Fostering Interaction Maximizing Productivity

48. Tips #1 to Inspire Attendance:Phone Contact Prior to Launch High Show Rate At Board Low None One or More Number of Phone Calls During Recruitment

50. Reassurance if they are late

51. URL, Username, Password, Room #

52. Contact Info for Support

53. Incentive Amount

55. Three Tips to Maximize Productivity Inspiring Attendance Fostering Interaction Maximizing Productivity

56. Tip #1 to Maximize Productivity:Check for Complete Answers

57. Tip #2 to Maximize Productivity:Moderator Posts Frequently

59. Question “sets”

60. Facts (easy) vs. explanations (difficult)

61. Brainstorming

62. Fill-in-the-blank

63. Sentence completion

64. Photo sorts

65. Letter-writing

67. Tip #1 to Foster Interaction:Train Respondents to Interact Set expectations Point out commonalities “Force” them to react to others’ posts Applaud interaction

68. Tip #2 to Foster Interaction:Acknowledge and Encourage Humor Maggi If the program isn't taking into account where I've been, I won't find it very helpful in narrowing my focus to truly useful items. I think it has the "sexy" appeal, of a graphic interface with flying words and all that. However, it's lipstick on a dog of a product. Jeff With the US presidential election in full swing, your "lipstick" comment made me laugh out loud. Thanks, Maggi, for injecting some humor into the discussion. :-) Maggi thanks Jeff, i do try!

70. 5-10 question sets, depending ...

71. Uninfluenced/Interview options reduce work load

72. Even the flow from day to day

73. Be honest with participantsMore Questions = Less Interactivity

74. Summary of Tips to Moderate Boards

75. STEP 9: Observers STEP 9 STEP 8 STEP 7 STEP 6 STEP 5 STEP 4 STEP 3 STEP 2 STEP 1

76. Tips For Managing Observers

77. STEP 10 STEP 10: Analysis STEP 9 STEP 8 STEP 7 STEP 6 STEP 5 STEP 4 STEP 3 STEP 2 STEP 1

78. Notes on Analysis & Reporting The process is exactly the same Some differences: Transcripts from text-based sessions are often easier to read than from offline sessions Vendors of text-based boards include useful tools: Record observations Post tentative hypotheses Flag good quotes

79. Ten Steps to Successfully Design and Execute Online Qualitative Research: A Two-Hour “Crash Course” Thank You! Jeff Walkowski jeff.walkowski@qualcore.com

80. Presented at the Asia-Pacific conference on Qualitative Research in Web 2.0 22 & 23 Feb 2011, Macau SAR For more information Please visit: http://www.merlien.org