The first in the Webinar Series for Mystery Shopping Providers Association Members. This Webinar focuses on how to create efficient questionnaires and great guidelines. We also share some of the pitfalls to avoid.

1. MSPA-AP
Questionnaires and Guidelines
Feb 27, 2013
1Thursday, 30 May 13

2. The heart of Mystery Shopping
Asking the right
questions.
2Thursday, 30 May 13

3. WEBINAR
• Approximately 50 registrations
• New initiative from the MSPA Board
• Complement face-to-face Workshops
• Different but similar to the workshops
• Will be recorded
• All materials belong jointly to the presenter and MSPA and
cannot be copied without authorisation.
3Thursday, 30 May 13

4. PRESENTER
• Steven Di Pietro
• Executive Director Service Integrity
Mystery Shopping
• Global and AP Board roles
4Thursday, 30 May 13

5. WEBINAR HOUSE RULES
• Everyone is on Mute
• You can hold up your hand
• Or type written questions
5Thursday, 30 May 13

6. AGENDA
• The core features of a great questionnaire
• The 12 17 deadly questionnaire mistakes
• Building good guidelines
6Thursday, 30 May 13

7. EXPERTS
• You’re all experts, maybe there’s one
hint, message or question you hadn’t
considered, or
• Something you disagree with.
7Thursday, 30 May 13

8. QUESTIONNAIRES
• Core features
8Thursday, 30 May 13

9. HAVE A GOAL
• Too many questionnaires are aimless. Just saying “Let’s find out
what the customer thinks” is insufficient. Ask yourself the
following questions:
• What will change once the results are received?
• Why am I collecting the data. For example – to get a broad grip
of service, to improve contact centre service, for marketing, for
retention, for staff KPI’s, to drive home strategies? etc.
9Thursday, 30 May 13

10. WHO IS IT AIMED AT?
• Senior Managers? Store Staff?
Regional Managers?
• Drives the detail and frequency.
10Thursday, 30 May 13

11. ACKNOWLEDGETHE ART
• Creating questionnaires is an art.
See section 2
11Thursday, 30 May 13

12. RELEVANCE
• Only ask what is relevant. Not because a
client thinks they can jam it in.
• > Accuracy
12Thursday, 30 May 13

13. DYNAMIC
• Balance trends with collecting relevance.
• If the store had clear signage, it won’t change by the next
round.
13Thursday, 30 May 13

14. CONTROLLABLE AND
ACTIONABLE
• Store reporting should be only
for actionable items.
• Un-actionable items should not
be reported at store level.
14Thursday, 30 May 13

15. MEANINGFUL SCORES
• Does the overall score make sense?
• You can say our satisfaction has increased 2% and people
know what it means. You cannot say the combined Service,
Complaints and Efficiency target has increased 2%, because it
means nothing. If they must be combined, consider using an
index instead of a percentage.
15Thursday, 30 May 13

16. MEASURETHE IMPORTANT
• Measure things important to the customer.
• Survey customers.
• E.g. Don’t ask about security guard uniforms if they are not
important for the scenario.
16Thursday, 30 May 13

17. UNREALISTIC BENCHMARKS
• Moving from 80% to 90%, then 90% to
92%.
• Enough is enough.
17Thursday, 30 May 13

18. FOCUS ONTHE SYSTEM, NOT
THE INDIVIDUAL
• Focus on the store system, or network
system.
• Not the person.
18Thursday, 30 May 13

19. THETREND ISYOUR FRIEND
• For restricted budgets with low
frequencies.
• Aim for less questions, with higher
frequency.
19Thursday, 30 May 13

20. SKIP LOGIC
• If Q3 = N then go to Q 15
• Reduces queries. Faster for shoppers.
20Thursday, 30 May 13

21. 12 17TRAPS
21Thursday, 30 May 13

22. 1.ASKTHE QUESTION ONCE
• Too many questions have repetition
• e.g. asking about satisfaction after every section
of the customer interaction
22Thursday, 30 May 13

23. 2.ASK ONLY ONE QUESTION
• Avoid joining statements like ‘and’.
• Was the staff member wearing the uniform and was it clean?
23Thursday, 30 May 13

24. 3. SMALL SCALES
• Not 1-10
• Use 1-5, or 1-4
• Better still 1 or 0...Yes/No
• Yes/No questions are the sign of a good question
24Thursday, 30 May 13

25. 4.AVOID NEGATIVE
• e.g Was the staff member untidy?
• Difficult to score
• Confuse shoppers, and staff
• They equate a tick with doing a good job.
25Thursday, 30 May 13

26. 5.AVOID LEADING
QUESTIONS
• Don’t set up the answer in the
question.
• We always try to ensure staff are well
presented, was the staff member
wearing the full uniform?
• Leads to shopper sympathy and bias.
26Thursday, 30 May 13

27. 6. REMOVE AMBIGUITY
• Be precise of positive attributes.
• Describe “clean”.
• Describe “Posters in good
condition?”
• Was the ticketing clear? Referring to
font? Size? Position? Consistent?
Present?
27Thursday, 30 May 13

28. 7. NO ENTRAPMENT
• Don’t set staff up for failure.
• Realistic scenarios.
• Don’t ask shoppers to try tricking staff.
28Thursday, 30 May 13

29. 8. BALANCE FREQUENCY
WITH REPORTING
• Infrequent shops e.g. 6 monthly should
be used only for System wide
reporting. Don’t even collect staff
names.
• Frequent shops e.g. monthly are
appropriate for store level reporting
with staff names.
29Thursday, 30 May 13

30. 9. LESS IS MORE
• Don’t ask too many questions.
• 30-35 questions for memorised shops.
• 3 physical attributes (e.g. 1 name, 1 time
measurement, 1 physical attribute such as
marketing material)
30Thursday, 30 May 13

31. 10. USE NORMAL SCENARIOS
• Scenarios should match normal consumer behaviour because:
• otherwise shopper can be exposed,
• otherwise store cannot perform,
• measures the majority of transactions.
31Thursday, 30 May 13

32. 11. STAY INTHE EXPERTISE
• Shoppers should focus on the mystery shop, not giving retail
advice.
• Would you employ this server?
32Thursday, 30 May 13

33. 12.AVOID MEASUREMENTS -
HUH?
• Measurements overload shoppers.
• Usually the information can be
obtained from business system.
33Thursday, 30 May 13

34. 13. NO SCOREKEEPING
• Don’t let shoppers score or even
see the scoring.
• Creates inconsistencies and bias.
34Thursday, 30 May 13

35. 14. BAN N/A
• Not Applicable (N/A) answers are a sign
of a poorly constructed question.
• And are an easy out for the shopper.
35Thursday, 30 May 13

36. 15. USETHE ADDRESS LISTING
• Use the address listing to answer questions
• Use the site listing to answer questions.
• Should not ask about the condition of the bathroom if the
store doesn’t have a bathroom.
36Thursday, 30 May 13

37. 16. REMOVE SIMILAR
ANSWERS
• Answers should be discrete and different.
• Was the staff member a) Polite b)Friendly, c)
Warm?
37Thursday, 30 May 13

38. 17. REDUCE FREETEXT
• Temptation (especially for clients) is to have free text after
each question.
• Creates inconsistencies, queries, longer QC, longer time for
shopper, shoppers usually fabricate the story just to fulfil the
requirements.
38Thursday, 30 May 13

39. SHOPPER GUIDELINES
39Thursday, 30 May 13

40. LENGTH OF GUIDELINE
• Long guidelines = poor questionnaire.
40Thursday, 30 May 13

41. GUIDELINE OVERLOAD
• Guidelines against each question = poor
questionnaire
41Thursday, 30 May 13

42. UNNECESSARY GUIDELINES
• Self explanatory questions
don’t need a guideline.
• Important guidelines lose
focus.
• e.g. - this is a clean
bathroom
• e.g. - this is an empty
garbage bin
42Thursday, 30 May 13

43. CONSISTENT WITH
QUESTIONNAIRE
• Obvious but some questionnaires are
inconsistent with the guidelines.
• e.g. Guidelines say take two photo’s and the
system questionnaire asks for 1 photo.
43Thursday, 30 May 13

44. LENGTH OF GUIDELINES
• Shoppers cannot be expected to read
40 pages of guidelines when they are
being paid $15 for the whole
assignment.
44Thursday, 30 May 13

45. The heart of Mystery Shopping
Asking the right
questions.
45Thursday, 30 May 13

46. NEXT
• Survey
• Contact me directly
• sdipietro@serviceintegrity.com.au
46Thursday, 30 May 13