1. Questionnaire Design
&
Administration
Lecture 6

2. Chapter 9
Questionnaire design
2

3. Learning Objectives
• Discuss the process of questionnaire design
• Draft, pre-test and revise questionnaires
• Identify the potential sources of error that can
occur when conducting a survey
• Explain the issues involved in the
administration of questionnaires using
different survey methods
• Discuss the ethical issues involved in
collecting data from respondents

4. Questionnaire Design & Administration

5. A survey is only as good as the
questions it asks
• Questionnaire design is one of the most
critical stages in the survey research process.
– Importance of question wording is easily
overlooked.
• Relevance and accuracy are the two basic
criteria for questionnaire design.
5

6. Questionnaires
• Standardise data collection for survey
• Ensure comparability of the data, increase
speed and accuracy of recording and
facilitate data processing.
Definition
• Formalised set of questions for obtaining
information from respondents

7. Questionnaire design
• Planning a questionnaire‟s design involves
several decisions:
– What should be asked?
– How should questions be phrased?
– In what sequence should the questions be
arranged?
– What questionnaire layout will best serve
the research objectives?
– How should the questionnaire be pre–
tested? Does the questionnaire need to be
revised?
7

8. What should be asked?
• A questionnaire is relevant if no unnecessary
information is collected and only the
information needed to solve the marketing
problem is obtained.
• A questionnaire is accurate if the information
obtained is reliable and valid.
8

9. How should questions be phrased?
• Open–ended response questions pose some
problem and ask the respondent to answer in
his or her own words.
– Most beneficial when conducting
exploratory research.
• Fixed–alternative questions give the
respondent limited–alternative responses
and ask him or her to choose one closest to
his or her viewpoint.
– Takes less time and easier to answer.
9

10. Objectives of a Questionnaire
• Translate the information needed into a set of
specific questions that the respondents can and
will answer
• To uplift, motivate, and encourage the respondent
to become involved in the interview, to cooperate,
and to complete the interview
• Questionnaires should minimise response error

11. Questionnaire Administration
• Personal interviews
– Lengthy, complex and varied
• Telephone interviews
– Short and simple, conversational style
• Mail surveys
– Simple questions, detailed
instructions

12. Questionnaire Design
• Formulate questions to obtain the required
information
• Question order and wording
• Sequence and layout
• Pre-testing
– correct and pre-test again
Suggest questions/statement you could
ask to measure:
•Satisfaction with event/ establishment
•Motivation to participate
•Age
•Determinant attributes

13. Formatting the Question
• Open-response questions
• Closed-response questions

14. Open-ended Response
Questions
• Open-response (unstructured) questions
• Open-ended questions that respondents
answer in their own words
• Express general attitude
• Interviewer bias is high
• Coding is costly and time consuming
What were you expecting by attending this
event?

15. Open-ended Response
Questions cont.
• Used for:
– introduction to a survey
– when responses are unforeseen or
there are too many to list
– when verbatim responses are
important
– when behaviour being measured is
sensitive

16. Closed Response Questions
• Specify the set of response alternative and
response format
• Two alternative formats:
– choice from a list of responses
– choice on a rating scale

17. Closed Response Questions
cont.
• Advantages
– Can be administered quickly
– Coding and processing is less costly and
time consuming
– Easy to answer and easier for interviewer
– Tabulation and analysis easier
– Less potential error
– Responses are directly comparable

18. Closed Response Questions
cont.
• Limitations
– Effort required in designing alternatives
(decision on type of responses)
– May require exploratory research for
determining choices (difficult to develop
good questions)
– Bias response because people pick
alternatives offered (less ability for self-
expression)
– Potential order bias

19. Closed Response Questions cont.
• Multiple-choice questions
– Respondents are asked to select one or more
alternative
Which of the following people had an
influence on your choice of university?
None 
Parents 
Friends 
Ex-Uni student 
Teachers at secondary school 
Careers teachers at secondary school 
Colleagues 
Other please specify…………………. 

20. Closed Response Questions cont.
• Scales
– Comparative and non-comparative
(covered last week)
• Dichotomous questions
– Only two response alternatives
– Mainly used for demographic and
behavioural data
Have you holidayed in Asia in the last 3
years?
Yes No 

21. Question Wording
• Vocabulary should be simple clear and direct
• Words should not be vague or ambiguous
• No „double-barreled‟ questions
• No leading or loaded questions
• Instructions should be clear
• Respondents should only answer applicable
questions
• Test several alternatives of question wording
• Questions should be appropriate length
• Word sensitive questions carefully

22. Question Wording cont.
Is the question necessary?
• Every question should contribute to the
information needed or serve some specific
purpose.
• Question wording should address the
research objectives in order to make better
decisions

23. Question Wording cont.
Is the respondent informed?
• Filter questions are often used to ascertain
familiarity, product use and past experience
– Are you the main grocery buyer in the
household?
– Have you seen the Holden Cruze ad?
– Have you purchased car tyres in the last 3
months?
• If respondents answer “No”, they may not be
suitable respondents (i.e., they have not met the criteria for
inclusion in the study)

24. Question Wording cont.
Can the respondent remember?
• Inability to remember can lead to
– Errors of omission
– Telescoping
– Creation
• The ability to remember events are influenced
by
– the event itself
– time elapsed since the event
– presence/absence of other events that
would help memory

25. Question Wording cont.
Can the respondent articulate?
• Respondents may be unable to describe the
situation accurately
• Inability to articulate may lead to
– respondents ignoring questions
– refusing to respond to the rest of the
questionnaire

26. Question Wording cont.
Legitimate Purpose
• Unwilling to divulge information which they do
not see as serving a legitimate purpose
• Specify why the question is being asked

27. Sequence and Layout Decisions
Opening Questions
• Important in gaining confidence and co-operation
of respondents
• Questions should be interesting and non-
threatening
• Could be qualifying question

28. Sequence and Layout Decisions
cont.
Type of information
Basic
Classification
Identification

29. Sequence and Layout Decisions
cont.
Difficult questions
• Sensitive, embarrassing, complex or dull
questions should be placed late in the
sequence
Effect on subsequent question
• Questions asked early in a sequence can
influence the responses to subsequent
questions
• Use funnel or flowerpot approach

30. Sequence and Layout Decisions
cont.
Logical order
• All questions that deal with a particular issue/topic
should be asked before proceeding to a new topic
• Use branching questions to direct respondents to
other questions

31. Form and Layout
• Format, spacing and position are important
• A questionnaire can be divided into several
parts
• All questions should be numbered
• Questionnaires are sometimes pre-coded
where each possible response to a question
is associated with a unique number (or
letter)
• Questionnaire should be serially numbered

32. What sort of strategies do you use to ensure high response
rates in surveys?
Michael Sparks, Director
Intuitive Solutions

33. Reproduction of the Questionnaire
• Use good quality paper and have a professional
appearance
• Consider using booklet if the questionnaire runs to
several pages
• Avoid splitting a question and response categories
over two pages
• Use single columns for response categories
• Avoid overcrowding questions
• Directions or instructions for individual questions
should be placed close to the questions as
possible e.g., [PROBING]

34. Pre-testing and Correcting
Problems
• Testing the questionnaire on a small sample of
respondents to identify and eliminate potential
problems
• Use protocol analysis or debriefing
• After each significant revision of the questionnaire,
another pre-test should be performed with a
different sample

35. Pre-testing
• Debriefing approach
– respondent completes questionnaire with
interviewer observing
– interviewer then debriefs respondent
• Protocol approach
– respondent asked to think aloud as they
complete questionnaire
– interviewer takes notes and asks for any
further clarification at the end

36. Pre-testing cont.
•Specific •The
Questions Questionnaire
–Variation –Sequence
–Meaning –Form and layout
–Task difficulty –Skip patterns
–Respondent –Length
interest and –Instructions
attention

37. Cover Letter and Openers
• Main role : to win over
respondent‟s co-operation and
willingness to participate
• Respondents must be aware that a
bonafide survey is taking place
• Identify the sponsor
• Purpose of survey should be
described clearly and simply

38. Cover Letter and Openers cont.
• Prospective respondents must be made aware of
how and why they were selected
• Assure respondent confidentiality
• Specify the completion date
• Cover letter should solicit the prospect
respondent‟s help in the survey
• Thank you statement for willingness to
participate

39. Observational Forms
• Specify the who, what when, where, why and
way of behaviour to be observed
• Offers consistency, structure, completeness

41. Administering the Questionnaire:
Personal Interviews
• Identification of location/venue
• Recruitment of field staff
– Develop job specifications for the project
– Decide on the characteristics the fieldworkers
should possess
– Recruit appropriate individuals
• Training of field staff
– Conducted at central location or by mail
– To ensure questionnaire is administered in the
same manner

42. Training of Field Staff
Making the initial contact
• To encourage respondents to participate
• Convince potential respondents that their
participation is important

43. Training of Field Staff cont.
Asking the question
• Be thoroughly familiar with the questionnaire
• Ask the question in the order in which they
appear
• Use exact wording given in the questionnaire
• Read questions slowly
• Repeat questions that are not understood
• Ask every applicable question
• Follow instructions

44. Training of Field Staff cont.
Probing
• Motivate respondents to enlarge on, clarify, or
explain their answers
• Should not introduce any bias
• How to probe??
– Repeat the question
– Repeating the respondent‟s reply
– Using a pause or silent probe
– Boost or reassure respondents
– Eliciting clarifications
– Using objective / neutral questions or comments

45. Training of Field Staff cont.
Recording the answers
• Same format should be used
• Structured Tick box that reflects the respondent‟s
answer
• Unstructured Record the responses verbatim
– Record responses during the interview
– Use the respondent‟s own words
– Do not summarise or paraphrase the respondent‟s answers
– Include everything that pertains to the question objectives
– Include all probes and comments
– Repeat the response as it is written down

46. Training of Field Workers cont.
Terminating the Interview
• Terminate after all information is obtained
• Record spontaneous comments
• Interviewer should answer the respondent‟s
questions about the project
• Thank respondent and express appreciation

47. Field Editing and Checking
Quality control and editing
• Check that questions have been completed,
do not accept incomplete answers, writing is
legible
• Keep record of hours worked and expenses

48. Field Editing and Checking cont.
Sampling Control
• Ensure interviews are strictly following
sampling plan.
• Keep record of:
– number of calls made,
– number of not-at-homes,
– number of refusals,
– number of completed interviews per interviewer.
Control of cheating
• Falsifying part of a question or entire
questionnaire.
• Cheating can be minimised through proper
training, supervision and validation of field
work.

49. Field Editing and Checking cont.
Validation
• Check for authenticity.
• 10 – 25% of respondents are contacted.
• Ask about length and quality of the interview,
reaction to the interviewer, and basic demographic
data.

50. Field Editing and Checking cont.
Evaluation
• Provide feedback to interviewers on:
– Cost and time
– Response rates
– Quality of interviewing
– Quality of data

51. Telephone Interviews
• Similar to personal surveys plus:
– selecting the telephone numbers
– call outcomes
– the introduction
– when to call
– [6-8pm week days, 10-8 weekends]
– Call report

52. Mail Surveys
• Identification of respondents
• Mail interview package
– Outgoing envelope
– Reply paid envelope
– Postage
– Method of addressing
– Cover letter
– Questionnaire length, content, layout etc
– Method of notification
– Incentives
– Follow up

53. Online Surveys
– Similar to mail surveys
– Design the cover letter/email
– Technology issues should be addressed

54. Salaries and Wages
Classification Per hour rate*
Market Research Interviewer $21.72
Door-to Door $23.72
Editor/Coder/Key Punch Operator $22.12
Field Supervisor $26.32
Research Assistant $26.32
Research Officer $29.58
Research Manager $42.20
* Casual rates (Australian dollars)
Source: NUW and AMRO (2005) Report on the Market Research Industry Agreement 2005 - 2008

55. Non-response Issues
• Non-response error arises when potential
respondents included in the sample do not
respond.
• Major problem is whether non-respondents
differ from respondents in terms of
demographic, psychographic, personality,
attitudinal, motivational and behavioural
variables.
• To reduce non-response bias: improve the
response rate and/or adjust for non-
response.

56. Improving Response Rates

57. Sources of Error
• Inaccuracy in response
– inability to respond
– unwillingness to respond
• Interviewer error
– lack of interviewing skills
– fraud and deceit

58. Unethical Practices in Data Collection
• Using of surveys as a disguise for selling purposes
• Using details in survey to develop a prospect list
• Disguising the purpose of the measurement
• Deceiving the respondent as to the true duration of
the interview
• Misrepresenting the compensation in order to obtain
co-operation
• Not informing respondents of follow-up interviews
• Using hidden taping without respondent‟s consent
• Conducting simulated product test in which identical
products are given
• Not debriefing the respondent

59. Types of fixed–alternative
questions
• Simple–dichotomy: choose one of two
alternatives.
• Determinant–choice: choose one and only
one from among several alternatives.
• Frequency–determination: asks for general
frequency of occurrence.
• Checklist: choose multiple answers to a
single question.
59

60. Phrasing questions for surveys
• Means of data collection will influence the
question format and phrasing.
– Questions for mail, Internet, and telephone
surveys are less complex than those used
in personal interviews.
– Questionnaires for telephone and personal
interviews should be written in a
conversational style.
60

61. The art of asking questions
• Some guidelines to help prevent the most
common mistakes:
– Avoid complexity: use simple,
conversational language.
– Avoid leading and loaded questions.
– Avoid ambiguity: be as specific as
possible.
– Avoid double–barrelled items.
– Avoid making assumptions.
– Avoid burdensome questions that may tax
the respondent‟s memory.
61

62. What is the best question sequence?
• Order of questions serve several functions.
– Opening questions influence respondents‟
cooperation and involvement.
– Demographic questions that may
embarrass or threaten respondents should
be asked at the end of the questionnaire.
• Order bias may distort survey results.
– Funnel technique: asking general
questions before specific questions.
62

63. What is the best question sequence?
• A filter question screens out respondents who
are not qualified to answer a second
question.
– A pivot question is used to determine
which version of a second question will be
asked.
63

64. What is the best layout?
• Layout should be neat, attractive and
interviewer instructions easy to follow.
– Decent margins, white space to separate
blocks of print, etc.,
• Should appear as short as possible.
– Multiple–grid question
• Layout is also important for Internet
questionnaires.
– Discrepancies in screen configuration,
push button, status bar, radio button, etc.,
64

65. What is the best layout?
65

66. What is the best layout?
66

67. How much pre–testing and revising
are necessary?
• Questionnaire is usually tried out on a group
that is similar in make–up to the ultimate
sample.
• A preliminary tabulation of pre–test results
helps determine whether the questionnaire
will meet the objectives of the research.
• Pre–tests provide means for the testing
sampling procedure.
67

68. How much pre–testing and revising
are necessary?
• Pre–tests typically are conducted to answer
the following questions about the
questionnaire:
– Can the format be followed by interviewer?
– Does it flow naturally and conversationally?
– Are the questions clear and easy to
understand?
– Can respondents answer the questions
clearly?
– Which alternative forms of questions work
best?
68

69. Designing questionnaires for global
markets
• Researchers must take cultural factors into
account when designing questionnaires.
• The most common problem involves
translating a questionnaire into other
languages.
– International marketing researchers often
have questionnaires back translated.
69