The seven principles of persuasion by Dr. Robert Cialdini
The Do’s and Don’ts Of Survey Design
1. The Do’s and Don’ts
of Survey Design
Tips, tricks, and techniques for
making your survey research
more effective
2. What do surveys measure?
Surveys
measure characteristics of people
Behaviors
Attitudes
Beliefs
Opinions
Demographics
Sociographics
3. What do surveys measure?
Surveys
measure characteristics of people
Behaviors
Actions that are directly observable by others.
4. What do surveys measure?
Surveys
measure attributes of people:
Behaviors
Actions that are directly observable by others.
Demographics
Physical characteristics that are directly observable
and directly verifiable.
5. What do surveys measure?
Surveys
measure attributes of people:
Behaviors
Actions that are directly observable by others.
Demographics
Physical characteristics that are directly observable
and directly verifiable.
Sociographics
Social characteristics that are directly observable but
not directly verifiable (i.e., requires confirmation).
6. What do surveys measure?
Surveys
Attitudes, Opinions, Beliefs
Surveys
measure preferences of people:
Wants and needs
Surveys
measure mindsets of people:
measure constructs of people:
Satisfaction, Confidence, Persistence, etc.
7. Types of survey questions
Open response
Partially open response
Closed response
Semantic differential scales
Agreement and rating scales
Ranking scales
Checklists
8. Open response
What was your entry status when you
started your first semester at UNF?
____________________________
9. Partially open
response
What was your entry status when you
started your first semester at UNF?
1.
2.
3.
RECENT HIGH SCHOOL GRADUATE
TRANSFERRED FROM COMMUNITY
COLLEGE OR OTHER INSTITUTION
OTHER________________________
10. Closed
If you could do it all over again,
would you choose to come to
UNF?
1.
2.
3.
YES, I WOULD CHOOSE UNF
NO, I WOULD CHOOSE ANOTHER
UNIVERSITY
NO, I WOULD NOT GO TO
COLLEGE
11. Semantic differential
On a scale of 1 to 5, please indicate
the quality level of your instructors
by checking one of the boxes below:
11
The Best
11
The Worst
11
11
11
12. Agreement scale
The quality of my instructors is
excellent.
1.
2.
3.
4.
5.
Strongly Agree
Agree
Neutral
Disagree
Strongly Disagree
13. Rating scale
Overall, how would you rate the
quality of your instructors?
1.
2.
3.
4.
5.
Very Good
Good
Fair
Poor
Very Poor
14. Ranking scale
Write the letter of each instructor in
the numbered spaces below in order
of their relative quality:
A.
B.
C.
D.
E.
Instructor Jones
Instructor Smith
Instructor Brown
Instructor Lane
Instructor White
1. __________
2. __________
3. __________
4. __________
5. __________
15. Checklist
In which activities did you
participate? (Check all that apply):
STUDENT GOVERNMENT
POLITICAL ACTIVITIES
INTRAMURAL SPORTS
INTERCOLLEGIATE ATHLETICS
COMMUNITY SERVICE
16. Why people answer surveys
Theory
of Social Exchange
Perceived importance
Costs and rewards
Established trust
Anonymity
Peer pressure
18. Common wording problems
Objectionable questions
How old are you? ___________ (WRONG)
Please indicate your age range: (RIGHT)
a. 21 and under
b. 22-35
c. 36-49
d. 50 and over
19. Common wording problems
Unequal comparisons
Which of the following poses the greater threat
to peace in the Middle East? (WRONG)
a. Palestinian attacks against Israeli citizens
b. Israeli attacks against Palestinian militants
c. Both a and b threaten peace
Which of the following must cease for there to
be peace in the Middle East? (RIGHT)
a. Palestinian attacks against Israeli citizens
b. Israeli attacks against Palestinian militants
c. Both a and b must cease
20. Common wording problems
Bias in more than one direction
Are you against same sex marriage and in favor of a
constitutional amendment to ban it? (WRONG)
a. Yes
b. No
c. Unsure
What is your view on same sex marriage? (RIGHT)
a. I think marriage is a matter of personal choice
b. I’m against it but don’t want a constitutional amendment
c. I want a constitutional amendment banning it
21. Common wording problems
Unbalanced categories
Please indicate your age range: (WRONG)
a. 18-20
b. 21-30
c. 31-50
d. 51 and over
Please indicate your age range: (RIGHT)
a. 21 and under
b. 22-35
c. 36-49
d. 50 and over
22. Common wording problems
Emotionally charged words
Was the FDC negligent by ignoring the warnings about
Vioxx during testing and approving it for sale? (WRONG)
a. Yes
b. No
c. Unsure
If the FDC knew that Vioxx caused serious side effects
during testing, what should it have done? (RIGHT)
a. Ban it from ever being sold.
b. Require more testing before approving it
c. Unsure
23. Common wording problems
Threats to self-esteem
How often have you driven drunk? (WRONG)
a. Never
b. Once
c. Two or more times
Have you ever driven a vehicle after you have
had too much to drink? (RIGHT)
a. Never
b. Yes, on one occasion
c. Yes, on more than one occasion
24. Common wording problems
Personalization
Dear Mr. Jones: How would you rate the level of service
that you received from Salesman Mr. Smith? (WRONG)
a. Excellent
b. Good
c. Fair
d. Poor
Dear Valued Customer: How would you rate the level of
service that you received from your salesperson? (RIGHT)
a. Excellent
b. Good
c. Fair
d. Poor
25. The problem with scales of agreement
Artificial position statements are made to fit these formats.
“Agree/Disagree” statements require respondents to
take a position “For” or “Against” an issue.
There is no clear-cut way to interpret one’s agreement
or disagreement with a statement when the proper
response choices are unrelated to agreement.
Too many items using agreement scales are built upon
“face validity” alone. The underlying assumption is that
agreeing with an item indicates a person has a specific
opinion about that item.
“Agreement-disagreement” is not a semantic
differential.
26. Agree-Disagree is not a
semantic differential scale!
Agreeing
with a statement worded positively
may have a different meaning than
disagreeing with the same statement worded
negatively.
True semantic differential items:
Hot
Weak
Very Good
Frequently
Very Helpful
Cold
Strong
Very Poor
Never
Not Helpful at all
27. 1. There should be designated
parking areas for students.
SA
A
N
D
SD
Strongly agree
Agree
Neutral
Disagree
Strongly disagree
“I shouldn’t have to
waste time looking
for parking spaces!”
What was the student thinking when he chose “Agree?”
28. 1. There should be designated
parking areas for students.
SA
A
N
D
SD
Strongly agree
Agree
Neutral
Disagree
Strongly disagree
“I should have a
place to park just
like faculty have!”
not
What was the student thinking when he chose “Disagree?”
29. You can improve survey items by
using a more direct statement of
outcome and a more direct scale
of measurement.
30. Here is a common example
taken from the ISQ survey:
Professor clearly explains complex concepts and ideas.
Strongly Agree
Agree
Neutral
Disagree
Strongly Disagree
No response
And now….a better way to ask this question:
How would you describe your professor’s explanations of
complex concepts and ideas?
1.
2.
3.
4.
5.
Always clear and concise
Mostly clear and concise
Sometimes clear and concise & other times obscure and vague
Mostly obscurely and vague
Always obscure and vague
31. The problem with scales of
satisfaction:
To classify data as "satisfaction information," survey items
are often “shoehorned” into a Satisfaction-Dissatisfaction
format. These items use the same scale format for expediency
sake only, and not because the scale fits the meaning of the
statement.
Satisfaction surveys are good at identifying general problem
areas, but do not lend themselves to specific solutions.
All of the statements are typically couched in positive terms
thus increasing the likelihood of a positive response bias.
In a vague attempt to not be overly positive, many items
overuse the words “Adequate” and “Reasonable.”
32. Keys to making your surveys
better and more useful
Use a scale covering the whole range of responses.
Don’t be afraid of criticism. Make your items balanced.
Use a scale or response list that matches the statement
Identify a potential problem first, and then use survey
data to define its scope and suggest possible
solutions.
Use a survey only for its intended purpose.
Refine your surveys each time you use them.
Don’t forget to include free response input.