2. Research Methodology and
Evidence Based Healthcare (EBHC481)
Why is it important?
1. Statistical analysis can be repeated
2. If done incorrectly initially, there is
seldom a second chance to obtain
information by questionnaire from large
numbers of participants (Methods in
observational epidemiology)
3. Research Methodology and
Evidence Based Healthcare (EBHC481)
What Is A Questionnaire?
A tool for collecting information to
describe, compare, or explain an event or
situation, as well as, knowledge, attitudes,
behaviors, and/or socio-demographic
characteristics on a particular target group.
4. Research Methodology and
Evidence Based Healthcare (EBHC481)
Procedure for Development
โข Specify what information will be sought (back to
objectives)
โข Determine the type of questionnaire and
method of administration
โข Determine the content of individual questions
โข Determine the form of response to each
question
โข Determine the wording of each question
5. Research Methodology and
Evidence Based Healthcare (EBHC481)
How To Obtain Valid Information
โข Ask purposeful questions
โข Ask concrete questions
โข Use conventional language
โข Use complete sentences
โข Avoid abbreviations
โข Review questions with experts and potential
respondents (piloting)
โข Use shorter questions
7. Research Methodology and
Evidence Based Healthcare (EBHC481)
Steps to develop questionnaire
1. Decide on the information required.
2. Define the target respondents.
3. Choose the method(s) of reaching your target
respondents.
4. Decide on question content.
5. Develop the question wording.
6. Put questions into a meaningful order and format.
7. Check the length of the questionnaire.
8. Pre-test the questionnaire9. Develop the final
survey form.
8. Research Methodology and
Evidence Based Healthcare (EBHC481)
Points for consideration
1. A well-designed questionnaire should meet the research
objectives.
2. It should obtain the most complete and accurate information
possible. A good questionnaire is organised and worded to
encourage respondents to provide accurate, unbiased and
complete information.
3. A well-designed questionnaire should make it easy for
respondents to give the necessary information and for the
interviewer to record the answer, and it should be arranged so
that sound analysis and interpretation are possible.
4. It would keep the interview brief and to the point and be so
arranged that the respondent(s) remain interested throughout
the interview.
9. Research Methodology and
Evidence Based Healthcare (EBHC481)
Questions Context
โขStandardize the interviewer
โขStandardize the response format
โขAsk questions in a social, cultural, and
economic context
โขKeep confidentiality and voluntary
participation
โขInclude a letter of introduction or
presentation
10. Research Methodology and
Evidence Based Healthcare (EBHC481)
Questions Format
โข Open questions - more information but
difficult to codify, enter, and analyze
โข Closed questions - less information but easy to
codify, enter, and analyze
โข Fixed-response (closed-ended):
โข Yes-No
โข True-False
โข Multiple Choice
โข Rating Scale/Continuum (e.g. a Likert-type scale)
โข Agree-Disagree
โข Rank ordering
11. Research Methodology and
Evidence Based Healthcare (EBHC481)
Questionnaire: type
โข Structured: same questions in the same way
โข closed-ended questions
โข open-ended questions
โข precoded (self-coded) questions
โข Unstructured: questions may vary
Written document used to obtain information from respondents
12. Research Methodology and
Evidence Based Healthcare (EBHC481)
Type Examples
Open-ended question What is your current marital
status ?
Closed-ended question
(pre-coded/ not pre-
coded)
Are you currently (circle one):
married ? [1]
widowed ? [2]
divorced ? [3]
separated ? [4]
or have you never been married ?
[5]
13. Research Methodology and
Evidence Based Healthcare (EBHC481)
Questionnaire: Scaling
โข Rating scales
โข along a continuum according to the
strength of his or her attitude
โข Magnitude scaling
โข pick up an numeric score
No pain Worst possible pain
14. Research Methodology and
Evidence Based Healthcare (EBHC481)
14
Principles of Questionnaire Writing
โข Be clear and precise.
โข Response choices should not overlap.
โข Use natural and familiar language.
โข Do not use words or phrases that show bias.
โข Avoid double-barreled questions.
โข State explicit alternatives.
โข Questions should be reliable and valid.
15. Research Methodology and
Evidence Based Healthcare (EBHC481)
15
Principles of Questionnaire Writing
โข Be clear and precise.
โข Bad: โTell me what you think about this product?โ
โข Good โOn a scale from 1 to 10 how fresh is this bagel?โ
(circle one)
Very Stale 1 2 3 4 5 6 7 8 9 10 Very Fresh
โข Response choices should not overlap.
โข Bad: What is your income level?
$0 - $10,000
$10,000 - $20,000
$20,000 - $30,000
$30,000 or more.
16. Research Methodology and
Evidence Based Healthcare (EBHC481)
16
โข Use natural and familiar language.
โข Bad: โMoribund 1 2 3 4 5 6 7 8 9 10 Livelyโ
โข Do not use words or phrases that show bias.
โข Bad: โWhy isnโt this brand better liked?โ
โข Good: โWhat is it that you like least about this brand?โ
โข Avoid double-barreled questions.
โข Bad: โDo you like the taste and the feel of this brand?โ
โข State explicit alternatives.
โข โDoes it seem possible or does it seem impossible to
you that starting cigarrete smoking never happened?โ
Principles of Questionnaire Writing
17. Research Methodology and
Evidence Based Healthcare (EBHC481)
Things to avoid in questionnaires
What to avoid Example
Ambiguous questions Are you currently single, married, divorced, separated, or
widowed ?
Single ? Divorced?โฆโฆ..
Have you ever smoked ?
Cigarette? Pipes? Marijuana?โฆ.
Questions that are
not self-explanatory
What kind of home do you have ?
Nice? Large? An apartment? โฆโฆ..
Two questions in one Do you get out of breath when you walk up a hill ?
โNoโ means โฆโฆ..
Use of words that
some respondents do
not understand
What was your age at menarche ?
? โฆโฆ...โฆโฆ..
Have you ever worked with polycyclic aromatic hydrocarbons ?
? โฆโฆ...โฆโฆ..
18. Research Methodology and
Evidence Based Healthcare (EBHC481)
Things to avoid in questionnaires
What to avoid Example
asking about events
that most people will
not remember
How many times per week did you eat carrots when you
were between the ages of 6 and 10 ?
hypothetical
questions
Would you like to move to another place of residence ?
Of course. I wish I could โฆโฆโฆ..
insufficient numbers
of categories
When you go camping do you sleep in a camper, trailer, or
tent ?
I usually sleep in the open air โฆ...
Useful phrase: โothers specifyโฆ..โ
questions that are too
long, with too many
ideas
Have you ever used birth control pills, estrogen
replacement therapy, tranquilizers, or diuretics ?
โRepeat againโฆโฆโฆโฆโ
19. Research Methodology and
Evidence Based Healthcare (EBHC481)
Things to avoid in questionnaires
What to avoid Example
overlapping intervals How many times a day do you lift objects weighing 25 pounds or
more on the average ? Never ? 5 times or less per day ? 5-25
times per day ? 25 or more times per day ?
questions that
require too much
detail
Tell me each brand of cigarettes that you have smoked and for
exactly how long you have smoked each one since you first
started smoking.
Give me a break โฆโฆโฆโฆโฆ..
leading questions Have you had any infectious disease in the past year, such as
mononucleosis ?
Mononucleosis ? โฆโฆโฆโฆโฆ..
degree of accuracy
not specified degree
of accuracy not
specified
How tall are you ?
170 cm
171.8 cm
about 7 feet
6 feet and 10 inches โฆโฆโฆโฆโฆโฆ..
20. Research Methodology and
Evidence Based Healthcare (EBHC481)
Examples of BAD questions
โข Biased/leading question
โข Incorrect: Community organizing is hard. Do
leadership trainings help you feel prepared for
community organizing?
โข Much more prepared [ ]
โข Somewhat more prepared [ ]
โข Slightly more prepared [ ]
โข Not more prepared [ ]
โข Correct: The leadership trainings prepare me for
community organizing.
โข Strongly agree [ ] Agree [ ] Disagree [ ] Strongly disagree [ ]
21. Research Methodology and
Evidence Based Healthcare (EBHC481)
Piloting the questionnaires
โข It is necessary to pre-test the questionnaire before it is used in
a full-scale survey, to determine whether :
1. the questions as they are worded will achieve the desired
results
2. the questions have been placed in the best order
3. the questions are understood by all classes of respondent
4. additional or specifying questions are needed or whether
some questions should be eliminated
5. the instructions to interviewers are adequate.
โข Usually a small number of respondents are selected for the
pre-test. The respondents selected for the pilot survey should
be broadly representative of the type of respondent to be
interviewed in the main survey.
22. Research Methodology and
Evidence Based Healthcare (EBHC481)
A take-home message
โข Good questionnaires require many, many
drafts, and thus several months may be
necessary for questionnaire development.
โข Also, it is essential to try out the
questionnaire on a variety of people,
including those similar to the people who
will actually be in the study.
23. Research Methodology and
Evidence Based Healthcare (EBHC481)
Practical Task
โข Based on the topic and objectives you have
chosen in the previous sessions:
โข Start designing your questionnaire fulfilling the
following:
โข Decide on the sections/subheadings of the
questionnaire
โข Add 2-3 questions under each subheading
โข In pairs, start asking each other these
questions
โข Give feedback