1. Conducting, Analyzing, and Reporting
In-Depth Interviews for Program
Evaluation
David Filiberto, Ph.D.
2013

2. Developing an In-Depth Interview Guide
2

3. What is a Structured In-Depth Interview (IDI)?
 Questions are the same for each respondent
 Directly linked to evaluation goals
 Written accounts are created of respondents
answers
 Questions asked usually in same order with careful
consideration given to the wording and order
Ample opportunity for discussion
3

4. ‘Five Ws and one H’
(Who, What, Why, When, Where and How)
• What is the specific purpose of the interviews?
• What information is the client interested in
exploring?
• Who needs this information, and what are they
going to do with it?
Hint: Scope of Work will provide many of the answers
to the questions above
4

5. Steps to Developing the Structured Guide
• Decide what information is needed
• Search for existing questions
• Draft new questions
• Order questions effectively
• Pre-test and pilot the questions
• Revise and pre-test until deemed acceptable
5

6. General
Question Order
• Opening question non-threatening, broadly worded
• Early questions do not restrict what respondents feel they
can say later
• Structure from general to specific
• Ask about present before the past or future
• Group questions according to thematic clusters (work
plan for guidance)
• Before asking about controversial matters, first ask about
some facts
• Last questions allow respondents to provide any other
information - impressions of the interview
Specific
6

7. Conducting an In-Depth Interview
To improve consistency
• Always read the instructions to each
participant as stated
• Initially ask every question as stated in the
order from the guide
To improve neutrality:
• Avoid agreeing or disagreeing with a
respondent
• Avoid indicating that a participant’s answer is
‘right,’ ‘wrong,’ ‘good,’ ‘poor,’ or ‘interesting.’
• Avoid suggesting an answer or interpreting a
question for a respondent
• Avoid giving opinions during the interview
7

8. Summary of the interview process
• Become familiar with the instruments - study the interview
guide
• Revise guide if necessary based on pre-test
• Organize and label all data documentation materials
• Test recording equipment
• Greet the respondent in a friendly manner to begin
establishing positive rapport.
• Briefly describe the steps of the interview
• Conduct the interview according to the interview guide
• Give the respondent the opportunity to ask questions. Thank
them
• Expand your notes and transcribe interview within 24 hours if
possible
8

9. Analysis
Inherently Qualitative – carried out without statistical methods
- detailed descriptions, direct quotations, and observations from the interview
are provided as the foundation of the analysis
Several types of non-statistical or qualitative analysis exist.
– content analysis
– analysis of case studies
– generation of typologies (classification by common traits)
– and logical analysis to name a few.
Content analysis involves a process designed to condense raw
data into categories or themes based on valid inference and
interpretation.
9

10. Overview of Qualitative Analysis
Transcription
Preliminary and Primary
Analysis
Coding
Content
Analysis
Reporting
10

11. Coding: Example
Q: Please describe the current program approach?
A: We have an upstream delivery method, primarily to
retailers, a 3rd party vendor keeps track and sends the bulbs
out. The idea is to lower the price on shelves. The major
constraint is its difficult to know our customer- hard to
evaluate who is buying the bulb and what sockets they are
putting them into. However there are some advantages. We
can control consumer choice, working with retail partners, we
have dominant displays of incentivized bulbs. Yet we think
EISA is going to really impact future savings…
11

12. Coding: Example
A: We have an upstream delivery method, primarily to retailers, a
3rd party vendor keeps track and sends the bulbs out. The idea is
to lower the price on shelves. The major constraint is its difficult to
know our customer- hard to evaluate who is buying the bulb and
what sockets they are putting them into. However there are some
advantages. We can control consumer choice, working with retail
partners, we have dominant displays of incentivized bulbs. Yet we
think EISA is going to really impact future savings…
Respondent Excerpt Code Themes or Categories
1 Upstream delivery method UP Type of Program
Constraint is its difficult to know our Weaknesses of
1 customers NEG Program
1 Control consumer choice POS Strengths of Program
EISA is going to really impact future
1 savings EISA Impact of EISA
12

13. Analysis of IDI Data
1. Primary review – take an interview and develop categories
or themes.
2. Go back to the complete set of interview data. Try out the
preliminary organizing scheme to see if new categories and
codes emerge.
3. Find the most descriptive wording for the topics and turn
them into further categories (or nodes if using NVivo).
4. Reduction - look to reduce the total list by grouping related
topics, illustrating interrelationships.
5. Assemble the data material belonging to each category, or
node, together looking for apparent overarching themes to
abstract to a final report.
13

14. Writing-up Results: Forming the Narrative
• Determine the central message - know your
audience
• Remember the narrative is defined as a story
that will lead the reader through the data,
helping them understand the key issues and
leading them to appropriate conclusions
• Preface report with a summary
• Utilizing the background you have of the
program now try to tell a story and paint a
picture
• Have a plan: know your material and budget
adequate time to write, review, revise and edit
14

15. A Note or Two on Verbatim
Not to do:
• Try not to use quotes to make your points.
• Try not to overuse colorful quotes or examples.
• Changing quotes. Try not to edit or revise quotes.
To do:
• Note the recording time (number of minutes into the
interview) as key points arise.
• Find a balance between the amount of quoting.
• Use quotes to emphasize or as examples that support your
analysis.
• Try to ID quotes which do not require edits or revisions.
15

16. General Rules When Writing
• Empathize: put yourself in the client’s shoes.
• Clarity: avoid jargon, define the unfamiliar.
• Brevity: use words efficiently, most important facts
first, remove redundancy.
• Simplicity: balance detail with audience needs for clarity—
significance is more important.
• Word Choice: avoid needless complexity, avoid ambiguity.
• Active Voice: Technical writers want to communicate as
efficiently as possible and active voice is more straight
forward and is stronger than passive voice.
• Committing to Writing as a Process: requires
planning, drafting, rereading, revising, and editing, self-
review, peer-review, subject-matter expert feedback, and
16
practice.

17. Thank You
David Filiberto
dmf22@cornell.edu