1. Using Data for Impact: A Critical Examination
Through the Lens of Implementation Science
David Pelletier
Division of Nutritional Sciences
Cornell University
Spring CORE Group Meeting, Wilmington, DE
May 1, 2012

2. Outline
1. Why implementation science
2. Definition and state of the field
3. Broadening the field
4. A Consolidated Framework
5. An Example: Training frontline workers
6. An Example: Infant feeding and HIV

3. Why Implementation Science?
p2
• Efficacy-to-Effectiveness
• Knowledge-to-Action
• Evidence-Based Interventions
• Evidence-Based Decisions
• Evidence-Based Policy Making
• Scaling Up
• Etc.

4. Why Implementation Science?
Fixsen et al., p74
U5 Deaths Preventable Through:
Achieving 100% Coverage of Existing Interventions 62%
Discovering and Delivering New Interventions 21%
NIH/BMGF Research Funds (2000-04) Focused on:
Improving delivery and utilization 3%
Mechanistic and discovery research 97%
Source: Leroy et al., AJPH 97(2), 2007

5. Some Overall Observations from the Literature
1. New: Implementation science is a new and still-emerging
field with little consistency in terms, concepts and methods
2. Complex: There are ~40 categories of factors (and hundreds
of variations on these) that affect the quality and impact of
intervention delivery
3. Contextual: The importance and changeability of these
factors varies widely across countries, organizations and
communities. Context is King.
4. Conclusion: The focus of our research must shift from
“technical interventions” to “implementation interventions”

6. What is Implementation Science?
Definition of Implementation (or Delivery) Science
(from NIH Fogarty International Center)
“Implementation science is the study of methods to promote the integration of
research findings and evidence into healthcare policy and practice.
It seeks to understand the behavior of healthcare professionals and other
stakeholders as a key variable in the sustainable uptake, adoption, and
implementation of evidence-based interventions. […]
The intent of implementation science and related research is to
• investigate and address major bottlenecks ,
• test new approaches to prevent, detect and/or resolve bottlenecks
• determine a causal relationship between the intervention and its impact”
(at scale)

7. Current State of the Field
Fixsen et al., p74

8. Current State of the Field

9. Broadening the Definition
1. Beyond “health care”
– To community and prevention
– To other sectors (ag, livelihoods, social protection, etc)
2. Beyond “evidence-based interventions”
– To include IIGPP (next slide)
3. Beyond “research findings and evidence” in the
usual (privileged scientific) sense
– To include practice-based and local knowledge and
experience

10. Beyond “Evidence-Based Interventions”
The Narrow View: Implementation refers to efforts to introduce
evidence-based interventions into the practices of organizations
and professionals (NIH, WHO, etc)
A Broader View: Implementation refers to efforts to fulfill a goal or
the intent of a policy, even when the evidence is lacking
concerning how to do so (European Journal of Social Work 13(1):109, 2010)
An Inclusive View: Implementation refers to efforts to introduce
interventions, innovations, guidelines, practices or policies (IIGPP)
into organizations, systems or communities

11. Beyond “Scientific” Knowledge
Detached,
Body of
Highly-
Generalizable Formal
Structured or
Principles/ M&E
Experimental
Scientific/
Research
Theoretical
Knowledge Integration
Reflection, Application of
Learning, Knowledge in Improved
Documentation, Program Design Action
Codification & Implementation
Contextual
Knowledge Systematized
Practice-Based & Tacit Experience &
Experience & Knowledge Feedback
Engaged During
Research Implementation

12. Opening the Black Box
Toward a Comprehensive Framework for
Implementation Science
New
Intervention Contexts
Proven in Outcomes
A Few Contexts At-Scale
(RCTs) Dissemination &
Implementation
Processes

13. A Consolidated Framework of Implementation
(factors operate at multiple levels, from provider to organization and country)
Context
D. Outer Setting: Participant needs &
resources, govt and donor policies &
regulations, peer/ network influences
, societal & cultural influences, etc.
A. Intervention/ Implementing Organization
Innovation /
E.
Guideline/ Implementation
B. Inner Setting: org
Practice /
size, structure, maturity, manage
Processes:
Policy (IIGPP) •Formative research
(unadapted) ment, informal Implementation
networks, culture, climate •Planning
•Core components •Implem Strategy Outcomes
•Peripheral (tension, compatibility, priority, in
centives, goal clarity, learning •Change Theory Acceptability
components Service
climate), readiness •Engaging 1. Fit Client
Outcomes
(leadership, resources, access to -opinion leaders 2. Feasibility Outcomes
Perceived and
knowledge & info) -formal leaders 3. Costs 1. Efficiency -Health
Actual:
source, IIGPP (adapted) -champions 2. Effectiveness - Function
Application
evidence, advant •Core components -facilitators 3. Equity -Symptoms
age, adaptability, • Execution 1. Adoption 4. Safety -Satisfaction
trialability, compl •Peripheral -components 2. Adaptation 5. Patient-
exity, design components -sequence 3. Fidelity/Quality centered
quality and -intensity 4. Penetration 6. Timeliness
packaging, cost C. Individuals (providers -duration 5. Sustainability
and managers): knowledge -quality
& beliefs, self-efficacy, stage of •Feedback
change, identification with •Evaluation
organization, motivation, values, •Reflection Adapted from:
•Decisions Damschroeder et al., Implem Science 4:50, 2009;
Proctor et al., Admin Pol Mental Hlth 38:65-76, 2011
intellect, competence, capacity, l •Adjustments

14. Do We Need a Framework?
A Simple (Probably Painful) Example
Which of the following best describes the training approach
typically used with frontline workers in your programs?
1. Lectures and Discussion
2. Lectures and Discussion + demonstrations
3. Lectures and Discussion + demos + hands-on practice
and feedback

15. Effectiveness of Training Methods:
Results of a Meta-Analysis
Outcomes for Assessing Effectiveness
Teacher’s Teacher’s Teacher’s
Training Knowledge Skill Use of New
Method Practices
Lectures and 10% 5% 0
Discussion
Why?
+ Demonstrations 30% 20% 0
+ Practice and 60% 60% 5
Feedback
+ Coaching in the 95% 95% 95%
Classroom
Fixsen, D.L., et al., Implementation research: A synthesis of the literature. 2005,
National Implementation Research Network: Tampa, FL.

16. A Consolidated Framework of Implementation
(factors operate at multiple levels, from provider to organization and country)
Context
D. Outer Setting: Participant needs &
resources, govt and donor policies &
regulations, peer/ network influences
, societal & cultural influences, etc.
A. Intervention/ Implementing Organization
Innovation /
E.
Guideline/ Implementation
B. Inner Setting: org
Practice /
size, structure, maturity, manage
Processes:
Policy (IIGPP) •Formative research
(unadapted) ment, informal Implementation
networks, culture, climate •Planning
•Core components •Implem Strategy Outcomes
•Peripheral (tension, compatibility, priority, in
centives, goal clarity, learning •Change Theory Acceptability
components Service
climate), readiness •Engaging 1. Fit Client
Outcomes
(leadership, resources, access to -opinion leaders 2. Feasibility Outcomes
Perceived and
knowledge & info) -formal leaders 3. Costs 1. Efficiency -Health
Actual:
source, IIGPP (adapted) -champions 2. Effectiveness - Function
Application
evidence, advant •Core components -facilitators 3. Equity -Symptoms
age, adaptability, • Execution 1. Adoption 4. Safety -Satisfaction
trialability, compl •Peripheral -components 2. Adaptation 5. Patient-
exity, design components -sequence 3. Fidelity/Quality centered
quality and -intensity 4. Penetration 6. Timeliness
packaging, cost C. Individuals (providers -duration 5. Sustainability
and managers): knowledge -quality
& beliefs, self-efficacy, stage of •Feedback
change, identification with •Evaluation
organization, motivation, values, •Reflection Adapted from:
•Decisions Damschroeder et al., Implem Science 4:50, 2009;
Proctor et al., Admin Pol Mental Hlth 38:65-76, 2011
intellect, competence, capacity, l •Adjustments

17. An Example for Group Work:
The 2010 WHO Guidelines on Infant
Feeding in the Context of HIV

18. The 2010 Guidelines “The Challenges”

19. Policy Choice
ARV+Breastfeeding No Breastfeeding
Basic a. Widespread HIV testing a. Safe water and sanitation are assured
Requirements b. Reliable ARV supplies at household and community level
if child survival is c. Full ARV adherence b. Sufficient infant formula is assured
to be maximized d. Proper EBF adherence c. Can prepare it cleanly and frequently
and transition to mixed d. Exclusive formula for first six months is
feeding possible
e. Family is supportive of this practice
f. Access to health care that offers
comprehensive child health services
•What factors in the framework will affect the policy choice of national policy makers?
•What factors in the framework will affect the policy choice of regional and district managers?
•What factors in the framework will affect the counseling practices of front-line staff?
Note: Factors affecting practices of mothers not included here

20. Group Instructions (30 mins)
1. Each table has been assigned one of the following domains:
A. The intervention, innovation, guideline, practice or policy (IIGPP)
B. Individuals (providers and managers)
C. The Inner Setting (the implementing organization)
D. The Outer Setting (actors and conditions outside the organization
that affect implementation)
E. Implementation processes (activities and practices for “rolling out”
the guideline)
2. Identify some of the characteristics of your domain that may
positively or negatively affect the implementation of the guideline
(“implementation” here refers to adoption, adaptation, penetration to
all clinics, and quality of delivery)
3. How would you collect data or information before implementation in
order to plan your implementation strategy?
4. Chose one example to report out.

21. Further work…
– Does the framework identify “hidden bottlenecks?”
– Are any factors missing from the framework?
– How can the framework be made more user-friendly?
– How can we be innovative in assessing and re-assessing the factors?
– Who needs to be assessing and acting on various ones?

22. Thank You!

23. • Title: Data for Impact: A Critical Examination Through the Lens of Implementation
Science
• Implementation science seeks to improve the effectiveness of large-scale programs
by strengthening the frameworks, tools and evidence base for identifying and
minimizing implementation bottlenecks. Implementers can make major
contributions to this field because of their extensive knowledge, experience and
influence in the implementation process. This session provides an overview of
this emerging field and introduces one of the many conceptual frameworks in the
literature for guiding the assessment and improvement of the implementation
process. It also seeks participants’ views on the forms and meaning of “data” in
the context of real-world implementation by engaging participants in a rapid
application of this framework, using as a case study the WHO Guidelines on infant
feeding in the context of HIV.

24. Assuming that “Breastfeed + ARVs” is the “National Policy”
What factors will influence the counseling practices of health workers
and the feeding practices of mothers?
CFIR Domain CFIR Element
Characteristics of the intervention Perceived strength of evidence
Perceived advantage
Complexity
Characteristics of the individuals Knowledge and beliefs
Identification with the organization
Characteristics of the inner setting Goal clarity
Compatibility
Characteristics of the outer setting Perceived social, cultural and economic
situation of the mother/caretaker
Donor influences on MOH policy
Characteristics of the Implementation strategy (train and hope?)
implementation process Evaluation
Adjustments