1. Designing for Dissemination
and Implementation
Making Research More Relevant &
Actionable for Translation to Other
Populations and Settings
Lawrence W. Green

2. The Challenges & Opportunities
• The two biggest challenges:
– To close the gap between what policy makers,
program planners, practitioners and
communities need & what they are getting from
our research
– Reform some peer review & editorial tendencies
• The two biggest opportunities
– Extend CBPR principles to work with policy
makers, program planners & practitioners in
use of natural experiments—e.g., evaluation
– Combine PR with multi-site RCT methods that
expand the external validity of the results

3. Where Have All the Data Gone? Longtime Passing…
17 yrs
―It takes 17 years to turn 14 per cent of
original [applied] research
to the benefit of patient care‖ *
*Balas & Boren, 2000.
Reviews, guidelines, textbooks
Unknown 0.5 year 0.6 year 0.3 year 6. 0 - 13.0 years 9.3 years
Bibliographic databases
Kumar, 1992 Kumar, 1992 Poyer, 1982 Antman, 1992
Original research
Implementation
Acceptance
Submission
Publication
18% Negative
46% Lack of Lack of 35% 50%
results numbers, numbers, *Balas, 1995 Inconsistent
Poynard, 1985
Dickersin, 1987 Koren, 1989 indexing
Design issues Design issues

4. From the Levy Arrow to the Pipeline Fallacy of
Producing & Vetting Research to Get
Evidence-Based Practice*
Basic
Research The 17-year odyssey
Guidelines for Practice -
Research Evidence-Based Funding/patient or
Publication Synthesis Practice Population needs,
Peer Review Priorities & demands; local
Of Grants Peer Review practice or policy
Priorities for Evidence-based circumstances;
Research Funding Medicine professional
Academic appointments, movement discretion;
promotion, & tenure credibility & fit of
Impact Factor criteria the evidence.
Scoring** Blame the practitioner
*Green LW (2008). If it’s an evidence-based practice, where’s the
or blame dissemination
practice-based evidence? Family Med 25 (suppl_1): 20-24;

5. CDC’s Model for Research to Practice
& Policy
The research to
The CDC Wedge practice gap is here?
Adapted from Green LW, Popovic T et al, CDC Futures Group on Research, Atlanta: CDC, 2004;
Sleet DA et al. Health Promot Pract 2003;4:98-102; & Hanson et al PHR 2012

7. Mediating and Moderating Variables
Mediator
Intervention Outcome
or Program Variable(s)
Mediator
Moderators Moderators
Green L & Kreuter M. Health Program Planning: An Educational and
Ecological Approach. 4th ed. New York: McGraw-Hill, 2005.

8. The Prevailing Standard of Evidence:
The Randomized Controlled Trial
Change in
Mediating
outcome
variables
Intervention variable(s)
expected to
tested by measured
change, based
comparison with and
on previous
a control compared
evidence and
condition ? between
theory
experimental
& control
Context groups
Context
--Interventions highly standardized. --Comparison based on average
--Interventions reduced to simplistic form change for each group
--Everything else held constant. --Subgroup analysis discouraged
--Clients randomized, no choice. --Dropouts discounted, ignored
--Interventionists highly trained, restrained & --Cut-off date for outcomes often
supervised; no discretion. too soon for change to occur

9. Problems Identified by IOM Report*
(www.nap.edu)
• Narrow focus: Lack of attention to larger
systems context
• Lacking details of implementation process
• Lack of relevance to real world
• Many studies focus on one intervention, but
obesity may require a combination of
interventions; in fact, some things appear not to
work when tested alone, but are essential
ingredients in a more comprehensive program
*Institute of Medicine. Bridging the Evidence Gap in Obesity Prevention: A Framework to
Inform Decision Making. Washington, DC: The National Academies Press,
2010. Full text online at www.nap.edu.

10. IOM Conclusions about
Status of Evidence
• The current evidence lacks the power to set a
clear direction for obesity prevention across a
range of target populations
• This lack of evidence for effectiveness seen
as a lack of effectiveness
• It is difficult to fund, conduct & publish
research on community, environmental, and
policy-based obesity prevention initiatives
• Assessing or reporting on generalizability of
research results to other populations or
settings has not been given priority

11. Scientists: Evidence-based Practice
Is it valid?
Is it important?
Is it useful?
High internal validity + Low external validity = Diminished
G. Ramirez,
7/30/2009 relevance for practice or policy!

12. Paradigm Shift: Practice-based Evidence
What’s useful?
and important?
What is
sufficiently valid?
High external validity + Relaxed internal validity = Increased
G. Ramirez,
7/30/2009 relevance for practice!

13. Types of Community-Engaged
Evidence for Health Research
• Participatory research evidence
– Community-Based Participatory Research
(CBPR)
– Practice-based or action research
• Surveillance evidence
• Population diagnostic evidence
• Program evaluation evidence
– Multi-component
– Continuous quality improvement
– How context effects (moderates) outcomes

14. Change in Per Capita Cigarette Consumption
California & Massachusetts vs Other 48 States, 1984-1996
5
0
Percent Reduction
-5
-10
-15
-20
-25
Other 48 States California Massachusetts
1984-1988 1990-1992 1992-1996

15. http://www.cdc.gov/tobacco

16. The Spheres of Practice-Based, Community-
Based, Academic & Participatory Research
Practice-
Participatory
Based
Research
Research
The sweet spot
Highly for implementation CBPR
Controlled
Community-
Academic Based
Research
Research

17. Definition and Standards of Participatory
Research for Health*
Systematic investigation…
Actively involving people in a co-learning
process…
For the purpose of action conducive to
health
--not just involving people more intensively
as subjects of research or evaluation
*Green, George, Daniel, et al., Participatory Research…Ottawa: Royal
Society of Canada, 1997. www.lgreen.net/guidelines.html

18. Three Paradoxes
• The internal validity–external validity paradox
– The more rigorously controlled a study testing the efficacy
of an intervention, the less reality-based it becomes, so it
cannot be taken to scale or generalized
• The specificity – generalizability paradox
– The more relevant and particular to the local context, the
less generalizable to other contexts, yet the more it is
perceived to be practice-based ―like us‖
• The homophily–social distancing paradox
– The effectiveness of indigenous health workers uses their
commonalities with the community, but they seem to lose
that with increased professionalization;
– obverse paradox for scientists…

19. Number of Publications on CBPR
Based on Scopus Search*
Publications on CBPR
450
400
350
300
250
200 Publications
150
100
50
0
1987 1990 1992 1994 1996 1998 2000 2002 2004 2006 2008 2010
*Based on unpublished Scopus review by Doug Brugge, Tufts U., 2011.

20. The Lenses of Scientists, Health
Professionals and Lay People
Subjective
Indicators
of Health
Professional, Layperson
Scientific
Objective
Indicators
of Health

21. The Multi-Site Translational Community
Trial (mTCT) Proposal*
• Blends the internal validity advantages of
– Cluster randomized trial or multi-site RCT
– Fidelity to the function (but not the form) of an
efficacy-tested intervention
• With the external validity advantages of
– Diversity of settings, cultures, circumstances
– Adaptation of the form (not the function) of the
efficacy-tested intervention
• With some sacrifice of CBPR degrees of freedom
*Katz DL et al. From controlled trial to community adoption…Am J Public
Health. Published online ahead of print, June 16, 2011: e1–e11. .

22. Aligning Evidence* with (and deriving it from)
Practice: Matching, Mapping, Pooling & Patching
• Matching ecological levels of a system or community
with RCT evidence of efficacy for interventions at
those levels
• Mapping theory to the causal chain to fill gaps in the
evidence for effectiveness of interventions
• Pooling experience to blend interventions to fill gaps
in evidence for the effectiveness of programs in
similar situations
• Patching pooled interventions with indigenous
wisdom and professional judgment about plausible
causes & interventions to fill gaps in the program for
the specific population
*Green & Kreuter, Health Program Planning: An Educational and Ecological
Approach. 4th ed. NY: McGraw-Hill, 2005, Chap 5. Green & Glasgow, 2006.

23. • Take-home points of ―Designing for
Dissemination and Implementation‖
– Consider the trade-offs between internal and
external validity of original research and
evaluation
– If we want more evidence-based practice, we
need more practice-based evidence
– Extend CBPR principles to work with policy
makers, program planners & practitioners in
use of natural experiments—e.g., evaluation
– Combine RCTs with CBPR in multi-site trials
(Katz et al., AJPH, 2011)

24. Some Applications
• Katz DL, Murimi M, Gonzalez A, Nijike V, Green LW. From clinical trial to community
adoption: The Multi-site Translational Community Trial (mTCT). Am J Public Health.
Published online ahead of print June 16, 2011: e1–e11.
doi:10.2105/AJPH.2010.300104, 2011.
• Green LW, Travis JW, McAllister RG, Peterson KW, Vardanyan AN, Craig A.
Male circumcision and HIV prevention: Insufficient evidence and neglected
external validity. Am J Prev Med. Nov 2010;39(5):479-82.
• Green LW, Travis JW, McAllister RG, Peterson KW, Vardanyan AN, Craig A. Author
responses. Am J Prev Med. 2011; 40(3): e9-e10 and e13-4.
• Green LW, & Glasgow R. Evaluating the relevance, generalization, and applicability
of research: Issues in external validation and translation methodology. Evaluation &
the Health Professions 2006; 29(1): 126-153.
• Green, L.W. Public health asks of systems science: To advance our evidence-based
practice, can you help us get more practice-based evidence? American Journal of
Public Health 96(3): 406-409, Mar. 2006.
• Mercer, S.M, DeVinney, B.J., Fine, L.J., Green, L.W., Dougherty, D. Study designs for
effectiveness and translation research: Identifying trade-offs. American Journal of
Preventntive Medicine. 2007; 33(2): 139-154.
• Sanson-Fisher, R.W., Bonevski, B., Green, L.W., D’Este, C. Limitations of the
randomized controlled trial in evaluating population-based health interventions.
American Journal of Preventive Medicine. 2007; 33(2): 155-161.

25. Some References
• Glasgow RE, Green LW, Taylor MV, and Stange KC. An evidence
integration triangle for aligning science with policy and practice. Am J
Prev Med. 2012; 42: 646.
• Garfield SA, Malozowski S, Chin MH, Naryan K M, Glasgow R, Green
LW, Hiss RG, & Krumholz HM. Considerations for Diabetes
Translational Research in Real-World Settings. Diabetes Care 26(9):
2670-2674., Sep 2003.
• Green LW, Glasgow RE, Atkins D, Stange K. Making Evidence from
Research More Relevant, Useful, and Actionable in Policy, Program
Planning, and Practice: Slips ―Twixt Cup and Lip‖. Am J Prev Med. Dec
2009;37(6S1)S187- S191. Full text online:
http://rwjcsp.unc.edu/resources/articles/S187-S191.pdf
• Jagosh J, Macaulay AC, Pluye P, Salsberg J, Bush PL, Henderson
J, Sirett E, Wong G, Cargo M, Herbert CP, Seifer SD, Green LW, and
Greenhalgh T. Uncovering the Benefits of Participatory Research:
Implications of a Realist Review for Health Research and Practice.
Milbank Quar. 2012;90(2):311-346.