1. Economic Evaluation of Public Health
Interventions: Challenges &
Recommendations
Helen Weatherly
University of York, UK

2. Elements of my presentation
• Undertaking economic evaluations of public
health interventions
– Challenges for analysts and options for dealing
with the challenges
• Using economic evaluations of public health
interventions
– Challenges for decision makers and options for
dealing with the challenges

3. Challenges for analysts in the economic
evaluation of public health interventions
• Review of the literature
• Focus on four methodological challenges
• Options for dealing with the challenges

4. Review of the literature
• NHS EED online database
• Free access
• Summaries of economic evaluations studies
by health economists
• Papers published 2000 – 2005
• 1,264 studies found
• 154 relevant studies

5. Public Health Areas
0%
2%
4%
6%
8%
10%
12%
14%
16%
18%
20%
Accidents HIV/AIDS Low birth
weight
Smoking Alcohol Sexually
transmitted
infections
Ante & post
natal
visiting
Teenage
pregnancy
Drug abuse Obesity &
physical
activity
Youth
suicide
prevention
Public health area
% of
studies

6. Types of Economic Evaluation
Cost consequence
analysis (CCA)
36%
Cost-effectiveness
analysis (CEA)
37%
Cost-utility analysis
(CUA)
27%

7. Economic Evaluation Setting
0
5
10
15
20
25
30
35
40
45
50
Community Medical Home Workplace School Transport Prison
%
Setting

8. Target Group
General population
35%
Working age population
27%
Children
14%
Women
10%
Older people
10%
Ethnic minority
3%
Men
1%

9. Study Perspective
Not stated
24%
Societal
31%
Heatlh service, health
care payer, third party
payer
32%
Multiple
3%
Patient
1%
Government
1%
Hospital, local,
health
department,
provider
8%

10. Types of Costs
0
20
40
60
80
100
120
140
160
180

11. Analytical Challenges Posed by the
Evaluation of Public Health
Interventions
• Estimates of relative treatment (programme)
effects
• Measurement and valuation of outcomes
• Equity
• Inter-sectoral effects

12. Estimates of Relative Treatment
(Programme) Effects
• Aim to obtain unbiased estimates of effect
• Focus on populations/communities rather than
specific individuals
• Preference for Randomised Controlled Trials
(RCTs)
– In practice, few RCTs
– RCT measured outcomes tend to be short term.
Benefits from public health interventions may be
realised over the longer term

13. Measurement and Valuation of
Outcomes
• Requires measuring and valuing outcomes
over the longer term
• QALYs increasingly popular
• Are other outcomes relevant to public health
interventions?
– Effects beyond target individuals/groups
– Non-health benefits
• Consider which outcomes measurement and
valuation methods are valid

14. Equity Considerations
• Growing literature, although not generally
applied in the context of economic
evaluations
• Standard economic evaluations focus on
efficiency (i.e. health maximisation) rather
than on equity (i.e. the distribution of health
gains)

15. Inter-Sectoral Effects
• Current literature does not cast the net very
broadly
• Evaluations of PH interventions need to
consider both public and private costs as
effects of public health interventions are often
wide-ranging
• Need to explore the ‘ripple effects’ (e.g.
bicycle helmets)

16. Recommendations
• Attribution of outcomes
• Measuring and valuing outcomes
• Equity considerations
• Inter-sectoral costs and consequences

17. Attribution of Outcomes
• Conduct RCTs. If cannot be undertaken, fill gaps
using natural experiments & non-experimental
data
• Match outcomes in RCTs with those available in
long-term observational studies
• Build on quantitative methods for data
synthesis and extrapolation
- extending time frames
- combination of effects from different designs
- econometric modelling

18. Measuring & Valuing Outcomes
• Debate about theoretical & value propositions
underlying various forms of economic evaluation
- money valuation
- superQALY
- cross-sector compensation test
• Perform a cost-consequences analysis alongside
• Continue research on developing a more general
measure of well-being

19. Equity Considerations
• Narrative discussion
• Health inequality impact assessment
– Impact of an intervention on health inequality
• Opportunity cost analysis of equity
– Where the most cost-effective option is judged
inequitable, calculate the opportunity cost of not selecting
that option
• Equity weighting of health outcomes
– Is a QALY is a QALY is a QALY?
• Conduct pilot studies on methods for incorporating
equity concerns

20. Inter-sectoral Costs & Consequences
• Quantify inter-sectoral impacts of interventions in a
CCA. Could assess need for budgetary transfers
• Consider broader impact of public health
interventions on the voluntary sector & private
individuals
• Conduct an analysis of costs & consequences by
beneficiary group, e.g. by health status etc
• Assess whether a general equilibrium approach
would be more suitable for the evaluation of PH
interventions

21. Use of economic evaluations of public
health evaluations by decision makers:
Challenges & recommendations
• Decision makers objectives
– Comparing public health interventions with standard
health care interventions
• Motivating and sustaining behavioural change
• Enhancing transferability of studies

22. Adapted from: Guide to NICE methods, NICE 2008
Element of Assessment NICE HTA reference case NICE public health reference case
Defining decision
problem
The scope developed by NICE The scope developed by NICE
Comparator Therapies routinely used in NHS Therapies routinely used in public sector
Perspective on costs NHS and Personal Social Services (PSS) Public sector, including the NHS & PSS
Perspective on effects All health effects on individuals All health effects on individuals
Type of economic
evaluation
Cost-effectiveness analysis (CEA) Primary analysis CEA
Secondary analysis CCA, CBA
Synthesis of evidence
on outcomes
Based on a systematic review Based on a systematic review
Measure of health
effects
Quality-adjusted Life Years (QALYs) QALYs
Source of data for
measurement of HRQoL
Reported directly by patients &/or
carers
Reported directly by patients &/or carers
Source of preference
data for valuation of
changes in HRQoL
Representative sample of the public Representative sample of the public
Discount rate Annual rate 3.5%, costs & health
effects
Annual rate 3.5%, costs & health effects
Equity position Additional QALY same weight
regardless of other characteristics of
individuals receiving health benefit
Additional QALY same weight regardless
of other characteristics of individuals
receiving health benefit

23. Motivating and sustaining
behavioural change
• Beyond neoclassical economics
– Overcoming market failure
• Implementing public health interventions
– Targeting provision
• Concerns about the distribution of health
• Use of e.g. tariffs to reward implementation
– Incentives for public health and individual
behaviour
• E.g. Nudge economics

24. Transferability of studies
• CE studies of public health interventions are rare
compared to evaluations of standard care
• Maximise use of the few studies available
– Comparability of methods for economic evaluation of
public health interventions vs. those for standard care
– Encourage transparent, full, consistent approaches to
reporting of study methods including quantifying
uncertainty
• Reference case
– Support research to improve methods of assessment of PH
interventions
– Incentives to improve data availability

25. Summary (i)
• In principle the general methods of economic
evaluation can be applied to public health
interventions
• The current literature is disappointing and
represents many missed opportunities
• Efforts need to be made in improving the
effectiveness evidence base, through RCTs and
observational studies
• Economic evaluations in this area need to pay a
lot more attention to inter-sectoral effects and
equity considerations

26. Summary (ii)
• Prior to using the studies, decision makers need
to identify their goals
• Stakeholder involvement. Incentives to engage in
the process
• Transferability issues
– Suitability for use in different settings/commissioning
tasks
– Complexities inherent in each study, quantification of
uncertainty
– Communicating accessible information to users
– Role of technical analysts in use of studies

27. References
• Cookson R, Drummond M, Weatherly H. Explicit incorporation of equity
considerations into economic evaluation of public health interventions. Health
Economics Policy and Law. 2009;4(2):231-45.
• Marks L, Cave S, Hunter D, Mason J, Peckham S, Wallace A, Mason A, Weatherly H,
Melvin K. Public health governance and primary care delivery: a triangulated study.
Report prepared for the NIHR Service Delivery and Organisation Programme, 2010.
• Neumann PJ, Jacobson PD, Palmer JA. Measuring the value of public health
systems: The disconnect between health economists and public health
practitioners. American Journal of Public Health 2008, 98, 12, 2173-2180.
• Phillips C, Fordham R, Marsh K et al. Exploring the role of economics in
prioritization in public health: What do stakeholders think? The European Journal
of Public Health 2010.
• Weatherly H, Drummond M, Claxton K, Cookson R, Ferguson B, Godfrey C, Sculpher
M, et al. Methods for assessing the cost-effectiveness of public health
interventions: key challenges and recommendations. Health Policy 2009;93:85-92.