Presented by Jui Shah, MEASURE Evaluation/ICF International, as part of a symposium organized by MEASURE Evaluation and MEASURE DHS at the 6th MIM Pan-African Malaria Conference.

1. Developing a framework for
in-country impact evaluations of
malaria control efforts
Jui Shah, MPH
Team: Achuyt Bhattarai, Christie Hershey, Eline Korenromp, Elizabeth Ivanovich, Erin
Eckert, Holly Newby, Liliana Carvajal, Richard Cibulskis , Mike Lynch, Ryuichi Komatsu,
Thom Eisele, Yazoume Ye

2. Outline
 Background
 Development process
 Overview of the framework
 Next steps

3. Background: Defining the need
 Last decade of malaria control
 Renewed attention and resources
 Scale up of key interventions
 Current need to assess the effect of this scale up

4. Background: Building on past
work
 Monitoring & Evaluation Reference Group
(MERG)
 Plausibility design proposed by Alex Rowe and
MERG colleagues in 2007
 Need to address 2015 MDG measurement needs
 Incorporate new evidence, experience, and
thinking

5. Purpose of the document
 Review and update the 2007 evaluation
framework
 Make recommendations for evaluating the impact
of scaling up malaria control interventions
 Summarize recent experiences, including impact
evaluations of PMI priority countries

6. Partners
 MEASURE Evaluation
 USAID
 CDC
 UNICEF
 WHO
 Global Fund to Fight AIDS, Tuberculosis and
Malaria

7. Target audience
 National Malaria Control Programs
 Ministries of Health
 Funding agencies

8. Development process
 Identify core writing team
 Develop outline of the guidance document
 Assign sections to authors
 Craft preliminary drafts
 Complete internal and external reviews
 Finalize text and formatting
 Print and disseminate document

9. Content








Definition of key concepts
Executive summary
Introduction and objectives
Process for implementing
impact evaluation
Evaluation design
Program description
Measuring the coverage of
malaria control interventions
Measuring transmission
intensity
 Measuring malaria morbidity
 Measuring mortality
 Measuring and accounting
for non-malaria programs
factors
 Data synthesis, triangulation
and interpretation
 Challenges
 Way forward
 Appendices (case studies)

10. Process of implementing
evaluation
Step 2.
Malaria
Control
Program
Description
Step 5.
Ensure Use
and
Disseminate
Findings
Step 1.
Stakeholder
Engagement
Step 4.
Generate
Credible
Evidence
Step 3.
Develop
Evaluation
Design

11. Conceptual framework
Confounders
Coverage of primary malaria control
interventions
Coverage of other child survival
interventions
Case management
Vector control (ITNs and IRS)
Malaria prevention in pregnancy
(ITNs and IPTp)
ORT , EPI
Access / demand for healthcare
Nutrition
IMCI, PMTCT
Rainfall /
temperature
Malaria
transmission
Confounders
(Confounder)
Impact on malaria
Malaria-related morbidity
(mostly among children and
pregnant women in stable
transmission areas)
Malaria-related mortality
in children
Child morbidity and mortality due
to other cause than malaria
Diarrheal disease
ARI (Pneumonia)
Nutrition
HIV
Impact on all-cause child mortality

12. Measuring coverage
 9 indicators for
vector control
 1 indicator for
malaria prevention
during pregnancy
 3 indicators for
diagnosis and
treatment practices

13. Measuring morbidity
 Different data sources
 2 population-level indicators
 4 facility-level indicators

14. Measuring mortality
 Malaria-specific mortality
 Health management and information system
 Civil registration and vital statistics
 Verbal autopsy
 All-cause under-five mortality

15. Measuring contextual factors
Category
Child survival
interventions
Examples
Data sources
EPI coverage, micro-nutrient
supplementation coverage
WHO, UNICEF,
DHS, MICS
Environmental
factors
Rainfall, temperature, land
cover, flood, drought
National
meteorological
agency, online
datasets
Health systems
factors
Per capita expenditure on health WHO, World Bank
Socioeconomic
factors
Household asset and income,
parental education, economic
crises, GDP per capita,
population living below poverty
line
DHS, MICS
World Bank

16. Triangulation and interpretation
 Primary analysis
 Trends over time
 Stratification
 Secondary analysis
 Meta analysis (multi-country)

17. Next steps
 Finalize text
 Complete internal and external reviews
 Print and disseminate document

18. Thank you!