Improved Performance of the Malaria Surveillance, Monitoring, and Evaluation System from 2015 to 2018 in Madagascar
1. Improved Performance of the Malaria Surveillance, Monitoring, and Evaluation System
from 2015 to 2018 in Madagascar
Conclusions
MEASURE Evaluation’s support in strengthening malaria SME in Madagascar has likely contributed to significant improvements
in malaria and overall IDSR data reporting, both at facility and community levels, from 2015 to 2018. The improvements span
various areas, such as the availability of guiding documents and improvements in data quality in terms of increases in completeness
and timeliness of reporting and data accuracy at the health facility and community levels. MEASURE Evaluation’s support has also
contributed to establish a culture of data dissemination and use at the operational level by skilled professionals who have benefited
from the project’s country-specific malaria SME training workshops. The observed trends need to be maintained and improved,
especially because Madagascar has started strategies for progressive geographical elimination.
Background
Malaria poses a public health challenge in Madagascar, where the entire
population is at risk, especially in epidemic-prone areas. Given that malaria
cases and deaths reported through the national health management
information system between 2003 and 2013 have decreased, Madagascar
is considering pre-elimination strategies. This requires an effective
surveillance system to monitor cases, detect potential epidemics, and
investigate cases and foci as appropriate. Starting in 2015, after conducting
a baseline assessment of key malaria surveillance indicators in the
national integrated disease surveillance and response (IDSR), MEASURE
Evaluation, funded by the United States Agency for International
Development (USAID), has worked with the National Malaria Control
Program (NMCP) to strengthen malaria surveillance in Madagascar with
support from the U.S. President’s Malaria Initiative (PMI) and partners. In 2018, the project conducted a
follow-up survey to measure and document improvements in the performance of the malaria surveillance,
monitoring, and evaluation (SME) system in the country to inform future support to the NMCP.
Jean-Marie N’Gbichi,1
Maurice Yé,1
Alain Rakotorisoa,2
Léa Randriamampionona,2
Solo Harimalala,3
Andriamananjara N. Mauricette,3
Laurent Kapesa,4
Yazoumé Yé1
1
MEASURE Evaluation, ICF; 2
Direction de la Veille Sanitaire, de la Surveillance Epidémiologique et Riposte, Ministry of Public Health, Madagascar; 3
National Malaria Control Program, Ministry of Ministry of Public Health, Madagascar;
4
President’s Malaria Initiative/United States Agency for International Development, Madagascar
Acknowledgments—This publication has been supported by the President’s Malaria Initiative
(PMI) through the United States Agency for International Development (USAID) under the terms
of MEASURE Evaluation cooperative agreement AIDOAA-L-14-00004. MEASURE Evaluation is
implemented by the Carolina Population Center at the University of North Carolina at Chapel Hill,
in partnership with ICF International; John Snow, Inc.; Management Sciences for Health; Palladium;
and Tulane University. Views expressed are not necessarily those of PMI, USAID, or the United
States Government.
For information, contact:
JeanMarieNGbichi@icf.com
https://www.measureevaluation.org/
Methods
Similar to the 2015 baseline assessment, the 2018 follow-up survey covered
national, regional, and district levels. It consisted of a desk review of existing
malaria SME and IDSR documentation in the 22 regions and 114 districts
of the country through a self-applied data collection form. Data quality
checks were performed in a randomized sample of health facilities, in which
community-based data reporting was assessed in the facilities’ catchment areas.
The performance of the system was evaluated in terms of data quality (data
accuracy, timeliness, and completeness of reporting), data dissemination and
use, and staff capacity in SME and IDSR.
Data accuracy was assessed by comparing data reported to data recounted and re-
aggregated from health facilities’ registers with a 95–105% percent validity scale. Timeliness
of reporting was assessed by comparing the number of reports submitted on time to the
total number of reports expected. Completeness of reporting was assessed by comparing
the number of reports submitted to the total number of reports expected. For comparison
of 2015 and 2018 data, the z-tests for proportions from two samples were estimated, and
p-values were calculated. Differences were considered significant if p<0.05.
Results
2015 2018
Malaria data quality assurance protocol No Yes
Malaria data analysis and presentation protocol No Yes
Malaria SME training curricula No Yes
IDSR training curricula No Yes
Malaria quarterly bulletin No Yes
IDSR monthly bulletin Yes* Yes
* A template for the IDSR bulletin was observed but has not been finalized and disseminated
Table 1. SME guiding documents and data dissemination
tools at national, regional, and district levels
In the 2015 baseline assessment, the Madagascar malaria SME system lacked
primary documents, such as the malaria data quality assurance protocol, malaria
data analysis and presentation protocol, malaria SME training manuals, and a
malaria bulletin. These documents were present in 2018.
2015 2018 p-value
National
Quarterly malaria data discussion meetings held in the last two years
0%
(0/8)
100%
(8/8)
p=0.00060
Monthly IDSR data discussion meetings held in the last two years
0%
(0/24)
79.1%
(19/24)
p<0.00001
Regional
Health regions that received a malaria quarterly bulletin each quarter in the
last two years
0%
(0/10)
81.8%
(18/22)
p<0.00001
Regions that received an IDSR monthly bulletin in the last six months
0%
(0/10)
86.3%
(19/22)
p<0.00001
Regions that held malaria data discussion meetings in the last six months
0%
(0/10)
77.2%
(17/22)
p<0.00001
District
Districts that received a malaria quarterly bulletin each quarter in the last
two years
0%
(0/44)
85%
(97/114)
p<0.00001
Districts that received an IDSR monthly bulletin in the last six months
0%
(0/44)
64%
(73/114)
p<0.00001
Districts that held malaria data discussion meetings in the last six months
0%
(0/44)
80.7%
(92/114)
p<0.00001
Table 3. Data use by levels
None of the 2 national directorates (NMCP and the Direction de la Veille Sanitaire, la Surveillance Epidemiologique
et la Riposte [DVSSER]), 10 regions, and 44 districts assessed reported holding data discussion meetings in 2015.
In 2018, the NMCP reported holding 100 percent of the quarterly malaria data discussion meetings in the last two
years, and the DVSSER reported holding 79.1 percent of the monthly IDSR data discussion meetings in the last two
years. In addition, 77.2 percent of the 22 regions and 64 percent of the 114 districts reported holding malaria monthly
data discussion meetings in the last 6 months (p<0.00001). In 2018, the malaria quarterly bulletin was disseminated to
81.8 percent of regions and 85 percent of districts, and the IDSR monthly bulletin was disseminated to 86.3 percent
of regions and 64 percent of districts (p<0.00001).
Table 2. Data quality (completeness, timeliness of reporting, data accuracy)
Table 4. Capacity building
2015 2018 p-value
Completeness and timeliness at health facility level
Completeness
62.2%
(821/1,320)
73%
(998/1,368)
p=0.00044
Timeliness
48.3%
(638/1,320)
68.3%
(935/1,368)
p<0.00001
Completeness and timeliness at community level
Completeness
8.6%
(225/2,616)
94.3%
(18,920/20,064)
p<0.00001
Timeliness
5.2%
(136/2,616)
55%
(11,035/20,064)
p<0.00001
Data accuracy at health facility level
73%
(964/1,320)
87%
(396/456)
p<0.00001
Completeness of routine IDSR data reporting, including malaria, by centres de santé de base (CSB) health facilities
was significantly higher in 2018 (73%), compared to 2015 (62.2%) (p=0.0044). Timeliness of CSB reporting was
68.3 percent in 2018, compared to 48.3 percent in 2015 (p<0.00001).
Completeness of community-based reporting was significantly higher in 2018 (94.3%), compared to 2015 (8.6%)
(p<0.00001), and timeliness was 55 percent in 2018, compared to 5.2 percent in 2015 (p<0.00001).
Data accuracy, the comparison of data from weekly reporting forms to weekly malaria data in registries, was at
87 percent in CSB in 2018, compared to 73 percent in 2015 (p<0.00001).
2015 2018 p-value
National
NMCP SME unit technical staff trained in malaria SME
25%
(4/12)
93.3%
(14/15)
p=0.00024
DVSSER technical staff trained in IDSR
33.3%
(3/9)
79.1%
(15/17)
p=0.00386
Regional
Regional directors trained in malaria SME
50%
(5/10)
95%
(21/22)
p=0.00228
District
District directors trained in malaria SME
34%
(15/44)
99.1%
(113/114)
p<0.00001
At the national level, 93.3 percent of the NMCP SME unit technical staff were trained in malaria SME in 2018,
compared to 25 percent in 2015 (p<0.00001); 79.1 percent of DVSSER technical staff were trained in IDSR in 2018,
compared to 33.3 percent in 2015 (p<0.00001). At the regional level, 95 percent of regional directors were trained in
malaria SME in 2018, compared to 50 percent in 2015 (p<0.00001). At the district level, 99.1 percent of district team
leads were trained in malaria SME in 2018, compared to 34 percent in 2015 (p<0.00001).