The document summarizes efforts to build monitoring and evaluation (M&E) capacity in community-based HIV programs in Tanzania. It found that existing M&E systems were inadequate and data quality was poor. A new approach was developed using data quality assessments and community tracing of beneficiaries to identify weaknesses and improve M&E skills through tailored training. This led to measurable improvements in M&E plans, performance, and data quality. However, fully transitioning capacity building to local organizations remained a challenge.

1. Building M&E capacity in
community-based HIV programs in
Tanzania:
From diagnosis to assessing impact
Karen Foreit1
, Dawne Walker2
,
Mari Hickmann1
, Zaddy Kibao1
1
Futures Group, 2
Effective Development Group
Africa Evaluation Association
March 2014, Yaounde, Cameroon

2.  Total Population: 45 million
 5.1% of adults age 15-49
are HIV-positive.
Background - Tanzania

3.  Funded by the United States Agency for
International Development to improve monitoring
and evaluation in population, health and nutrition.
 Operational in Tanzania since 2003.
 Supports M&E system strengthening and capacity
building in health and social welfare programmes.
Background – MEASURE Evaluation

4. The challenge(s)
 Inadequate, poorly coordinated M&E systems
 Insufficient human & organizational capacity
 Poor-quality data
 Limited data use for planning & management
 Community-based programs staffed with
semi-literate volunteers

5. Community volunteers
 Limited literacy
 Do not understand why they
have to collect so much
information
 Often do not fill out forms
until after the visit

6. Data Quality Assessments (DQA)
 Measure accuracy of data reports
 Assess underlying M&E systems
 Originally designed for facility-based programs
 Sampling proportional to size
 “Trace and Verify” of client records

7. What we found
 Good results for facility-based programs
 Serious defects in community-based programs
 Few had M&E plans in place
 Households “claimed” as served reported they had
not been visited
 Declining data quality with distance to HQ

8. What we did
 Re-design DQA to support M&E
capacity-building
 New sampling procedures
 New data collection instrument
 Baseline needs assessment
 Tailored training and mentoring
 Objectively measure change
 Plan for sustainability

9.  Stratify sites:
 By size (large vs. small) and
 By location (close to vs. far
away from regional office/HQ)
 Randomly select from all 4
strata
 10 or fewer sites, visit all sites
New sampling procedures

10.  Community Trace and Verify
 Visit “claimed” beneficiaries
 Ask what services they received
 Compare beneficiary answers
to activity reports
New data collection instrument

11. Baseline assessment
 Questions specific to the level being
assessed
 Document every question
 4 point scale for system assessment for
greater precision
 Section on data use for all levels

12.  Move workshops closer to sites
 Group participants by common
problems
 Follow training with
individualized mentoring
 Advanced workshops for
partners with strong basic M&E
skills
Tailored training and mentoring

13. Objectively measure change
 “Mini” DQA
 Initial focus on low-scoring areas
 Expand from sites visited at baseline to other
sites in the program
 Expand to re-assess all areas of the system
assessment

14. Plan for sustainability

15. Standard Operating Procedures
 Team structure, roles and
responsibilities
 Procedures for implementing
at different levels
 Debrief templates
 GIS guidance
 Spot check guidance
 Report Writing

16. Structured report format
 Text and analysis template
 Standardized color scheme
and lay-out
 External editor

17. Build local capacity
 Transition to local organization with
backstopping from MEASURE Evaluation
staff
 Report writing course for contractor staff
 Greater participation in sampling and fieldwork
plan

18. Technical Support to Government
 Technical assistance to CHMTs mainly on
enhancing data quality and data use at District
level
 Strengthening of Community Care National M&E
system in Mainland and Zanzibar

19. Results
 CTV effectively pinpoints strengths and
weaknesses in data collection
 Measurable improvements in M&E plans,
performance and data quality
 28 IPs assessed, 23 received multiple rounds
 10 undertake internal DQA
 Slow progress in building local capacity to
independently conduct DQA

20. Also…
 Partners allocate
specific M&E budget
 Hired M&E staff
 Supportive supervision
and capacity building to
lower levels

21. Conclusions
Routine DQA augmented with CTV is an
effective platform to assess data quality and
program coverage of community programs,
guide assistance to strengthen individual and
organizational M&E capacity, and measure
the impact of training and mentoring for M&E.

22. MEASURE Evaluation is a MEASURE project funded by the
U.S. Agency for International Development and implemented
by the Carolina Population Center at the University of North
Carolina at Chapel Hill in partnership with Futures Group
International, ICF, John Snow, Inc., Management Sciences for
Health, and Tulane University. Views expressed in this
presentation do not necessarily reflect the views of USAID or
the U.S. Government.
MEASURE Evaluation is the USAID Global Health Bureau's
primary vehicle for supporting improvements in monitoring and
evaluation in population, health and nutrition worldwide.