A presentation from the 2014 Annual Results and Impact Evaluation Workshop for RBF, held in Buenos Aires, Argentina.

1. RBF PROGRAMS PERFORMANCE –
A CROSS COUNTRY OVERVIEW FROM
OPERATIONAL DATA
BUENOS AIRES, MARCH 2014

2. OBJECTIVE
To provide a cross-country overview of
performance to assess progress and identify areas
for further inquiries
2

3. COUNTRY PROGRAMS IN REVIEW
Country Start date Program areas Catchment population
Benin Mar 2012 8 districts 2.2 million (22%)
Burkina Faso Dec 2011 3 districts 813 thousand (5%)
Burundi Mar 2010 Countrywide 9.8 million (100%)
Cameroon Littoral: Apr 2011
3 other: Jul 2012
4 regions 2.8 million (13%)
Kenya Dec 2011 1 sub-county 200 thousand (0.5%)
Nigeria Dec 2011 3 LGAs 416 thousand (0.2%)
Zambia Apr 2012 11 districts 1.5 million (11%)
Zimbabwe Mar 2012 18 districts 4.2 million (30%)
Afghanistan April 2009 11 provinces
Laos Mar 2013 5 provinces 2.2 million (33%)
Sierra Leone Oct 2010 13 districts 5.9 million (100%)
Total population is for 2012 (WDI)
3

4. UNDERSTANDING OPERATIONAL DATA
Verified data from contracting facilities
 Not available for non-participating facilities (private sector,
hospitals in some cases)
 Do not reflect true population coverage
 Usually not available for control groups or for non-incentivized
indicators (have to rely on HMIS)
 May still be prone to errors despite verification
 Not IE, (lack of) effect is suggestive but not conclusive
Lack of precise information to calculate coverage
 Size of catchment population
 Parameters to calculate population “at risk” (i.e., denominators)
Cross-country analysis issues
 Challenges in (lack of) comparability of definitions and designs
 Only looks at program level, not sub-program level
4

5. ESTIMATED COVERAGE OF
SELECTED KEY SERVICES
5

6. COVERAGE OF INSTITUTIONAL DELIVERIES
-
10
20
30
40
50
60
70
80
90
100
2 3 4 1 2 3 4 1 2 3 4 1 2 3 4
2010 2011 2012 2013
Burundi
%
6

7. COVERAGE OF INSTITUTIONAL DELIVERIES
Each bar represents a quarter of implementation
0
10
20
30
40
50
60
70
80
90
100
Benin B Faso Cameroon Kenya Nigeria Zambia Zimbabwe
%
7

8. COVERAGE OF POSTNATAL CARE
0
10
20
30
40
50
60
70
80
90
100
2 3 4 1 2 3 4 1 2 3 4 1 2 3
2010 2011 2012 2013
Benin
Burkina Faso
Nigeria
Zambia
Zimbabwe
Burundi
%
8

9. COVERAGE OF FULL VACCINATION AMONG
CHILDREN AGED 1 AND UNDER
%
-
20
40
60
80
100
120
2 3 4 1 2 3 4 1 2 3 4 1 2 3
2010 2011 2012 2013
Burundi
Benin
B Faso
Cameroon
Kenya
Nigeria
Zambia
Zimbabwe
9

10. MODERN FAMILY PLANNING USE AMONG
WOMEN OF REPRODUCTIVE AGE
%
%
-
10
20
30
40
50
60
70
80
90
2 3 4 1 2 3 4 1 2 3 4 1 2 3
2010 2011 2012 2013
Burundi
Benin
B Faso
Cameroon
Kenya
Nigeria
Zimbabwe
0
5
10
15
20
25
30
4 1 2 3 4 1 2 3
2011 2012 2013
10

11. ANNUALIZED RATE OF CURATIVE CARE
OUTPATIENT CONTACT (PER CAPITA)
-
0.50
1.00
1.50
2.00
2 3 4 1 2 3 4 1 2 3 4 1 2 3
2010 2011 2012 2013
Burundi
Benin
B Faso
Cameroon
Nigeria
Zambia
Zimbabwe
-
0.20
0.40
0.60
0.80
1.00
1.20
1 2 3 4 1 2 3
2012 2013
11

12. TRENDS IN QUALITY
12

13. TOTAL QUALITY SCORE IN HEALTH FACILITIES
0
10
20
30
40
50
60
70
80
90
100
Burkina
Faso
Benin Cameroon Kenya Nigeria Zambia Zimbabwe
%
Scores are averages of health centers and hospitals, technical and
subjective where applicable
Each bar represents a quarter of implementation 13

14. TRENDS IN KEY COMPONENT QUALITY
SCORES
-
10
20
30
40
50
60
70
80
90
100
4 1 2 3 4 1 2
2011 2012 2013
%
Drugs management
Benin
Nigeria
-
10
20
30
40
50
60
70
80
90
100
4 1 2 3 4 1 2
2011 2012 2013
%
Financial management
Benin
Nigeria
14

15. KEY ISSUES IN
PROGRAM PAYMENT
15

16. PER CAPITA RBF PAYMENT ON SERVICE
DELIVERY PER YEAR
Paymentpercapita(US$)
“Year” means complete 12 calendar months counting from the month when program started
Value for the most recent year is extrapolated if duration is less than 12 months
Payment components consist of quantity, quality, and equity bonus where applicable
0
0.5
1
1.5
2
2.5
3
3.5
Kenya Cameroon B Faso Nigeria Zambia Benin Zimbabwe Burundi
Year 1 Year 2 Year 3 Year 4
16

17. SHARE OF PAYMENT TO HEALTH CENTERS AND
LOWER LEVEL IN TOTAL PAYMENT ON SERVICE
DELIVERY
%
Figures reported are averages of all quarters to date
0 10 20 30 40 50 60 70 80 90 100
Kenya
Zambia
Nigeria
Burkina Faso
Benin
Zimbabwe
Burundi
Cameroon
17

18. THREE SERVICES ABSORBING LARGEST
SHARE OF PAYMENT
OP >5
11%
OP
<=5
15%
Inst.
Delive
ries
17%
Other
s
57%
Burundi
Zambia
Cameroon
Zimbabwe
OP
contac
t
6%
Inst.
Delive
ries
35%
FP
40%
Others
19%
OP
contact
35%
Inst.
Deliveri
es
15%
FP
21%
Others
29%
OPC
21%
Hosp.
days
15%
VCT
12%
Other
s
52%
Figures reported are averages of all quarters to date 18

19. KEY SUMMARY POINTS
1. There is a large variation in programs performance, both at
baseline and over time
2. Overall, there has been good progress in performance of key
services and quality as measured by the programs
3. Annual per capita payment ranges from US$ 0.3 to US$3. Some
“one dollar per capita” programs perform rather well (Zambia,
Nigeria)
4. Curative care (especially OP), institutional deliveries, and FP are
typically the largest cost items
5. Most programs place strong emphasis on the low level of health
system (health centers and below)
6. Some areas requiring further investigations:
- Consistently slow progress in FP in some countries: why?
- Coverage >100%: measurement problem, reporting, population?
- What to do if performance is high and plateau?
- Is fee-for-service a right method for OP contact?
19

20. STRENGTHENING WORKS ON
ADMINISTRATIVE DATA
1. Regularly monitoring program progress to identify
candidates for adjustment (indicators and tools)
2. Taking advantage of HMIS data to compare with control
facilities and assess performance on non-incentivized
services
3. Developing online dashboard to facilitate use of data and
promote transparency
4. Developing automated data analysis software to lessen
burden of data analysis for teams and encourage focus
on results (ADEPT RBF)
20