A presentation from the 2014 Annual Results and Impact Evaluation Workshop for RBF, held in Buenos Aires, Argentina.

1. Operational
Lessons from RBF
H E A LT H R E S U LT S I N N OVAT I O N T R U S T F U N D
Learning from Implementation
Petra Vergeer & Hadia Samaha

2. The Why and How of
Operational Lessons
• Review of ongoing PBF operations suggests
some useful lessons
• Focusing on the most important lessons will
facilitate enhanced design and
implementation
2

3. Lesson
“Math-Phobes of the World Unite!”
– Use your data
• Data is vital but under-utilized, despite a lot of effort
invested into collecting, verifying and putting
payment data on the web
17

4. Coverage of full vaccination among children under 1
0
10
20
30
40
50
60
70
80
90
100
4 1 2 3 4 1 2 3
Benin
Burkina Faso
Cameroon
Kenya
Nigeria
Zambia
Zimbabwe
%

5. Total quality score in health facilities
0
20
40
60
80
100
Burkina
Faso
Benin Cameroon Kenya Nigeria Zambia Zimbabwe
%
Scores are averages of health centers and hospitals, technical and subjective where applicable
Each bar represents a quarter of implementation

6. Internet applications with public front-end
displaying performance & financial information
18
Burundi
Benin
Nigeria

7. 3 services absorbing largest share of payment
7
OP >5
11%
OP
<=5
15%
Inst.
Delive
ries
17%
Other
s
57%
Burundi
Zambia
Cameroon
Zimbabwe
OP
contac
t
6%Inst.
Delive
ries
35%FP
40%
Others
19%
OP
contact
35%
Inst.
Deliveri
es
15%
FP
21%
Others
29%
OPC
21%
Hosp.
days
15%
VCT
12%
Others
52%
Figures reported are averages of all quarters to date

8. Strengthening work on administrative data
1. Regularly monitoring program progress to identify candidates for
adjustment (indicators and tools)
2. Taking advantage of HMIS data to compare with control facilities
and assess performance on non-incentivized services
3. Developing online dashboard to facilitate use of data and
promote transparency
4. Developing automated data analysis software to lessen burden
of data analysis for teams and encourage focus on results
(ADEPT RBF)
8

9. Lesson
“Keep moving the goalposts!”
Continuous Quality Improvement
(CQI) implies Changing the Quality
Indicators
• Many facilities make rapid improvements in
quality and then plateau
6

10. At different levels:
• Facility Staff (managers, providers, staff -hospitals and clinics):
Need quality measures to assess and continuously improve services. Is care
improving?
• Regional/District & Program Managers:
Need measures to assess and continuously strengthen essential system
functions (e.g. competent workforce). Are essential system functions performing
to standard?
--------
Understanding what different stakeholders want:
 Clients (users of care)
 National Policy-makers (value, policy)
 Global Stakeholders (leadership, advocacy, accountability)
Integrating Quality into RBF Projects: Prioritizing
Health Conditions/Services for Improvement

11. Consider phasing improvement priorities: “impossible to improve everything at once”
Involve local and international experts to:
• Review country standards against global evidence: evidence is constantly changing
• Distill standards into minimum “intervention bundles”: focus attention on essential
high-impact interventions
• Illustrative quality of care process measures based on minimum standards:
o % cases adherent with standards – “all or nothing adherence” (e.g. % PPH cases
managed per minimum standard; % cases pediatric pneumonia treated per
standard)
o Average % adherence with minimum standards (e.g. average % adherence with
newborn sepsis case-management standards; N=30 cases)
Integrating Quality into RBF Projects: Selecting
standards and Defining Quality of Care Measures

12. Illustrative quality measure: Quality of Partogram
Completion (not so simple!)
Quality
Measure
Opera/onal
Defini/on

%
partograms
in
last
quarter
completed
per
standard

NUMERATOR:
Number
partograms
documen/ng
cervical
dila/on,
maternal
BP,
pulse,
temperature
at
admission
and
at
least
every
4
hours
un/l
delivery

DENOMINATOR:
Total
number
of
partograms
reviewed

13. Lesson
No commodities : No program
Worry about supply chains and drug
availability !!!!!!
6

14. Improving Medicine supply chains to
debottleneck RBF Programs
• A useful first step is to diagnose the root causes of poor availability
using an appropriate diagnostic tool. Poor quantification/
requisitions, lack of transport, and procurement delays are common
reasons.
• Range of options for RBF programs to tackle these challenges. E.g.,
better requisitioning tools, contracted transport for obtaining
supplies from district or regional stores, negotiated prices with
private sector supply sources etc.

15. Lesson
Seeing is Believing !!!!!
Verify and Counter Verify to Ensure You
pay for real outputs !!!!
6

16. Factors influencing verification: A Conceptual
Framework
Context
Verifica/on
Characteris/cs
Impact
on
accuracy,
cost,
sustainability
RBF
Characteris/cs
RATIONALE
FOR
RBF
CONTRACT
TYPE
USE
OF
RBF
RESULTS
Improving
health
outcomes/HSS
RelaFonal
Payment,
improving
performance
Financial
accountability/Cost
control
Classic
Transparency,
Naming
and
Shaming
Monthly
Annual
Yes
Large
Whole
universe
Risk-­‐based
approach
Internal
Verifica/on
Results
and
Their
Use
FREQUENCY
ALLOWABLE
ERROR
MARGIN
SAMPLE
SIZE
INSTITUTIONAL
SETUP
ADVANCE
WARNING
No
Small
Third
party
Learning,
Error
correcFon
Cost
recovery,
SancFon
PAYMENT
FREQUENCY
Monthly
Annual
POLITICAL
ENVIRONMENT
GOVERNANCE
CULTURE

17. Key Recommendations Verification
1. Consider context to determine whether merging
functions is appropriate (be mindful of conflict of
interest)
2. Analyze and use data available from verification
and counter-verification
3. Verification strategies should be dynamic, not
static, and use a risk-based approach

18. Lesson
Capable People Matter !!!!!
Invest in Your RBF Institutions & Teams
6

19. Scaling up capacity building and human
resources for RBF and its sustainability
• South-South TA with appropriate backstopping can lead to a successful
home bred PBF pilot experience
• Faculty members in Medical Schools are keen to embrace PBF, teach PBF
and spread its principles and success stories, provided that they got the
opportunity to be exposed to PBF in theory and in practice
• Creating local contract management and verification capacity by selecting
local non-governmental organizations and training and coaching them in
PBF can be an attractive strategy in some countries.
• Use of locals has made it possible to increase knowledge and capacity on
RBF, research, and MNCH in-country. Ensure there is no bias and missing
out on other international experiences

20. Lesson
Let RBF Leave its Mark !!!
Think beyond tomorrow
6

21. RBF Institutional Set up and
Ensuring Buy in
• There are 3 main stages in the integration of RBF into a
national health system
 Adoption: to move from PILOT to SCHEME
 Institutional: to move from SCHEME to POLICY
 PERPETUATION: to move from POLICY to SYSTEM
• Key issues are in terms of: Actors, Resources (including $$
$$), RBF design, and Process
• Context shapes the trajectory