A presentation from the 2014 Annual Results and Impact Evaluation Workshop for RBF, held in Buenos Aires, Argentina.

1. 1
Assessment Tool for Cross- Country Learning on Results Based Financing (RBF)
A. Introduction
RBF is a promising strategy that is increasingly being used by national health programs in
developing countries to accelerate progress towards the achievement of the health Millennium
Development Goals (MDGs). In order to ensure real time learning while waiting for the formal
impact evaluation results to become available, it is important to regularly review ongoing RBF
experiences and identify key implementation challenges and factors for success.
While the evidence for the importance of particular design elements is usually sub-experimental,
it may still be compelling. For example, experience so far suggests a few key features are critical:
Paying for results regularly, predictably, and using a simple to understand formula;
Verifying and counter verifying results regularly by independent agencies, e.g. monitoring of
actual receipt of services and quality of care;
Provision of substantial autonomy to health facilities in the use of resources they earn
including being able to purchase drugs independently from approved private sector suppliers;
Regular supervision and the use of robust tools for quality assessments, i.e., quantitative
supervisory checklists
Monthly or quarterly performance review meetings at health facility, local government, and
higher levels
B. Scope of the Assessment Tool
Countries may find it useful to visit another country and assess their RBF programs and/or impact
evaluations in order to learn from each other’s experience and adapt their program accordingly to
ensure better results. The assessment tool is meant to guide and structure such an RBF program
assessment. It will thereby allow for a comprehensive assessment of the program, and help
identify successes and current challenges of the RBF programs as well as possible solutions. At
this point, the assessment tool does not aim to cover the review of the impact evaluation of RBF
programs.
The assessment tool aims to ensure (i) all areas relevant to the success or failure of a RBF
program are covered in the assessment, (ii) the assessing team can use the assessment tool with
various stakeholders at all levels of the program, from service delivery to policy-making, (iii)
using the tool, the assessing team is able to identify the strengths and weaknesses of a program,
and propose ways to adjust it to remedy or prevent issues.
Please note that the assessment tool is currently framed to apply mostly to a Performance-Based
Financing (PBF) scheme, as PBF is implemented in most countries currently implementing RBF
within the HRITF. However it can easily be modified to apply to other types of RBF programs.
C. Assessment Tool Questions
This assessment tool aims to evaluate the implementation of a RBF program by focusing on the
areas that are most critical for achieving the intended results. The following table highlights these
core areas. For each area, the assessment tool proposes questions that may be explored with
various stakeholders to understand the program, and pinpoint its strengths and weaknesses. Not
all questions will be relevant to a given stakeholder, however all core areas can be explored with
a variety of respondents, at the health facility level, administrative level, etc.

2. 2
It is important to note that for such an assessment to be most useful you want to focus as much as
possible on what happens in reality. You are not just interested in how it should (as may be
described in the RBF manual) but what happens in reality in the RBF program implementation.
This will help focus on possible challenges and can help highlight potential solutions and
recommendations that may be needed.
Note the questions are there to guide you; you may inquire further on specific aspects when
deemed necessary. However, please keep in mind that you are here to understand how the
program works and be a listener: use open-ended questions (not yes/no questions), try to
understand how things are done and leave your own opinions aside during the interviews.
Core area Exploratory Areas
The RBF
program overall
1) What is the general description of the RBF program, including whether
it focuses on the supply or demand side?
2) What has been the timeline of the program?
3) Who are the different actors involved in the RBF program and what are
their functions, from the local to national level?
4) Which actions or services do the RBF indicators cover at all levels?
How are results paid for (e.g. per service, per target achieved, by weight
in overall budget…)? How do quality and quantity of care interact in
the payment for results? How have indicators or their tariffs changed
since the beginning of the program?
Data Quality
and Verification
5) What are the data sources for service outputs (quantity)? What is the
interaction of RBF data and Health Management Information System
(HMIS) in the country?
6) How is the quality of care measured; by whom and what does it focus
on?
7) Is there a way client satisfaction is assessed? How is this incorporated?
8) Who carries out the quantity and quality verification, what are the
periodicity and sampling procedure?
9) What is the ex-ante mechanism for verification, i.e. the mechanism for
verification used prior to making the payment?
10) How is this data on quantity and quality counter-verified, i.e. what is
the mechanism that checks that the verification is done correctly (often
ex-post, after the payment has been made)?
11) What happens when verification and counter-verification show
discrepancies between reported and achieved results?
12) Are there any possible conflicts of interests (e.g., between the providers
and the verifiers or between the verifiers and the counter-verifiers)?
Results
Achieved
13) Taking into account quality of the data, what are the results that have
been achieved so far?
14) Which indicators (of quantity and quality) have improved and which
have not?
15) What geographical areas are lagging behind or surging ahead?
16) Are there other disparities?
17) What does the trend look like? Is there evidence of plateauing?

3. 3
Core area Exploratory Areas
RBF Financing
and
Sustainability
18) What are the sources of financing for RBF?
19) How much is the Government contributing?
20) Which development partners are contributing? How are the relationship
and dialogue between the partners (including the Bank) and the
Government?
21) What is the level of financing per capita?
22) How does the RBF scheme fit into the referral system? What proportion
of funding is going to hospitals?
23) Which indicators absorb the largest percent of the RBF budget?
24) What percentage do RBF payments contribute to health workers’ take-
home pay? How motivated is the staff?
25) What are the views of stakeholders in RBF, at all levels including
beneficiaries?
26) How sustainable is the RBF financing scheme, financially or
otherwise?
Capacity
Building
27) What has been the effort in Capacity Building at the health facility level
and at other levels (initial and ongoing)?
28) To what extent have these efforts worked?
29) How are costs for capacity building distributed?
30) What kind of mechanism for Technical Assistance is in place?
Supervision and
Monitoring
31) How is the RBF program supervised at all levels?
32) What does supervision entail and how often does it happen?
33) How structured is the supervision?
34) To what extent are the operational data or HMIS data used for program
monitoring, supervision and adjustments?
Project
Management
35) What are the management structures for RBF at the different levels,
what is their role and who do they include?
36) How effective are the management structures – Steering Committee,
health center committee etc.: how regularly do they meet, what do they
focus on, what attention do they give to client satisfaction?
37) How effective is the RBF Implementation Unit (often a Project
Implementation Unit- PIU) and how have the key people been trained?
38) Does the project have other support elements (e.g. human resources,
Cash-on-Delivery arrangements etc.) and are they moving in tandem?
Institutional
roles and
functions
39) Is there a separation of roles of purchaser, fund holder and verifier?
40) Are there clear contractual relationships between them?
Facility
Autonomy
41) How is facility autonomy operationalized (e.g. how does the facility use
the RBF and possible other resources; what is the decision making
process for the use of the funds)?
42) Do facilities have bank accounts and who is the signing authority?
43) To what extent do approval processes or procurement rules support or
hinder actions (to be) taken at the facility?

4. 4
Core area Exploratory Areas
Payment for
Results
44) What is the per capita cost of RBF?
45) Are funds transferred directly to facilities for outputs received?
46) What are the time lag quarters between performance and payment?
47) Are there cost containment measures in place?
48) How are payments for results being used?
Community
Involvement and
Beneficiaries
49) What kinds of community management structures are in place at the
health service delivery level?
50) How effectively does the facility link with the community management
structure?
51) How and how often does the facility check on beneficiaries’
experiences at the facility and satisfaction with the services?
52) How does the facility integrate the views of the community and
beneficiaries within its own management?