This PowerPoint was presented at the Triple-S research seminar, taking place in Kampala, Uganda from 24-26 September 2012.

1. WATER SERVICE DELIVERY INDICATORS
Research seminar
Kampala
24-26 Sept 2012
WATER SERVICES THAT LAST …1

2. INTRODUCTION
• Development and use of Service Delivery
Indicators:
– WASHCost in Ghana, Burkina, India, Mozambique
– Triple-S Ghana and Uganda (and Burkina Faso)
– IADB support to monitoring in Latin America
– USAID/Rotary Club sustainability assessment
• Common indicator sets:
– Service level indicators
– Service provider indicators
– Service authority indicators
– (Enabling environment indicators)
WATER SERVICES THAT LAST …2

3. SERVICE LEVEL INDICATOR(S)
• Commonalities:
– Similar indicators, include: functionality, quality, quantity, accessibility,
reliability
– Indicator details and scoring based on national norms and standards
• Issues:
– Inclusion of user satisfaction indicator(s)
– Differentiating between service provided (facility level) and service
accessed (household level)
– Different units of analysis: users –facilities
• Advantages of taking users as unit of analysis:
– More accurate assessment of level of service (no risk of double counting)
– Takes into account the use of multiple sources of water (especially relevant in rural
settings)
– Takes into account the unserved within an area
– Easy to aggregate for an area (based on percentage of people with access to different
level of service)
• Advantages of taking facility as unit of analysis
– Less units / data point -> possible lower costs
– No sampling required or more straightforward
– More useful to inform asset management
– Easier to link service level data to service provider data
WATER SERVICES THAT LAST …3

4. SERVICE PROVIDERS INDICATORS
• Commonalities:
– Similar clusters of indicators:
• Organisation / governance
• Administrative maintenance
• Financial management
• Technical / operational management
– Based on sector norms and standards, where
available
WATER SERVICES THAT LAST …4

5. PERFORMANCE OF SERVICE AUTHORITIES
• Commonalities:
– Indicators are related to:
• Resourcing of service authority (human
resources, logistics, etc.)
• Planning functions
‘Real’ authority functions
• Coordination functions
• Monitoring functions
• Direct (post construction) support Direct support functions
functions
• Issue: need to differentiate clearer
between direct support functions and
‘real’ authority functions
WATER SERVICES THAT LAST …5

6. WHY COLLECTING DATA USING THE SERVICE
DELIVERY INDICATORS?
Reason for collecting monitoring data Type of data Frequency
Monitor impact of (project) activities All or depending Periodically (e.g.
on activity annually)
Information on state of assets to inform asset Facility data Once
management Functionality Real time
Information on service levels and performance of Service provider Periodically (e.g.
service providers to inform direct (post quarterly)
construction) support
Information on performance of service authorities Service authority Periodically (e.g.
to inform capacity support annually)
Tracking performance and holding service Service level
providers and authorities accountable (regulation) Service provider Periodically (e.g.
Service authority annually)
Finding correlations between different indicators all Once (baseline)
to inform sector discussions on how to do things
better (single loop) and different (double loop)
(Learning)
WATER SERVICES THAT LAST …6

7. SCORING OF INDICATORS
• Commonality:
– Use of QIS tables, to make
qualitative information
quantitative
– Scoring levels: none – some –
acceptable – better than WS-Op3a: Corrective is executed in an
acceptable – ideal effective way
– Mix of scoring logics: Mostly use Is corrective maintenance
carried out?
No
Score: 0
of likert scale, but also
Yes
summations of scores per sub- Is corrective maintenance No
indicator carried out within 24
hours?
Score: 50
Yes
Score: 100
WATER SERVICES THAT LAST …7

8. COMPARING SCORES
Comparing scores
Reason for collecting over between between between between
monitoring data time Units areas SDMs indicator
sets
Monitor impact (needs
control
group)
Asset management
Direct (post construction)
support
Capacity support
Regulation
Do things better (single
loop) and different (double
loop) (Learning)
WATER SERVICES THAT LAST …8

9. ANALYSIS OF SCORES
• Average score
– per indicator Most useful for research
– per indicator cluster
• Benchmarking
– Units that meet the benchmark per indicator
– Units that meet a certain number of benchmarks per
indicator cluster Most useful for informing
asset management, direct
support, capacity support,
regulation
WATER SERVICES THAT LAST …9

10. SCOPE OF DATA COLLECTION
Difference:
• Full coverage or sampling
WATER SERVICES THAT LAST …10

11. COMMON CHALLENGES
• Coding of facilities and service providers
and matching them
• Obtaining accurate information on service
level sub-indicators like quality and
quantity provided and used
WATER SERVICES THAT LAST …11

12. DATA COLLECTION COSTS
• Ghana:
– Triple-S baseline 3 districts: approx 0.12 US$ / capita
– Unicef baseline 10 Northern districts: approx: 0.48
US$ / capita
– WASHCost: approx 0.50 US$ / capita
• El Salvador:
– Baseline: approx 0.30 US$ / capita
– Repeat data collection: approx 0.10 US$ / capita
• Costs mostly related to logistics (fuel, per diems)
• Tension between usefulness of data (including
accuracy) and costs of data collection
WATER SERVICES THAT LAST …12

13. WHAT NEXT?
• Action: Feed results of data analysis into (national level) sector discussions
and discourse
• Research:
– Process documentation of sector change processes, influenced by monitoring
data (double loop learning)
• Action: Scaling-up of use of indicators (e.g. by Unicef in 10 districts in
Northern region of Ghana)
• Research:
– Analysis of collected data
– Documentation of scaling up process: Costs, what works well, what does not?
– Process documentation of use of data for improving asset management, direct
support, capacity support, regulation (single loop learning)
• Action: follow-up / continuous data collection
• Research:
– Analysis of collected data for monitoring progress
– Analysis of collected data to influence sector change processes
WATER SERVICES THAT LAST …13

14. QUESTIONS
• How to use SDI data in future experiments?
• How to use data to reflect on usefulness and
relevance of national norms, standards and
guidelines?
What needs to be done to refine SDI?
• Do we need to include user satisfaction?
• Do we need household level data? For what? Is it
worth the extra effort (and costs)?
• How to aggregate scores for an area with multiple
SDMs?
• ……?
WATER SERVICES THAT LAST …14