1. Impacts of post-construction support on the
performance of rural water supply in Colombia

2. Water MDG
Water services that last 7 March 2012

3. Background
• IADB re-entering into rural water supply and sanitation
• Encountering sustainability issues around rural water supply in Latin America
– Lack of statistics and data-bases on the state of rural water supply systems
– Recognition of the need for post-construction support, but lack of detailed insight
into key characteristics: how, what, when, who?
– Lack of quantitative data to support claims for post-construction support
• Request to IRC and CINARA to support this through research in Colombia:
• Assess the effectivity and efficiency of various modalities for post-construction
support to community-based water providers on the quality and sustainability
of the water services delivered
Water services that last 7 March 2012

4. Context
• Different rural coverage figures depending on
definitions: 57% - 73%; if water quality is included in
the definition, only 12%
• Decentralization since 1991:
– Community-based service providers
– Municipalities as guarantors of sustainable service
provision
• Role of post-construction support not made explicit
– Some municipalities have taken this up; others not
– National and regional government have set-up support
programmes
– Civil society initiative (AQUACOL and coffee growers
association)
Water services that last 29 Agosto 2011

5. Methodology
• Quantitative analysis to link characteristics of post-construction support to
service levels and performance and governance of service providers
• Applied to 40 rural water systems, selected using stratified sampling in
Caldas, Cauca and Valle del Cauca, supported by 7 different post-
construction support models:
– Business Culture Programme – national government
– Housing Secretariat of Caldas – departmental government
– Health Secretariat of Cali – municipal government
– Aguas de Manizales – urban utility
– Aguas Manantiales de Pacora – urban utility
– AQUACOL – association of community-based service providers
– Coffee Growers Association Caldas
– No post-construction support
Water services that last 7 March 2012

6. Indicator sets
• Indicator sets for: service levels, governance and
performance of service providers, support models
• Scoring tables
– Comparison
– Aggregation
– Quantifying qualitative data
– Data collection: 1 day per system
Water services that last 7 March 2012

7. Service ladder
Coverage Continuity of Quality Net quantity User satisfaction Total score Service level
(%) supply received
(l/p/d)
>90 Equivalent to IRCA between 0 Between 130-170 80% of respondents More than 4,5 High
>23 hours/day and 5% (No l/p/d satisfied with
risks) quality, quantity,
continuity and tariff
80-90 Between 20 IRCA between Between 100 -129 70% of respondents Between 3,75 Acceptable
and 23 hours 5,1% - 14% (low l/p/d, or between satisfied with at and 4,5
risk) 171-200 l/p/d least three of the
indicators
60-79 Between 12 IRCA is Bewteen 50-99 50% of respondents Between 3 Deficient
and 19 hours measured but l/p/d or between satisfied with at and 3,74
not met : 14,1% 201 -250 l/p/d least three of the
- 80 (Medium to indicators
high risk)
<59 Less then 12 No water Less then 50 or Less than 50% of Below 3 Very deficient
horas quality analysis more than 250 respondents
or IRCA very l/p/d, or no water satisfied with three
high, >80% quantity analysis indicators
Water services that last 7 March 2012

8. Indicators for the governance and
performance of service providers
• Three sub-categories
Performance level Score
– Internal governance and legality:
compliance with legal requirements,
organisational structure, decision-
making and accountability High performance More than 80
– Administrative management (incl Acceptable Between 60 and 79,9
commercial aspects and accounting) performance
– Technical and operational Deficient Between 40 and 59,9
management (including water performance
resources management tasks) Very deficient Below 40
• Total 21 indicators, each with a performance
maximum score of 1
• Weighing factor per category
Water services that last 7 March 2012

9. Indicators for post-construction
support
• 3 characterizing variables
without score (direct/indirect, Score Performance level
demand/supply-driven, costs) More than 6 High
Between 4 and 6 Medium
• 10 indicators with score to Less than 6 Low
measure degree of formality
• Based on semi-structured
interviews and survey
Water services that last 7 March 2012

10. Findings: service levels
• Most limiting factors are
Qualification Number of systems
water quality and quantity: High 4
– Lack of information (22 Acceptable 16
systems had no data on Deficient 15
quality; 9 had no data on Very deficient 5
quantity)
– Very high consumption levels
• Not necessarily a problem
for users; their main reason
for (dis)satisfcation is
continuity
Water services that last 7 March 2012

11. Findings: service provider
performance
• Less than half (16) has an
Governance and Administrative Technical
acceptable of high score legality management management
for performance Most common Customer Register of Water
• Above all high scores in low scores relations materials metering
administrative Gender Capacity of Catchment
management (non- balance in the personnel management
payment rate of only water of the water
committee committee
15%)
Most common Inter- Non-payment State of the
• Low scores in “advanced” high scores institutional rates infrastructure
technical management, relationships Accounting Autonomous
particularly catchment Renewal of operations
protection water
committee
Water services that last 7 March 2012

12. Contextual factors
• Settlement size has no impact on service
level; performance differs significantly
according to settlement size
• Technology type impacts on water quality
and performance: more complex systems
have better management
• Age of systems mainly affects performance:
older systems have significantly better
management
Water services that last 7 March 2012

13. 90.0
80.0
Score on performance
70.0
60.0
50.0
40.0
30.0
20.0
10.0
0.0
Menos de 100 (n=14) 101-300 (n=16) Mas de 300 (n=10)
Calificacion promedia en gestion Number of user families
14.0 16.5 23.2
tecnico-operativa
Calificacion promedia en gestion
17.7 20.9 26.0
administrativa
Calificacion promedia en
15.2 19.8 23.8
gobernanza interna y legalidad
Water services that last 7 March 2012

14. Findings: relation between performance of
service provider and service level
• Low level of correlation
5
• Mainly at the extremes: best 4.5
performing service providers
Score on service level
4
have best services and worst 3.5 R² = 0.156
performers have poorest 3
2.5
service, but blur in the middle 2
• Performance of service 1.5
providers closest related to the 1
indicator of water quality 0.5
0
0 20 40 60 80 100
Score of service provider performance
Water services that last 7 March 2012

15. 5.00
4.50
4.00 Calificacion promedia de
satisfaccion de usuarios
3.50
Score on service level
Calificacion promedia de
3.00 calidad
Calificacion promedia de
2.50
cantidad
2.00 Calificacion promedia de
continuidad
1.50
Calificacion promedia de
1.00 Cobertura
0.50
0.00
muy deficiente deficiente aceptable alto (n= 3)
(n=6) (n=18) (n=13)
Performance of service provider
Water services that last 7 March 2012

16. Findings: impact of post-construction
support
• Except for 2, all systems had received some external support the last
year, some ad hoc, some structural post-construction support
• Analysis with original classification and after re-classifying
• Post-construction support does lead to statistically significant better
performance of service providers, but not to better service levels
• But, even with support the average performance is barely acceptable
Number of systems Average performance Average score for service
score of the service level
provider
Systems linked to post- 27 61.1 3.63
construction support model
Systems without structured 13 48.1 3.52
post-construction support
Water services that last 7 March 2012

17. Findings: impact of post-construction support
100.0
Calificacion promedia de desempeno del prestador 90.0
80.0
70.0
60.0
50.0
40.0
30.0
20.0
10.0
0.0
Programa
Aguas Secretaria
Sin modelo Comite de UES Rural Aguas de Cultura
Manantiale AQUACOL Vivienda
de apoyo Cafeteros Cali Manizales Empresaria
s Pacora Caldas
l
Calificacion promedia en gestion tecnico-
14.7 14.7 17.2 17.1 19.0 17.7 20.2 26.9
operativa
Calificacion promedia en gestion
18.6 20.7 20.1 19.9 19.5 25.6 21.6 30.9
administrativa
Calificacion promedia de gobernanza y
17.4 15.0 17.7 18.3 17.8 23.7 25.2 25.6
legalidad
Water services that last 7 March 2012

18. Findings: impact of post-
construction support
• Factors explaining degree of impact of post-
construction support:
– None of the originally identified factors appeared to be
significant (neither scored nor other explicative variables)
– Low correlation with the degree of formality of the
support model (staff profiles and institutionalization)
– Frequency of support activities
– Inter-institutional set-up of models (models acting as
node, referring to dedicated entitites)
Water services that last 7 March 2012

19. 90.00
80.00
70.00
60.00
50.00
40.00
30.00
20.00
10.00
0.00
0 (n=2) 1-4 (n=16) 5-8 (n=13) 9-12 (n=4) Más de 12 (n=5)
• Performance of service provider in relation to frequency of
support activities
Water services that last 7 March 2012

20. Conclusions on the use of indicator
sets
• The sets at system level worked well; above the use of scoring tables was
considered positive both by water committees and officials:
– Main drawback is that standard regression analysis is not possible with
categorised data
– 5 + 21 indicators requires over 75 sub-indicators! – easy to collect much more
information than needed
– Key indicator missing is information management
• The set of variables for the post-construction support model can identify
the degree of formality of the model and describe it, but is not a set that
can explain or predict its possible impact
Water services that last 7 March 2012

21. Conclusions
• Half of the surveyed systems had acceptable
levels of service; more than half had under-
performing service providers
• Low scores particularly due to lack of data on
water quality and quantity, and poor technical
management
• Relatively high scores in financial management
and organisation – result of high emphasis given
to this in Colombia
• In different degrees explained by contextual
factors (settlement size, type of technology, age)
• Low correlation between performance of service
provider and service received
Water services that last 7 March 2012

22. Conclusions
• Nearly all systems receive some support, albeit it ad hoc
• Those with structured post-construction support have
significantly better performing service providers; impact on
service levels is positive but not significant
• High variability within and between support models: not clear
that one model works better than another; rather look at the
underlying factors that explain effectivity of models:
– degree of instutionalisation
– frequency of support
– inter-institutional character of support model
• No clear data on costs or personnel to draw conclusions on
efficiency
Water services that last 7 March 2012

23. Recommendations
• Strengthen performance of providers and indirectly service
levels, partially through post-construction support
• Clarify and specify mandates for post-construction
support, whilst recognising the variety of mechanisms that
already exists, with key role for municipalities
• Focus on institutionalizing support roles of those models that
score low currently
• Use monitoring as tool to identify both generic (water
quality, technical management) and provider-specific support
needs
• Strengthen devolution of information to service providers
Water services that last 7 March 2012

24. Gracias
Stef Smits (smits@irc.nl)