What did WSP Learn from Our External Review? (Evaluation Week)
1. What did WSP Learn from Our External Review?
IEG Evaluation Week
October 24, 2011
Jae So
Manager
Water and Sanitation Program
Transport, Water and Information & Communications Technology Department
Sustainable Development Network Vice Presidency
2. About WSP
WSP is not the same organization it was in 1979
• Evolve with changing WSS sector needs
• Evolve with changing business needs
Core mission unchanged
• Help the poor gain sustained access to
improved water supply and sanitation services
• Road test solutions on the ground
Quick Facts About WSP
• Established in 1979
• Core work in economic and sector work, capacity
building and technical assistance
• Most decentralized program in WBG: 24 field offices
in 4 regions with small DC office
• FY budget $46 million
• Guided by Council, chaired by the director of TWI
3. WSP External Evaluation FY2004-2008
Purpose
Provide WSP Council and Management with:
• Independent evaluation of WSP performance during 2004-2008
• Identify lessons to improve the program in the future
• Provide input for future planning and programming
.
4. WSP External Evaluation FY2004-2008
TOR based on IEG’s sourcebook for evaluating GRPPs
Started in Fall 2008,completed in June 2009
• Involved desk reviews, field visits and interviews with WSP staff, donors,
partner organizations and clients, assessed structure of WSP
WSP World Bank Group/SDN/TWI
Council
WSP
WSP-LAC WSP-SA WSP-EAP WSP- AFR WSP-HQ
Clients Clients Clients Clients Clients
5. Outcome of External Evaluation
External evaluation helped to strengthen the realignment of
WSP Programming in four distinct ways:
Institutional arrangements
Relevance
Sustainability
Effectiveness
6. WSP External Evaluation FY2004-2008
#1 Institutional Arrangements
Stronger Governance
• Greater Actions • Council is
engagement for more actively
• Created smaller involved in
Council
working groups programming
to focus on key
issues
Outcome
Recommendation
7. WSP External Evaluation FY2004-2008
#2 Relevance
Strategic Five-Year Business Plan Developed
• Cautious Actions • Results
approach to achieved
expanding • Five-year under each
scope of business plan business area
services identified six core
business areas
Recommendation Outcome
8. WSP External Evaluation FY2004-2008
#3 Sustainability
Predictable Funding
Actions
• Increase • WSP maintains
predictability in • Programming
a strong range
funding determined of donors
before funding vs.
funding before
programming
Recommendation
Outcome
9. WSP External Evaluation FY2004-2008
#4 Effectiveness
Monitoring Framework Developed
• Better systems Actions
• Countries report
to monitor and on results
report on • Established a
framework
results more refined
results
framework
Recommendation
Outcome
10. Outcomes and Challenges
Outcomes
Results Focused
•Results framework
Strategic
• Five-year business plan
Strong Enabling
Environment
• Aligned with CAS and
Water Sector Board
• Strong donor relations
• Predictable funding
Challenges
Lack of standardized evaluation tools among GRPPs limits learning from
other evaluations.
11. Result Focused
Poor-Inclusive Sector Reform
In Kenya, WSP developed a
gender toolkit that was used to
Five-year business plan (FY11-15) train 24 water utilities.
•Help 50 million people gain access to
improved rural sanitation and leverage
US$200 million for sanitation financing; After the training, Githunguri
•With domestic private sector, help 2.5 Water Company no longer
million people gain improved WSS required women to provide title
services and leverage US$100 million for deeds to secure a connection.
WSS financing;
•Support poor inclusive sector reform;
Within six weeks of the policy
•Improve access in urban and peri-urban
change, 50 women had
areas;
signed up for new
•Help WSS services for poor respond to connections and 29 had been
climate change; connected. Our projections
approximate that 300 women a
•Rebuild WSS services in fragile states. year will be connected due to
this new policy.
WSP has evolved over timeWSP is decentralized, solutions are customized and then scaled up globallyWSP core mission remains unchanged.WSP was very infrastructure and technology focused initially Realized that lack of technology and infrastructure was not the issue- need to work through institutions and governments to scale up WSS services WSP has learned how to work at scale--with 2.6 billion people lacking access to improved sanitation and 884 million people lacking access to improved sources of water HUGE Access Gaps still remain. WSP works at scale in three distinct ways: technical assistancecapacity building leveraging knowledge and partnerships
Periodic external evaluations every five years are a part of WSP’s governance arrangements. Evaluation specialist Universalia was selected through competitive bidding
IEG’s five criteria were helpful for us to structure the evaluation. No recommendations on efficiencyIn TOR we included the five criteria that IEG has developed at that time for GRPPs- the criteria were helpful in structuring our own external evaluation Because WSP is such a decentralized program- Universalia conducted field visits across the world
large council- hard to come up with consensus some people not part of the processBroke down the council into smaller groups to focus on key issues ie. Sub-committee on business planning and reporting Council is now presenting plans to each other-and the donors also ceased to see themselves as just financiers or auditors.
Responding to demand from clients– WSP grew from 2004-2008 especially in the areas of rural sanitation and domestic private sector participation$20 million to $40 million in annual fundingNumber of staff doubled External review helped us to manage growth and consolidate programming within 6 business areas
The evaluation ended as the global financial crisis was starting- we were told that we cannot expect the same funding levels and we need to establish a more reliable flow of funding. WSP established a five year business plan , gave clarity to the donors as well as clients, on WSP’s proposed business activities. Since there was predictability on the business planning side, donors were much more comfortable to fund WSP . We have streamlined our financing structure as well…we went from 29 TF in 2007 to 20 current TF
Universalia had a hard time conducting the assessment because prior their was no results framework, and they had no benchmarks from which to assess WSP progress External review helped us establish a clear results framework which was not conceived by us but through a grassroots approach with inputs from focus countries
How did the four aspects link to three outcomes Recommendations regarding sustainability and institutional arrangements has helped WSP to strengthen relations with donors which has helped with predictable and flexible funding Recommendations regarding relevance has helped us develop a strategic 5-year business plan based on our comparative advantage Recommendations regarding effectiveness has helped us to develop our results framework to link activities to results- has improved accountability
WSP supported the Water and Sanitation Service Improvement Project a World Bank credit of USD 150 . In support of WASSIP, WSP developed a gender mainstreaming toolkit targeting gender issues at project, utility, and institutional levels. The toolkit was used to train 24 water utilities.