Strategic performance management_system_and_strategic_planning_cycle

April 2007 UI Response to UNDP
Co
PERFORMANCE MANAGEMENT:
A PRACTICAL GUIDE FOR
MUNICIPALITIES
Prepared by
Dr Malik Khalid Mehmood

Assisted by
Katharine Mark
Ritu NayyarStone
Sonia Ignatova

UNDP, Bratislava Regional
Centre
Grosslingova 35, 811 09
Bratislava
Slovak Republic March 2007
Project 08013-000
Contract No. PS-2006/10

Table of Contents

INTRODUCTION AND SCOPE................................................................................................. 6
The Millennium Development Goals and their Role in this Guide .......................................................... 7 Focus of the
Guide.................................................................................................................................... 8 Remainder of
Guide.................................................................................................................................. 8

STEP 1: BEGIN GOVERNING FOR RESULTS IN YOUR MUNICIPALITY
(ORGANIZE THE EFFORT AND DETERMINE THE SCOPE) ........................................ 10
What is Performance Management? How Can It Help Local Governments? ........................................ 10 Focus on Service
Outcomes ................................................................................................................... 11
Limitations.............................................................................................................................................. 11 Select the Scope and
Coverage of the Performance Measurement Process ........................................... 13

STEP 2: IDENTIFY OUTCOMES AND OUTCOME INDICATORS ................................. 18
Identify the service/program objectives, and customers......................................................................... 18 Select the important
outcomes for your service/program....................................................................... 21 Categorize performance
indicators ......................................................................................................... 26
Select outcome indicator breakouts (disaggregation) of each outcome indicator by key characteristics
................................................................................................................................................................ 28

STEP 3: SELECT DATA COLLECTION SOURCES AND PROCEDURES..................... 33
Use agency records................................................................................................................................. 34 Survey Citizens (including
Businesses).................................................................................................. 35 Data Quality
Control .............................................................................................................................. 48 The Cost of Performance
Measurement ................................................................................................. 51

STEP 4: ANALYZE PERFORMANCE DATA....................................................................... 52
The Importance of Analyzing Performance Data ................................................................................... 52 How Performance Data
Can Help .......................................................................................................... 52 Do Some Preliminary
Work ................................................................................................................... 54 Examine the Aggregate Outcome
Data .................................................................................................. 54 Examine "Breakout"
Data ...................................................................................................................... 56 Examine Findings Across
Indicators ...................................................................................................... 59 Make Sense of the
numbers.................................................................................................................... 60

STEP 5: REPORT PERFORMANCE RESULTS................................................................... 65
The Importance of Good Reporting........................................................................................................ 65 Internal
Reporting................................................................................................................................... 65 External
Reporting.................................................................................................................................. 69
Other Information that should be Included When Reporting Performance Data - In Both Internal and
External Reports ..................................................................................................................................... 70 What If the Performance
News Is Bad? ................................................................................................. 70 Dissemination of Performance
Reports .................................................................................................. 71

STEP 6: SET MUNICIPAL TARGETS ................................................................................... 77
The Importance of Setting Targets ......................................................................................................... 77 Set
Targets .............................................................................................................................................. 79 Connect Local Targets to
National Targets ............................................................................................ 83

2

Relation of Municipality Targets to Millennium Development Goals (MDGs) and Strategic Plans ..... 83
Concerns About Targets and Target Setting........................................................................................... 84 Final
Comment ....................................................................................................................................... 84

STEP 7: USE PERFORMANCE INFORMATION TO IMPROVE SERVICES ................ 86
The Importance of Using the Information .............................................................................................. 86 Undertake Service
Improvement Action Plans....................................................................................... 86 Analyze Options/Establish
Priorities...................................................................................................... 91 HOLD HOW ARE WE DOING?
SESSIONS ....................................................................................... 93 Performance
Budgeting .......................................................................................................................... 95 Capital
Budgeting ................................................................................................................................... 99 Strategic
Planning................................................................................................................................. 100 Motivate Your
Employees.................................................................................................................... 100 Performance
Contracting ...................................................................................................................... 101 Contribute to National and Regional
Information Sources .................................................................. 102 Final Comment on Using Performance
Information ............................................................................ 103

STEP 8: BUILD MUNICIPAL CAPACITY.......................................................................... 105
Decide what training is required, to whom, and how much ................................................................. 106 FINAL
WORDS ................................................................................................................................... 109

APPENDIX A: Set of sample service outcome/quality indicators for a variety of municipal services

APPENDIX B: The millennium development goals and likely data collection sources

APPENDIX C: Sample customer questionnaire for municipal services, user survey for water service,
patient feedback form

APPENDIX D: Sample procedures for rating certain quality elements of municipal services using
trained observer ratings: street cleanliness

APPENDIX E: Health status, mortality, country data for MDG 7, Target 10

3

PREFACE

Preparation of this guide has been sponsored by the UNDP as part of an effort to support the achievement
of the Millennium Development Goals.

The Millennium Development Goals (MDGs) were derived from the United Nation's Millennium
Declaration, and adopted by 189 countries in 2000. The MDGs, as they are called, focus on the most critical aspects
of poverty, and the factors that contribute to both income and human poverty. They were re-confirmed in 2005 as the
Global Development Agenda. The MDGs are a set of global goals whose achievement depends on the
implementation by countries of national MDG agendas aimed at achieving nationally adjusted global goals. The
MDGs have played a special role in developing and transition countries as they provide one set of goals that can help
them and their cities as well set concrete targets and concentrate resources on meeting those. There are a large number of
other examples and resources around the world now available, with more and more national governments and city
governments
choosing to measure their performance and improve the services they provide their citizens.

UNDP advocates the adoption by both national and local governments of an MDG agenda (see the
Millennium Declaration, UN, 2000). While this Guide was first targeted at those countries and specifically the
municipalities of such countries which have decided to adopt an MDG agenda (for further discussion of the relationship
between national and local MDG agendas see Capacity Development for Localizing the MDGs, UNDP 2006), the
performance management methodology in this Guide can be used by all municipalities irrespective of whether they
have adopted the achievement of the MDGs as
their goals.

UNDP defines localizing the MDGs as the process of designing (or adjusting) and implementing local
development strategies to achieve the MDGs (or more specifically, to achieve locally adapted MDGs targets). This
implies either adapting and sequencing the indicators and targets of existing local
development strategies as needed, or elaborating a MDGs-based development strategy that reflects local priorities and
realities. For this approach to be successful, it should be locally owned and participatory/ UNDP and other UN
agencies have produced Toolkits and Guides to help municipalities localize the MDGs and thus address regional
disparities and marginalization at the sub-national level. (link to UNDP
and UN-Habitat toolkits).

There is a compelling logic to believe that unless the type of goals included in the MDGs are brought to
the local level (localized), national and global achievements will be skewed. National targets and indicators
represent national averages. Achieving them would require targeted interventions in pockets of poverty which are often
very context specific. In order to impact the lives of people, goals, such as those in the MDGs need to be adapted to the
current level of development and translated into local realities, and
embedded into local planning processes.

A Special Note to Municipalities:

Integrating the MDGs in the municipal performance framework can bring several advantages to your City
and to your country, such as:

- Make sure performance of services related to key development problems such as poverty, health,
education and environment, is monitored and weak performance is identified and addressed;
- Link your local development agenda to the national MDG agenda (if it exists) thus ensuring that

4

the work of your city contributes to nationally set goals (the setting of which you should have
ideally participated);
- Ensure that the work of your City contributes to the global development agenda and contributes
to better and safer world which also benefits you city (poverty and the related crime and diseases,
environment have no frontiers nowadays);
- Increase the chances of benefiting from financial support to your City from central government or
the donor community.

This Guide is intended to support you in this effort.

5

INTRODUCTION AND SCOPE

This guide is aimed at helping all municipalities accomplish their goals by monitoring their own
performance and using the information they get to improve the lives of all their citizens.

Why establish a performance monitoring system to improve results? Many countries of Europe, the CIS
region, and elsewhere in the world have initiated political and administrative decentralization processes. Decentralization
means transparency and accountability to taxpayers, as well as local governments that
strive to continually improve the services they provide to their citizens.

Local authorities play a vital role in improving the well-being of their citizens. They provide the most
basic everyday services - such as solid waste collection, road maintenance, or access to water - as well as working in many
other ways to help their citizens out of poverty and improve their quality of life. Integrating a sound performance
measurement process is only a first step. Often faced with very limited resources, poor quality infrastructure, and
historically weak trust and communication between citizens and local government, local government officials and staff
in Eastern Europe and elsewhere frequently
feel especially handicapped in trying to implement improved services that meet citizen needs.

Setting up a system to monitor performance of municipality programs and policies will enable the
municipality and its agencies to:

• Establish strategic plans, such as City Development Strategies
• Regularly, monitor progress in meeting strategic plan and annual performance targets
• Use performance budgeting as a means to links resources with results
• Identify weak performing services and programs so that the necessary actions can be taken to
improve performance
• Allocate their own resources (not just city budget funds, but also city staff and equipment) in the
most effective way
• Help motivate public employees to continually focus on improving the quality of their services to
citizens
• Identify best practices in order to learn from high performing entities
• Compare performance across localities, regions, and countries to help identify achievable targets
for the municipality's own performance and identify areas that need additional strengthening or
resources

As suggested above, and as will be made clearer in this Guide, establishing a performance monitoring can
have multiple benefits for the work of a municipality, ranging from strengthening strategic planning,
learning what service approaches work well and which do not improve budgeting and justifying the
allocation of funds for initiatives to improve service delivery, to encouraging the reporting of results to citizens. The
Guide suggests steps a municipality can take both to improve collection of information on performance - "performance
measurement" - and to use that information to help get better results -
"performance management."

The use of performance indicators and targets to improve conditions for citizens has increased over the
last decade as local and national governments around the world have become increasingly aware of the
value of results-based decision-making.

6

THE MILLENNIUM DEVELOPMENT GOALS AND THEIR ROLE IN THIS GUIDE

The MDGs provide a clear framework for national and local development efforts taking a holistic multi-
dimensional approach to poverty reduction and human development. They link the global, the national and local levels
through the same set of goals and provide a target-based, measurable framework for accounting for national and local
development results. As already indicated, the MDGs, focus on critical
aspects of poverty, and the factors that contribute to poverty. These include essential areas such as health,
education, access to drinking water, or adequate shelter. The eight MDGs listed below:

Goal 1: Eradicate extreme poverty and hunger
Goal 2: Achieve universal primary education
Goal 3: Promote gender equality and empower women
Goal 4: Reduce child mortality
Goal 5: Improve maternal health
Goal 6: Combat HIV/AIDS, malaria and other diseases
Goal 7: Ensure environmental sustainability
Goal 8: Develop a global partnership for development

The Goals are accompanied by "indicators" and "targets" which are more specific, to allow countries to
focus on particular areas that are important to monitor. Governments and municipalities that have chosen
to adopt an MDG agenda can adapt the global targets and indicators to national and local circumstances. They are should
also include additional outcomes and indicators important to the specific local
conditions.

All the MDGs, with associated targets and indicators, are listed in Appendix B. We have marked the
indicators that might be of particular interest to local governments as part of the indicators they choose to
monitor. Some examples of MDGs and the indicators used to monitor their progress are:

Indicator 7. Proportion of pupils starting grade 1 who reach grade 5
Goal 5: Improve maternal health
Indicator 17. Proportion of births attended by skilled health personnel
Goal 7: Ensure environmental sustainability
Indicator 29. Proportion of population with sustainable access to an improved water source
Indicator 30. Proportion of people with access to improved sanitation

Because of the international interest in MDGs, they have helped focus national attention on the
importance of identifying priority outcomes and using performance measurement tools to reach those outcomes. Most
international donors, including the UN system as a whole naturally support the MDGs
and are aware of the importance - more broadly - of monitoring performance in key areas of public service. The
adoption of an MDG agenda and initiatives related to improving public service delivery can act as effective advocacy
tools and help the mobilization of resources for local development. As cities begin to establish their own
performance measurement systems, it will be useful to have allies at the national and international level, who may be
able to provide resources and data to strengthen the local
efforts.

This Guide will refer to the MDGs throughout, identifying associated resources for local governments,
and suggesting opportunities for using some aspect of the MDGs.

7

This guide shows cities, first, how municipalities can use performance measurement and
performance management to improve their services and their responsiveness to citizens; how they
might use outcome indicators, including MDG indicators, as part of their performance
monitoring; how they can set local targets for each of their indicators and reach the desired outcomes;
and how they can also contribute to the efforts of the country itself to set and meet country-wide targets
to mitigate poverty and improve the quality of
life at the national level.

FOCUS OF THE GUIDE

The material in this guide focuses on "Governing for Results." Most suggestions contained here are
intended to encourage municipalities and their agencies to seek and use information on the results (the benefits) that
their services and programs are achieving for their citizens. This means explicitly
considering the specific likely effects on their citizens when making policy and program decisions.

Achieving results at as low a financial cost (that is, being efficient) is another important area of municipal
performance. However, this is a subject for another guide.

This guide focuses on the process of regularly monitoring the outcomes of a municipality's services. By
regularly, we mean at least annually for purposes such as budgeting, and more frequent monitoring of performance by
individual agency managers, such as quarterly or monthly. The focus is on developing a practical process that
municipalities can adapt to their own situation for regularly, and reliably, tracking their own progress on outcomes of
importance to their citizens. The Guide does not discuss more in- depth evaluation studies, in which, if resources are
available, the government can sponsor more in-depth look into how well a particular program or service is performing
and why. Such in-depth studies can on occasion be very useful to municipalities. However, the regular monitoring of
outcomes discussed here
report can often provide considerable information for such studies.

This guide is complementary to two Guides that have been produced by UN Agencies: 1) Toolkit for
Localizing the MDGs (UNDP); 2) Localizing the MDGs: a Guide for Municipalities and Local Partners (UN-
Habitat). The two publications deal with strategic planning which integrates and localizes the MDGs. As will
become clear from this guide, performance management can be an element of an MDG- based strategic planning and
implementation processes, or any strategic planning process, and a tool for monitoring the implementation of strategic
plans. The readers of this guide are therefore encouraged to
also consult at least one of the two documents mentioned above.

REMAINDER OF GUIDE

This Guide will help you develop high quality performance management systems or improve the ones
already in place. The guide is organized in accordance with the basic steps for instituting performance
management in your local government, as listed below.

Step 1. Organize the Effort and Determine the Scope
Step 2. Identify Outcomes and Outcome Indicators
Step 3. Select the Data Collection Sources and Procedures
Step 4. Analyze the Performance Data
Step 5. Report Performance Results

8

Step 6. Set Municipality Targets for Each Performance Indicator
Step 7. Use the Performance Information to Improve Services and Establish Accountability
Step 8. Build Municipal Capacity for Governing for Results

This guide takes the reader through each of these steps, with each chapter corresponding to one step. The
flow chart below depicts the usual order of these steps. It is important to note, however, that in many
cases these steps are iterative. For example, after setting targets (Step 6) usually data will again be collected (Step 3)
and analyzed (Step 4) in order to monitor progress, and then reported (Step 5), perhaps
leading to the establishment of new targets (Step 6).

STEP STEP 2 STEP 3 STEP 4 STEP 5 STEP 6 STEP 7
1 Choose
Select Outcomes Data Analyze Report Set Use
Servic & Identify Collection Targets
es Indicators

STEP 8 - Build Municipal Capacity

A number of Appendices are provided. These include:

Appendix A: A candidate set of outcome indicators for a number of municipal services. Use these to
give you ideas for the kinds of indicators you might want to select in your municipality.
Appendix B: Millennium Development Goals with associated targets and indicators and likely data
collection sources for indicators applicable to municipalities.
Appendix C: Examples of a sample customer questionnaire for municipal services, an illustrative user
survey for a water service, and a patient feedback form from a hospital in India.
Appendix D: Procedures for rating certain quality elements of municipal services using "trained
observer ratings"
Appendix E: Data for a number of performance indicators to illustrate data that could help a
municipality set its own targets for those indicators.

9

Step 1. BEGIN GOVERNING FOR RESULTS IN YOUR

MUNICIPALITY (ORGANIZE THE EFFORT AND
DETERMINE THE SCOPE)

WHAT IS PERFORMANCE MANAGEMENT ? HOW CAN IT HELP LOCAL GOVERNMENTS?

Performance management - sometimes referred to as "governing for results" - is a system of regularly
measuring the results (outcomes) of public sector programs, organizations, or individuals, and using this information to
increase efficiency in service delivery. Public officials need regular feedback on the
effectiveness of their services in order to make improvements, while at the same time the public wants to
know that the government is spending their tax money in the best way to meet citizens' needs.

What is the difference between Performance Measurement and Performance Management? - Performance
Management goes one step further, using the measurements of performance to manage
services better.

Traditionally, this kind of information has been hard to get - emerging only piecemeal through
complaints or occasional anecdotes. Over the last three decades, performance management has become increasingly
popular way for governments around the world - both local and national governments - to
manage their programs and services to achieve the results their citizens expect.

BENEFITS OF A PERFORMANCE MANAGEMENT SYSTEM TO LOCAL GOVERNANCE

• Improving service quality and outcomes;
• Improving resource allocation and justifying agency budgets or service cuts;
• Making public agencies accountable for results to elected officials and the public;
• Increasing the citizens' trust in the local government; and
• Making work more interesting and satisfying for public employees because of its
citizen focus.

Governing for Results has encouraged governments both to make the results they are seeking explicit and
to design programs and policies to actively and directly seek those results. Monitoring progress towards those results
provides a constant feedback into the policy and implementation process to improve efforts
to achieve their objectives.

FOCUS ON SERVICE OUTCOMES

Performance management is based on a simple concept: a focus on service outcomes, or actual results,
rather than only on the quantity of service that an agency provides. This implies assessing the performance of
the government based on the implications of services to customers, not on physical outputs. The work of the local
government is measured by what the citizen or user of the service actually experiences - are the roads in good enough
condition so that children can get to school in the rainy season? Do the children stay in school until they graduate? Do
pregnant women visit the primary health clinic during pregnancy, and does that result in healthier babies? Is garbage
collected regularly, and does
that have an effect on health?

This simple idea, however, means that many people in local government need to think in a different way.
It will not be enough to measure how many kilometers are paved, or whether the clinic has the right staff. It will be
important to see what the results of those efforts are in order to know whether they are working
well.

Through tracking performance indicators, and clearly linking those indicators to the results that the local
governments wants to see, the system provides decision makers with better information. With this information
they can make better decisions—and show why they made those decisions. Using
performance management, local governments can demonstrate their commitment to providing quality
service.

This way of thinking can proceed at several levels. For example, a city may believe it is very important to
provide a "safe, health, and clean environment for all the citizens of the municipality." That may lead to the identification of
a number of outcomes that will be sought in several different services: good quality service at the primary health clinic,
public awareness about health hazards, better solid waste collection, clean water sources. Each of those in turn can require
a number of different outcomes. For example, the head of solid waste management may want to ensure that streets
are clean, citizens satisfied with collection service, landfills are well managed, and that the service has full cost
recovery so that good
service is sustainable.

LIMITATIONS

Municipal officials need to recognize important limitations of the performance information
that would come from the steps discussed here. These include:

• The regularly collected annual performance information discussed in this report will
not tell you WHY the recorded performance levels were good or bad. (However, a well-
designed performance measurement process can provide useful clues. For performance
indicators that indicate unexpectedly low, or high, results, more in-
depth evaluations will be needed to get at the causes).
• Similarly, the performance information does not tell municipality officials what has
to be done to correct problems identified by the performance measurement data.
• Performance measurement information provides information about past
performance. It does not by itself tell what future result will be. However,
information on past performance provides a major source of information for estimating
future results, such as needed for making budget, policy, and program
decisions.

11

GET STARTED

Depending on their size, governance system, and capacity constraints, governments around the world are
using different tools to govern-for-results. One or more of the following approaches might be used to start
developing a performance management system in your city:

Develop and track selected performance indicators in each service sector. Make policy decisions
based on the information and disclose this information in city performance reports and its budget.

Develop service improvement action plans in priority sectors.

Apply performance management to the internal processes of the local government, for example to
increase municipal revenue, or reduce the time it takes for citizens to register births or marriages.

Implement a comprehensive performance management system in your city that combines strategic
planning, setting goals and objectives for each service sector, citizen participation and using performance
information. A municipality can choose to adopt an MDG agenda and integrate MDG Goals in its strategic
plan. For each of the MDG Goals this municipality should set targets and
indicators which reflect local circumstances.

Whoshouldbeinvolved?

Many stakeholders play a role in the process of implementing and using a performance management
system. Some of the key actors are described here.

Mayor
The Mayor should be a principal user of performance information especially in establishing major policies and
in reviewing city programs and its budget. In addition, the Mayor will play a major role
in setting the climate for the shift to a results-orientation. The Mayor's support is important, for
making sure that adequate resources are allocated to implementing and, later, sustaining the process..

City Council
The support of the council will be essential to the success of the enterprise, not only through the provision of
funding when necessary, but also through underlying the importance of performance information by requesting it
and using it. Elected council members will find outcome information to be very useful in carrying out their
responsibilities, enabling them to more easily understand the impact of city services on their constituents, and
to make decisions in their appropriation and
oversight roles.

DepartmentHeads
The heads of different departments or institutions play a crucial role in facilitating and using
performance data.

National and Regional Government
These entities should have their own performance management processes. Many of their agencies are likely to need
performance data (such as on health, education and welfare) to make their own policy and program decisions - and to
provide information that will enable them to set national targets (such
as for MDG indicators).

12

National and regional agencies may also use the performance indicators as a basis for identifying
local areas that require special assistance, training, and help in achieving equity in the country, or in
identifying best practices that can be shared with other localities to improve service everywhere.

International Donors
Donor loans or grants sometimes stipulate the condition that infrastructure projects or grants-in-aid be
subject to detailed performance monitoring, or tied to achievable results. For example, the World Banks output-
based-aid involves delegating service delivery to a third party (private firm or NGO)
under contracts that tie payment to particular outputs or results delivered. Other agencies like USAID,
CIDA, and DFID also use performance monitoring for specific projects, own inter-agency
performance, and in some cases link them to country plans. A number of international donors, including the
UN system, are focusing assistance efforts on helping countries attain the MDG targets and may want to see the
connections between municipal programs and those targets. These donors can provide valuable assistance in
capacity building within municipalities, by providing training on performance management systems, and creating
incentives for performance monitoring in service
delivery.

Non-Governmental Organizations
Different types of NGOs can play two valuable roles in improving service delivery: (1) in providing important
quality public services themselves -- in which case they should themselves have their own performance management
process, and (2) playing a watchdog or advocacy role in increasing citizen
awareness of their rights to better quality public services.

Business Community
Businesses are a major consumer of government services (such as water, transportation, and, economic
development services). In addition, many services are delivered through contracts. To ensure quality services, the
municipality can use performance contracts with these businesses. s Data for performance monitoring will, thus, also
have to be obtained from, or at least with the cooperation
of, these businesses.

Citizens
Citizens are the major consumers of public services. And they pay (via their tax payments or fees),
for many services. Citizens are also a major source of information needed to evaluate services and are an important
source for identifying the outcomes that should be tracked, both annually and for
strategic plans.

SELECT THE SCOPE AND COVERAGE OF THE PERFORMANCE MEASUREMENT PROCESS

Which services should you include?

You might choose to start with one service (or program), several, or cover all municipal services and
programs. It is recommended here that you attempt to cover all your services so that all municipality staff are encouraged to
focus on results. Realistically, however, you may need to start with a few services at a time, so that the successes in one
area can serve as motivation to introduce performance management
more widely.

13

There are several different ways to decide where to start. One method is to identify the departments that
might be "easiest" - for example, areas where data is easily available, or some inexpensive improvements are most likely to
yield rapid results. Another might be to start with departments in which the leadership
is already very interested in adopting the new approach, which is also likely to make the pilot effort
easier.

Another approach to choosing a starting place can be to look at citizen priorities. To do so might slow
down the process, but does identify an area that is likely to yield improvements in citizen satisfaction in the short term.
International donors are often extremely supportive of such consultations with citizens, and can help fund such efforts.
Some more detail about such an approach is provided in Step 7 on Using Performance Management, under the description
of Strategic Planning. If there is strategic planning, the
strategic goals identified will determine the services that are to be monitored.

The Millennium Development Goals can also provide guidance on what outcomes to select. A step by
step approach on how a municipality should select services that contribute to the MDGs might be as
follows:

Step 1 - Review the MDG Goals and select those global Goals which are relevant to local realities (in
a transition or developing country context, all should normally be relevant)
Step 2 - Identify the services which are related to those Goals
Step 3 - Adapt the indicators to local circumstances
Step 4 - Review the national (if they exist) and global indicators related to the selected (and adapted)
targets and select those which are relevant to local circumstances
Step 5 - Identify additional outcomes and indicators which are relevant to local circumstances and
contribute to the MDG goals
Step 6 - Collect baseline data for all indicators
Step 7 - Set for each indicator targets appropriate for the locality, given the priorities, citizen
preferences, needs, and available resources, and also bearing in mind as possible benchmarks,
such sources as national MDG targets, performance in other localities, and adapting those
targets to local circumstances
Step 8 - Identify non-MDG related city goals, identify indicators and set targets

Local governments in different countries are responsible for different types of functions. In addition, in
some countries those functions may be subject to change, especially where a process of decentralization may be
underway. It makes sense to start with services that are fully under the control of the locality, because that is where
improved decisions will have the greatest impact, but there have also been instances when performance measurement can
also be applied to functions that are mixed - i.e., shared between central and local government -- or even largely central
functions. It can be especially difficult in shared
functions to ascertain the effectiveness of the service, and measuring performance can provide useful input on the
various aspects of the service. Thus, the central government might gain information about how well local governments
are carrying out a task, or performance information might show that central
funding or regulations aren't yielding the results that were expected.

Exhibit 1-1 provides a list of exclusive and shared functions in Albania as of 2006. It can be noted that
while most cities that have used performance management in Albania have focused first on exclusive functions, such
as solid waste collection, parks, street cleaning, or water provision, there have also been
several efforts to measure performance in areas of shared functions, such as in education and social
assistance. Some of those examples will be provided in this Guide.

Exhibit 1-1. Functions of Communes and Municipalities in Albania

14

Exclusive Functions of Communes and Municipalities

I. Infrastructure and Public Services
a. Water supply
b. Sewage and drainage system and [flood] protection canals in the residential areas;
c. Construction, rehabilitation and maintenance of local roads, sidewalks and squares;
ç. Public lighting;
d. Public transport;
dh. Cemeteries and funeral services;
e. City/village decoration;
ё. Parks and public spaces;
f. Waste management;
g. Urban planning, land management and housing according to the manner described in the law.

II. Social Cultural and Recreational Functions
a. Saving and promoting the local culture and historic values, organization of activities and management of
relevant institutions;
b. Organization of recreational activities and management of relevant institutions;
c. Social services including orphanages, day care, elderly homes, etc.

III. Local Economic Development
a. The preparation of programs for local economic development;
b. The setting [regulation] and functioning of public market places and trade networks;
c. Small business development as well as the carrying out of promotional activities, as fairs and advertisement in
public places;
ç. Performance of services in support of the local economic development, as information, necessary structures and
infrastructure;
d. Veterinary service;
dh. The protection and development of local forests, pastures and natural resources of local character.

IV. Civil Security
a. The protection of public order to prevent administrative violations and enforce the implementation of commune
or municipality acts;
b. Civil security.

Shared Functions of Communes and Municipalities

a. Pre school and pre university education;
b. Priority health service and protection of public health;
c. Social assistance and poverty alleviation and ensuring of the functioning of relevant institutions;
ç. Public order and civil protection;
d. Environmental protection;
dh. Other shared functions as described by law.

Source: Law on Organization and Functioning of Local Governments, No. 8652, dated 31.07.2000

A Good Approach: Establish a Municipal Steering Committee and Working Groups

Once you have determined the scope of your performance measurement process, a good way to begin
implementing it is to establish a high-level, across-government, Steering Committee to oversee the process. The
Steering Committee can then establish a Working Group to lead work on the details of
implementation. The Steering Committee should include such persons as:

- A representative of the Mayor
- A high-level official of the finance/budget office
- A high-level official of the human resources (personnel) office
- Several department heads
- Information technology high-level official

15

The Working Group should have representatives from the departments carrying out or overseeing the
work in question, from the financial department, and from a number of related areas. Encourage each
participating municipal department to have its own working group

Exhibit 1-2 provides examples of the types of people that might be included in these department working
groups. Such groups should consider including a representative from outside the government to obtain a
broader, consumer, perspective.

Exhibit 1-2. Example of Working Group Composition

Solid Waste Working Group Head of Technical and Land Administration
City Manager Municipal Services Manager
Social economic department Head of Planning and Information
Director of financial department Construction and Design Team
Head of the solid waste collection company Municipality administrators
Sanitation team Representatives from business firms
Health offices administration Financial Management team
Municipality level social team
Municipality level economic team Construction and Maintenance of Asphalt and
Environmental protection authority Gravel Roads Working Group
Representative from an NGO or citizen group Technical and Land Administration Department
interested in city cleanliness Technical team
Administrative Support services
Education Working Group Urban Development and Construction Bureau -
Deputy Mayor local branch
School principal
Representative from the parent-teacher Municipality-Wide Working Group
association Deputy Mayor
Representative from the Education Committee Head of public works department
of the City Council Director of finance department
Other department heads
Land Management Working Group Representatives of selected NGOs

What should be the functions of these working groups?

The government-wide working group should have such task as:

- Developing a government time table for implementation
- Identifying and defining the types of indicators to be included
- Identifying staff training needs and making arrangements for the initial training efforts - for both
management and line personnel.
- Develop a communication strategy
- Communicate with local government bodies, City Council, civil society and ordinary citizens
- Arranging for the development of guidelines for major data collection procedures
- In general, guiding such steps as those described in this guide in Steps 2-8

16

The department working groups should have similar tasks but focused on their specific needs.

17

Step 2. IDENTIFY OUTCOMES AND OUTCOME INDICATORS

IDENTIFY THE SERVICE/PROGRAM OBJECTIVES, AND CUSTOMERS

For each service or program included in the municipality's performance measurement process, the
municipality should start by identifying the service's objectives. What is the service intended to do for
the city and its citizens? What are the primary benefits desired?

A good statement of objectives should identify the key intended benefits and who are the intended
beneficiaries (such as all the municipality's citizens or some particular segment). This process should also identify the
possible unintended effects, both beneficial and negative effects. Each of these will help
formulate the outcomes that will be tracked. Ask such questions as:

• Who benefits from the program - in what ways? Which demographic groups are particularly
affected by the program?
• Who might be hurt by program activities - in what ways?
• What persons who the program does not directly target might be significantly affected by the
program?
• Is the public-at-large likely to have a major interest in what the program accomplishes?

Exhibit 2-1 provides examples of the objectives and affected citizen groups for a few services.

Exhibit 2-1. Service, Objective, and Customers

Program or Service Objective Customers or Users
Solid waste collection Clean city and neighborhoods City residents
Schools Better education Children and parents;
employers
Financial Department Increase municipal revenue All municipal services and
citizens
Road maintenance Safe and rideable roads City residents and city visitors
All services Improved collection of fees All municipal services and
citizens
Land management Adequate housing City residents City residents
Water authority Healthy population Elderly people, their families,
Social services Healthy and secure elderly their caretakers
Housing people
Health
NGOs

Examples of key customer groups in different programs are shown in Exhibit 2-2.

18

Exhibit 2-2.
Examples of Key Customer Groups in Different Programs

A road construction program Citizens and transportation companies
A water treatment plant Citizens, businesses, and visitors to the community
A vocational school program: Students, parents, and local businesses who recruit the school's
graduates
A sports facility Athletes and the general public
A municipal park Adults, children, and senior citizens in the community, and
visitors

SELECT THE OUTCOMES TO TRACK FOR EACH SERVICE

Identifying the specific outcomes that you will try to achieve, given the service's objectives, is one of the
most important parts of this process.

What isanoutcome?

An outcome is the result of a service, from the point of view of the citizens, especially the
customers for the service. We can start by thinking about the various steps that go into
delivering a service:

First, there are inputs - these are the resources we use, for example, money or employees of the
municipality.

Second, there are outputs - these are the products that the city department, contractor, or
agency produces, such as kilometers of road repaired, tons of garbage collected.

Third are the outcomes - these are the results of the service: the roads are in good condition,
the city streets are clean, etc.

It is useful to identify two primary levels of outcomes; "intermediate" outcomes and "end"
outcomes. We can think of the higher outcomes - the "end outcomes" or "ultimate" outcomes • as the real purposes of
what we are doing. For instance, the improved health of citizens that comes from a clean city, or the ability to
go to work or school quickly and safely that is made possible by good roads. We call these "end outcomes."
An intermediate outcome is also a result, not just an output, but the accomplishment of something that is
likely to lead to an end
outcome.

Exhibit 2-3 diagrams the "causal" relationship among these categories. Funding and people are needed to
implement activities. Those activities are expected to produce outputs that are expected to lead to
intermediate outcomes and then to end outcomes.

Exhibit 2-3. Building Towards Results

19

End
Outcome

Intermediate
Outcomes

Outputs

Inputs

Below are some sources of information that can help you identify what outcomes your municipality
should track. Each source is likely to have its own perspective on what is, or should be, important to citizens and the
community a whole. Most, probably all, services and programs will each need to consider multiple outcomes in
order to be comprehensive as to what is important to citizens and the
community.

• Discussions or meetings with customers and service providers
• Customer complaint information
• Legislation and regulations
• Community policy statements contained in budget documents
• Strategic plans
• Program descriptions and annual reports
• Discussions with upper level officials and their staff, to identify future directions, new
responsibilities, new standards at the national or regional level
• Discussions with legislators and their staff
• Input from program personnel
• Goal statements by other governments for similar programs
• Poverty Reduction Strategy Document (or other national strategy)
• Sector Strategies
• Regional Development Strategies

You can obtain information on program results through meetings with customers (known as "focus
groups"); meetings with program staff; and meetings with other local government personnel.

Exhibit 2-4 provides several different examples of outputs, intermediate outcomes, and end outcomes.

Exhibit 2-4.
Examples of Outputs, Intermediate Outcomes, and End Outcomes

Output Intermediate Outcome End Outcome
Roads are repaired Roads are in good condition Citizens can reach work,

20

school, markets and services
Clinics are built and staffed Pregnant women visit clinic Children are born healthy
Garbage is collected Neighborhoods are clean Lower incidence of disease
Customers are billed Fees are paid Cost recovery enables
adequate services to be
provided
Water is supplied Citizens have access to water Citizens are healthy
Schools have desks and Children attend school Children are educated
textbooks

How are outputs and outcomes different?

An important element of performance measurement is that it differentiates between outputs and
outcomes. In measuring what government does, the traditional focus has been on tracking
expenditures, number of employees, and sometimes their physical outputs.

An Output or an Outcome?

Sometimes people are confused about the difference between an output and an outcome. A
key question is how likely it is to be important to citizens and service customers. Outputs are usually the
physically things that services and their employees did (e.g., paved 200 square meters of road), while
outcomes are what those things are expected to accomplish from the
viewpoint of the "recipient" of the service (e.g., road condition is good).

The outcome focus of performance measurement, however, connects performance to the
benefits of services for citizens and the community. For example, performance measurement is concerned not
with the number of teachers employed, but with the reduction in student dropout rate. Of course, focusing on
outcomes does not mean that you neglect outputs. Instead, a focus
on outcomes provides a framework for you to analyze outputs in a meaningful way.

In the above example, hiring more teachers or increasing the number of lessons taught does not
necessarily reduce the number of students dropping out of school. It may mean that you also need special
programs to improve the employment opportunities for parents for students who are dropping out of school.
Or you might set up a preventive counseling program to help those
students who are the most likely to drop out. Measuring the performance of programs targeted
at decreasing the dropout rate would then tell you how successful or unsuccessful these
programs are.

Another example: Focusing on the percentage of your municipality's roads that are in good,
rideable condition, rather than on the number of square meters of road maintained, helps
identify specific areas that most need maintenance attention.

SELECT THE IMPORTANT OUTCOMES FOR YOUR SERVICE/PROGRAM

21

It is not always obvious what outcomes should be selected, but the best way to decide is to think about
what is most important. Whoever is responsible for selecting the outcomes should brainstorm about outcomes before
making a final decision. Remember, there can be several "layers" of outcomes - ranging from the end outcomes (e.g., the health
of citizens) to a number of intermediate outcomes (e.g., access to water, sufficient water pressure, and adequate cost
recovery). Usually, you will want to track both intermediate outcomes and end outcomes to determine both whether the
end outcome has been reached and which intermediate outcomes have been successful or might need to be adjusted. End
outcomes are more important, but intermediate outcomes provide services/programs with earlier information on
progress and, thus, usually need to be tracked.
One important source of outcomes that should not be neglected is consultation with end-users, that is, those who will
benefit from the service. This might include meetings with citizens in different
neighborhoods, or the use of information from a citizen survey.

How do you "brainstorm" for outcomes?

Brainstorming is a technique to help a group think creatively to come up with new ideas. The central
"rule" is that everyone should say what he or she thinks openly and without inhibition. No one will be critical. A good
way to start might be to have a large piece of paper (maybe on a flip chart) and ask everyone to suggest outcomes
they would like to see. Go ahead and shout them out. Write everything
down. After the "brainstorm", the group will discuss the choices to decide which outcomes they will
focus on.

Multiple outcomes are to be expected for any public service.

Here are some examples of some outcomes that have been selected in cities in Eastern Europe for selected
services:

Solid waste collection:
- Areas around collection points are rated "clean"
- Citizens are satisfied with cleanliness in their neighborhoods
- Full cost recovery via collection of garbage fees

Water
- More households are connected to the city water system
- Citizens feel they have enough water when they need it
- Increased water quality
- Full cost recovery through water tariffs

Outcomes contributing to the Millennium Development Goals

Some local governments may want to consider in what ways their local functions contribute to reaching
the Millennium Development Goals. As a starting point, it is useful to note that most local government services are
essential contributions to the Millennium Development Goals, although they are not specifically identified by the
Goals, the Targets, or the Indicators. For instance, maintaining the adequacy of roads - a key local function in most
countries - is essential to ensuring access to many primary services (clinics, schools, markets, water). Appendix
B provides an annotated version of the MDGs,
suggesting ways in which local services might be contributors to the Goals.

22

Local governments may choose to start their performance management efforts in areas related to one or
more Millennium Development Goal, choosing outcomes over which the local government has some
control and that are important to the community. An example might be:

Supporting outcomes:
• Good condition roads to allow access to schools
• School facilities are in good condition (in countries where local governments are responsible
for school facilities)

IDENTIFY AND DEFINE THE SPECIFIC PERFORMANCE INDICATORS THAT WILL BE MEASURED -
FREQUENCY OF COLLECTION , UNITS, LEVEL OF DISAGGREGATION

For each outcome you identify, you also need one or more specific outcome indicators, specific ways to
measure progress toward that outcome. Outcomes Indicators are at the heart of performance management. They are the
elements you will measure and track to see whether your local government is achieving the results it wants. For each
outcome that is sought, measurable indicators need to be selected that permit
the government to assess the progress being made towards the outcome.

An indicator must first of all be measurable. Not all outcomes of programs are measurable, or at least
directly measurable. You need to translate each outcome of the program into performance indicators that
specify what you will measure. In some cases you may want several indicators for one outcome.

Typically indicators start with the words "number of" or "percent of." In some cases, you will want to
measure both the number and the percent; for instance you might want to measure the total number of children who have
received vaccinations, and also the percent of children that represents, so that is clear
how many children are still at risk.

Sometimes, when you cannot measure directly a particular outcome, you can use a substitute indicator, a
proxy indicator. For example, for outcomes that seek to prevent something from occurring, measuring the number of
incidents prevented can be very difficult, if not feasible. Instead, governments track the number of incidents that
do occur - a proxy indicator. These proxies are not ideal but they can be the
only practical approach.

Each indicator needs to be fully and clearly defined so that data collection can be done properly and
produce valid data.

For example, consider the important indicator: "Proportion of population with sustainable access to an
improved water source." What do the words "sustainable," "access," and "improved water source," mean? Different people
responsible for collecting the data for the indicator can easily define each of these terms differently. Next year
different staff might interpret the terms differently than those
collecting the data last year.

An excellent source for definitions, especially for MDG indicators, is that provided by the United Nations
Development Group. The block below presents the definition provided by UNDG for the water-access
indicator used in the above example. The MDG indicators most likely to be directly relevant to
municipalities are listed in Appendix B.

23

Exhibit 2-5
Definition of the Indicator "Proportion of Population With Sustainable Access to an Improved
Water Source, Urban and Rural"

The percentage of the population who use any of the following types of water supply for drinking: piped
water, public tap, borehole or pump, protected well, protected spring or rainwater. Improved water sources do not
include vendor-provided water, bottled water, tanker trucks, or unprotected wells and
springs.

Access to safe water refers to the percentage of the population with reasonable access to an adequate
supply of safe water in their dwelling or within a convenient distance of their dwelling. The Global Water Supply
and Sanitation Assessment 2000 Report defines reasonable access as "the availability of 20 litres per capita per day at a
distance no longer than 1,000 metres." However access and volume of drinking water are difficult to measure, so
sources of drinking water that are thought to provide safe
water are used as a proxy.

Source for this and the other MDG indicator definitions: Indicators for Monitoring the Millennium Development
Goals. Definitions, Rational, Concepts, and Sources. 2003. New York: United Nations. This publication is
available at undp.un.org -in six languages.

Such available definitions should provide your municipality with a very good starting point. However, as
the definition in the block indicate, at least some tailoring to own local situation is likely to be necessary
to fully define each indicator.

Appendix A provides a candidate set of outcome indicators for a number of typical municipality services.
(The MDG indicators included in Attachment B are included in the set of candidate outcome
indicators presented in Appendix A.)

You need to consider several factors when selecting performance indicators. Exhibit 2-5 suggests a set of
criteria for selecting them. Rate each indicator according to these criteria.

Exhibit 2-5.
Criteria for Selecting Performance Indicators

Relevance. Choose indicators that are relevant to the mission/objectives of the service and to what they
are supposed to measure.

Importance/Usefulness. Select indicators that provide useful information on the program and that are
important to help you determine progress in achieving the service's objectives.

Availability. Choose indicators for which data can likely be obtained and within your budget.

Uniqueness. Use indicators that provide information not duplicated by other indicators.

Timeliness. Choose indicators for which you can collect and analyze data in time to make decisions.

24

Ease of Understanding. Select indicators that the citizens and government officials can easily
understand.

Costs of Data Collection. Choose indicators for which the costs of data collection are reasonable.

Exhibit 2-6 is an example of indicators that you could use as a starting point for two different programs.
This exhibit also contains a fourth major category of performance indicator: efficiency indicators. These are usually
defined as the ratio of the cost of a particular service to the amount of product that was
produced with that amount of expenditure.

The unit of product traditionally has been one of the outputs. The efficiency indicator usually is of the
form "cost per unit of output." However, a sole focus on output efficiency can tempt employees to speed up their work,
sacrificing quality. A municipality that also collects outcome data can in many cases then
use a much more true indicator of efficiency: "cost per unit of outcome." For example, the public works agency can then in
addition to tracking cost per meter of road repaired also track "cost per meter of road
repaired that was improved from an unsatisfactory condition to a good condition."

Exhibit 2-6.
Illustrative Performance Indicators
City of Bangalore, India

Water Supply Environment
Input Input
Cost Cost
Staff Staff
Materials, equipment Materials, equipment

Output Output
Average number of hours of water supply per Number of persons per hospital bed, including both
day government and private sector hospitals
Ratio of number of stand-posts in slums to total Percentage distribution of waste water treated by
slum household each method
Daily consumption of water in litres per capita Percent of waste water treated and re-cycled for non-
per day (LPCD) consumption purposes

Outcome Outcome
Percentage of water lost during distribution Noise pollution in decibels at selected locations
total water supply Percentage of population suffering from pollution-
Average citizen satisfaction rating with water resultant respiratory diseases
quality Percentage of population suffering from pollution-
Percentage of households having safe or resultant water-borne diseases
potable water source located within 200 Pollution load per capita per day
meters of the dwelling

Efficiency Efficiency
Cost of installing water harvesting equipment Average cost, per kilolitre, of waste water treatment
(per kilo litre) Cost per person treated in hospitals by pollution-
Cost per metered household resultant diseases

25

Adapted from "Bangalore City Indicators Programme." (December 2000). Government of Karnataka, Bangalore Metropolitan
Region Development Authority

CATEGORIZE PERFORMANCE INDICATORS

It is good practice for a municipality and its agencies to categorize each of its indicators by such
categories as those given above. This will help users of the performance information keep in mind the
relative importance to the city and its citizens of the individual indicators.

Input, output, and efficiency indicators are relatively familiar to program managers. Governments
regularly use them to track program expenditures and service provided. Indicators of outcomes are much rarer even though
they are more helpful in determining the consequences or results of the program.
Categories of performance indicators are described below, and examples are shown in Exhibit 2-7.

It is important for you to recognize the differences between the following categories of
information:

Inputs
Input data indicate the amount of resources (amount of expenditures and amount of personnel
used in delivering a service.

Outputs
Output data show the quantity of work activity completed. A program's outputs are expected to lead to desired
outcomes, but outputs do not by themselves tell you anything about the outcomes of the work done. To
help identify outcomes that you should track, you should ask
yourself what result you expect from a program's outputs.

Outcomes (intermediate and end outcomes)
Outcomes do not indicate the quantity of service provided, but the results and accomplishments of those
services. Outcomes provide information on events, occurrences, conditions, or changes in attitudes and behavior
(intermediate outcomes) that indicate progress toward achievement of the objectives of the program (end
outcomes). Outcomes happen to groups of customers (e.g., students or elderly persons) or to other
organizations (e.g., individual schools and/or businesses) who are affected by the program or whose
satisfaction the government wishes to
attain.

Efficiency and Productivity
These categories relate the amount of input to the amount of output (or outcome). Traditionally, the ratio of the
amount of input to the amount of output (or outcome) is labeled "efficiency."
The inverse, which is the ratio of the amount of output (or outcome) to the amount of input, is
labeled "productivity." These are equivalent numbers.

Exhibit 2-7.
Examples of Performance Indicators

Input Number of positions required for a program

26

Cost
Supplies used
Equipment needed

Output Number of classes held
Number of projects completed
Number of people served
Number of letters answered
Number of applications processed
Number of inspections made

Outcome Crime rate
Employment rate
Incidence of disease
Average student test scores
Percent of youth graduating from high school
Number of successful rehabilitations
Number of traffic accidents

Efficiency Cost per kilometer of road repaired (output based)
Cost per million gallons of drinking water delivered to customers (output
based)
Cost per number of school buildings that were improved from "poor" to
"good" condition (outcome based)

Exhibit 2-8 contrasts output and outcome indicators for specific services or activities.

Exhibit 2-8.
Contrast Between Output and Outcome Indicators

Output Indicators Outcome Indicators
1. Number of clients served. 1. Clients whose situation improved.
2. Lane kilometers of road repaired. 2. Percentage of lane kilometers in good condition.
3. Number of training programs held. 3. Number of trainees who were helped by the program.
4. Number of crimes investigated. 4. Conviction rates of serious crimes, and crime rate.
5. Number of calls answered. 5. Number of calls that led to an adequate response.

As a summary of selecting performance indicators, Exhibit 2-9 provides an example of Objectives,
Outcomes, and Indicators for a road maintenance program. The example also provides targets for
improving performance. Targets will be addressed later in this manual, in Step 5, Data Analysis.

Exhibit 2-9.
Example of an Objective, Outcomes, Indicators, and Targets
Road Maintenance Program

Objective:
Provide safe, rideable roads to the citizens, by regular renovation and maintenance of existing roads and
by upgrading of any unpaved roads in the municipality.

27

Outcomes:
(1) Maintain municipality's road surface in good, better, or excellent condition.
(2) Reduce traffic injuries or deaths by improving the condition and clarity of road signs.

Indicators for Outcome (1):
Input: cost of paving the road, personnel, equipment; amount of equipment used.
Output: kilometers of road paved; number of households having paved roads.
Outcome: kilometers of road surface in good or excellent condition; percent of citizens satisfied with road
conditions.
Efficiency: cost per kilometer of road paved; cost per kilometer of road in excellent condition.

Indicators for Outcome (2):
Input: cost of new road signs, personnel costs.
Output: number of road signs improved; number of new road signs installed.
Outcome: traffic injuries or deaths; road signs in good or excellent condition.
Efficiency: cost per new or improved road signs.

Target for Outcome (1):
Ensure that 90 percent of the road surface is in good or excellent condition.

Target for Outcome (2):
Reduce traffic injuries or deaths during the year by 10% through improved road condition and clarity of
road signs.

An important element of selecting performance indicators is to define each indicator thoroughly so that
measurements will be made in a consistent way by different personnel and over time. For example in the above
measurement of roads in various conditions, The municipality agency needs to define how to
determine whether a meter of road is in excellent, good, fair, or poor condition.

SELECT OUTCOME INDICATOR BREAKOUTS (DISAGGREGATION) OF EACH OUTCOME INDICATOR
BY KEY CHARACTERISTICS

Your municipality and your agencies will find the outcome information considerably more useful for
making improvement if you breakout the outcome data by key customer and service characteristics. This will much
better enable users of the data to identify more precisely where problems, and successful
practices, are present. Consider breaking our the outcome data into categories such as the following:

• By geographical location;
• By organizational unit/project;
• By customer characteristics;
• By degree of difficulty (in carrying out the task in question); and
• By type of process or procedure you use to deliver the service.

Each of these recommendations for indicator breakouts is discussed below.

By geographical location
Break out data by district, neighborhood, etc. The presentation of data by geographical area gives
users information about where service outcomes are doing well and where they are not.

28

Exhibit 2-10 shows the percentage of respondents who rated the cleanliness of their neighborhood in
Püspökladány, Hungary, as very clean and somewhat clean. Overall (for the entire city), 45 percent of respondents
stated their neighborhood was very clean or somewhat clean. However, when you break
up responses geographically (by districts) you begin to see interesting variation. While most of the districts got a
similar rating on neighborhood cleanliness, only 26 percent of respondents in district 1 rated their neighborhood as
very clean or somewhat clean. This shows that district 1 is a problem area, and the city needs to examine why
residents in that district rated cleanliness so low. (Note: The seven districts in the city were categorized based on
socioeconomic conditions. Respondents were asked, "How would you rate the cleanliness of the neighborhood you
reside in from 1 to 5, where 1 is
very dirty, and 5 is very clean?").

By organizational unit/project
Separate outcome information on individual supervisory units is much more useful than information on several
projects lumped together. For example, it is useful to have separate performance information on each public
works departmental unit, not only for all the units together. Another useful application of breakouts by
organizational unit would be to have separate performance information on the different units of the police
department. For example, response times could be
examined for individual units that specialize in particular crimes or other emergencies.

By customer characteristics
Breakouts by categories of customers (e.g., age, gender, education) can be very useful in highlighting
categories of customer services that are or are not achieving desired outcomes. For example, if the government
finds that the daytime hours of operation for reporting a problem with city services are too limited, the government
may consider opening a hotline in the evenings for citizens to contact them who otherwise are not able to call
during the day. For another example, park staff may find that they have put too much effort into satisfying parents
with children and that their parks are lacking
facilities that the elderly can enjoy.

By degree of difficulty
All programs have tasks that vary in difficulty. A more difficult program will have a harder time achieving
the results you desire, and therefore distinguishing the degree of difficulty of a program will drastically change
your perception of its outcomes. To show good performance an organization is sometimes tempted to attract easier-
to-help customers, while discouraging service to more difficult (and more expensive) customers. Reporting
breakouts by difficulty will eliminate this temptation. Exhibit 2-11 gives an example of considering the
difficulty factor in presenting performance
information.

By type of process or procedure you use to deliver the service
Presenting performance information by the type and magnitude of activities or projects being
supported by the program is very useful for you. For example, a street cleaning program can comprise
sweepers, garbage cans and dumpsters, and garbage trucks. You should present data on each project in the program by
(1) the type and amount of each activity; and (2) the indicators resulting from each
project's efforts.

29

Exhibit 2-10.
Geographic Location Breakout

30

Exhibit 2-11.
Workload (Client) Difficulty Breakout

Unit No. 1 Unit No. 2
Total Clients 500 500
Number Helped 300 235
Percent Helped 60% 47%

Difficult Cases 100 300
Number Helped 0 75

Non-Difficult Cases 400 200
Number Helped 300 160

Note: If you only looked at aggregate outcomes for Units 1 and 2 together, you would unfairly evaluate Unit 2, which had
a higher proportion of difficult cases.

You can use breakouts for purposes such as the following:

• To help pinpoint where problems exist as a first step toward identifying corrective action;
• As a starting point for identifying "best practices" that might be disseminated to other program
areas, by identifying where especially good outcomes have been occurring; and
• As a way to assess the equity with which services have been serving specific population groups

A summary checklist of these breakout categories is given in Exhibit 2-12.

Exhibit 2-12.
Possible Client and Service Characteristic Breakouts

Service Characteristics
Gender Examine outcomes for men and women separately.
Age Examine outcomes for different age ranges. Depending on the program, the age
groups might span a large range of ages (such as examining clients under 21,
between 21 and 59, and 60 and older), or the program might focus on a much
smaller age range (such as youth programs wanting to compare outcomes for youth
under 12, 13-14, 15-16, and 17 or older)
Race/Ethnicity Examine outcomes for clients based on race/ethnicity.
Disability Examine outcomes based on client disability. For example, some programs might
want to determine whether clients with disabilities rate services differently than
those without disabilities, as well as the outcomes for clients with various types of
disabilities.
Educational level Examine outcomes for each client based on the educational level achieved before

31

starting service.
Income Examine outcomes for clients grouped into specific income ranges based on the
latest annual household income at the time clients began service.
Household Examine outcomes for households of various sizes, generations, and number of
children.
Difficulty of Examine outcomes by incoming statute based on expected difficulty in being able to
problem at intake help the client. Inevitably, some clients are more difficult t help than others. For
example, an employment program might want to consider the literacy level of its
new clients. An adoption program might want to relate outcomes to the age and
health of the children.
Service Characteristics
Facility/Office Examine outcomes for individual facilities or offices.
Service provider Examine outcomes for clients of individual service providers, such as caseworkers.
Type of Examine outcomes for clients who were served using each distinct procedure. For
procedure example, a youth program might have used workshops, field trips, classes, etc.
Amount of Examine outcomes for clients who received varying amounts of service. This might
service be expressed as number of sessions a client attended, the number of hours of service
provided each client, or whatever level of service measurement the program uses.

Source: Analyzing Outcome Information: Getting the Most from Data, The Urban Institute, 2004.

32

Step 3. SELECT DATA COLLECTION SOURCES AND
PROCEDURES

IDENTIFY A COLLECTION METHOD

A performance indicator is not very useful until a feasible data collection method has been identified. For
MDG indicators, a United Nations publication (2003) provides general suggestions for data collection and sources. (See
Appendix B for a list of the MDG indicators likely to be directly applicable to municipalities, and their
data sources.) However, your municipality will need to work out the data
collection procedure details.

For the MDG indicator used as an example in Step 2, "Proportion of the population with access to
an improved water source," the UN report notes that the usual sources have been administrative records on facilities
and surveys of households. (It states that the "evidence suggests that data from surveys are more reliable than
administrative records and…provide information on facilities actually used by the population.) We note that
another possible data collection procedure is the
use of trained observer rating procedures to help determine what is available to households.

Data for most of the MDG indicators are obtained from national census and survey (usually conducted
every two to five years) or agency records from national line ministries. In some cases data are also
computed directly by the countries National Statistical Offices, the World Bank or UNESCO Institute for Statistics. In
addition to these national surveys or agency records you will need to track disaggregated
values for the indicators for your municipality.

There are four primary sources of performance data:

• Agency records
• Surveys of citizens
• Ratings by trained observers
• Use of special measuring equipment

In this guide we discuss the first three in some detail below.

Several factors will affect your decisions of which sources to use for which indicators:

• How applicable is it to the information you seek? (For instance, outcome information such as
citizen satisfaction or ratings of service quality can only be obtained from surveys of citizens.)
• What is the availability of sources from which you can obtain the information?
• How much time and resources would it take to regularly collect the data?
• What is the likelihood that reasonably accurate data can be obtained from the procedure?

33

USE AGENCY RECORDS

Examples of performance data obtainable from agency records (sometimes call "administrative records")
include the following (some of these records will be available locally, others from the national
government):
• Incidence of illnesses and deaths in a hospital Tracking Citizen Calls and
(end outcome indicator) Response Times in Indjija,

Serbia
• Results of test scores in schools (end outcome The key feature of Indjija's Sistem48 is a call center
indicator) for citizens to make complaints, comments, or

• Total percent of owed fees collected requests concerning any local government service.
(intermediate outcome indicator) After the call is received, several things take place:
• Number of complaints received (intermediate • Callers are guaranteed a response within 48
hours
outcome indicator)
• The complaint or request is forwarded to the
• Percent of time equipment is operational (such as service in question for resolution
equipment for street cleaning or public transit vehicles • Data about the call is logged and reported,
(internal intermediate outcome including:
indicator) — Time of call
• Response time to respond to citizen requests for — Content

Length tion
a service, such as to determine eligibility for a Data on — calls areof time until resoluMayor in bi-
the reviewed by the
public welfare benefit, to obtain a business weekly meetings with the departments. Receiving
permit, to receive emergency medical attention, and recordfing citizen calls provides anasimponrtangt
measure o citizen satisfaction, as well poi tin
etc. (intermediate outcome indicator) to specific areas of particular concern.
• Cost per kilomet er of road maint ained
(efficiency indicator)

• Size of workload, for example number of buildings needing inspection or number of kilometers
of street that need to be repaired (used for calculating outcome and efficiency indicator values)

Why is it useful to use agency records?

The advantages of using agency records as data sources are their availability, low cost, and program
personnel's familiarity with the procedures. Since agency record data are already collected and available, this has been the
major source of performance data used by local governments. This information can,
thus, form the starting point for your performance measurement system.

For some performance indicators, an agency might need to obtain information from another municipal
agency or even from another level of government. For example, one of your public welfare agencies
might need record data from the health department to process an application for disability benefits.

Can you use existing processes?

For performance measurement purposes, however, your agencies are likely to need to modify their
existing processes. For example, you may have to modify the forms and procedures to enable you to
calculate service response times. This involves:

• Recording the time of receipt of a request for service;

34

Strategic performance management_system_and_strategic_planning_cycle

Strategic performance management_system_and_strategic_planning_cycle

Recomendados

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente (20)

Destacado

Destacado (12)

Similar a Strategic performance management_system_and_strategic_planning_cycle

Similar a Strategic performance management_system_and_strategic_planning_cycle (20)

Más de Malik Khalid Mehmood

Más de Malik Khalid Mehmood (20)

Strategic performance management_system_and_strategic_planning_cycle