The document discusses the importance of evaluating outcomes in service delivery. It notes that while services often focus on outputs like clients served, evaluating outcomes is important to determine if a service is making a meaningful difference. Evaluating outcomes can improve services by providing insights into who benefits most and which program elements are most effective. Both monitoring data and evaluations are important, with evaluations helping to assess effectiveness, processes, and cost-effectiveness. The document outlines different evaluation methodologies and challenges in measuring outcomes.
Call Girls Mussoorie Just Call 8854095900 Top Class Call Girl Service Available
The Importance of Measuring Outcomes
1. The Importance of outcomes and
evaluation in service delivery
Yfoundations
10th July 2012
Prof Ilan Katz (SPRC)
2. Will cover
Why outcomes are important
Evaluating for outcomes
Monitoring and evaluation
Types of evaluation
Some key methods
Conclusions
3. Why are outcomes important
Most services focus on outputs:
Numbers of clients seen
Numbers of service events
Unit costs
However this does not measure impacts ie
whether the service makes a difference
can create perverse incentives.
Outcome measurement can improve service
delivery.
4. How does evaluation improve
services
Important to know what impact the service is
actually making on clients and why.
Better understanding of who benefits most
from the service and which aspect of the
service is most beneficial.
Results are fed back into the service so that
lessons can be learned about elements of
effective practice.
5. Outcomes and accountability
Accountability increasingly focused on
outcomes rather than outputs.
Funding decisions now rely on cost
effectiveness as well as assessment of need.
Outcomes and evaluation can also increase
accountability to service users, as participants
and recipients.
6. Evaluation and monitoring
Monitoring – regular collection of data which
is used for quality assurance, adherence to
standards and compliance.
Evaluation – one off or episodic activity
focusing on effectiveness of implementation,
processes, outcomes and/or cost
effectiveness.
Both are important and can build on each
other.
7. Theory of change or program logic
Links inputs processes, outputs and
outcomes.
Explains why activities should lead to
particular outcomes.
Needs to be backed up by theory and
empirical evidence and be specific.
Explains the link between inputs, processes
and outcomes.
8. Process evaluation
Is service doing the things it is supposed to
do in the most efficient way possible?
Has the program been implemented as
planned?
Is it targeting the right people?
What are the barriers to accessing the
service?
How well is it working with other services?
What are clients’ and other stakeholders’
views of the service?
9. Ultimate Outcome:
Young people find stable accommodation and are able to participate actively in society to their maximum potentioal
Intermediate Outcome:
Young people at risk of homelessness are provided with safe accommodation and given the
opportunity to address issues leading to homelessness
Impacts
For children and adolescents:
Improvement in housing situation
Outputs Safety
Situation What services will provide: Improvements in educational/vocational
counselling participation
Limited access to initial assessments Improvements in social functioning
Community specialist Inputs Temporary accommodation Improvements in financial stability
services brief interventions
$X put into youth Reduction in wait time for service
Benefits advice
High demand for service homelessness Better interaction with family
Peer support
Staffing resources Improvement in health and mental health
providing education and support to
Young people unable to
Policies – referrers For homeless Service System:
access mainstream
housing services Commonwealth and Lower numbers of homeless youth
NSW Who the services will interact with:
Young people with acute housing
Better inter-agency collaboration
Young people leaving issues Longer term engagement with vulnerable
Other services
families and becoming families and carers clients
vulnerable to exploitation other members of the broader youth
Client Group: Young
and abuse services For Referrers:
people at risk of
homelessness primary referrers and other service Clear referral pathway
partners Improvement in collaborative care
9
10. Methodologies
Administrative data (employment, health,
housing)
Worker assessments, case files
Use of standard validated surveys
Self completed surveys
pre and post questionnaires
Satisfaction, wellbeing, circumstances
Qualitative methodologies/action research
11. Outcomes
Immediate
Satisfaction, appropriate referral
Intermediate
Improved wellbeing, skills, housing
Long term
Home ownership/stability, employment, social
networks
12. Challenges of measuring
outcomes
When?
Should you follow up clients?
How?
Questionnaire, feedback, admin data
What?
How to define a good outcome
Comparison and benchmarking
Attribution
How do you know it was your service making the
difference?
13. Who should do the evaluation?
In house
Advantages
• Cheaper and more tailored to agency needs.
Disadvantages
• Lack of credibility and independence.
• Need organisational expertise
External
Advantages
• Independent, authoritative – provides feedback
Disadvantages
• Expensive and potentially burdensome, potential
embarrasment
14. Ilan Katz
Social Policy Research Centre
Matthew Gray
CAEPR, ANU
Ilan.katz@unsw.edu.au
www.sprc.unsw.edu.au
www.anu.edu.au/caepr
G2 Western Campus
University of New South Wales
Kensington 2052
NSW, Australia
+61 2 9385 7810