This document provides a guide to reading cost-benefit analysis reports. It outlines a 4-step process: 1) get your bearings by identifying the bottom line and what is covered, 2) survey the landscape by understanding the data, perspectives, and breakdown of costs and benefits, 3) look at the details such as costs, time periods, and impacts not monetized, and 4) make sense of uncertainties, limitations, and how everything adds up. The guide is illustrated using a cost-benefit analysis of the Center for Employment Opportunities transitional jobs program.

1. Making Sense of the Bottom Line:
A Guide to Reading Cost-Benefit Reports
April 24, 2012
Gary VanLandingham, Director, Results First, Pew Center on the States
Joshua Rinaldi, Policy Analyst, Cost-Benefit Analysis Unit, Vera Institute of Justice
Slide 1

2. This webinar will teach you to:
• Use a structured framework to read cost-benefit
reports.
• Quickly identify the key components of a cost-
benefit analysis that affect the bottom line.
• Understand how these details and the CBA
are relevant to policy discussions and decisions.
Slide 2 • April 25, 2012

3. Why is cost-benefit analysis important?
• Resources are scarce and getting scarcer.
• People want to know how to use resources more
efficiently to get the most “bang for the buck.”
• Cost-benefit analysis (CBA) provides information to
help weigh the pros and cons of investments.
Slide 3 • April 25, 2012

4. Making Sense of the Bottom Line
Gary VanLandingham Joshua Rinaldi
Director Policy Analyst
Results First Vera Institute of Justice
Slide 4 • April 25, 2012

5. Results First
Results First is partnering with states to assess and
advance policy options that benefit residents and
improve states’ fiscal health. Results First is an initiative
of the Pew Center on the States and the John D. and
Catherine T. MacArthur Foundation, with additional
support from the Annie E. Casey Foundation.
Slide 5 • April 25, 2012

6. The Cost-Benefit Knowledge Bank for Criminal Justice (CBKB)
is a project of the Vera Institute of Justice funded by the
U.S. Department of Justice’s Bureau of Justice Assistance.
• Website (cbkb.org)
• Cost-benefit analysis toolkit
• Snapshots of CBA literature
• Podcasts, videocasts, and webinars
• Roundtable discussions
• Community of practice
Slide 6

7. Agenda
• CBA basics 5 minutes
• A guide to reading CBA reports 30 minutes
• Recap 5 minutes
• Questions and answers 15 minutes
Slide 7 • April 25, 2012

8. Housekeeping items
• If you need Webinar support or for troubleshooting:
 Type questions into the chat box
 Call 800-843-9166
 Send e-mail to help@readytalk.com
• This webinar is being recorded.
• The recording and PowerPoint slides will be posted
on cbkb.org.
Slide 8 • April 25, 2012

9. Housekeeping items
Questions
 Use the chat feature to send us your
questions at any time during the webinar.
 We will address your questions midway
through the presentation and again at
the end.
Slide 9 • April 25, 2012

10. CBA Basics
Slide 10 • April 25, 2012

11. What is cost-benefit analysis?
A type of economic analysis that
compares the costs and benefits of
policies and programs, and
• Identifies efficient uses
of resources
• Uses dollars as a common
measurement
• Presents a long-term picture
Slide 11 • April 25, 2012

12. The CBA process
1. Determine the impact of the initiative.
2. Determine which perspectives to include.
3. Measure costs and benefits (in dollars).
4. Compare costs and benefits.
5. Assess the reliability of the results.
For a glossary of CBA terms:
cbkb.org/basics/glossary
Slide 12 • April 25, 2012

13. A Guide to Reading
Cost-Benefit Reports
Slide 13 • April 25, 2012

14. A guide to reading cost-benefit reports
1. Getting your bearings
2. Surveying the landscape
3. Looking at the details
4. Making sense of your surroundings
Slide 14 • April 25, 2012

15. Cost-benefit analysis of CEO
• The Center for Employment Opportunities (CEO) is a
New York City–based employment program for
formerly incarcerated people.
• We will refer to examples from a cost-benefit analysis
of CEO to illustrate how this guide takes you through a
CBA report.
Slide 15 • April 25, 2012

16. Step 1
Getting Your Bearings
Slide 16 • April 25, 2012

17. Getting your bearings
• What’s the bottom line?
• What does the report cover?
Slide 17 • April 25, 2012

18. What’s the bottom line?
• Look at the executive summary for the “bottom line”
comparison of costs and benefits, expressed as:
• Net Present Value (NPV) or Net Benefits
total benefits – total costs
• Benefit-Cost Ratio (BCR)
total benefits/total costs
• Return on Investment (ROI)
total benefits/(total benefits – total costs)
• For more information:
cbkb.org/toolkit/reporting-cost-benefit-results
Slide 18 • April 25, 2012

19. What’s the bottom line?
• Look at the executive summary for the NPV,
net benefits, BCR, or ROI.
ES, page 18
ES, page 19
Slide 19 • April 25, 2012

20. What does the report cover?
The executive summary may discuss:
• Measurements and components of the costs
and benefits
• Assumptions on which costs and benefits are based
• Omissions from the study
Slide 20 • April 25, 2012

21. What does the report cover?
• Table of contents
• Section headings
• Tables
• Appendices
Slide 21 • April 25, 2012
Page vii

22. Step 2
Surveying the Landscape
Slide 22 • April 25, 2012

23. Surveying the landscape
• What data did the CBA use?
• Which perspectives were included?
• What is the breakdown of costs and benefits?
Slide 23 • April 25, 2012

24. What data did the CBA use?
Identify:
• How recent the data is
• How specific the data is to the jurisdiction
• How costs and benefits were measured
Slide 24 • April 25, 2012

25. Data for the CEO cost-benefit analysis
Evaluation
• Three-year random-assignment study
Budgets
• CEO financial statements
• New York City budget documents
• New York State budget documents
Interviews
• CEO staff
Literature
• Research literature on victimization costs
The data section is on page 56 of the CEO report.
Slide 25 • April 25, 2012

26. Which perspectives were included?
• CBAs examine several perspectives to present a
comprehensive picture of a program’s net impact
to society.
• In criminal justice, CBAs often consider the
perspectives of:
 Taxpayers
 Victims
 Program participants
Slide 26 • April 25, 2012

27. Perspectives in the CEO analysis
Page 67
Slide 27 • April 25, 2012

28. What is the breakdown of costs and benefits?
• Understanding which perspectives are included and
what data was used will help you identify the benefits
included in the analysis.
Page 67
Slide 28 • April 25, 2012

29. Questions
Use the chat
feature to send us
your questions.
Slide 29 • April 25, 2012

30. Step 3
Looking at the Details
Slide 30 • April 25, 2012

31. Looking at the details
• Does the report use marginal costs?
• What time period was analyzed?
• What discount rate was used?
• Which impacts were not monetized?
Slide 31 • April 25, 2012

32. Does the report use marginal costs?
• Marginal costs (and benefits) are incurred because
of changes in units of activity from an existing level
of operations.
Cost per day of keeping one inmate
in prison in New York in 2011
Average cost Marginal cost
$164.59* $27.53**
* Source: Vera’s “The Price of Prisons” report (2012)
** Source: CEO report (2012)
• For more information:
cbkb.org/toolkit/marginal-costs
Slide 32 • April 25, 2012

33. What time period was analyzed?
• Identify the period used in the report to collect or
estimate costs and benefits.
• Costs and benefits should be measured for as long
as they last.
Page 6
Slide 33 • April 25, 2012

34. What discount rate was used?
• Discounting is a technique that translates future costs and
benefits into present-day values.
• Criminal justice CBAs usually use rates of 3% or 5%.
• The CEO report used a 3% discount rate.
Slide 34 • April 25, 2012

35. Why is discounting important?
• A higher discount rate (relative to a lower discount
rate) will more greatly reduce future benefits or costs
in present-value terms.
• The discount rate selected is of particular importance
in CBAs with long time frames where a large share of
the costs and benefits accrue in future years.
• For more information: cbkb.org/toolkit/discounting
Slide 35 • April 25, 2012

36. Which impacts were not monetized?
• If a CBA does not monetize certain impacts, it may
discuss them qualitatively.
Page 57
Slide 36 • April 25, 2012

37. Step 4
Making Sense
of Your Surroundings
Slide 37 • April 25, 2012

38. Making sense of your surroundings
• How does the study address uncertainty?
• What are the study’s limitations?
• How does this all add up?
Slide 38 • April 25, 2012

39. How does the study address uncertainty?
• CBAs rely on estimates and assumptions.
• Uncertainty in estimates or assumptions means
uncertainty in the results.
Slide 39 • April 25, 2012

40. How does the study address uncertainty?
• Sensitivity analysis provides information about
the degree to which results will change if the CBA’s
underlying assumptions are modified.
• Sensitivity analysis can be used to present a range
of results.
Slide 40 • April 25, 2012

41. Uncertainty in the CEO analysis
• In a sensitivity analysis, the CEO results were tested
by replacing the benefits calculated with the “worst-
case” values.
• The CBA was performed again using these worst-case
values to determine whether it still resulted in a
positive bottom line.
Slide 41 • April 25, 2012

42. Uncertainty in the CEO analysis
Page 122
Slide 42 • April 25, 2012

43. What are the study’s limitations?
The report should describe any limitations of
the analysis.
Page 56
Page 57
Slide 43 • April 25, 2012

44. How does all this add up?
• Limitations are a part of any analysis and do not
invalidate a CBA. But it is important that the limitations
are clear.
• Uncertainty is a part of any analysis. Still, it is
important that the range of possible cost-benefit
results is clear, so that policymakers can make
well-informed decisions.
• Benefits are not necessarily budget savings.
Slide 44 • April 25, 2012

45. Recap
Slide 45 • April 25, 2012

46. Putting it all together
• Don’t panic. There is a logical approach to reading
cost-benefit reports.
• Use this guide to determine whether the CBA you’re
reading is based on sound data and assumptions.
• Use the guide to ask questions to better understand
the CBA results.
Slide 46 • April 25, 2012

47. A guide to reading cost-benefit reports
1. Getting your bearings
2. Surveying the landscape
3. Looking at the details
4. Making sense of your surroundings
Slide 47 • April 25, 2012

48. 1. Getting your bearings
• What’s the bottom line?
• What does the report cover?
Slide 48 • April 25, 2012

49. 2. Surveying the landscape
• What data did the CBA use?
• Which perspectives were included?
• What is the breakdown of costs and benefits?
Slide 49 • April 25, 2012

50. 3. Looking at the details
• Does the report use marginal costs?
• What time period was analyzed?
• What discount rate was used?
• Which impacts were not monetized?
Slide 50 • April 25, 2012

51. 4. Making sense of your surroundings
• How does the study address uncertainty?
• What are the study’s limitations?
• How does this all add up?
Slide 51 • April 25, 2012

52. Questions
Use the chat feature to send us your questions.
Slide 52 • April 25, 2012

53. Contact information
Gary VanLandingham
gvanlandingham@pewtrusts.org
Joshua Rinaldi
jrinaldi@vera.org
212-376-5201
Slide 53 • April 25, 2012

54. Sources
MDRC. (2012). More Than a Job: Final Results from the
Evaluation of the Center for Employment
Opportunities (CEO) Transitional Jobs Program
Vera Institute of Justice. (2012). The Price of Prisons:
What Incarceration Costs Taxpayers
Slide 54 • April 25, 2012

55. This project is supported by Grant No. 2009-MU-BX K029
awarded by the Bureau of Justice Assistance. The Bureau of
Justice Assistance is a component of the Office of Justice
Programs, which also includes the Bureau of Justice Statistics,
the National Institute of Justice, the Office of Juvenile Justice
and Delinquency Prevention, and the Office of Sex Offender
Sentencing, Monitoring, Apprehending, Registering, and
Tracking. Points of view or opinions in this document are those
of the author and do not represent the official position or
policies of the United States Department of Justice.
Slide 55

56. Thank you!
Slide 56 • April 25, 2012