Globally, organisations waste US$122 million for every US$1 billion invested due to poor project performance. Daniel Galorath, the world’s leading expert in project estimation, explains why - and how to create better outcomes.

1. 1
E S T I M A T E
Maximising Capital
Investments:
IS GUESSWORK ERODING YOUR BOTTOM LINE?
E S T I M AT I O N B I A S A N D M I T I G AT I O N 2 0 1 7 E D I T I O N
Dan Galorath for
Unico Enterprise Services
© 2016 Copyright Galorath Incorporated
Contact Details:
Michael McKeon
Unico Enterprise Services
+61429054486
michael.mckeon@unico.com.au
E S T I M AT E • A N A LY Z E • P L A N • C O N T R O L

2. 2
Key Points
Every forecast is
subject to estimation
bias… a major cause
of program failure and
corporate misspending.
Experts Bias is probably
costing - in cost,
schedule, and less than
hoped for benefits.
When bias is managed
better decisions
are made and more
successes follow.

3. 3
Your People Are
Optimism Biased
HARVARD BUSINESS REVIEW EXPLAINS THIS
NOBEL PRIZE WINNING PHENOMENON:
- Humans seem hardwired to be optimists
- Routinely exaggerate benefits and discount costs
- Bias permeates opinions & decisions & causes waste & failure
Delusions of Success: How Optimism Undermines Executives’ Decisions (Source: HBR Articles | Dan Lovallo, Daniel Kahneman | Jul 01, 2003)
Solution:
Temper with “outside view”: past measurement Results, traditional
forecasting, risk analysis and statistical parametrics can help.
Don’t remove optimism, but balance optimism and realism.

4. 4
IT Failure Can Impact
Business DramaticallyCASE STUDY: LEVI STRAUSS
• $5M ERP deployment contracted
• Risks seemed small
• Difficulty interfacing with customer’s systems
• Had to shut down production
• Unable to fill orders for 3 weeks
• $192.5M charge against earnings on a $5M IT project failure
“IT projects touch so many aspects of organization
they pose a new singular risk”

5. 5
Cognitive Bias: tendency to
make systematic decisions
based on PERCEPTIONS
rather than evidence.
“Perception has more to do
with our desires—with how
we want to view ourselves—
than with reality.”
Behavioral Economist Dan Ariely
Researchers theorize in the
past, biases helped survival.
Our brains using shortcuts
(heuristics) that sometimes
provide irrational conclusions.
Bias affects everything:
- From deciding how
to handle our money
- To relating to other people
- How we form memories
Bias Guesses & ForecastsSOURCE: BEINGHUMAN.ORG

6. 6
The Planning Fallacy(KAHNEMAN & TVERSKY, 1979)
• Manifesting bias rather than confusion
• Judgement errors made by experts
and laypeople alike
• Errors continue when estimators are
aware of their nature
JUDGEMENT ERRORS
ARE SYSTEMATIC AND
PREDICTABLE, NOT RANDOM
OPTIMISTIC DUE TO
OVERCONFIDENCE
IGNORING UNCERTAINTY
ROOT CAUSE:
EACH NEW VENTURE
VIEWED AS UNIQUE
• Underestimate costs, schedule, risks
• Overestimate benefits of the
same actions
• “inside view” focusing on
components rather than outcomes
of similar completed actions
• FACT: Typically, a past more
similar assumed
• Even ventures may appear
entirely different

7. 7
Bias Mitigation Reference
Class Forecasting
PREDICTS OUTCOME OF PLANNED ACTION BASED ON ACTUAL
OUTCOMES IN A REFERENCE CLASS:
SIMILAR ACTIONS TO THAT BEING FORECAST.
Attempt to force the
outside view and
eliminate optimism and
misrepresentation
Choose relevant
“reference class”
completed analogous
projects
Compute
probability
distribution
Compare range of new
projects to completed
projects
Best predictor of performance is actual performance of implemented comparable projects
(Nobel Prize Economics 2002)

8. 8
Bias Mitigation Reference
Class Forecasting
PREDICTS OUTCOME OF PLANNED ACTION BASED ON ACTUAL
OUTCOMES IN A REFERENCE CLASS:
SIMILAR ACTIONS TO THAT BEING FORECAST.
Attempt to force the
outside view and
eliminate optimism and
misrepresentation
Choose relevant
“reference class”
completed analogous
projects
Compute
probability
distribution
Compare range of new
projects to completed
projects
Provide an
“outside view” focus
on outcomes of
analogous projects
Best predictor of performance is actual performance of implemented comparable projects
(Nobel Prize Economics 2002)

9. 9
E X A M P L E :
Reference Class Forecasting
With SEER Estimate

10. 10
E X P E R I M E N T :
Anchoring Biases Estimates
SOURCE: MYWEB.LIU.EDU/~UROY/ECO23PSY23/PPT/04-ANCHORING.PPTX
Result:
Those who got higher
numbers on the wheel
of fortune guessed bigger
numbers in Step 3
1
2
3
Subject witnesses the number
that comes up when a wheel
of fortune is spun
Is asked whether the number of
African countries in the U.N. is
greater than or less than the
number on the wheel of fortune
Is asked to guess the number of
African countries in the U.N.
If given a number, that biases estimates

11. 11
Flaw of Averages
Case Studies(SOURCE: HBR)
• U.S. Weather Service forecast that North Dakota’s rising
Red River would crest at 49 feet.
• Made flood management plans based on this average figure
• In fact, the river crested above 50 feet, breaching the dikes, and
unleashing a flood that forced 50,000 people from their homes.
EXAMPLE: $2 BILLION
PROPERTY DAMAGE IN
NORTH DAKOTA

12. 12
M C K I N S E Y - O X F O R D
2013 Study on IT
Program Performance

13. 13
A G I L E :
Detailed Software Development
Life Cycle Management
(SCRUM EXAMPLE)
- Focus is on what features can be delivered per iteration
- Not fully defined what functionality will be delivered at the end?
- Iterations are often called a “Sprint”

14. 14
Agile Is Not a Silver Bullet
SOURCE: HTTP://WWW.JAMASOFTWARE.COM/BLOG/RETHINK-AGILE-MANIFESTO-PROJECTS-STILL-FAIL/
- Projects still fail at roughly the same rate as 2001
- Dr. Dobbs: Agile is not a productivity revolution
0%
20%
40%
60%
80%
Agile (72%) Traditional (64%)8% Improvement
not Revolution
Agile 8% More Success

15. 15
Software Project Performance
Still Disappointing
(McKinsey)
-80%
-60%
-40%
-20%
0%
20%
40%
60%
45% Over Budget 56% less Functionality than
planned
McKinsey Study: Projects over
$15m
So does skipping estimating solve this?
Well: If commitments aren’t made there can be no disappointment

16. 16
Estimation & PlanningA COMPLETE ESTIMATE DEFINED
An estimate is the most knowledgeable statement you
can make at a particular point in time regarding:
• Effort / Cost
• Schedule
• Staffing
• Risk
• Reliability
• Estimates more precise with progress
• A WELL FORMED ESTIMATE IS A DISTRIBUTION

17. 17
If A Promise or Hope Is Good Enough
You Don’t Need Viable Estimates
NO NEED TO ESTIMATE IF:
• A promise of 5 sprints with 4 people is good enough
• You are willing to spend whatever it costs in whatever time it takes
• Estimates don’t impact planning or decision processes
• You don’t need to know the probability that it will be complete or when
• You aren’t concerned if it overruns substantially
• You aren’t concerned with failure and contingency
• You don’t need to consider total ownership costs
• You are ok if this is an overestimate and resources are not optimized
• There will be no system testing on top of development
• If lack of documentation is ok for maintenance

18. 18
Agile Trying To Kill Estimates
Without Considering Business
Needs For Estimates#NOESTIMATES

19. 19
#noestimates is Viable For Detailed Development
BUT SHOULD NOT ABDICATE BUSINESS ISSUES FOR SUBSTANTIAL DEVELOPMENTS
Business Case Evaluation of
alternatives
Agile or Hybrid
Agile Software
Development
System Test
(when appropriate)
Maintenance
and Support
FOR SUBSTANTIAL SYSTEMS
Hybrid Agile:
Requirements
& Design
How Much?
How Long?
Ownership Cost
Go / No Go
• Agile development = root level software development management…
• Story point estimating is short term productivity management
• It is not a business decision making process

20. 20
Mitigate Planning Fallacy & Bias
Additional Historical Data
Trend line
Trend line +1 sigma
Trend line -1 sigma
Matches Platform/Application
Recently added
Used For Calibration
Current Estimate
1
10
100
1,000
10,000
100,000
1,000,000
DevelopmentEffortHours(Log) 1 10 100 1,000 10,000100,0001,000,00010,000,000
Effective Size (Log)
Development Effort Hours vs Effective Size
Additional Historical Data
Trend line
Trend line +1 sigma
Trend line -1 sigma
Matches Platform/Application
Recently added
Used For Calibration
Current Estimate
1
10
100
1,000
10,000
100,000
1,000,000
DevelopmentEffortHours(Log)
1 10 100 1,000 10,000100,0001,000,00010,000,000
Effective Size (Log)
Development Effort Hours vs Effective SizeEstimate Agile Projects With SEER
TO UNDERSTAND WHAT COST / SCHEDULE TO EXPECT

21. 21
STEP
1
2
3
4
5
6
7
8
9
10
EXPECTED
30
50
80
50
90
25
35
45
70
25
500
Adding Reality to Estimates
EXAMPLE – 2 (SOURCE SEI)

22. 22
STEP
1
2
3
4
5
6
7
8
9
10
BEST
27
45
72
45
81
23
32
41
63
23
EXPECTED
30
50
80
50
90
25
35
45
70
25
500
WORST
75
125
200
125
225
63
88
113
175
63
Adding Reality to Estimates
EXAMPLE – 2 (SOURCE SEI)
What would you
forecast the schedule
duration to be now?

23. 23
Monte Carlo Analysis
TO QUANTIFY AND MITIGATE BIAS
90% confidence,
project under
817 days
50% confidence,
project will be
under 731 days
Almost guaranteed to
miss the 500 days

24. 24
SEER:
Risk Driven Estimates
THE ENGINE FOR PROJECT EVALUATION
• SEER predicts outcomes
• SEER uses inputs to develop probability distributions
• The result is a probabilistic estimate
• SEER will predict a likely range of outcomes
• Monte Carlo provides project-level assessments of risk
Least, likely, and
most inputs provide
a range of cost and
schedule outcomes
Confidence
(probability) can be
set and displayed for
any estimated item

25. 25
Key Points
Every forecast is subject
to estimation bias…a
major cause of program
failure and corporate
mis-spending
Experts Bias is probably
costing in cost, schedule,
and less than hoped
for benefits
When bias is managed
better decisions
are made and more
successes follow

26. 26
• FLYVBJERG, BENT, Curbing Optimism Bias and Strategic
Misrepresentation in Planning: Reference Class Forecasting in
Practice, European Planning Studies Vol 16 No 1, Jan 2008,
• Kahneman, Daniel, and Amos Tversky. “Prospect Theory: An
Analysis of Decision under Risk.” Econometrica 47, no. 2,
March 1979.
• Johnson, H. Thomas. Relevance Regained. Free Press.
• Mislick, Gregory K.; Nussbaum, Daniel A.. Cost Estimation:
Methods and Tools (Wiley Series in Operations Research and
Management Science) (p. 143). Wiley.
• Rose, Todd. The End of Average: How We Succeed in a World
That Values Sameness, HarperCollins.
• Taleb, Nassim Nicholas. Fooled by Randomness: The Hidden Role
of Chance in Life and in the Markets (Incerto). Random House
Publishing Group.
Bibliography (Partial)
• Glen, Paul; McManus, Maria. The Geek Leader’s Handbook:
Essential Leadership Insight for People with Technical
Backgrounds. Leading Geeks Press.
• Kogon, Kory; Merrill, Adam; Rinne, Leena. The 5 Choices: The Path
to Extraordinary Productivity. Simon & Schuster.
• Patterson. Crucial Conversations Tools for Talking When Stakes
Are High, Second Edition. McGraw-Hill.
• Kahneman, Daniel. Thinking, Fast and Slow. Farrar, Straus and
Giroux.
• Hubbard, Douglas W.. The Failure of Risk Management: Why It’s
Broken and How to Fix It . Wiley.
• Hubbard, Douglas W.. How to Measure Anything: Finding the Value
of Intangibles in Business .
• Galorath, Evans, Software Sizing, Estimation, and Risk
Management: When Performance is Measured Performance
Improves, Auerbach Publications, 2007