1. Thank you for attending this session. My talk today is about using Earned Value to increase the
probability of success for your program. The Probability of Program Success (PoPS) is a DoD .
From the US Army’s PoPS manual
Secretary Bolton directed this effort as a result of his concern that, although Army programs
were heavily measured and reported via existing metrics systems, he repeatedly saw programs
identify significant problems (unreported in the metrics) prior to decision reviews at his level, or
USD (AT&L)’s.
Starting with the core concepts of Earned Value, I d like to explore how these can be used to increase
Starting with the core concepts of Earned Value I’d like to explore how these can be used to “increase
the probability of success.”
1

2. We all acknowledge that Cost, Schedule, and Technical Performance are inseparable. The NDIA Earned
Value Management Intent Guide (EVMIG) states this.
The challenge is to put this “intent” into practice. To use EV is the presence of the underlying statistical
nature of program activities.
What we need to do next is to put this understanding to work in the pursuit of increasing the
probability of success for a program.
The term “probability of success” itself is statistical. The underlying variables of Earned Value and the
underlying engineering processes in a development program are statistical.
underlying engineering processes in a development program are statistical
Everything is statistical.
So why do we speak in words around CPI, SPI, TCPI, and all the other PI’s in static, single point values?
We are ignoring the very nature of programs. They are statistcial process
2

3. In order to make the connection between Cost, Schedule, and Technical Performance Measures, we
first need to establish the domain in which we will be measuring things.
In order perform measurements we need to know what “done” looks like, so some assessment of
Physical Percent Complete can be determined. Measures of Physical Percent Complete at the detailed
work package level may be different than physical percent complete at the system level. And different
again at the Mission Capabilities level.
3

4. At each of these levels there are a standard set of questions around Earned Value. The scale of each
question may be different, but the questions are still the same.
These questions need to be asked and answered often. On most of the program we work, these are
asked and answered weekly.
4

5. Using 748B there are two approach to gaining compliance. Bottom up and Top down. Let’s look at
both, but let’s start with the bottom up.
The compliance I’m talking about is not the full DCMA type compliance, because even with that full
compliance the program may not be success.
This compliance is the work processes that increase the Probability of Program Success (PoPS). If you
have not heard of PoPS, then I have something to offer. Go to Google and search for the phrase
“Probability of Program Success.” Each DoD Service has a hand book for what to do in this area.
This is more than EV compliance, this is how to manage a program using EV.
This is more than EV compliance this is how to manage a program using EV
We have to remember, all the EV compliance is necessary, but many times not sufficient for program
success
5

6. It is the discovery of the “uncertainty” in the cost, schedule, and technical measures that is the starting
point.
The necessary conditions of establishing the performance measurement baseline (PMB), collecting the
Earned Value performance measures, and managing changes to the baseline, must be accompanied by
the sufficient conditions:
 What are the natural variances on the cost and schedule metrics?
 How do these variances impact the cost and schedule outcomes?
 What are the correlations between the schedule elements and how do these correlations impact the
h h l b h h l l h h l h
forecasting confidence?
This correlation analysis of developed in the cost estimating world, but has not reached the same level
of maturity in the scheduling world.
6

7. Earned Value can provide answers to the standard questions of Program Planning and Controls only if
there are measures of Physical Percent Complete at the Work Package level.
These measures are rolled up to the Control Accounts and the normal Contract Performance Reporting
process results in the monthly CPR.
But there are no fields in the CPR for the “probability” or “confidence” intervals on the numbers.
It’s like predicting the path of a hurricane and its target in the Gulf of Mexico, when the storm is
forming in the open Atlantic.
We all know the statement that CPI at the 15% point in the program is a good predictor of the ETC. But
ll k h h h h f h
this is like saying that the storm will strike the coast of North America above or below Florida. It’s
useful information but is provided at the macro scale level.
The question is “what can we do to change the outcome, once we know we’re not going to make it?”
7

8. Especially when there is an unexpected – or at least undesirable event – on the program.
Dealing with unexpected events is part of a Credible IMS and Cost Plan. So this is nothing new to the
g p p g
mature PP&C organization.
But how is the IMS impacted downstream when Plan‐B replaces Plan‐A?
Modeling the impact on schedule and cost for Plan‐B is a start. But these models need to include the
emerging statistical nature of the programmatic performance as well as the technical performance of
the program
8

9. We have instructions to include Cost, Schedule, and Technical Performance Measures from the NDIA EV
Intent Guide.
The challenge is to put them to work on a program. To use EV to Increase the Probability of Success.
9

10. Here’s a set of promises provided in the EVMIG that need actionable work to cause them to appear.
While each of these is a recognizable beneficial outcome that no one would argue is critical to a
g g
program’s success, the steps needed are not included in the EVMIG.
The specifics of each of these statements need to be worked out in enough detail so they can be
generally applied to a broad set of programs.
10

11. The ultimate outcome of applying the processes is the ability to forecast the future in some way.
The forecast has to have a confidence interval assigned to it.
g
It has to have correctable processes if the outcome is not as desirable as you would like.
So knowing just the statistical nature of the processes is necessary but is not sufficient for increasing
the probability of program success.
11

12. Let’s first the core problem with Earned Value. The units of measure are dollars or what ever currency
you use in your native country.
With a simple slight of hand, money is turned into time. We’ve monetized time in a linear manner. With
little or no consideration if this is actually the case. There is no solution to this dilemma at this point.
There are alternatives to this issue, but they are controversial or obvious depending on your point of
view.
12

13. After the time is not money and money is not time discovery, the next eye opener is the natural
statistical variability of the time elements.
The Monte Carlo simulations are one way to assess the behavior of the network of activities, but there
subtleties beyond just the simulation of the network.
The Monte Carlo tools don’t easily model the coupling and correlations between the work activities in
a proactive manner. There are other tools for modeling the network, but they are also outside this
presentation.
13

14. So when we make simple assumptions, even if they appear complex, when we try to simplify what is
actually a complex problem, when want to bound the variability to fit inside our expected outcomes –
we are setting ourselves up for naïve outcomes.
tti l f ï t
Maybe outcomes that are useful, but may be naïve all the same.
Naïve in the sense that they don’t, or many times, can’t increase the probability of the program's
success.
14

15. Let’s go back to the foundation of Earned Value in the context of the business of project success.
Many times Earned Value is seen and used as reporting tools for past performance. Of course there are
y p g p p
forecasts that can be derived – TCPI is one. Other predictive measures can be used as well.
But the business processes, the program management processes, are more than just the SV and CV
derived variables.
15

16. When we look at the 32 criteria for EV we can count 7 to 9 items are actually related to “earning value”
or the measurement of “earning this value.”
While this is obvious when you look at this chart, it may not be obvious in the “heat of the battle,” to
get the CPR out the door on month end.
In order to earn value from Earned Value we need to bring the understanding about the business
elements into focus.
16

17. So let’s extract the management processes from 748B
17

18. This is the never ending story of program management. The funding spreads are assigned to the PMB.
For all but a trivial project, these spreads and the work they fund changes at the detailed level as part
of the normal variance of a program. So the question of “how much” has a variance itself.
f th l i f S th ti f “h h” h i it lf
This variance is expressed in a confidence interval.
The management reserve is outside the Performance Measurement Baseline. But how much do we
actually need in management reserve. The answer comes from the question “how much unidentified
scope is contained in the program, that will change the PMB, managed itself by the Contract Budget
Baseline, guide by the Change Control Process and reported in Form 3 and 5 of the CPR.
18

19. The answers to the previous questions and many more in the program management processes starts
with “how can we get value” out of Earned Value processes, beyond just mandatory reports to the
customer?
t ?
By value I mean business value. The return on our investment. The addition of business value for the
cost of implementing earned value.
Someone has to pay for all this earned value effort. How can the sunk cost be recovered in some
measureable way.
19

20. The concept of ROI for Earned Value seems odd. EV appears to be an expense.
So we’ll have to think a bit about “returning” the invested value from our EV system.
g y
20

21. The DCAS Gold Practice is one source of Earned Value advice beyond just the formulas and compliance
documents.
The Gold Card suggests there are four value generating outcomes from applying Earned Value.
 Better Visibility means we can see what has been happening and what will happen in the future.
 Reduce cycle time means we can get our work done faster while maintaining the needed quality and
cost.
 Fostering Accountability means there is no ambiguity as to who is accountable for the planned
outcomes
 Reduced Risk means that the potential impediments to progress can be identified, mitigated or
retired/
21

22. From the DAU Gold Book, we are told about these benefits of Earned Value.
Let’s look at how to get to the beneficial outcomes of these promises.
g p
With each statement is some outcome.
But with each statement we need to understand what activities need to take place to produce that
outcome.
How much effort is needed to produce the outcome?
Does this effort get paid back from the benefits of the outcome?
g p
22

23. Gained better visibility into the program’s performance is the starting point for improvement. If we
don’t know how we’re doing we can know if we need to improve.
But more importantly we can’t know if these improvement actually result in business value for the
program.
Without the units of measure of the improvement process, we can’t monetize these units in some way
that we can use for comparison.
23

24. Reducing cycle time means reducing the time it takes to produce a product for the customer.
The detailed planning that comes with EV can be the basis of reducing rework, missed efforts, and
p g g
other contributors to delays
24

25. The accountability aspects come as part of the standard implementation of Earned Value. The OBS /
WBS produces the RAM
25

26. Risk reduction is part of the Performance Measurement Baseline (PMB)
26

27. Now let’s look at the top down approach for earning the value from Earned Value
27

28. With the Performance Measurement Baseline established for the management of the program, the
bulk of the sunk cost for Earned Value is complete. You have the sequence of work packages, a time
phased budget, a description of the deliverables.
h db d t d i ti f th d li bl
What remains is getting all this information into some “system” so analysis can be made.
But if you have decided to manage the program in some credible way to start with – building the
Performance Measurement Baseline – then you’re almost there.
28

29. The completion of the Performance Measurement is necessary but not sufficient. As participants in the
Earned Value community there is sometime the notion that having a PMB is all that is needed. The
PMB is necessary, but not sufficient. The sufficient attributes are:
PMB i b t t ffi i t Th ffi i t tt ib t
 Identification of the needed system capabilities – if we don’t know what the system is supposed to
do from the technical or business mission point of view, then the PMB can not be credible. Because
not matter the accuracy of the PMB or the proper representation of the work package sequencing, or
the labor spreads, if the PMB describes the wrong work, then the result will be disappointing.
 Only after there is a clear and concise description of the needed technical or mission capabilities –
usually in the form of a Concept of Operations (ConOps) and supporting systems engineering models
usually in the form of a Concept of Operations (ConOps) and supporting systems engineering models
– can the technical, operational, programmatic, and business requirements be developed.
 These two sources form the raw material for the Performance Measurement Baseline (PMB). All
ConOps elements must flow down in a properly structured “tree” to requirements. The
Requirements must properly flow down to a Work Breakdown Structure (WBS) to properly describe
the deliverables. Only then can the PMB be developed. An Integrated Master Plan / Integrated
Master Schedule (IMP/IMS) paradigm should be used to structure the PMB, no matter the size of the
program.
 The program then needs to be executed
 All these activities must be performed inside the context of continuous risk management
29

30. Translating System Capabilities into System Requirements begins with the a narrative description of the
needed Capabilities. This sounds like a tautology – a Chicken or the Egg problem. But discovering the
system requirements is difficult in the absence of some higher level description of the needed
system requirements is difficult in the absence of some higher level description of the needed
“Capabilities” of the desired system.
The concept of a “Capability” is a capacity or potential [Davis]:
 Provided by a set of resources and abilities, To achieve a measureable result, In performing a
particular task, Under specific conditions, To specific performance standards
One approach to capturing the system capabilities is:
 Start with the customers understanding the current and future operational needs of their system
 Aligning those needs with industry standards and trends
 Translating the needs into system capabilities in the form of system requirements specification or a
Concept of Operations. The Systems Requirements Specification is not the same as a Technical
Requirements Specification.
 Establish a high–level system concept to identify system components and their interfaces
 Guide the Integrated Product and Process Development (IPPD) Teams, mentoring, and working with
providers of system components (both custom built and COTS) to ensure they adhere to the overall
system view
t i
 Work with the customer to identify and mitigate technical risks through a structured Risk
Management process at the Systems Requirements level
 Verify each system capability through a System Integration and Qualification Test Plan
 Work with the customer to develop a Fielding and Product Support Plan of the delivered system
30

31. Poorly formed requirements have been shown to contribute as much as 25% to the failure modes of
programs and projects.
Requirements engineering can be decomposed into the activities of requirements elicitation,
specification, and validation. Most of the requirements techniques and tools today focus on
specification, i.e., the representation of the requirements. The Deliverables Based Planningsm method
concentrates instead on elicitation. This method addresses problems found with requirements
engineering that are not adequately addressed by specification techniques. [Christel]
This method incorporates advantages of existing elicitation techniques while addressing the activities
performed during requirements elicitation. These activities include fact–finding, requirements
gathering, evaluation and rationalization, prioritization, and integration. The requirements baseline is
established starting with a Fact Finding activity. This does not search for requirements, but instead
defines the boundaries of the requirements space.
With these boundaries established, the actual requirements can be gathered and classified. The
classification process is an important state for assessing the need, evaluation, and tradeoff processes.
Evaluating the requirements is done against the background of the system boundaries and system
capabilities. With these evaluation, prioritization can start. These prioritizations assess the order in
p ,p p
which the requirements will be deployed against the funding and resource limitations.
With the requirements prioritized, they can then be integrated and validated. The integration process
defines the interdependencies between requirements that guide the development of the Work
Packages that produce the deliverables that fulfill the requirements.
31

32. The Performance Measurement Baseline (PMB) is the primary assessment document for assuring the
credibility of a project plan. The PMB is the baseline of the cost, schedule and deliverables for each
Work Package (WP) in the plan.
W kP k (WP) i th l
Constructing the PMB requires knowledge of the system requirements, skill in developing the WPs that
produce the deliverables for these requirements, and discipline in assembling the cost, schedule and
relationships between the WPs. It is the discipline that requires the most focus for the planners and
project controls staff. Without this discipline, the development of a credible baseline is simply not
possible.
In the end the planners and project controls staff must “know” in intimate detail each WP, its
I th d th l d j t t l t ff t “k ” i i ti t d t il h WP it
deliverables and resource requirements, the performance measurement criteria and the dependencies
that form the critical path through the project schedule. The concept of a Deliverables Based Plan
(DBP) is at the core of the Performance Measurement Baseline (PMB).
Deliverables are the units of measure of progress to plan.
 Deliverables are what the customer has paid money for.
 Deliverables contain the system capabilities, the associated value that fulfill the requirements of the
li bl i h bili i h i d l h f lfill h i f h
master plan
The first critical success factor in building the Performance Measurement Baseline (PMB) is the
decomposition of the system requirements into technical capabilities, then into deliverables that
enable those technical capabilities, and finally into the WPs that produce those deliverables.
32

33. The focus of the Performance Measurement Baseline execution steps is to physically assess the
progress of the program in units reflecting the progress using the three independent variables:
 C t
Cost
 Schedule
 Technical Performance
The traditional Earned Value Management approach uses three data sources, the budget (or
planned) expenditures (BCWS), the actual expenditures (ACWP), and the Earned Value (BCWP)
captured from the Control Account or Work Package Manager’s. The comparison of budget versus
actual expenditures indicates what was planned to be spent versus what was actually spent at any
p p p y p y
given time. The use of Earned Value (BCWP) indicates what was produced for that expenditure.
With this approach the use of physical percent complete for the amount of work performed is a
starting point. It does not indicate anything about the conformance to specification of the work
produced for the amount of money spent.
By adding Technical Performance Measures (TPM) to the analysis of Earned Value Management, the
program manager can assess the actual progress of the program. Non–compliance with the planned
Technical Performance Measures dilutes the Earned Value proportionally. This dilution is in the form of:
Technical Performance Measures dilutes the Earned Value proportionally This dilution is in the form of:
 Rework of non–compliant deliverables
 Deferred work not completed during the planned period of performance
 With the period of performance complete, the unused funds – if any – are used to adjust the Earned
Value (BCWP) to reflect the unfinished or deferred work.
 If the work is deferred and there is remaining funds, a change order is issued to move the funding.
 If the work is non–compliant and the funding is exhausted, a dilution of BCWP is needed to reflect
the true Earned Value
33

34. Let’s review. These processes represent a sequence of activities that increase the maturity of the
programmatic aspects of a project or program.
The plans that result from these efforts describe the increasing maturity of the product or services
Th l th t lt f th ff t d ib th i i t it f th d t i
that are the deliverables from the project.
In order to develop and execute a Plan, a set of requirements is needed. Before these requirements
can be developed, an understanding of the system capabilities should be developed.
1. Identify Business Needs ‐ Define the set of capabilities needed to achieve the program objectives or
a particular end state for a specific scenario using the Concept of Operations (ConOps). CONOPs is a
verbal or graphic statement, in broad outline, of an assumption or intent in regard to an operation
g p , , p g p
or series of operations, mission, or system.
2. Establish Requirements Baseline – defines the technical , organizational, and operational
requirements that must be in place for the system capabilities to be fulfilled. Define these
requirements in terms isolated from the implementation details. Only then, bind these
requirements with the technical alternatives.
3. Establish Performance Measurement Baseline – build a time‐phased network of scheduled activities
describing the work to be performed, the budgeted cost for this work, and the organizational
describing the work to be performed the budgeted cost for this work and the organizational
elements that produce the deliverables from this work. Define the technical performance measures
showing how this work proceeds according to the technical and programmatic plan.
4. Execute the Performance Measurement Baseline – execute the Performance Measurement Baseline
Work Packages, while assuring all performance assessments represent measures of Physical Percent
Complete.
5. Continuous Risk Management – identify, plan, and budget risk mitigation or retirement activities at
assure impediments to progress are handled.
34

35. Principles are the source of guidance for Practices .
A principle is a “general truth, a law on which other are founded or from which others are derived.”
[Webster]
For the principles of program and project management to be effective they must :
 Express a basic concept
 Be universally applicable
 Be capable of straightforward expression
 Be self evident
The 10 Principles of Deliverables Based Planningsm guide the application of the four process areas.
These principles encompass the entire life cycle of a project or program, from inception and the
discovery of the business or system capabilities, through requirements elicitation, to the creation of the
Performance Measurement Baseline (PMB), to the execution of this baseline.
The principles provide several feedback loops to assure that subsequent activities provide measurable
information to correct gaps that exist in the previous activities. This iterative and incremental approach
to program management assures the periods of assessment for corrective actions are appropriately
spaced to minimize risk while maximizing the delivered value to the program.
35

36. ANSI – 748 is essential a business management framework.
The Earned Value portions are minimal compared to the “good management” aspects.
p p g g p
36

37. These good management processes are evidenced by documents and reports.
These are connected to the 748B process areas in the following way.
p g y
37

38. So we’ve arrived at the end of our short time here. What did we learn?
38

39. CPI/SPI are the first level outputs from Earned Value, but they are only indicators of past performance.
And their units of measures are money, not time.
y
All variables on the program have statistical behaviors.
When we don’t speak in terms of confidence intervals, then our credibility is reduced.
39

40. But there are steps that can be taken to start down the road of producing value, from Earned Value.
40