1) Current EVMS measures like CPI and SPI provide visibility into cost and schedule performance but do not capture other important factors like technical performance and risk. 2) A balanced approach is needed that connects EVM measures to integrated master plans which include measures of effectiveness, performance, technical milestones, and key parameters. 3) The recommended solution is to make integrated master plans mandatory, update guidance to show how to connect EVM to plans, use statistical analysis to forecast estimates, and require risk-adjusted estimates and regular risk register reviews.

1. Beyond CPI/SPI
Forecasting the Corrective Actions
Needed to Keep the Program GREEN
Glen Alleman and Tom Coonce
The Situation
We’re a Program Controls Manager on ACAT1 and ACAT2 programs using DCMA
Validated Earned Value Management, with monthly reporting of CPI and SPI
along with the other variables in the IPMR (DI-MGMT-81861).
Our government Program Manager is using our IPMR information to assess the
performance of our program, along with CDRL submittals, and other
performance documentation.
These measures are used to provide the visibility to program performance, the
Estimate At Completion, and Estimate to Complete for the program.
The Risk Register is not a submittal, nor the reporting of Measures of
Effectiveness, Measures of Performance, Technical Performance Measures, and
Key Performance Parameters as part of the submittal for Earned Value
Management.
 Root Cause Analysis of the situation
 Known missing processes and data
The Protagonist
The Program Manager is a key participant. In the EV community we think of
ourselves as the ones providing the information to help manage the program,
there are other participants in the effort to increase the probability of program
success.
These participants start with Systems Engineers. The Program Controls staff are
the planners, scorekeepers, and advisors to those performing the work.`
The Imbalance
The use of Earned Value may be necessary, but it is far from sufficient to plan,
measure, and control the performance of the program to increase the probability
of success.
There are several core reasons for this gap in the ability of Earned Value:
 While EAI 748-C states in the open page that Earned Value should
Objectively assess accomplishments at the work performance level, there is
no guidance on exactly how to do that.

2.  As well in §3.8 of EAI-748-C, it says, Earned value is a direct measurement
of the quantity of work accomplished. The quality and technical content of
work performed is controlled by other processes. Earned value is computed
based on the budget assigned to the completed work scope.
 The units of measure in Earned Value are dollars. So the meaning of
Schedule Variance (SV) in the IPMR is not defined without looking at the
Integrated Master Schedule (IMS). Knowing where to look is also not
defined in any of the IPMR fields.
The imbalance is obvious everywhere. From OTB’s to Nunn McCurdy’s, to simple
cost and schedule overruns. Add to that, the less than acceptable technical
performance of many of our programs, the imbalance is we can’t show up on
time, using the money we’ve been given, and have some acceptable probability
that the thing we’re producing will actually work as planned.
The source of the imbalance starts when we measure our progress to plan with
the “Gold Card” CPI and SPI that is not based on measures associated with the
technical performance of the product, the effectiveness of that product in the
“mission space,” and the compliance with the Key Performance Parameters
needed to assure the outcome will be useful to those paying for the program.
As well, when we forecast the future performance of the program in the absence
of the risks to that future performance we are not being credible.
Finally when we make any forecast (an estimate of the future) in the absence of
the statistical behavior of the past performance and the statistical performance
of the future performance (risk is an uncertainty that is statistically based), then
the credibility of our numbers is suspect.
 Specific assessment of RCA on ACAT1’s
 Traceability to missing parts in the policy and practice
Aiming for Balance
We must start by recognizing that Earned Value, the BCWP part of Earned Value,
must be informed by more than the passage of time and consumption of
resources. The Control Account Managers assessment of Physical Percent
Complete is not sufficient. Tangible Evidence – as 748 says Objective assessment
of the accomplishments at the work performance level.
But more is needed. Measures of Performance, Measures of Effectiveness,
Technical Performance Measures and their Quantified Backup Data, and the Key
Performance Parameters all contribute to informing BCWP.
These attributes – each a random variable – are related in the following manner:

3. The result of this picture is that the Integrated Master Plan (IMP) needs to be the
framework for our credible Earned Value Management System. It’s the IMP –
built by Systems Engineering – where the MOE’s, MOP’s, KPP’s, TPM’s, and Risks
live. The Integrated Master Schedule shows the sequence of work traceable to
the IMP.
This vertical traceability is called out in DI-MGMT-81861, starting in §3.7.1.3.
 How do we connect the dots
 What are the units of measure for informing BCWP
 What are some scenarios for doing this – TPMs, MOEs, MOPs, Risk adjusted
everything
The Recommended Solution
 Put the Integrated Master Plan (IMP) back in the mandatory category
with a DID for ACAT1’s – the IMP is the framework for assessing the
increasing maturity of the deliverables. Measuring progress toward those
deliverables in the IMS is necessary but is far from sufficient. CPI/SPI,
later starts, late finishes, labor absorption are all valuable. But without an
assessment of the Effectiveness and Performance of the produces products
and services, being on time, and on budget, doesn’t mean the delivered
outcomes actual work for the end user.
 Update the DOD IMP/IMS Preparation and Use Guide, from V0.9 to V1.O,
and show through examples how to make the connections between MOE,

4. MOP, TPM, KPP, and risk as the mechanisms for informing of BCWP to
represent Physical Percent Complete in the IMS.
 Develop EAC and ETC forecasts based on sound statistical analysis using
ARIMA (Autoregressive Integrated Moving Average) – Box-Jenkins, using
historical data from Format 1 in the EVMS database.
 Use this statistical data to calculate EAC and ETC
 Start thinking more like a Systems Engineer and less like a cost and schele
analyst – can we recognize what Done looks like in units of measure
meaningful to the decisions makers. These units start with Effectiveness
and Performance and include Risk Retirement, Technical Performance and
the array of …illities associated with program success.
 Require the Risk Register to be submitted and reviewed at IBR and
undergo a review quarterly at least and monthly at best.
 Require a risk adjusted EAC instead of a Management EAC