WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
No CAM Left Behind: APL's Incremental Approach to EVMS Implementation
1. No CAM Left Behind:
APL’s Incremental
Approach to EVMS
Implementation
Steve Shinn
Howard Hunter
2009 NASA Project Management Challenge
2/24/2009
2. Agenda
Brief APL Overview
In the beginning
Factors that precipitated EVMS development
APL Space Department approach to EVMS
Management support
Graduated approach
Implementation details
Architecture
“Practice” during mission phases A/B
Training - “No CAM left behind”
Reporting
EV system highlights
Where are we now?
Closing
Questions
3. APL, a Division of The Johns Hopkins
University
School of Arts and Bloomberg School Applied Physics Nitze School of The Peabody
Sciences of Public Health Laboratory Advanced Institute
Whiting School of School of Medicine International Studies
Engineering School of Nursing
Carey Business
School
3
4. APL Profile
• Not-for-profit DoD-chartered “University
Affiliated Research Center” (UARC)
• Staffing: 4,500+ employees (70% scientists
and engineers)
• $960M annual portfolio
• Business areas:
MAIN Air and Missile Defense
CAMPUS Biomedicine
Civilian Space
SOUTH MONTPELIER Homeland Protection
CAMPUS RESEARCH PARK Infocentric Operations
National Security Space
Precision Engagement
Science and Technology
Strategic Systems
• Main campus in Maryland: 400 acres, 50+ buildings Undersea Warfare
• Thirty locations across the United States Warfare Analysis
4
5. Civilian Space
World-Class Science and Innovative Engineering
Key Programs:
TIMED – Thermosphere, Ionosphere, Mesosphere
Energetics and Dynamics
MESSENGER – MErcury Surface, Space
ENvironment, GEochemistry, and Ranging
CRISM – Compact Reconnaissance Imaging
Spectrometer for Mars
New Horizons – Mission to Pluto/Charon
STEREO – Solar Terrestrial Relations Observatory
RBSP – Radiation Belt Storm Probes
Critical Challenge: Answer fundamental space and earth science
questions for our sponsors, our nation, and humanity
5
6. APL Spacecraft: 1996–2007
MSX
= From PDR to I&T
NEAR = Arrival at I&T faclity through launch
ACE
TIMED
CONTOUR
MESSENGER
New Horizons
STEREO
1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007
6
7. In the Beginning
Real Quotes Prior to EVM Implementation
“Earned Value doesn’t work, and I have the empirical evidence to prove it.”
“If [EVMS] takes up too much of my time, I’m going to leave. I really don’t have
time for this. I came here to do science!”
“EV is a waste of time – I have my own system that I developed.”
“EVMS money should be spent on science!”
“Our engineers and scientists are sensitive to new processes. We can’t have a
single engineer quit because of your EVM system.”
“I don’t control the resources. How am I supposed to be responsible for this?”
“I don’t need this earned value stuff. All I need is a functional WBS and a good
system engineer.”
7
8. Factors That Precipitated EVM
Implementation
From 1979 to 1996, APL was singularly successful in arriving at program
costs that were largely within a few percentage points of the planned costs
at completion.
Eight Spacecraft Bus Programs were within ±8% of the initial cost
estimate at the start of phases C/D.
Recent missions and instruments experienced cost growth.
To combat future cost/schedule growth, APL implemented several new
processes and systems.
EVM was introduced as a project management tool.
State-of-the-practice capabilities for cost/schedule analysis and estimating
were needed to complement EVM and attack the cost growth challenge.
5-year vision: Create an organic cost/schedule estimating
and analysis function within the SD that incorporates state-
of-the-practice methods for resource estimation and analysis.
8
9. EVMS Part of Mission Delivery
The APL Quality Management System states that:
The Laboratory’s policy is to make critical contributions to critical national
challenges through the application of science and technology.
All products for our sponsors meet requirements for intended use as well as
schedule and cost. We are committed to continual improvement.
Cost and Schedule Development and Management Standards are intended to
define standard cost and schedule planning and management practices for all SD
projects and programs. This includes:
Cost planning
Cost management
Schedule planning
Schedule management
EVMS
This charter formed the basis for our move to EVMS.
9
11. APL Vision for EVMS
Provide project management with a consistent, standard framework for assessing project
performance on major projects.
Proposal/Initial Project Concept: Provide up-front planning and a clearer definition of
work using a standardized Work Breakdown Structure (WBS).
For project initiation through delivery:
Integrate cost and schedule for each phase of the project; utilize the Responsibility
Assignment Matrix (RAM) and the Work Statement Authorization (WSA).
Utilize management best practices.
Monitor and control the project.
Provide an early warning system.
Make use of regular, rigorous estimates at completion (EACs) to ensure better
understanding of future work.
Think of EVMS as one part of the CAM project management tool suite.
Exploit EVMS, where practical, for integrated, efficient collection and reporting of 533,
contract performance reporting (CPR), risk management, and CADRe data.
11
12. Making the Vision a Reality: What Worked
Management support – critical to success
“What’s in it for me?” – show value to users
Keep it simple
Incremental approach
Don’t expect to have an ANSI 748A-compliant system on day 1
Always show utility
“One-System Approach” – collaborate closely with sponsors and subcontractors
Open Communication
12
13. APL Management Support
With a foundation in place, management took a series of steps to support the
EV implementation.
Promoted a cultural shift from Senior Department Managers through
Program Managers to Lead Engineers (CAM) with the use of an EVMS to
manage a project (part of the program management discipline)
Implementing an EVMS prior to external requirements
Use of EVMS prior to key decision point (KDP) C
SD utilization of EVM on projects >$15M
EV steering committee established
“We use EVM not because we have to but because it’s the right thing to do.”
As people witness senior management “buy in,” the process change gets
easier.
13
14. Our Approach: One CAM at a Time
New system users (or processes/policies) can be organized into three groups:
The Advocates – These are the people who recognize a good idea and
embrace new systems that may help them do their jobs.
Get as many of these people on your side. They become the “town-criers”
who will ultimately help get the system embraced.
The Uninformed/Indifferent – This group may be either unaware of the
concepts or unsure if the system will ultimately help them.
You need to show the value of the system to this group ASAP. If they like
what they hear, they will become advocates. If you are unable to show
value, this group will join the group below.
The Naysayers – This group often resists any change. Perhaps they have been
successful for a long time and see no reason for change. This group is often
sarcastic and/or skeptical of any communications.
Minimize this group’s effectiveness. You can’t completely ignore them (think
termites in wood), but you can’t devote too much time to them. Limit their
damage while keeping others from joining.
14
15. Graduated Approach
EV utilized ($3M) -Time of Flight board (2005)
Managed by exception
EV deemed beneficial to management of project
Secondary project ($4M) (2006)
Leveraged this project to demonstrate the discrete measurement of
performance for Subcontractor effort and material
Aid in developing the change control process (CBB,UB, BCR & MR)
First Mission - RBSP (2007)
IMP/IMS and related schedule reports
Cost & schedule integration
WBS/Dictionary, WSA’s, RAM, CPR’s
Data reviewed each month with CAM’s – workshop format
Supported monthly sponsor meetings. Reviewed schedule & CPR data
EVMS was used through phase B of the RBSP project. Phase C/D cost
and schedule baselines are being established.
Small successes built positive “press”.
15
16. Graduated Approach (cont.)
Project Management Control System (PMCS) Description
The description explains the processes, procedures, and methodologies used
by the SD for the planning and control of projects through the use of the
EVMS.
Training held:
The initial training course has been completed.
Over 175 individuals have been trained.
Hands-on workshops are held every month on EVMS-active programs during
data reviews.
Ensures just-in-time training
Lower training cost than intensive 3/5/7-day EVMS courses
16
17. Graduated Approach for ANSI Compliance
First: Implement the most meaningful pieces of the system.
Second: Ensure that each portion of your system is ANSI-748A compliant from
the start.
You may not implement all elements, but make sure that the ones you do
implement are reasonably sound.
Third: Perform a gap analysis to determine holes.
Fourth: Determine a plan/path to complete compliance.
Fifth: Plan remaining activities and stick to your schedule!
Earned Value Report
Project - XXXXX
Month End 10/31/2005
Cumulative to Date
Actual Start Small!
Budgeted Cost Cost Variance
Work Work Work
WBS
1.110 Management & Administration
%Com Scheduled Performed
16% 63,791 42,618
Performed Schedule
44,264 -21,173
Cost
-1,646
Budgeted
269,959
A simple
1.120 Reliability & Quality Assurance;
1.210 Engineering
12%
17%
24,755
242,632
31,098
156,780
15,711 6,344
72,923 -85,853
15,388
83,857
251,862
915,936
spreadsheet
1.220 Integration & Testing
1.240 Instrument Calibration
0%
0%
0
0
0
0
0
0 0
0 0
0
35,488
21,154
summary can get
1.310 Lo Science
1.320 Hi Science
0%
0%
0
0
0
0
0
0
0
0
0
0
109,057
81,573 you started.
1.810 Phase E 0% 0 0 0 0 0 263,604
1.820 Phase E 0% 0 0 0 0 0 374,848
1.830 Phase E 0% 0 0 0 0 0 404,338
1.RMA Rocket & Mission Analysis 46% 16,427 16,944 22,019 518 -5,075 37,000
Total 361,978 261,813 168,691 -100,165 93,122 2,779,194
17
18. Collaboration and Cooperation
Too many systems are “private” and/or stove-piped.
Involve stakeholders in the system.
Minimize the “us versus them” mentality.
APL PMs have worked with NASA counterparts to utilize EV data.
Collaborative environment
System is based on NASA policies (7120.5D) to ensure that projects don’t
“reinvent the wheel.”
Work with subcontractors and instrument providers early and often.
Early engagement is critical to receiving timely information.
Make EVMS part of a larger, integrated project management suite.
Integrate risk management, scheduling, funding, project milestones, and
system engineering tools
18
19. Our Solution to Common EVMS Pitfalls
To keep EVMS from being an administrative burden, focus on value first
and then on administrative elements.
To minimize “gotcha” audit mentality, maintain a collaborative
environment.
To lower EVM system implementation costs, leverage existing project
management processes.
Bottom Line: The data are only as good as the people reviewing it.
Many of the elements of EVMS are simply good project management.
19
20. APL EVM Long-Term Goals
Continue to foster organizational change with demonstrated EV
capability
Integrated approach to the reporting of the 533, CPR, and CADRe
Enterprise system in place
EVMS compliant with ANSI/EIA-748A guidelines
Provide a meaningful management tool to the APL project management
community
20
22. EVMS Architecture
Accounting Interface SharePoint Toolkit
Central online repository for project
performance data
Data
Warehouse
RMIS Actua
ls Tra
nslat
ion Data
Accru base
Actual Cost of Work als E
s timat
Performed (ACWP) or S ystem
Budgeted Cost of Work
Resources into Scheduled (BCWS)
Operating Plan
wInsight
Cobra
MS Project Project
Server EV trend
.MPP analysis and
ForProject
reporting
Schedule data EV data analysis,
Schedule data (multiple projects, reporting and
Resource (Budgeted Cost of access and control, baseline control,
loaded Work Performed alerts through web, and rate and
schedules [BCWP] calculation) status update) efficiency variances
22
23. “Practice, Practice, Practice” (Phases A/B)
RBSP project
Phase A
IMP/IMS and related schedule reports
Cost and schedule integration
WBS, WBS dictionary, WSAs, RAM, and CPRs
Schedule management plan and PMCS/draft
CAM training
Phase B
Data reviewed each month with CAM (workshop format)
SharePoint data repository established for SD projects
IMS developed
Operating Plan/EAC demonstrated in the CPR
Supported monthly sponsor meetings. Reviewed schedule and CPR
data.
Monthly EV review with PM and Deputy PMs (management by
exception)
Baseline status review completed successfully
23
24. Baby Steps
Focus on value:
Agreement between stakeholders (functional supervisors and CAM)
Processes developed
Not a time burden
Not a punitive system
Management by exception
Ability to drill down into the data
Information readily available
24
25. “No CAM Left Behind”
Work closely with the CAM
Schedule development
Understanding the data from the EVMS
Frequent interactive meetings
Mentoring relationship
Part of the team
Work with the PM
He/She is our most important customer
Value added to the PM team (internal and sponsor)
Provide key insight and analysis to the data
The focus of the EV team should be mission success
25
26. Training
Generic EVMS training
Basic EV
Control Account Manager (CAM) duties/description
EV types/accounting considerations
Analysis and reporting
Emphasis placed on advanced skills through CAM/PM workshops
Variance analysis
MR
Baseline change control
CPR
EAC/estimate to complete (ETC)/Operating plan update
Integrated baseline review
26
27. Reports Available Through EVMS
S e c tio n S - S c h e d u le Re p o r ts
Section E - Earned Value Reports
# D e scription Fre que ncy Format
S1 S c hedule Health C hec k Monthly XLS
# Description Frequency Format
Contract Performance Report Format
S2 C AM S tatus Update R eques t Monthly XLS E1 Monthly XLS
1, 2, 3, 4
S3 C AM S c hedule Update Monthly XLS E2 Contract Performance Report Format 5 Monthly XLS
S4 R es ourc e O rphan R eport Monthly .P D F
E3 Cost Variance Chart (wInsight) Monthly .PDF
S5 R es ourc e Mis m atc h R eport Monthly .P D F
S6 3 Month Lookahead Monthly .P D F E4 Schedule Variance Chart (wInsight) Monthly .PDF
S7 C ritic al P ath Monthly .P D F
Section T - Trending
S8 S c hedule Update Narrative Monthly .D O C # Description Frequency Format
S9 B as eline S c hedule Varianc e Monthly .P D F T1 Total Milestones Monthly XLS
S 10 D etailed S c hedule Monthly MP P T2 IMP/Contract Milestones Monthly XLS
S 11 D eliverables - 3 Month Lookahead Monthly XLS T3 Total Slack Monthly XLS
S 12 P has e B - Miles tones Monthly .P D F T4 Estimate At Completion Monthly XLS
S 13 P has e B - D eliverables Monthly .P D F
T5 Management Reserve Monthly XLS
Section F - Financial Reports
# Description Frequency Format
533 Monthly Financial Management Report Monthly XLS
533 Quarterly Financial Management Report Quarterly XLS
27
28. CPR – Sponsor
CLASSIFICATION (When Filled In)
CONTRACT PERFORMANCE REPORT FORM APPROVED
FORMAT 1 - WORK BREAKDOWN STRUCTURE DOLLARS IN $ OMB No. 0704-0188
1. CONTRACTOR 2. CONTRACT 3. PROGRAM 4. REPORT PERIOD
a. NAME a. NAME a. NAME a. FROM (YYYYMMDD)
The Johns Hopkins University Applied Phy G-MIG G-MIG Phase B
b. LOCATION (Address and ZIP Code) b. NUMBER b. PHASE 2008 / 07 / 01
Johns Hopkins Road, Laurel, MD 20723 NAS5-012345 TO 10 B b. TO (YYYYMMDD)
c. TYPE d. SHARE RATIO c. EVMS ACCEPTANCE
CPFF NO X YES (YYYYMMDD) 2008 / 07 / 31
5. CONTRACT DATA
a. QUANTITY b. NEGOTIA c. ESTIMATED COST OF d. TARGET PROFIT/ e. TARGET f. ESTIMATED g. CONTRACT h. ESTIMATED CONTRACT i. DATE OF OTB/OTS
COST THORIZED UNPRICED WO FEE PRICE PRICE CEILING CEILING (YYYYMMDD)
1 75,500,000 0 0 75,500,000 0 0 0
6. ESTIMATED COST AT COMPLETION 7. AUTHORIZED CONTRACTOR REPRESENTATIVE
MANAGEMENT ESTIMATE CONTRACT BUDGET VARIANCE a. NAME (Last, First, Middle Initial) b. TITLE
AT COMPLETION BASE
(1) (2) (3) Shinn, Steve O. Program Managers
a. BEST CASE 0 c. SIGNATURE d. DATE SIGNED
b. WORST CASE 0 (YYYYMMDD)
c. MOST LIKELY 0 75,500,000 75,500,000
8. PERFORMANCE DATA
NASA WBS[21] CURRENT PERIOD CUMULATIVE TO DATE REPROGRAMMING AT COMPLETION
ACTUAL ACTUAL ADJUSTMENTS
BUDGETED COST COST VARIANCE BUDGETED COST COST VARIANCE
WORK WORK WORK WORK WORK WORK COST SCHEDULE BUDGETED ESTIMATED VARIANCE
ITEM SCHEDULEDPERFORMEDPERFORMED SCHEDULE COST SCHEDULED PERFORMED PERFORMED SCHEDULE COST VARIANCE VARIANCE BUDGET
(1) (2) (3) (4) (5) (6) (7) (8) (9) (10) (11) (12a) (12b) (13) (14) (15) (16)
2.1.1 Project Management 170,200 170,200 165,000 0 5,200 675,000 675,000 670,420 0 4,580 0 0 0 1,428,028 1,693,400 -265,372
2.10.1 Satellite Integration, Assembly and Test 122,500 121,644 98,000 -856 23,644 559,425 519,616 348,750 -39,808 170,866 0 0 0 1,208,500 1,094,200 114,300
2.10.4 Logistics Preparations & LV Support 11,500 0 0 -11,500 0 25,000 0 0 -25,000 0 0 0 0 61,590 38,400 23,190
2.10.5 Launch Vehicle Interface 2,200 2,200 2,300 0 -100 11,500 11,500 12,700 0 -1,200 0 0 0 22,483 26,500 -4,017
2.11.1 Education/Public Outreach Activities (Phases A-D) 23,800 16,500 31,800 -7,300 -15,300 110,230 70,410 88,600 -39,820 -18,190 0 0 0 220,551 170,758 49,793
2.2.1 System Engineering 205,000 205,000 208,500 0 -3,500 821,000 821,000 775,200 0 45,800 0 0 0 1,850,000 2,000,500 -150,500
2.2.2 Spacecraft System Engineering (lead, deputy electrical, depu 69,000 69,000 44,100 0 24,900 284,200 284,200 255,000 0 29,200 0 0 0 630,250 460,450 169,800
2.2.4 Contamination & Configuration Eng'r 60,400 53,700 54,632 -6,700 -932 318,950 231,700 225,500 -87,250 6,200 0 0 0 579,000 735,125 -156,125
2.2.5 Deep Dielectric Parts Testing 17,575 25,550 85,000 7,975 -59,450 182,530 118,410 230,520 -64,120 -112,110 0 0 0 220,700 544,444 -323,744
2.3.1 Performance Assurance Engineering 70,926 70,926 109,500 0 -38,574 286,971 286,971 322,612 0 -35,641 0 0 0 634,759 786,416 -151,657
2.4.1 Project Scientist-APL 79,880 79,880 90,475 0 -10,595 325,800 325,800 330,500 0 -4,700 0 0 0 723,754 735,700 -11,946
28
31. SharePoint
SharePoint Toolkit – Central online repository for project performance data
(EVMS Portal)
Includes all projects using EV
Data available through SharePoint:
WBS, WBS dictionary
IMS
Work authorizations
EV and scheduling reports by month
Internal and external
Baseline versus Operating Plan versus Actuals Report
EV report – indices, S, P, A, SV, CV etc.
An electronic, interactive, real-time CAM notebook!
31
32. SharePoint – CAM Notebook
Categories of
monthly reports
Monthly Earned Value
Reports
32
34. Program Manager and SD Management
Involvement in Our Current EVMS
The EVMS:
Is providing the PM with a consistent, standard framework for assessing project performance
Is providing the PM with a sound program management tool that generates data in a format
that enables management by exception
Is providing the PM with early detection of problems, enabling faster response and corrective
action
The PM:
Is the single point of integrated responsibility for project technical, schedule, and cost
performance
Sign-off on final monthly EV data
The CAM:
Is establishing and maintaining the schedule and budget
Reports performance and assesses earned value
Monitors costs charged to the control account
Generates an Estimate at Completion
Is reviewing variance analysis and developing and implementing corrective action, as required
APL Senior Management:
Are provided with an EV analysis via weekly executive management meeting
34
35. EVMS: Comments from Our PMs
Helped to pinpoint areas where development was behind schedule
and there was potential risk for cost growth
Weighted milestone tracking has helped me understand how far
behind/ahead the team is regarding performance
The EVMS will be even more beneficial as we move through
design, build and test phase of the project
Allowed insight into trending across the project and enhanced
management decision process
Benefit of oversight far outweighs burden of implementation
35
36. Closing: Making EVMS Work
Don’t implement EVMS for the sake of implementation.
Show value at every step of the process.
Integrate EVM with all of your project management tools.
EVM is NOT an island!
Avoid an audit-like mentality.
EVM is a management tool, not an accounting tool.
Use GAO, OMB, and NASA documents as a guide to ensure that you’re on
solid footing.
EVMS really is a good thing!
Reap the benefits of the only management system in the world that is both
analytical and predictive!
Rest assured: You won’t be leaving your CAMs behind.
36
37. Questions?
It must be remembered that there is nothing more difficult to plan,
more doubtful of success, more dangerous to manage, than the
creation of a new system.
For the initiator has the enmity of all who would profit by the
preservation of the old institutions and merely lukewarm
defenders in those who would gain by the new ones.
Niccolo Machiavelli
First attempt to implement an
EVMS for a defense system
Florence, Italy circa 1509
37