Wheatcraft hill.lou terry

Developing Requirements for
Constellation’s Next Generation
Space Suit

Presented at PM Challenge 2010
Presenter's:
Terry Hill, NASA/JSC
Lou Wheatcraft,
Compliance Automation
February 2010

Used with Permission

Overview
 Part 1: The Basics
 Risk and Requirements
 Requirement Validation
 Part 2: Application Case Study
 CxP Suit Requirement Development Process
 Results
 CxP Suit Continuing Requirement Management
Process
 What we could have done better

National Aeronautics and Space Administration 2

Risk and Requirements

WHAT’S COMING NEXT


NASA OIG
 “NASA must be vigilant in its process of establishing and
validating project requirements.”
 “Program risks increase when NASA awards contracts
before developing a sound business case and clearly
defining requirements;”
 Placing “the project at risk of significant cost overruns, schedule
delays, and performance shortfalls.”
 “Effective risk management, safety, and mission
assurance controls are key to supporting robust and
reliable operations in the context of very challenging
launch and mission schedules.”
NASA’s Most Serious Management
and Performance
Challenges – Nov 2008

GAO
 “The start of product development represents the point at which
program managers make a commitment to provide a product that
will perform as required and be delivered on time and within
estimated costs.”

 “Programs are more likely to succeed if program managers are able
to achieve a match between user needs, which eventually become
requirements, and resources (technology, design and production
knowledge, money, and time) at the start of product development.”

 “Conversely, if they do not match requirements with resources, cost
overruns and schedule delays are likely to occur, reducing the
organization’s buying power in other areas.”

GAO-03-598

Effect Of Requirements Definition
Investment On Program Costs
200
OMV

180

GRO 78
160 GALL Why are you running so fast
TDRSS
when you don’t know where
Actual – Target Cost

140
CEN
IRAS HST you are going?
120
Target Cost

100 GOES I-M
TETH German proverb
MARS
LAND 76
MAG
80
ACT STS LAND 78
ERB 77 COBE
60

ERB 80
40 SEASAT
UARS GRO 82 HEA
VOYAGER
EUVE/EP ISEE
20
DE
SMM ULYSSES IUE
PION/VEN

5 10 15 20 25 30

Requirements Definition and Preliminary Design
Target Total Cost

Cost to fix requirement defects
1,000-
1000

70

60-

50-

40- 40 40

30- 30

20-
15
10
10-
6
1 3
0-
Requirements Design Coding Development Acceptance Operations
Phase Phase Phase Testing Testing


Importance of Best Requirement Practices on
Project Success
 “The companies using best requirements practices will estimate a
project at $3 million and better than half the time will spend $3
million on that project. Including all failures, scope creep, and
mistakes across the entire portfolio of projects, this group will spend,
on average, $3.63 million per project.”

 “The companies using poor requirements practices will estimate a
project at $3 million and will be on budget less than 20% of the time.
50% of time, the overrun on the project both in time and budget will
be massive. Across the entire portfolio of successes and failures,
this company with poor requirements practices will (on average) pay
$5.87 million per project.”

2008 Study by Keith Ellis, IAG Consulting of 100
companies with projects in excess of $250,000

Setting Yourself Up for Failure
 Project success is “improbable” for 68% of the companies Ellis
studied

 “Projects might succeed – but not by design. Based on the
competencies present, these companies are statistically unlikely to
have a successful project.”

 While these companies indicated they recognized that requirements
are important to project success, they still failed to take effective
actions to insure a good set of requirements.

 By doing so, they tripled their chances of project failure


What Needs to be Done
 “Organizations understand conceptually that
requirements are important, but do not internalize this
understanding and change their behavior as a result.
 The most successful of companies do not view
requirements as a document which either existed or
didn’t at the beginning of a project, they view it as a
process of requirements discovery.
 Only companies that focus on both the process and the
deliverables are consistently successful at changing
project success rates.”


No Surprises

People who write bad requirements
should not be surprised
when they get bad products…

but they always are.

Ivy Hooks


Words of Wisdom

“Putting forth the same effort, or
using the same approach, then
expecting different results
is...insanity”

- Charles Bolden,
NASA Administrator, July 2009


A Winning Product
 Delivers what’s needed
 Within budget
 Within schedule
 With desired quality

Risk: Anything that can
prevent you from delivering
a winning product!


Requirement Validation



Requirement Validation vs. Verification
 Requirement validation  Verification confirms that
confirms the the designed and
completeness and built product meets the
correctness of the requirements
requirements
 Done after design and build
 Starts with first requirement
and continues through life
cycle

Requirement validation
makes sure you are Verification makes sure
building the right thing. you built it right.


Requirement Validation
 Helps ensure our requirements are:
 Needed, verifiable, achievable
 Clear, concise
 Correct, consistent, and complete
 Two types
 Continuous
 Discrete


Continuous Validation Process
 Requirement Writer:
 Write requirement and attributes
 Check requirements against standards
 Submit to gatekeeper for review in “chunks”
 Gatekeeper (person or inspection points)
 One or more experts
 Reviews requirements against standards
 Accepts defect-free requirements for input to
database or document
 Return requirements to author if defects found


Who Does Requirement Validation?
Writers Managers

Everyone is
accountable

Developers Reviewers


Continuous Validation Process
 Continuous validation holds everyone
responsible
 Requires standards and checklists
 Requires training
 Management has to enforce discipline and
accountability
 Benefits
 Stops the creation of BIG bad documents
 Most effective way to realize process improvement
 Reduces time for milestone reviews
 Prevents lost time due to rework


Discrete Validation Process
 Key milestone that requires  Effective IF
time and resources  The right people are involved

 Formal process  The products are ready for review
 Complete document  The participants know what to do
 Involves a wide range of  Management ensures
stakeholders compliance
 Requires standards and
feedback mechanisms
 Requires training
 Management has to ensure
responsiveness
 SRR results in a requirement
baseline


System Requirement Review (SRR)

Scope

Requirements

Design
MCR Manufacture
or Code
Verification
SRR
Operations
SDR
Upgrade/
PDR Maintain
CDR
TRR

MCR: Mission Concept Review
 Baseline requirements SRR: System Requirements Review
 Assess feasibility SDR: System Definition Review
PDR: Preliminary Design Review
 Set expectations CDR: Critical Design Review
TRR: Test Readiness Review

CxP Suit Element Requirement
Development Process



Background: Case Study
 Shuttle / ISS Extra Mobility Unit (EMU)
 Shuttle was designed to be a shirt sleeve environment
not requiring space suits of any kind.
 EMU developed after Shuttle was designed to address a
late in design possible failure scenario with the Shuttle –
risk that payload bad doors would not close and would
have to do it manually.
 Suit dimensions were driven by Shuttle hatch sizes.
 Many, many requirements and later functionality of the suit were
not addressed or defined early in the process.
 The Shuttle EMU was later augmented and recertified to
work at and around the ISS to perform station assembly
and repair.

Shuttle/ISS EMU Case Study –
Design and Development Cycles

Pre-declared Cert.

Initial Cert.

Re-Certification
and Ops Con &
Requirements
evolved


Background: EMU Case Study
 Lessons Learned
 Any little design change will take you into some
level of re-certification.
 Always larger and more expensive and takes longer
than ever planned.
 The better the ConOps is thought through in the
beginning allows the writing of better and more
comprehensive requirements eliminating many
re-certification cycles later and thus saving Life
Cycle dollars.


Background: CxP Suit Element
Requirement Development Challenge
 The Task
 The team was asked to develop a CxP Initial
Capability (and Lunar requirements when common
hardware would be used) requirement set for the
Space Suit Element in time to support the CxP Suit
Element SRR and for release with the RFP for a
prime contractor in mid-fall.
 Schedule
 Very Short Schedule – 3 Months from initiation of
requirements generation to Major Project Milestone
review and 5 months to baseline set of functional
requirements.


Background: CxP Suit Element
Requirement Development Challenge
 The Philosophy
 Learn from past projects mistakes in how and when
requirement are written
 Clean sheet approach to developing a space suit and
writing the requirements
 Exercise the text book methodology of Systems
Engineering and Requirement generation in a NASA
project
 Produce quality requirements that are verifiable in a
cost effective manner that address the functionality
defined in the CxP EVA System Operations Concept
and EVA Systems Architecture documents.


Background: CxP Suit Requirement
Development Schedule
 Planned Schedule for 2007:
 May 31-Jun 5 – Requirement Training and Kick-off
 Jun 1-22 – Suit Element Requirements Generation Activities
 June 25-29 – Suit Element SRR Doc(s) review
 July 2-6 – Suit Element SRR Doc(s) update & SRR final prep.
 July 10 – Suit & Vehicle Interface Elements SRR Kick-off
 July 10-20 – Suit/VI Element SRR Doc review & RID submittal
 July 22-Aug. 7 – Suit/VI Element SRR Panels and Boards.
 Aug. 9-Oct 20 – Close SRR Actions and update ERD
 Oct. 23 – Suit ERD for Baselining at EVA PCB.
 Oct. 29 – Suit ERD rev. A draft submitted to EVA CM for pre-blackout CSSS
Tech. Library drop for Prime contract RFP release

* Final outcome, Element SRR slipped one week to ensure
quality of products where ready for review. Review
revealed products were ready and of the appropriate
fidelity by EVA project management.


Background: CxP Suit Requirement Development
Approach: Ground rules
 Co-located the team off-site in a conference  Requirements meet the criteria for good
room facility to enable a concentrated effort requirements in the Checklist for Good
– this reduces the risk of day-to-day Requirements,
distractions which is a risk to product quality
 Rationale for each requirement, no copying
and schedule.
parent requirement with a noun change,
 Provided task training in requirement
 All requirements verifiable, clear, concise
development and writing processes – this
and if it can be interpreted in more than one
reduces the risk to quality.
way, it is not ready for acceptance.
 Reduce risk of requirement development by
contracting out the training and using
consultation which provided standard
training as provided to CxP, and an
independent “fresh set of eyes” on how
requirements could be interpreted and
implemented – this reduces the risk to
quality.
 Used CRADLE tool to develop requirements
prior to baseline. Post baseline, the
requirements were managed out of
CRADLE, but draft revisions were handled
outside of CRADLE until change approval.


Putting Requirement Risk in the Proper
Perspective
 Not to put too much pressure on you….
 The Requirements Document is probably the single most influential
piece of paper that we have control over in the entire Constellation
Program.
 This is our chance to make sure that we are asking for what we
really want. Let’s get it right.
 This is a big, fat, hairy deal. If we don’t get this right, folks 20 years
from now will be shaking their heads and saying, “What were those
yahoos thinking?”
 I’ll be around and don’t want to go to that meeting.

CxP EVA Suit PGS Team Requirement Kickoff
Meeting 5/2007


Suit Requirement Development Process


Subsystem Teams
 Responsible for the drafting & modifying requirements
 Four independent teams
 Suit Element-level (integrated across subsystems)
 Subsystems
 Pressure Garment, Crew Survival, Power-Comm & Avionics, Life
Support, (Ground Support Equipment team was after the initial
requirement generation effort)
 Each team consisted of:
 Functional Lead
 NASA Subject Matter Experts (SMEs)
 “Core” Scrub team
 Teams given training during kick off meeting
 Briefed in the Goal, Schedule and Process
 Checklist for Good Requirements

Scrub Team
 “Core” Scrub Team:
 SE&I representative
 Compliance Automation representative – independent “goodness
assessment”, and guidance in requirement development.
 Tech Writer
 CRADLE Operator

 Review requirements from the Subsystem Teams
 Edit & “clean” Requirements based on Checklist for Good Requirements
 If they could “fix” requirement & verify the intent was understood, would do
so & pass onto the CRADLE Board
 If intent not understood and the requirement needed more work,
representatives from appropriate Requirement Team were called in for
clarification.

 Kept list of common defects and briefed Requirement Teams daily


CRADLE Board
 Had three functions:
 Determined if requirements where technically traceable and
appropriate
 Determined if requirements where technically appropriate and
the correct mission phases applied
 And determine any outstanding issues were resolved as the
CRADLE Board was used as a dry-run for final approval.
 Base Team:
 Subsystem representative or subject matter expert
 Secondary Team:
 Project Management
 Subsystem Leads
 Focus Meetings a.k.a. Offline
 Resolves outstanding issues
 Membership as required

Approval Board
 Function: Team Consensus/Final Approval for requirement to be
included in ERD
 Base Team:
 Project Management
 Subsystem Leads
 CRADLE operation
 Subject matter expert
 If discussions proceeded for longer than 15 minutes, the
requirement and discussion were sent to a Focus Meeting for
resolution. The idea was that if a requirement was not clear enough
to convey the intent and importance in 15 minutes of discussion,
then it was not written correctly.
 Focus Meetings a.k.a. Offline
 Resolves outstanding issues
 Membership as required


Results



Results in Project Reviews
 For the Suit ERD SRR, a ratio of 0.38 Review Item Descriptions
(RIDs) were received per requirement.
 In comparison, the parent document had a 2.94 RID/requirement ratio at
its SRR.

Potential bidders for the “… the most comprehensive and of the highest
development of the Suit Element
stated that the ERD was: quality they ever remember seeing.”

“I can't say enough about how amazed I am by
The JSC Engineering Directorate this set of requirements documents. As far as I
Crew and Thermal Systems
Division Chief was also very know, no other Cx project has allocated and
impressed with the quality of the decomposed anywhere near to this level of
Suit ERD, saying: depth. You are the first. I have also never seen
anything like these from previous programs.”


CxP Suit Continuing
Requirement Management Process



Post-baseline Requirement
Development and Validation
 As a result of the extremely compressed requirement development
schedule, there remained several areas that needed more work and
issues that needed to be resolved prior to preliminary design taking
place.
 These open areas included:
 resolving TBDs/TBRs in the requirement set,
 finishing the verification requirements,
 defining the internal interfaces and maturing applicable interface requirements
 and adding Ground Support Equipment (GSE) requirements to the ERD.
 Therefore there was requirement development and maturation
process needed to continue this process.


Requirement Review Process Post-
ERD Baseline


Requirement Review Process Post-
ERD Baseline: SEIWG
 The SEIWG (Systems Engineering and Integration Working Group)
 Meets once a week to discuss new and existing requirements and/or
requirements issues, such as TBDs/TBRs, action items, allocations, etc.,
 Serves as the SE&I pre-coordination for the Suit Control Board.

 SEIWG Procedure
 Step 1:
 All potential changes must be submitted via a change request form.
 Step 2:
 Scheduled SEIWG meeting to present/discuss change and any final modification
of the draft FROM/TO language.


Suit Control Board
 Functions:
 Detailed review of proposed FROM/TO language to
go into the ERD.
 A minimum of a 2 business day review by board members
prior to the Suit Control Board meeting.
 Functions as the official forum of stakeholder review,
concurrence or rejection
 Functions as the final approval gate for base-lined
requirements.

Note: Since Baselineing of ERD the SCB closed
approximately 180 TBD/Rs prior to levying the
document on the prime contractor.

Wrap up



What we could have done better
 Challenging schedule resulted in a lot of open work post baselining
of ERD.
 Large number of TBDs/TBRs
 Interfaces identification and definition incomplete
 Some of the external interfaces just didn’t exist due to sister projects were
evolving in parallel and at times without the same rigor demonstrated in the
Suit Element effort.
 Traceability incomplete
 Verification requirements
 GSE requirements
 Difficult to keep requirements at the right level
 Some requirements may not have been needed
 Some requirements reflected or assumed a design where NASA was
clear there was only one desirable functional solution.


Parting Thoughts
 Address Requirement Risk at the beginning of
your project
 Develop a formal requirement development
process that includes continuous requirement
validation
 Include continuous requirement validation into
your requirement management process
 Train your team
 Enforce the process
 Allocate the time and resources needed to do
the job right – the first time!!

Presenter Biographies


Terry Hill
NASA/JSC Constellation Space Suit System Engineering Project Manager
Terry Hill is NASA’s Johnson Space Center’s Engineering Project Manager and deputy CxP EVA Suit
Lead for the CxP Suit Element, responsible for the development of the functional, performance, and
quality requirements and preliminary design of NASA’s next generation space suit system.
Terry has a BS in Aerospace Engineering and a MS in Guidance, Navigation & Control Theory with a
minor in Orbital Mechanics and Mathematics from the University of Texas at Austin.
He began his career at NASA while working on his masters thesis
project in developing banks of simplified Kalman filters integrated
into an artificial neural network to obtain an optimal state solution for
precision landing on Mars.
While at NASA ,Terry has worked on projects and programs
spanning verification of ISS navigation software, Shuttle Design
Test Objectives (DTO) and back room mission support, X-38 Crew
Return Vehicle navigation algorithm development, Space Launch
Initiative technology development, Orbital Space Plane project
office ISS-Prime integration, STS-107 Return to Flight Tile Repair
capability development, to Constellation Program Space Suit
System leadership.
In leading the CxP Suit Element engineering team, Terry has
facilitated the development of system requirements for space suit
development and a clean-sheet design approach that has widely
recognized within and outside of NASA.


Lou Wheatcraft
Compliance Automation – Senior Consultant Trainer
Lou Wheatcraft is a senior consultant/instructor for Compliance Automation, who has over
40 years experience in the aerospace industry, including 22 years in the United States Air
Force. Lou has taught over 120 seminars in requirement development and management
for NASA’s APPEL Program and industry over the past nine years. He has worked with,
and provided intact team training and consultation to multiple NASA project teams at many
of the NASA Centers.
Lou has had articles published in the International Council of Systems Engineering
(INCOSE) INSIGHT magazine and in DoD’s magazine, CrossTalk. Lou has made
presentations at NASA’s PM Challenge, INCOSE’s International Symposium, and at the
local Project Management Institute (PMI) Chapter Meetings.
Lou has a BS degree in Electrical Engineering, an MA degree in Computer Information
Systems, an MS degree in Environmental Management, and has completed the course
work for an MS degree in Studies of the Future.
Lou is a member of INCOSE, co-chair of the INCOSE Requirements Working Group, a
member of PMI, the Software Engineering Institute, the World Futures Society, and the
National Honor Society of Pi Alpha Alpha. Lou is the recipient of NASA’s Silver Snoopy
Award and Public Service Medal and was nominated for the Rotary Stellar Award for his
significant contributions to the Nation’s Space Program.


Wheatcraft hill.lou terry

Recomendados

Recomendados

Más contenido relacionado

Similar a Wheatcraft hill.lou terry

Similar a Wheatcraft hill.lou terry (10)

Más de NASAPMC

Más de NASAPMC (20)

Último

Último (20)

Wheatcraft hill.lou terry