1. Orion
PDR Process -
What in the World
Happened?
NASA PM Challenge 2010
Used with Permission
Charles Armstrong
National Aeronautics and Space Administration
Orion VI&D Plans & Processes Manager
charles.h.armstrong@nasa.gov
281-244-6315
Tony Byers
Lockheed Martin Space Systems Company
Systems Engineering – Program Integration Manager
anthony.w.byers@lmco.com
303-977-4389 1
3. Management Challenges
Project Orion
• Planning Challenges
• Develop and execute a Preliminary Design Review (PDR) process
that ensures NASA’s next generation spacecraft can meet all
mission objectives and satisfy requirements with acceptable risk
• Develop a multi-tiered approach that ensures the integrated
subsystems, modules and vehicle are reviewed to ensure the
design approach is at a sufficient level of maturity to proceed to
the detailed design phase.
• Balance thoroughness and schedule. The desire to review every
aspect of the design down to the lowest possible level must be
balanced against the available time to execute the review as to
not delay the development activities unnecessarily.
• Balance inclusion and efficiency. Develop a process that
ensures all stakeholders have an adequate time to participate
and provide input helping to shape the design, ensure mission
success, and ultimately provide a safe crew exploration vehicle.
3
4. Management Challenges
Project Orion
• Execution Challenges
• Communicate the expectations to a diverse 2000+ personnel
organization consisting of NASA, prime contractor (LM), and
many support contractors
• Balance the desire to improve the design and need for a design
baseline. From the time the PDR baseline is struck, a lot of
potential energy develops to improve the requirements,
architecture, design, and configuration. This energy must be
managed to ensure that PDR products align as close as
possible to the PDR baseline.
• Monitor and control process execution throughout the duration
of the effort
4
5. Operational Excellence
Project Orion
• Teamwork
– Created a joint NASA/LM PDR Integration Team (PIT) to ensure common expectations are
communicated across the NASA, LM, and support contractor teams
• Ensured key personnel from each Module and IPT were included on team
• Enlisted the support of the Orion planning organizations
• Communication
– Focused on communicating early and often
– Conducted a series of TIMs with each of the subsystem leads to develop common
expectations and identify issues needing resolution prior to PDR
– Instituted multiple statuses and forums targeting all stakeholders
• Commitment
– With the goal of “Get it right the 1st time”, the Orion Management team routinely stressed
the importance of attaining the PDR level of design maturity
– Management routinely communicated the PDR expectations and monitored status weekly
–
• Accountability
– Each Subsystem Lead took ownership of their respective Subsystem Design Review
preparation and execution
– A detailed schedule was developed to manage the program DRD review and delivery
– Detailed metrics were developed for tracking all deliverables by System and Subsystem
7. PDR Process & Data Package
Project Orion
• PDR Definition
– A technically-focused design review which demonstrates that the preliminary system,
module and subsystem design of the Orion integrated vehicle, its concept of operations,
associated processes and applicable technical plans meet all system requirements with
acceptable risk and within cost and schedule constraints.
• PDR Process
– The PDR process included a series reviews which provide a solid foundation for
determining if the current design is at PDR level of maturity.
• 184 Technical Reviews
• 104 Peer Reviews
• 10 Combined Tech/Peer Reviews
• 18 Subsystem Design Reviews
• 1 System and Module Review
• 8 RID Teams responsible for RID Screening & Disposition
• 1 PDR Pre-Board
• 1 PDR Board
• PDR Data Package
– 170 DRDs delivered and reviewed
– 53 “Drafts Available” DRDs provided to NASA for comment
8. PDR Objectives
Project Orion
• Demonstrate that the Orion Project’s design baseline meets all Constellation Program
and Orion system requirements to the subsystem level and does so with acceptable
risk and within the cost and schedule constraints
• Ensure that all system requirements have been allocated to the component level, the
requirements are complete, and flow down is adequate to verify system performance
• Ensure all physical and functional interfaces for Orion system and subsystem design
have been identified and defined
• Demonstrate sufficient definition of integration and verification activities to allow the
Project to proceed with detailed planning of test, integration, and the manufacturing
of test articles
• Confirm that verification method requirements have been developed to the subsystem
level and are consistent with the requirements and the vehicle design
• Demonstrate that the PDR vehicle configuration is a closed vehicle and within all
specified margins
• Demonstrate that the vehicle hazards and hazard control methods are identified and
assessed, including safety features of the design and crew survival features of the
vehicle
8
9. PDR Objectives
Project Orion
• Establish the preliminary design, thus enabling the Project to baseline subsystem
specifications
• Ensure that technical and programmatic risks are identified, are categorized by
criticality, and that risk mitigation plans are approved and are being managed
• Ensure that the Project is meeting cost and schedule baselines and has the
infrastructure necessary to support management and reporting of cost and schedule
data
• Show appropriate maturity in the proposed design approach to proceed to critical
design
• Establish the basis and a viable path for proceeding to the Critical Design Review
(CDR) including interim milestones
9
10. PDR Entry & Success Criteria
Project Orion
• Performed a detailed analysis and mapping of Entrance and Success Criteria
• Orion is required to meet both NASA NPR 7123.1A and LM 2.1.6-T1-PgmMgt-1.2-P-
C3 PDR Entrance and Success/Exit Criteria
• Evaluated both sets of entrance and exit (success) criteria
• Integrated into an “Orion” set of PDR Entrance and Success Criteria
• Documented in CxP 72212 Orion PDR Process Plan
• Mapped PDR Products/DRDs to criteria
• Criteria reviewed by all NASA and LM Module/IPT Leads, Subsystem Leads, and
key others prior to formal documentation in CxP 72212
• Criteria reviewed and approved by LM SSC Central Systems Engineering
• NASA & LM IPTs developed a detailed self assessment against the entry and success
criteria
• Allowed team to better resolve issues prior to the PDR Board
• Assisted program management in documenting open issues needing to be resolved
post PDR
• First evaluation of PDR Entry and Success Criteria occurred at the PDR Pre-Board
• Final evaluation of the Success Criteria occured at the PDR Board
10
11. CxP 72212 Orion PDR Plan
Project Orion
• Comprehensive PDR plan documented in
CxP 72212 and placed under configuration
control
• All significant stakeholders
participated in review of document
prior to release
• Formalized the overall PDR process
• Documented the requirements and
associated documentation baseline for
PDR
• Incorporated tailored NPR 7123.1A criteria
into plan
• Served as the primary method of
documenting all agreements
regarding method to meet the SSDR
and PDR criteria
• Used as a reference throughout the
execution to ensure the process was
adhered to
11
12. Road to August 2009 PDR
Project Orion
2008 2009
Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May June July Aug Sep
Ares LC4 Results Final Drop 12/20 Remaining
Ares LC4 Loads Analysis Update Ares LA4 Loads Analysis Data Dropped
1/23 1/30 Lunar Data Drop 8/14 8/31 (TBR)
Orion LC-4 R&U Update Will be updated after Orion/Ares Coordination Orion Rev & Update
Sizing / Subsystem FEMs LA4 FEM Develop Sizing / SS FEM LC5 FEM Develop Szg
Orion Uncoupled Orion Uncoupled Orion Uncoupled Orion Uncoupled
DAC-3 Initialization Loads Loads Update (Test Data Updates)
606E Closeout ERB (DAC-2 Loads + 6 wk Loads Update (LC-4, Landing)
2/24-27 PDR Loads
LOC/LOM ERB Update for Landing,
GN&C SSDR Subsystem Design Reviews Status
Module Struct, Mech, & Sec Environments)
MELs Pyro, & TPS 10/23 10/24 Midterm 606F Integrated Loads SWELs Closeout 7/8
T-008 SAP
Complete MEL Update 9/29,30 ERB 606F Products ERB Complete ERB 606G T-009, T-010 DAC-3
DAC-3 POD Delivered
9/9 & 10/2 SBR ERB 11/19-20 12/22 Analysis Reports Due to
8/1 8/15 2/12-13 3/24 4/14 5/15 6/11&12
NASA
KO PDR Board
6-wk Asmts & 7/13 7/24 8/31
DAC-2 Sys Integ DAC-3 Sys Integ
DAC/PDR Planing
SMR Pre-Board
Integrated vehicle 10/29 Integrated vehicle Integrated vehicle
products on ProjectLink T-009, T-010 DAC-2 products available products available 2 wk Sys & Mod
1 week prior to ERB Analysis Reports Due to NASA Presentations 8/31
Tech Reviews & Peer Reviews S/C, Mod, & AV&SW Spec DRD Rev & RID All RIDs
S/C & Module Spec (DAC-2) 12/5 (C&T & Mid-DAC-3) – 5/5 Dispositioned
7/7
S/C & Module Spec Development Sys Product Develop Final PDR DRDs
Subsystem Spec Development delivered to NASA
Subsystem IRD & ICD Development
Subsystem Design and Databook Development 1-wk DRD
Master Verification Plan Development Delivery Margin
Drawing Release
Ph 1 TIM #2 1/20-23 2/9-13 Ph 1 3/30-4/3 Ph 1 5/4-8 Ph 1 6/15-26 Ph 1 TIM #6 Delta
CSERP Groundrules TIM 10/1 11/3-7 Ph 1 TIM #1 TIM #3 TIM #4 TIM #5 7/27-8/7 CSERP Review
Safety & Mission Assurance
Spiral Data Spiral Spiral
Cradle Snapshot 8/15 C&T SS Review 12/4-5
Pkg Delivery Review Data Rev
C&T SRR 10/29 2/28 Sys & Subsystems Cradle Snapshot 11/20 12/7-11 Board
1/22/10
Avionics & Software Activities
AA-2 7/16 PA1 DD250 3/3 4/10 T&V Sys B/L Rev
PTR2 EGSE DD250 2/11 CAIL PR 6/30
Test 12 Verification Activities
and 12
13. Orion PDR Process
Project Orion
Project Orion
June July August Sep Oct
Subsystem Design 2-Wk SMR Prep 8/4 Board
Dry-
Reviews (SSDRs) SSDRs Decision Gate
Runs Orion PDR
PDR KO
7/13 7/21 8/10 Forward
System & Daily Board Decision Gates
Module
Presentations RIDs Captured
SMR
Against Design
DRDs Delivered RID Tool Close (all RIDs Sponsored)
RID Tool Open 7/7 PDR -45 7/31
TBD
4 wk DRD Review
DRD Waterfall / Early Review
(RID Creation & S
(Tech Reviews & Peer Reviews)
Sponsor)
PIT Lead with Module/IPT Support Integ
PIT War Room
Starts 8/27 Team
8/10 NASA Screening
NASA Team Concludes RID Screening
NASA Lvl III Review & Filter RIDs Screening All RIDs receive product
(Work disconnects with external stakeholders) Team owner or lead recommended
(LM SEIT led SME team available to answer questions) Disposition
8/17
Starts 8/3 Product Owner
Recommend All RIDs
Disposition Dispositioned
8/20
Starts Review Team
8/12 Disposition
DRDs
8/31-9/1 PDR Board Redelivered
Starts 8/17 Pre Decision to Proceed to DAC-4 NLT 2/1/10
Board
5 1/2 Day Pre-Board to work issues in parallel
with completion of RID Disposition DRD Update & Redeliver
(Type 1 + Final Type 2)
14. Orion Subsystem Design Reviews
Project Orion
Project Orion
June 2009
Monday Tuesday Wednesday Thursday Friday
1 2 3 4 5
SSDR KO/ROE (2 ea.)
8 9 SSDR KO/ROE (2 ea.) 10 Chairs ROE (Makeup) 11 12 Chairs ROE (Makeup)
Houston 2:00 PM MT 2:00 PM MT
SCRAM Review
Sacramento, CA LAS Jettison Motor SSDR DAC-3 Closeout ERB
15 16 17 18 19
Includes
Denver Mechanisms SSDR Structures SSDR Sat 6/20
Denver Crew Systems SSDR Denver Wiring SSDR
2 wk Approach
Denver Denver Ends
Passive Thermal Control SSDR Landing and Recovery System SSDR Sat 6/20
Denver Propulsion SSDR
22 23 24 25 26
Denver LAS focus 6/20 & 6/22 AM Structures SSDR Overflow
Denver Environmental Control and Life Support SSDR
Denver Avionics and Software SSDR
Denver Denver Ends
Electrical Power System SSDR & Voltage/Distributed Architecture Updates Thermal Protection System SSDR Sat 6/27
Promontory, Utah LAS Abort Motor SSDR
29 30 July 1 2 3 - Holiday
Houston PDU Review
Houston Pyrotechnics SSDR
Elkton, MD LAS Attitude Control Motor SSDR
GN&C SSDR Conducted 2/24-27
LAS SEIT, Avionics & Power SSDR Conducted 4/14-15
15. RID Criteria & Ground Rules
Project Orion
Criteria
• A design-related issue intended to document:
• Inability of the preliminary design to meet requirements (e.g., design compliance
issues)
• Incomplete or incorrect requirement flow down or traceability
• Incomplete or inadequate maturity for PDR (e.g., incomplete/inadequate trades or
models)
• Undefined or inadequately defined technical interfaces
• Inconsistency between design, analysis, requirements, or operations concepts
• Incorrect or inadequate verification methods in support of the preliminary design
• Inconsistency with the primary characteristics that should factor into each design
decision as identified in CxP 70000, Constellation Architecture Requirements
Document, Section 3.1.2. These primary characteristics are: Safety and Mission
Success, Programmatic Risk, Extensibility and Flexibility, and Life Cycle Cost
• Unsafe design features
• Unidentified or inadequately mitigated risk
• Inconsistency with Project objectives
15
16. RID Criteria & Ground Rules
Project Orion
Ground Rules
• RIDs can be written against RIDable documents or against any aspect of the Orion
vehicle design and technical plans within the purview of the Orion Project
• RIDs will be recommended for withdrawal if the initiator is identifying a deficiency that
is out of scope of this review or does not meet the RID criteria listed above
• An assertion via RID that a design feature is inconsistent or deficient with respect to
the primary characteristics expressed in the Orion Requirements Baseline should be
supported by rationale that provides a credible case for the existence of a more
favorable alternative
• Where possible, RIDs should be tied to functional or performance requirements or
constraints
• RIDs will also be recommended for withdrawal if the requestor is identifying an area
of “forward work”
• “Forward work” is defined as work that is necessary for successful completion of the
Orion vehicle but not required to satisfy the success criteria for PDR
• Lack of sufficient information is a basis for a RID only if the RID initiator has
exhausted all reasonable means to obtain the information or if the information has
not been made available as required by the entrance criteria
16
17. RID Criteria & Ground Rules
Project Orion
Ground Rules
• Editorial RIDs will be demoted to comments and forwarded to the document
developer as information at the end of the review
• Initiators are expected to review the errata sheet associated with a specific DRD prior
to reviewing the DRD and/or prior to entering a RID
• RIDs are intended to identify design issues and are not intended to document known
issues as documented in the errata
• RIDs may be written if the errata description or details are insufficient to provide
confidence that the issue has been addressed or will be addressed in the appropriate
manner and timeframe
17
18. What is a Good RID?
Project Orion
• What is a good RID?
– A “new” design related issue
– Accurately describes the design related issue consistent with the RID criteria
– Provides the details of where the issue was found
• For DRD related RIDs (section, page, etc)
• For Architecture Element RIDs with no DRD referenced (chart, conversation, etc.)
– Provides a sufficiently detailed recommended resolution
– Stays within the scope of PDR expectations
– For requirements, provides correct “from” language and recommends “to” language
An issue that has not previously been identified.
• What is a bad RID?
– Incomplete issue identification
– No recommended resolution
– Recommended resolution of “fix this”
– Editorial comment (spelling, grammar, etc.)
– RID that expands on errata or duplicates a Pre-Declared RID
– Post PDR level of maturity / unrealistic expectations
– RID written against a baselined or reference document (Examples: SRD, S/C Spec)
– RID entered by someone other than the original initiator without documenting the original
initiator
– RID written against a previously made project decision (VICB, CPCB, etc.) without new data
or evidence to challenge the decision
Don’t tell us something we already know!!!
20. PDR Assessment
Project Orion
• Design Issues
– A detailed pre-assessment was performed by each of the module prior to entering PDR
– All known issues were presented at each SSDR and the SMR
• No new significant issues surfaced during the reviews
– Significant focus was applied to ensure known issues had agreed-to plans to proceed
– Top issues contributing to non-green assessments
• Mechanisms – FBC R&R – FBC Recontact, CM/SM R&R Maturity Parachute Architecture Study was underway prior to
• LRS – FBC Recontact / Parachute Architecture & Testability entering PDR. Assessment concluded in October
• TPS & Structures – Parachute Architecture Impacts to Baseline 2009.
• EPS – 120 V Incorporation
Driven by significant post SDR requirements changes
• AV&SW – C&T Incorporation, BFCS Maturity
• ECLS – ARS Suit Loop, Procurement Schedule Procurement schedules were being updated to align with
• CM Prop – Procurement Schedule project funding profile
• Pyro – Devices Dependent on Mechanism Maturity
• LAS ACM – Production Motor Maturity
• 18 Subsystem Design Reviews (SSDRs) executed successfully
– Each SSDR assessed against a common set of entry and success criteria plus a joint set of NASA/LM agreed to objective
evidence customized for each Subsystem
– 59 Total Review Days & approximately 500 review participants
• PDR executed successfully
– Each module assessed against a common set of entry and success criteria; ITAs assessment integrated
– 56 days of DRD review, 7 day System & Module review, 1 week pre-Board and a Board
– Approximately 1200 reviewers
• Project Management assessed PDR a success and authorized the program to proceed
Yellow / Green to the detailed design phase
• Assessment based on a Red, Yellow, Green scale
21. PDR Data Package Delivery
Project Orion
As of July 8, 2009 PDR Data Package
• NASA/LM jointly assessed PDR
deliverables to determine which
DRDs were necessary to meet PDR
expectations and satisfy criteria
• PDR –
• 146 Deliverable Documents
• SSDR & PDR –
• 26 Design & Data Books
• Data Package Total
• 172 Total PDR Documents
• 53 “Drafts Available” DRDs
provided to NASA for
comment
Waterfalled Review and Delivery
• Managed via the PDR Integration
PDR -45 days Team
• Captured in a 2000+ line MS
Project Schedule
• Metrics generated & reported on
a weekly basis
• Deliveries culminated on 7/7/09,
PDR-45 Days
22. Action Item & RID Closure
Project Orion
SSDR Actions PDR RIDs • Detailed Action and RID
AV&SW 140 150
screening occurred
LAS
Crew Systems 39
throughout the SSDR &
CM 241
ECLSS 49
PDR process
SM 90
EPS 97
• All actions and RIDs
GN&C 26 Av&Pwr&Wire 444
dispositioned and closure
LAS AM 60 38
Software aligned with project post
LAS ACM 96
T&V, Fac, GSE 411 PDR work plan
LAS JM 60
AI&P 18
LAS SE&I AVP 12 • All RIDs approved by PDR
LRS 42 SR&QA 91 Board
Mechanisms 56 566
Systems
Propulsion 150 • Project approved closure
33 Total 2049 plan implemented
PTC
Pyro 26
Structures 141 • Approximately 100,000 pages of design, process,
TPS 20 and plan documentation reviewed
Wiring 14 • Included requirements, interface definition,
Total 1061 verification details, general plans, etc.
24. Key PDR Lessons Learned
Project Orion
• SSDRs
– SSDRs executed in parallel and in a common location worked well
• Key personnel were in the building and available and could easily bounce between reviews when
needed
– Focused review teams allowed the reviews to move at a sufficient pace while ensuring all significant
information was reviewed to an adequate level
• On-site review teams of 50 to 100 personnel is appropriate
• Overflow rooms and remote participation were not an ideal way of participating in the on-site review
– Data Packages consisted of the core information necessary to evaluate design maturity
• Minimizing the data package to only those DRDs necessary to establish design maturity supported a
streamlined review process
– Early technical and peer reviews facilitated improved design awareness prior to the review
– Need to strengthen Software integration into SSDRs
• Evaluate what additional products need to be added to the SSDR Data Packages
• Note software was addressed to level required for NPR 7123.1a (project wants additional focus for CDR)
– Need to ensure adequate spacing between tech reviews and SSDRs to allow for sufficient time to close
TR actions
• Need SSDR co-chairs to review open TR actions prior to SSDRs
– Transferring SSDR actions to RIDs created a dual paper trail that required unnecessary effort to resolve
– SSDR Dry-runs need to be scheduled further in advance to allow subsystem team to make adjustments
prior to the review
– SSDR Caucuses were most effective when kept to a small attendance
– SSDR color ratings took more time than expected given that issues were identified ahead of the SSDR
25. Key PDR Lessons Learned
Project Orion
• PDR
– Data Package Preparation, Delivery, & Review
• Separate Technical and Peer Reviews were not the most efficient way to review DRDs
– Peer Reviews were not viewed as where the detailed review occurs
– Integrating Technical and Peer Reviews into a single review that meets the expectation of both
reviews
• Decouple DRDs not needed for design review from the design review
– Review and accept these documents through NASA’s normal DRD acceptance process
• Waterfalled delivery allows for early review of material prior to the face-to-face reviews
– System & Module Review (SMR)
• SMR value did not match level of effort required to prepare and execute
– Most technical review and discussion occurred at the SSDRs
– Weak in-person attendance
• Need to change this review to an integrated vehicle review focusing on the integrate module and
integrated stack design for CDR
– Include Assembly, Integration & Production
– Include Test & Verification
– Pull the integrated analysis and design products and review them up-front prior to the SSDRs
• Create a one-day CDR Kick-off that gives the requirements baseline, environments overview, etc.
prior to the CDR SSDRs
26. Key PDR Lessons Learned
Project Orion
• PDR
– RID Tool, ProjectLink, Portal, and Wiki
• Use one tool from the beginning of the process through completion
– Minimizes duplication of issues and the overhead of transferring from one tool to another
• Test the RID tool earlier in the planning process
• Choose a RID tool that allows for a detailed closure process tracking
• Portals and Wiki were a valuable resource to all reviewer
– Assign a NASA IT person to the PIT for CDR
• PDR hotline (outlook email box) served as a ongoing resource when reviewers did not know who to
contact regarding a process or technical issue
– RID Screening and Review Teams
• IPTs who integrated the screening/review effort completed activities earlier and reduced iteration
when dispositioning / resolving issues
• Improved direction regarding “Approve” and “Approve w/mod” needs to be developed and
communicated to review teams prior to start
• Training needs to better prepare the reviewers for what to review the design against (Rqmts, Docs)
– Consider adding an Orion familiarization to the training
– Pre-Board and Board
• Overall Board execution proceeded to plan
– Increased Pre-Board time needed in CDR schedule if a similar number of RIDable documents
are to be reviewed during CDR
• Ensure RIDs dispositioned by pre-board or board are updated in the tool needs to occur
• Do not push issue decisions to later forums (VICB, CPCB, etc.). Make the decisions prior to leaving
the Board even if this extends the Board
29. • All reviewers participate in the Orion CDR Kickoff (Orion
Familiarization) to educate reviewers on the:
• Design configuration Top-level CDR Plan
• Coordinate what metrics are needed with RID/Action Tool prior to CDR
• RID closure will be handled within IDE
• Requirements baseline DRAFT
• Errata and Pre-Declared RIDs all in one location (minimize number)
• Environments baseline
• Caucus serves the function of RID Screening Project Orion
• Design Work products
• Align tool with Process
• Significant design issues 3
Week 1 Week 2 Week and their closure plans
Week 4 Week 5 Week 6 Week 7 Week 8 Week 9 Week 10 Week 11 Week 12
Orion CDR Kickoff
DRD Review
• Don’t duplicate SSDR actions and RIDs.
RID Tool Opens • Issues captured in standard format throughout process (Action/RID Tool)
RID Tool Closes
• Standardize Etc.)
Interactive Reading Room (DRDs, Drawings, process for handling cost/schedule issues
• Ensure Action/RID template aligns with IDE tool
• Summarize SSDR content that has been covered in component CDRs
System Analysis Review (1 to 2 days)
SSDRs
Issue Identification (RID generation)
• Replace SMR with an Integrated Vehicle Design Review
SSDR Caucuses • Cover integrated module, integrated stack, AI&P, T&V, etc.
IVDR Integrated Vehicle Design Review
RID Disposition (Product Owners & Leads)
RID Review Teams
Pre-Board
Board
Notes from Orion Streamlining TIM 12/09 DRAFT
31. PDR Supporting Documentation
Project Orion
• PDR Plan
– CxP 72212 CxP Orion PDR Plan is available on PDR Wiki or PDR
Portal/Dashboard
• PDR Resources
– PDR Wiki
• Used as a resource to all Orion personnel and PDR stakeholders to communicate the
current planning status
• https://ice.exploration.nasa.gov/confluence/pages/viewpage.action?pageId=13828791
&maxRecentlyUpdatedPageCount=20
– PDR Portal
• Used to formally provide all SSDR and PDR details, data package, and review results
• Federal record of review
• https://ice.exploration.nasa.gov/ice/site/cx/reviews_orion_pdr/
32. PDR Integration Team
Project Orion
In addition to the core PIT members above, there were members from each IPT, Module, Planning, and Administration
incorporated into the planning effort and considered active PIT members.
35. Entrance Criteria
Project Orion
Orion Entrance Criteria (CxP 72212)
1) Successful completion and close-out of SDR. All prior SDR Actions and RIDs have been closed, or
approved closure plan exists for those remaining open.
2) PDR plan has been approved by the CPCB, which includes the PDR entrance and success criteria,
RID/Action process, Board Members, stakeholders and schedule.
3) All PDR applicable and supporting products have been delivered and made accessible for stakeholders
prior to review.
4) Updated baselined documentation, as required.
5) Preliminary design is established for each subsystem and supporting subsystem specifications are final
with supporting analysis and trade-studies. Spacecraft, Module and Subsystem Specifications are
complete and traceable. Subsystem specifications include Component definition. Component
specifications delivery and review schedules have been developed and aligned with procurement strategy,
including long-lead items. Preliminary software design and supporting subsystem specifications should
include a complete definition of the software architecture and preliminary database design description (if
applicable) which should include all analysis and data used for the development/maturation of the software
architecture.
6) Technology development plans are updated.
7) The project risks and opportunities have been developed. Acceptable risk mitigation plans have been
developed in accordance with the Orion Risk Management Plan.
8) Cost/schedule data are available to appropriate reviewers.
9) Updated logistics documentation, including initial support equipment list.
36. Entrance Criteria
Project Orion
Orion Entrance Criteria (CxP 72212)
10) Applicable technical plans (e.g., technical performance measurement plan, contamination control plan,
parts management plan, environments control plan, EMI/EMC control plan, producibility/manufacturability
program plan, reliability program plan, quality assurance plan).
11) Updated Applicable standards.
12) Integrated CEV flight safety analysis (including software safety analyses) and other applicable S&MA
analysis of preliminary design (Failure Modes and Effect Analysis, flight system reliability and
maintainability analysis, Probabilistic Risk Assessment, etc.) are complete and requirements are
implemented in system, module, subsystem, and component specifications (component specifications are
developed per the acquisition strategy). All associated plans have been developed.
13) Updated Engineering drawing tree.
14) Preliminary Functional and physical interfaces are developed and documented in the Interface Control
Documents (ICDs).
15) Test and verification and validation plans are defined.
16) Plans to respond to regulatory requirements (e.g., Environmental Impact Statement, as required)
17) Disposal Plans defining safe disposal of assets.
18) Technical resource utilization estimates and performance margins are updated.
19) Preliminary limited life items list is updated (LLIL).
20) Operational planning is proceeding to schedule.
37. Success Criteria
Project Orion
Orion Success Criteria (CxP 72212)
1) The Spacecraft, Module, and Subsystem Requirements, TPMs and any CxP-imposed requirements are
agreed upon, finalized, stated clearly and are consistent with the preliminary design.
2) The flow down of verifiable requirements to the subsystems and to the appropriate components are
complete. An adequate plan exists for timely resolution of all open items. Requirements are traceable to
Orion SRD, CxP IRDs and other applicable CxP requirements including any errata.
3) System, module and subsystem preliminary designs are complete and meet requirements at an
acceptable level of risk. All liens have been identified and closure plans developed.
4) Technical interfaces are defined, deemed adequate, and are consistent with the overall technical
maturity and provide an acceptable level of risk.
5) Adequate spacecraft technical margins exist with respect to TPMs.
6) Any required new technology has been developed to an adequate state of readiness or back-up options
exist and are supported to make them a viable alternative.
7) The project risks and opportunities are understood and have been credibly assessed. Appropriate risk
mitigation plans for all top risks.
8) Safety and mission assurance (e.g., safety, reliability, maintainability, quality, and EEE parts) have been
adequately addressed in preliminary designs and any applicable S&MA products (e.g., PRA, system safety
analysis, and failure modes and effects analysis) have been reviewed and concurred.
9) The operational concept is technically sound, is supported by the vehicle design, and includes (where
appropriate) human factors. Appropriate requirements identified by the ConOps have been flowed down
including requirements for ConOps execution.
10) Preliminary test and verification plans are updated. Manufacturing plans are developed.
11) System costs are within targets.
38. Product to Criteria Mapping
Project Orion
• Performed a detailed analysis and mapping of Entrance and Success Criteria
• The PDR Data Package is mapped to the criteria and documented in Appendix C of
CxP 72212 Orion PDR Process Plan
39. Orion PDR
rev 8/26/09 – 2200 MT
Project Orion
Project Orion
Subsystem Design Reviews (SSDRs)
June 18 Subsystem Reviews
July August Sep Oct
Subsystem Design 2-Wk • 3 SSDRs Executed Prior to the 2-wk SSDRs
8/4 Board
Reviews (SSDRs) SSDRs Dry- SMR Prep • GN&C SSDR Completed on 2/24-27
Decision Gate
Runs Orion PDR
PDR KO• LAS SEIT, Avionics & Power SSDR Completed on 4/14-15
7/13 • LAS Jettison Motor Completed on 6/8-10
7/21 8/10 Forward
System & Daily Board Decision Gates
Module • Helped iron out the process and allowed the team to take advantage of
RIDs Captured
Presentations SMR lessons learned
Against Design
• 13 SSDRs completed 2-wk set of SSDRs on 6/15-26
RID Tool Open
• At any given time 4Close (all RIDs Sponsored) in parallel in the Lockheed Martin
DRDs Delivered RID Tool
7/31
SSDRs were occurring
7/7 PDR -45
TBD Denver WB2 facility
2 remaining SSDRs completed the last week of June
• 4 wk DRD Review
DRD Waterfall / Early Review
(Tech Reviews & Peer Reviews) • SSDR ProductsS
(RID Creation &
Sponsor)
• 28 Design & Data Books Delivered & Reviewed
• 23 Draft Subsystem Specs, 21 Subsystem IRDs, 21 Subsystem ICDs, and
PIT Lead with Module/IPT Support Integ
PIT War Room
Starts 8/27 Team
12 Subsystem Volumes of MVPs Available for Comment
8/10 NASA Screening
• 59 Total Review Days Team Concludes RID Screening
NASA
• Over 1,000 review participants includingreceive product NESC, SRB, NASA Crew
NASA Lvl III Review & Filter RIDs Screening All RIDs support from
Office, NASA Team owner or lead recommended
(Work disconnects with external stakeholders) MOD, NASA Engineering, NASA S&MA, NASA Health & Medical,
(LM SEIT led SME team available to answer questions) Disposition
Constellation, Orion Project, & LM Central Engineering
8/17 Redelivery date will be
• 1,056 Action Items Generated
Starts 8/3 Product Owner recommended on 8/21 based
Recommend
• All actions recorded in either the ConCERT RID Tool on number and complexity of
All RIDs or LM IDE
Disposition Dispositioned RIDs.
8/20
Starts 8/12
Review Team
Disposition
DRDs
8/31-9/1 PDR Board Redelivered
Starts 8/17 Pre Decision to Proceed to DAC-4 NET 11/6
Board
5 1/2 Day Pre-Board to work issues in parallel
with completion of RID Disposition DRD Update & Redeliver
(Type 1 + Final Type 2)
40. Orion PDR
rev 8/26/09 – 2200 MT
Project Orion
Project Orion
Subsystem Design Reviews (SSDRs)
Subsystem Design Reviews (SSDRs)
June 18 Subsystem Reviews
July August Sep Oct
Subsystem Design 2-Wk
Each SSDR assessedBoard 2-wk SSDRs set of entry and success criteria
• 3 SSDRs Executed Prior to the 8/4
against a common
Reviews (SSDRs) SSDRs Dry- SMRplus a joint set of Decision Gate agreed to objective evidence customized for
Prep NASA/LM
• GN&C SSDR Completed on 2/24-27
Runs Orion PDR
each Subsystem (Assessed either Green, Yellow,on 4/14-15 or Red)
PDR KO• LAS SEIT, Avionics & Power SSDR Completed
7/13 • LAS Jettison Motor Completed on 6/8-10
7/21 8/10 Forward
Crew Systems
LAS SEI, AVP
System &
Mechanisms
Mechanisms
Daily Board Decision Gates
• Helped iron out the process and allowed the team to take advantage of
Module
Structures
LAS ACM
LAS ACM
RIDs Captured
AV & SW
CM Prop
SM Prop
SM Prop
lessons learned
LAS AM
LAS AM
Presentations
LAS JM
SMR
Wiring
Wiring
Against Design
GN&C
ECLS
ECLS
• 13 SSDRs completed 2-wk set of SSDRs on 6/15-26
Pyro
LRS
LRS
EPS
EPS
PTC
TPS
DRDs Delivered Subsystem Criteria4Close (all RIDs Sponsored) in parallel in the Lockheed Martin
RID Tool
• At any given time SSDRs were occurring
RID Tool Open 7/7 PDR -45 7/31
1. Requirements are complete and flowed to the appropriate
TBD Denver WB2 facility
products. Subsystem specifications are updated.
2. Subsystem design is complete to a level of detail that is
2 remaining SSDRs completed the last week of June
• 4 wk DRD Review
sufficient to demonstrate that all requirements are expected to
DRD Waterfall / Early Review
(Tech Reviews & Peer Reviews)
• SSDR ProductsS
be met at an acceptable level of risk. Subsystem preliminary
(RID Creation &
design baseline is complete and does not exceed subsystem
Sponsor)
mass and• Design & Data Books Delivered & Reviewed
power allocations. Subsystem preliminary design
includes hardware and software considerations. Test and
PIT Lead with Module/IPT Support Subsystem Specs, are
verification plans are defined. Technical interfaces Subsystem IRDs, Subsystem ICDs, and Subsystem
• Draft Integ
PIT War Room
Starts 8/27 Team
defined, deemed adequate, and are consistent with the
Volumesatof MVPs Available for Comment
overall technical maturity an acceptable level of risk.
8/10 NASA Screening
Resolution plans for any non-compliance have been including support from NESC, SRB, NASA Crew
• Over 1,000 review participants Concludes RID Screening
developed. NASA Team
3. The Office, NASA opportunities are understood and RIDs receive product LM Orion, & LM Central
subsystem risks and MOD, NASA Engineering, NASA Orion,
NASA Lvl III Review & Filter RIDs Screening All
(Work disconnects with externalbeen credibly assessed. Acceptable risk mitigation owner or lead recommended
have
Engineering where applicable.
stakeholders)
plans have been developed Team
• preliminary subsystem design is reliable and safe forDisposition
(LM SEIT led SME team available 4. The _________ Action Items Generated
to answer questions) use.
8/17
Reliability and safety analyses of the preliminary design have Redelivery date will be
• _____ 8/3 requirements flowed down “Pre-declared RIDs”
Actions classified as to
been performed and resulting
Starts Product Owner recommended on 8/21 based
• _____ Actions Recommend as “Complete Prior to PDR”on number and complexity of
the system, module, and subsystem. Preliminary plans for
classified
tests, analyses, demonstrations, and Disposition verify
inspections will
All RIDs
the reliability_____ Actions classifiedReliability, Dispositioned RIDs.
• and safety of the subsystem. Safety, as “Forward Work”
and Quality Assurance (SR&QA) has been adequately 8/20
addressed in preliminary designs, and applicable SR&QA the ConCERT RID Tool or LM IDE
• All actions recorded in either
Starts 8/12
products (e.g., PRA, system safety hazard analysis, and Team
Review
Disposition
failure modes and effects analysis) satisfy PDR product
maturation requirements (e.g. Phase 1 Safety Review criteria) DRDs
5. Test, verification and manufacturing planning are underway. 8/31-9/1 PDR Board Redelivered
Starts 8/17 Pre Decision to Proceed to DAC-4 NET 11/6
Board
5 1/2 Day Pre-Board to work issues in parallel
with completion of RID Disposition DRD Update & Redeliver
(Type 1 + Final Type 2)
41. Orion PDR
rev 8/26/09 – 2200 MT
Project Orion
Project Orion
June Data Requirements DescriptionAugust Review and Waterfall
July (DRD) Sep Oct
Subsystem Design 2-Wk 8/4 Board
Reviews (SSDRs) SSDRs Dry- SMR Prep
PDR Data Package Decision Gate
Runs Orion PDR
• NASA/LM jointly assessed PDR deliverables to determine
PDR KO which DRDs were
System &
necessary to meet PDR expectations and satisfy criteria
7/13 7/21 8/10 Forward
Daily Board Decision Gates
Module • PDR –
RIDs Captured
Presentations SMR146 Deliverable Documents
• Against Design
(Includes multiple volumes of specific DRDs, i.e. MVP, Specs, IRDs)
DRDs Delivered
• SSDR & PDR – Tool Close (all RIDs Sponsored)
RID
RID Tool Open 7/7 PDR -45 7/31
TBD • 26 Design & Data Books
4 wk DataReview
• DRD Package Total
DRD Waterfall / Early Review
(RID Creation Total PDR Documents
• 172 & S
(Tech Reviews & Peer Reviews)
Sponsor) 145 RIDable Documents (listed on PDR Portal)
•
PIT Lead with Module/IPT Support 12 Commentable Documents
• Integ
PIT War Room
Team
Starts 8/27 17 DRDs reviewed through NASA DRD Review Process Early
•
8/10 NASA Screening
(2 Documents Redelivered Prior to 7/7)
Team Concludes RID Screening
NASA
• 53 “DraftsScreening
NASA Lvl III Review & Filter RIDs
Available” DRDsAll RIDs receive product for comment
provided to NASA
• Total SSDR and PDR Data Package consisted of nearly 100,000 pages of
(Work disconnects with external stakeholders) Team owner or lead recommended
Disposition
(LM SEIT led SME team available to answer requirements, interface definition, verification details, general plans, and
questions)
8/17 Redelivery date will be
supporting design detail
Starts 8/3 Product Owner recommended on 8/21 based
Recommend All RIDs on number and complexity of
Disposition Dispositioned RIDs.
Waterfalled Review and Delivery
8/20
• Managed via the PDR Integration Team
Starts 8/12
Review Team
• Captured in a 2000+ line MS Project Schedule
Disposition
• Included Tech Reviews, Peer Reviews, C&DM Processing, and Delivery
8/31-9/1 PDR Board
DRDs
Redelivered
• Metrics generated on a weekly basis NET 11/6
Starts 8/17 Pre Decision to Proceed to DAC-4
• Reported to Management team through various forums
Board
4 Day Pre-Board to work issues in parallel with 7/7/09, PDR-45 Days
• Deliveries culminated on
completion of RID Disposition DRD Update & Redeliver
(Type 1 + Final Type 2)
42. Orion PDR
rev 8/26/09 – 2200 MT
Project Orion
Project Orion
June Data Requirements DescriptionAugust Review and Waterfall
July (DRD) Sep Oct
Subsystem Design 2-Wk Data Requirements Description (DRD) Review and Waterfall
8/4 Board
Reviews (SSDRs) SSDRs Dry- SMR Prep
PDR Data Package Decision Gate
Runs Orion PDR
PDR Data Packageassessed PDR deliverables to determine which DRDs were
• NASA/LM jointly
PDR KO
• 8th, 2009
7/13 Metrics Summary – July 8/10 Forward and satisfy criteria
necessary to meet PDR expectations
7/21
System & Daily Board Decision Gates
Module • PDR –
RIDs Captured
Presentations SMR146 Deliverable Documents
• Against Design
(Includes multiple volumes of specific DRDs, i.e. MVP, Specs, IRDs)
DRDs Delivered
• SSDR & PDR – Tool Close (all RIDs Sponsored)
RID
RID Tool Open 7/7 PDR -45 7/31
TBD • 26 Design & Data Books
4 wk DataReview
• DRD Package Total
DRD Waterfall / Early Review
(RID Creation Total PDR Documents
• 172 & S
(Tech Reviews & Peer Reviews)
Sponsor) 145 RIDable Documents (listed on PDR Portal)
•
PIT Lead with Module/IPT Support 12 Commentable Documents
• Integ
PIT War Room
Team
Starts 8/27 17 DRDs reviewed through NASA DRD Review Process Early
•
8/10 NASA Screening
(2 Documents Redelivered Prior to 7/7)
Team Concludes RID Screening
NASA
• 53 “DraftsScreening
NASA Lvl III Review & Filter RIDs
Available” DRDsAll RIDs receive product for comment
provided to NASA
(Work disconnects with external stakeholders) and PDR Data Package consisted of nearly 100,000 pages of
• Total SSDR Team owner or lead recommended
requirements, interface definition,Disposition
(LM SEIT led SME team available to answer questions) verification details, general plans, and supporting
8/17 Redelivery date will be
design Starts 8/3
detail Product Owner recommended on 8/21 based
Recommend All RIDs on number and complexity of
Disposition Dispositioned RIDs.
Waterfalled Review and Delivery
8/20
• Managed via the PDR Integration Team
Starts 8/12
Review Team
• Captured in a 2000+ line MS Project Schedule
Disposition
• Included Tech Reviews, Peer Reviews, C&DM Processing, and Delivery
8/31-9/1 PDR Board
DRDs
Redelivered
• Metrics generated on a weekly basis NET 11/6
Starts 8/17 Pre Decision to Proceed to DAC-4
• Reported to Management team through various forums
Board
4 Day Pre-Board to work issues in parallel with 7/7/09, PDR-45 Days
• Deliveries culminated on
completion of RID Disposition DRD Update & Redeliver
(Type 1 + Final Type 2)