SlideShare una empresa de Scribd logo
1 de 34
Descargar para leer sin conexión
Integration of Technical Risk
Management with Decision Analysis

  Presented at NASA Project Management
        Challenge 2007 Conference

                Galveston, Texas
               February 6-7, 2007


        Homayoon Dezfuli, Ph.D., NASA HQ (hdezfuli@nasa.gov)
        Robert Youngblood, Ph.D., ISL, Inc. (ryoungblood@islinc.com)
Acknowledgement

•   Some of the materials in this presentation are based on
    methodology developed by the Office of Safety and Mission
    assurance in support of the following activities:
    –   Revision of NASA requirements for system safety (NPR 8715.3A
        dated September 12, 2006)
    –   Revision of NASA System Engineering Handbook (to be released
        June 2007)
    –   Development of a new NASA standard entitled “Risk-informed
        Management of Safety and Mission Success” (planned for release
        before the end of fiscal year 2007)




                                                                         2
Why Integrate Technical Risk Management with
Decision Analysis?
Excerpt from 2006 Third Quarterly Aerospace Safety Advisory Panel
  (ASAP) Report

      “With regards to risk assessments that are being made to support launch
      decisions, it appears that a series of fragmented, non standardized tools and
      methodologies are in use. The result is that risk recommendations to senior
      management concerning individual hazards effecting launch are sometimes
      made in isolation without consideration of overall launch risk. For example, the
      most recent Shuttle launch focused heavily on two of the 569 potentially
      catastrophic hazards currently known to exist, without any assessment of the
      overall likelihood of such a catastrophic failure. A lack of confidence in the
      technical basis for the assessments also appears to sometimes exist and
      variations in risk matrix definitions among programs has been observed. Lastly,
      only limited guidance is available concerning agency policies on what risks
      should be accepted under what conditions. The ASAP recommends that a
      comprehensive risk assessment, communication and acceptance process be
      implemented to ensure that overall launch risk is considered in an integrated
      and consistent manner. The process should be sound, mature, consistently
      implemented to yield high confidence and consistent results that are generally
      accepted by the majority of the community.”


                                                                                         3
Key Points

• Decision analysis can be a powerful tool for aligning
  decisions with program objectives, given the
  current state of knowledge and the decision-maker’s
  preferences
• Modern risk analysis methodology can significantly
  improve on the one-at-a-time management of risk
  issues criticized by the ASAP report
• Therefore:
   – Perform Technical Risk Management within a decision
     analysis framework
   – Use modern risk analysis methodology, including
     probabilistic risk assessment (PRA)
   – Retain the track, control, and accountability strengths of
     the Continuous Risk Management (CRM) Process

                                                                  4
Outline

 •   Background
 •   State of Practice of Risk Management (RM) at NASA
 •   An Example of How Risks are Handled in Decision Processes
 •   Need for a More Rigorous Approach to Inform Risk
     Management Decisions
 •   An Analytical Framework to Inform Risk Management
     Decisions
 •   The Role of Probabilistic Risk Assessment (PRA) in Risk
     Management
 •   PRA Methodology Synopsis
 •   Summary




                                                                 5
Background

•   NASA is embarking on a comprehensive space exploration
    program
•   The goals of the exploration program are “safe, sustained,
    affordable human and robotic exploration of the Moon, Mars,
    and beyond ... for less than one percent of the federal budget”
•   Meeting these goals requires development of a constellation of
    new systems
•   The design and development of these systems will involve
    many decisions that require weighting/trading various
    competing programmatic and technical considerations against
    one another




                                                                      6
Programmatic and Technical Considerations
that Should be Factored into Our Decisions

•   Affordability
    –   Meet budget constraints related to design and development cost,
        technology development cost, operation cost, and facility cost
•   Mission Technical Objectives and Performance
    –   Accomplish technical objectives in a sustainable manner and on
        schedule
    –   Enhance effectiveness and performance
•   Safety
    –   Protect the health of public and workforce, the environment and
        mission assets
•   Stakeholder Expectations
    –   Meet needs of intra-agency and extra-agency stakeholders




                                                                          7
The Role of Risk Management

•   Purpose of risk management is to promote program success
    –   Incorporating risk-informed decisionmaking in the design and
        formulation of the program baseline
    –   Proactive identification and control of departures from the
        program baseline
•   An effective and proactive RM process should
    –   Provide input to determine the preferred decision alternative in
        light of programmatic objectives
    –   Assess risk associated with implementation of the selected
        alternative
    –   Assist in setting resource priorities (including prioritization of
        work to resolve uncertainties if warranted)
    –   Plan, track, and control risk during the implementation of the
        selected alternative
    –   Iterate with previous steps in light of new information


                                                                             8
State of Practice of Risk Management at NASA




                                               9
Continuous Risk Management (CRM) Process

•   Identify – Identify program risk “issues”
                                                                           Identify




                                                                 rol
•   Analyze – Estimate the likelihood and




                                                                                              Anal
                                                             Cont
    consequence components of the risk                                 Communicate and




                                                                                                  yze
    issues                                                                Document


                                                                Tra
•   Plan – Plan the Track and Control actions                      ck
                                                                                          n
                                                                                      Pla


•   Track – Track and compile the necessary
    risk data, measuring how the CRM process
    is progressing
                                                   Current emphasis on
•   Control – Determine the appropriate
    Control action, execute the decision linked    • “management” of individual
    to that action, and verify its effectiveness     “risks,” given a decision already
                                                     made somewhere else
•   Communicate and Document –
    communicating and documenting all risk         • monitoring and accountability for
    information throughout each program              action items associated with
    phase                                            “risks”


                                                                                                        10
Risk Matrix Paradigm
    Iso-risk contours




                                                                     LIKELIHOOD
    These two points
     have the same
     expected risk




•   Risk “issues” are mapped onto the matrix individually
     –   Interaction between risks is not considered
     –   Total risk from all “issues” is not evaluated
     –   Highly subjective; uncertainties are not formally accounted for
     –   Unsuitable for combining risks to obtain aggregate risk
•   Risk tolerance boundaries are defined based on iso-risk contours
     –   Expected consequences (probability times consequences) do not adequately inform
         decisions relating to safety
•   Limited ability to support risk-trade studies

                                                                                           11
An Example of How Risks are Handled in Decision
                 Processes




                                                  12
STS-121 (OV-103 Discovery) Contingency
Planning

Decision Alternatives
1. Crew Rescue:
   –   Success of contingency
       (rescue) flight (STS-300)
   –   Success of Contingency
       Shuttle Crew Support (CSCS)
       capability (on-orbit safe
       haven)
2. Repair/Entry:
   –   Successful application of
       patching materials (STA-54)
       to repair damaged heat shield
   –   Vehicle integrity during
       reentry (no downstream
       heating damage)                 Source of Figure: Shuttle Program, NASA-JSC

   –   Successful approach and
       landing (operations)

                                                                                     13
Pros and Cons Associated with Decision
Alternative 1 (Data Source: Shuttle Program)
                                          STS-300




               CSCS




                                                      OBSERVATIONS
                                       Adverse consequences of Interest are:
                                           • Loss of crew
                                           • Loss of vehicle (OV-103)
                                           • Evacuation of station
                                       It appears that Low (L), Medium (M), and High
                                       (H) are expressions of degree of belief in the
                                       truth of the proposition


                                                                                        14
Pros and Cons Associated with Decision
Alternative 2 (Data Source: Shuttle Program)

                 Materials
                                                       OBSERVATIONS
                                            •   Additional adverse consequence of
                                                interest: Public death or injury
                                            •   Acknowledgment of uncertainties
                                                (imperfect models and a limited
                                                state of knowledge)



                  Vehicle Integrity




                                       Operations




                                                                                  15
Which Decision Alternative is More Desirable?




                                     Source of Matrices: Shuttle Program, NASA-JSC



                                                       COMMENTS
 •   The shuttle program uses the above matrix to assign a level of risk to a hazard based on its likelihood and
     consequence severity
      • Catastrophic: hazard could result in a mishap causing fatal injury to personnel and/or loss of one or more
        major elements of the flight vehicle or ground facility
      • Infrequent: Could happen in the life of the program. Controls have significant limitations or uncertainties
 •   The matrix is used here to analyze decisions
 •   The definition of “catastrophic” does not discriminate between human safety and asset safety (defined as an
     inclusive consequence severity level)
                                                                                                                      16
Need for a More Rigorous Approach to Inform
Risk Management Decisions

• Characteristics of our Decision Situations
   – They are complex: Need a systematic framework to sort
     things out
   – Must deal with uncertainty: Need to assess uncertainty
     analytically, know when it is necessary to reduce it
   – Must deal with multiple Objectives: Need to employ analytic
     decision techniques to handle competing priorities (apples
     vs. oranges)




                                                                   17
An Analytic Framework to Inform Risk
Management Decisions

•   Model consequences of decision alternatives in terms of
    impact on fundamental objectives
    –   Use performance Measures (PMs) as metrics to characterize
        performance of the decision alternatives with respect to a
        particular fundamental objective. Examples:
         • PM for crew safety is the probability of loss of crew
         • PM for space asset safety is the probability of loss of space
           vehicle
         • PM for public safety is the probability of public death or injury
         • PM for station occupancy is the probability of evacuation
    –   Inclusion of uncertainties is critical in evaluating PMs
•   Compare the consequences of decision alternatives on the
    PMs
•   Collectively consider all PMs and associated uncertainties to
    inform risk management decisions (deliberation has an
    important role here)
                                                                               18
Decision Analysis

                         Formulation of
                    Objectives Hierarchy and
                     Performance Measures
                             (PMs)




                                                Input from Technical and Programmatic Processes
                        Proposing and or
                      Identifying Decision
                          Alternatives




                         Risk Analysis of
                     Decision Alternatives,
                        Performing Trade
                    Studies, & Quantification
                      of Performance Index




                        Deliberation and
                      Recommendation on
                      Decision Alternative


                                                                                                  Decision Making and
                                                                                                   Implementation of
                                                                                                  Decision Alternative



                    Tracking and Controlling
                    Performance Deviations




                                                                                                                         19
Integration of CRM Process with Decision
Analysis

    Probabilistic Risk                                                 Decision Analysis
   Assessment Plays a
    Major Role in this                                                      Formulation of
                                                                       Objectives Hierarchy and
        Activity                                                        Performance Measures
                                                                                (PMs)
                                                                                     See Chart 21




                                                                                                    Input from Technical and Programmatic Processes
                                                                           Proposing and or
                                                                         Identifying Decision
                                                                             Alternatives




                             Continuous Risk
                         Management (CRM) Process                          Risk Analysis of
                                                                        Decision Alternatives,
                                                                          Performing Trade
                                                                       Studies, & Quantification
                                                                        of Performance Index
                                                                                     See Chart 24
                                          Identify
                               rol




                                                             Analy
                           Cont




                                      Communicate,                         Deliberation and
                                      Deliberate, and
                                                                  ze



                                                                         Recommendation on
                                        Document                         Decision Alternative

                                                                                     See Chart 26
                              Tra
                                 ck                                                                                                                   Decision Making and
                                                         n
                                                     Pla                                                                                               Implementation of
                                                                                                                                                      Decision Alternative


                                                                       Tracking and Controlling
                                                                       Performance Deviations

                                                                                     See Chart 27




                                                                                                                                                                             20
Formulation of Objective Hierarchy and
Performance Measures (PMs)
                                                                                                         Mission Success




     Objectives
                                                                     Technical Objectives                                                                                               Other
     Hierarchy           Affordability                                                                                                              Safety
                                                                      and Performance                                                                                                Stakeholders
                                                                                                                                                                                       Support




                                                                                     Enhance             Provide             Protect                                   Protect       Realize Other
                         Meet Budget            Meet        Achieve Mission                                                                        Protect
                                                                                  Effectiveness/       Extensibility/      Workforce &                               Mission and     Stakeholders
                         Constraints          Schedules     Critical Functions                                                                   Environment
                                                                                  Performance          Supportability      Public Health                             Public Assets   Expectations




                           Design/                              Loss of                                                       Astronauts                                Loss of
                                              Schedule                              Mass/ Cargo         Commercial                                  Earth                                Public
                         Development                            Mission                                                        Death or                                  Flight
                                              Slippage                               Capacity           Extensibility                           Contamination                           Support
    Performance          Cost Overrun                          Function x                                                       Injury                                  Systems
     Measures
       (PMs)

    Representative
      PMs are                                                                                             Loss of                                  Planetary            Loss of        Science
                          Operation             ……..             ……..                Reliability/                             Public Death
       Shown                                                                                              Support                                Contamination           Public       Community
                         Cost Overrun                                                Availability                               or Injury
                                                                                                         Capability x                                                   Property       Support


                                                                                             Focus of Technical Risk Assessment


                      Economics and Schedule Models                                       Models Quantifying Performance Measures (PMs)                                              Stakeholder
    Model-Based                                                                                                                                                                        Models
                     Models to Assess Life Cycle Cost and
     Analysis of                                              Models Quantifying Capability Metrics Relating to Mission        Models Quantifying Metrics Relating to Frequency
                           Schedule Performance
    Performance                                               Requirements (Mass, Thrust, Cargo Capacity, …)                   or Probability of Failure to Meet High-Level Safety
      Measures                                                                                                                 Objectives (e.g., PRA)
                                                              Models Quantifying Metrics Relating to Probability of
                                                              Loss Mission Critical System / Function (e.g., PRA)




                                                                                                                 Decision
                                                                                                                Alternative




                                                                                                                                                                                                     21
PMs of Interest to the Decision Situation
Discussed Earlier


                                                                              Mission Success




 Objectives
                                                    Technical Objectives                                                                    Other
 Hierarchy     Affordability                                                                                    Safety
                                                     and Performance                                                                     Stakeholders
                                                                                                                                           Support




                                                                                                  Protect                   Protect
                                 Meet      Achieve Mission
                                                                                                Workforce &               Mission and
                               Schedules   Critical Functions
                                                                                                Public Health            Public Assets




                                               Loss of                                            Astronauts
                               Schedule                                                                                    Loss of
                                               Mission                                             Death or
                               Slippage                                                                                   Vehicles(s)
                                              Function x                                            Injury
 Performance
  Measures
    (PMs)

                                                                                                 Public Death               Station
                                                                                                   or Injury              Evacuation
                                                                Amenable to PRA




                                                                                                                                                        22
PMs of Interest to NASA's Exploration Systems
Architecture Study (ESAS)




                                           In ESAS study, the
                                           PMs were referred
                                           to as “Figures of
                                           Merit (FOMs)”




    Source: ESAS Report

                                                                23
Risk Analysis of Decision Alternatives

                                  Examples of Decisions
             Architecture A vs. Architecture B vs. Architecture C
             Technology A vs. Technology B
             Making changes to existing systems
             Extending the life of existing systems
             Changing requirements
             Responding to operational occurrences in real time                                                                Deliberation and
             Prioritization
             Contingency Plan A vs. Contingency Plan B
                                                                                                                             Ranking / Selection of
             Launch or No Launch                                                                                             Preferred Alternative


                    Decision
                  Alternatives
                  For Analysis               Identify               Qualitative
                                                                                                                                   Risk and PM
                                               Analyze              Techniques                                                       Results


                                                                      Available Techniques
                                                                          Spectrum of
                     Scoping &                                                                                        Is the
                   Determination of                                                            Preliminary
                                                                                                                     Ranking /     Yes
                   Methods To Be                                                               Risk & PM
                                                                                                                    Comparison
                        Used                                                                     Results
                                                                                                                     Robust?


                                                                                                                            No
                                               Identify             Quantitative                                        Cost-
                                            Analyze                                                                                 No
                                                                    Techniques                                       Beneficial
                                                                                                                     to Reduce
                                                                                                                    Uncertainty?

                                                                    Risk Analysis
                                                                     Techniques                                            Yes
   PRA                      Iteration
Techniques
                                                   Additional Uncertainty Reduction If Necessary Per Stakeholders




                                                                                                                                                      24
Implications of Uncertainty in Comparing
Alternatives


                      PDF(PM(Alt 1)) & PDF(PM(Alt2))
                                                                                                            PDF(PM(Alt1)) & PDF(PM(Alt2))
                     3




                                                                                         PDF(Performance
  PDF(Performance




                    2.5                                                                                      1
                                                                                                           0.8




                                                                                            Measure)
     Measure




                      2
                                                                         Alternative 1                     0.6                                          Alternative 1
                    1.5
                                                                         Alternative 2                     0.4                                          Alternative 2
                      1
                                                                                                           0.2
                    0.5
                                                                                                             0
                     0
                                                                                                                 0          1        2        3     4
                          0          1        2            3         4
                              Good                             Bad                                                   Good   Performance Measure   Bad
                                     Performance Measure




                          PDF (PM (Alt X)) : Probability Density Function of Performance Measure associated with Decision Alternative X



  Alt 2 is clearly better than Alt 1                                                     Alt 1 is slightly better on average, but
                                                                                         uncertainty in performance implies
  Assuming that the uncertainty                                                          some probability that Alt 2 is actually
  distributions reflect the actual state                                                 better than Alt 1
  of knowledge, it is very unlikely that
  uncertainty reduction will improve                                                     Uncertainty reduction will improve
  the decision                                                                           the decision
                                                                                                                                                                        25
Deliberation and RM Planning

                                                                                   Need for                      Need to
                              Risk & TPM        Present Preliminary                                No                           No
                                                                                  Additional                  Refine / Adjust
                                                    Results to
                                Results                                           Uncertainty                   Decision
                                                   Stakeholders
                                                                                  Reduction?                  Alternatives?


                                                                                 Yes                           Yes
                                     Analysis


•   Deliberation                                                             Selection

     –   To make use of collective wisdom to                          Furnish Recommendation to
                                                                      Decision-Maker
         promote selection of the best                                Capture Basis
                                                                                                        Decision Making and
                                                                                                         Implementation of
         alternative for actual                                       Develop / Update Risk             Decision Alternative
                                                                      Management Plan
         implementation                                               Refine Metrics and Develop
                                                                      Monitoring Thresholds
•   RM Planning
     –   Identified and analyzed risks are
         managed in one of four possible
         actions (Mitigate, Research, Watch,
         or Accept)
     –   A portfolio of observables and
         thresholds needs to be identified
           • Measurable (or calculable)
             parameters
           • provides timely indication of
             performance issues
     –   RM Protocols are determined
     –   Responsibility for Tracking
         activities is assigned

                                                                                                                                     26
Monitoring Performance Deviations

                                    Decision Making and
                                     Implementation of
                                    Decision Alternative




                                                 Performance Monitoring and Feedback / Control
•    Detection of significant                                Track                       Control

     deviations from program intent                     Trend Performance
                                                        Based on PMs and
                                                                                         Intervene if
                                                                                        Performance
     in a timely fashion, without                       Thresholds or Other
                                                          Decision Rules
                                                                                    Thresholds Exceeded


     over-burdening the program
•    Mitigation/Control of risk when                                                             Identify
                                                                                              Iterate Continuous
     a performance threshold is                  Metrics & Thresholds from “Plan”             Risk Management
                                                                                                    Process
     exceeded
     – A perceived insignificant risk
       “issue” becomes significant
     – Emergence of new risk “issues”




                                                                                                                   27
The Role of Probabilistic Risk Assessment (PRA)
              in Risk Management




                                                  28
PRA can be a Powerful Tool for Risk
Management
 •   Quantifies goal-related performance measures
     –   Probability of Loss of Crew (LOC) versus Reliability Analysis of
         sub-systems
     –   PRA metrics are integral risk metrics as opposed to limited
         metrics such as sub-system reliability
 •   Captures dependences and other relationships between sub-
     systems
 •   Works within a scenario-based concept of risk that best
     informs decision-making
     –   Identifies contributing elements (initiating events, pivotal events,
         basic events)
     –   Quantifies the risk significance of contributing elements, helping
         focus on where improvements will be effective
     –   Provides a means of re-allocating analytical priorities according to
         where the dominant risk contributors appear to be coming from
     –   Provides a framework for a monitoring / trending program to
         detect risk-significant adverse trends in performance
                                                                                29
PRA can be a Powerful Tool for Risk
Management (cont.)

 •   Quantifies uncertainty and ranks contributors to uncertainty
 •   Supports trade studies by quantifying
     –   The most goal-related metrics
     –   System interfaces and dependencies
     –   Responses to system and function challenges
     –   Effects of varying performance levels of different systems




                                                                      30
PRA Methodology Synopsis
Understanding of Consequence of Interest                                                                    Master Logic Diagram (Hierarchical Structure)                             Event Sequence Diagram (Inductive Logic)
(Performance Measures (PMs)) to Decision- maker

                                                                   End State: ES1                                                                                                IE                  A                   B     End State: OK
                                                               End State: ES2
                                                              End State: ES2
                                                                                                                                                                                                End State: ES2



                                                                                                                                                                                        C            D               E       End State: ES1



                                                                                                                                                                                                                             End State: ES2




                                                                                                      Modeling of Pivotal Events Using Techniques such as                       Modeling of Basic Events Using Various Techniques
                           Event Tree (Inductive Logic)
                                                                                                      Fault Tree                                                                       (Some Examples are Shown Below)
                                                                            End                                                       Not A
         IE            A         B           C         D           E
                                                                            State
                                                                                                                                                                                                  1
                                                                                                       Logic Gate                                                                 Q(km ) =             α Q
                                                                           1: OK                                                                                  Basic Event                 ⎛ m - 1⎞ k t
                                                                                                                                                                                              ⎜      ⎟
                                                                           2: ES1
                                                                                                                                                                                              ⎝ k - 1⎠
                                                                           3: ES2

                                                                           4: ES2

                                                                           5: ES2
                                                                                                                                                                                                Prf ( t ) = 1 − e − λt
                                                                           6: ES2
                                                                                                                             Link to another fault tree




               Probabilistic Modeling of Basic Events                                                                                                                                  Communicating Risk Results and Insights to
                                                                                                                  Model Integration and Quantification
                                 30                           50                                                                                                                                   Decision-maker
60
                                 25                           40
50
                                 20                                                             100              End State: ES2
40                                                            30
                                 15
30
20                               10                           20                                 80                                                                                   Reporting PMs and associated uncertainties
                                  5                           10                                                           End State: ES1
10
                                                                                                 60                                                                                   Ranking of risk scenarios
        0.01    0.02   0.03   0.04    0.02   0.04   0.06   0.08     0.02   0.04   0.06   0.08                                                        Quantification of PMs            Ranking of risk significance contributors (e.g.,
                                                                                                 40
                                                                                                                                                     and propagation of               hardware failure, human errors, etc.)
     Examples:
     Probability that the hardware x fails when needed
                                                                                                 20
                                                                                                                                                     epistemic uncertainties          Insights into how various systems interact
     Probability of nozzle burn-through                                                                   0.01      0.02    0.03    0.04      0.05                                    Tabulation of key modeling assumptions
     Probability of common cause failure of two redundant devices
                                                                                                                                                                                      Identification of key contributors to uncertainty
      The uncertainty in occurrence frequency of an event                                                                                                                             Proposing candidate risk reduction strategies
          is characterized by a probability distribution

                                                                                                                                                                                                                                               31
Integrated Nature of PRA Addresses Some
ASAP Concerns

•   Some of the pros and cons of the example discussed earlier
    would have been modeled explicitly in an integrated risk
    analysis
•   Performance uncertainties are included explicitly
•   Treating the issues as consistently as possible, within an
    integrated framework making use of all available information,
    is an improvement over mapping each issue separately into a
    5x5 and trying to trade them off against each other




                                                                    32
Summary

•   The proposed RM approach is based on an analytic-
    deliberative decision-making methodology
•   Embeds current Continuous Risk Management (CRM) process
    in a broader decision analysis framework
•   It is analytical because
    –   It promotes analyses of the consequences of decision options in
        terms of PMs relating to program fundamental objectives
    –   It promotes the use of analytical methods rather than judgment,
        wherever the methods are practical
    –   It promotes systematic incorporation of decision maker’s
        preferences (values) into decision-alternative ranking process
•   It is deliberative because
    –   Allows the consideration of elements that have not been captured
        by the formal analysis
    –   Provides an opportunity to scrutinize the modeling assumptions
        of the analysis and the relevant uncertainties

                                                                           33
Acronyms

•   Alt.     Decision Alternative
•   Cat.     Catastrophic
•   CDRA     Carbon Dioxide Removal Assembly
•   ASAP     Aerospace Safety Advisory Panel
•   Crit.    Critical
•   CRM      Continuous Risk Management
•   CSCS     Contingency Shuttle Crew Support
•   ECLSS    Environmental Control and Life Support Systems
•   ESAS     Exploration Systems Architecture Study
•   EVA      Extravehicular Activity
•   FOM      Figure of Merit
•   HAC      Heading Alignment Circle
•   IE       Initiating Event
•   ISS      International Space Station
•   LiOH     Lithium Hydroxide
•   LOC      Loss of Crew
•   Marg.    Marginal
•   PDF      Probability Density Function
•   PRA      Probabilistic Risk Assessment
•   PM       Performance Measure
•   RM       Risk Management
•   STA-54   Shuttle Tile Ablator 54
•   STS      Space Transportation System



                                                              34

Más contenido relacionado

La actualidad más candente

Terry.conroy
Terry.conroyTerry.conroy
Terry.conroyNASAPMC
 
Vonnie simonsen
Vonnie simonsenVonnie simonsen
Vonnie simonsenNASAPMC
 
Kirsch.mike
Kirsch.mikeKirsch.mike
Kirsch.mikeNASAPMC
 
Newman.steve
Newman.steveNewman.steve
Newman.steveNASAPMC
 
Gaydar.michael
Gaydar.michaelGaydar.michael
Gaydar.michaelNASAPMC
 
Charles.armstrong
Charles.armstrongCharles.armstrong
Charles.armstrongNASAPMC
 
Bauer.frank
Bauer.frankBauer.frank
Bauer.frankNASAPMC
 
Richards.robert
Richards.robertRichards.robert
Richards.robertNASAPMC
 
King richardson
King richardsonKing richardson
King richardsonNASAPMC
 
Geyer.m.sasaki.c
Geyer.m.sasaki.cGeyer.m.sasaki.c
Geyer.m.sasaki.cNASAPMC
 
Bladwin.kristen
Bladwin.kristenBladwin.kristen
Bladwin.kristenNASAPMC
 
C armstrong tbyers
C armstrong tbyersC armstrong tbyers
C armstrong tbyersNASAPMC
 
Sally godfreyheatherrarick
Sally godfreyheatherrarickSally godfreyheatherrarick
Sally godfreyheatherrarickNASAPMC
 
Homayoon.dezfuli
Homayoon.dezfuliHomayoon.dezfuli
Homayoon.dezfuliNASAPMC
 
Thomas.mc vittie
Thomas.mc vittieThomas.mc vittie
Thomas.mc vittieNASAPMC
 
Dezfuli.homayoon
Dezfuli.homayoonDezfuli.homayoon
Dezfuli.homayoonNASAPMC
 
Bilardo2 15-2012
Bilardo2 15-2012Bilardo2 15-2012
Bilardo2 15-2012NASAPMC
 
K.pagel.beene
K.pagel.beeneK.pagel.beene
K.pagel.beeneNASAPMC
 
Chen.tim
Chen.timChen.tim
Chen.timNASAPMC
 
Mike.ryschkewitsch
Mike.ryschkewitschMike.ryschkewitsch
Mike.ryschkewitschNASAPMC
 

La actualidad más candente (20)

Terry.conroy
Terry.conroyTerry.conroy
Terry.conroy
 
Vonnie simonsen
Vonnie simonsenVonnie simonsen
Vonnie simonsen
 
Kirsch.mike
Kirsch.mikeKirsch.mike
Kirsch.mike
 
Newman.steve
Newman.steveNewman.steve
Newman.steve
 
Gaydar.michael
Gaydar.michaelGaydar.michael
Gaydar.michael
 
Charles.armstrong
Charles.armstrongCharles.armstrong
Charles.armstrong
 
Bauer.frank
Bauer.frankBauer.frank
Bauer.frank
 
Richards.robert
Richards.robertRichards.robert
Richards.robert
 
King richardson
King richardsonKing richardson
King richardson
 
Geyer.m.sasaki.c
Geyer.m.sasaki.cGeyer.m.sasaki.c
Geyer.m.sasaki.c
 
Bladwin.kristen
Bladwin.kristenBladwin.kristen
Bladwin.kristen
 
C armstrong tbyers
C armstrong tbyersC armstrong tbyers
C armstrong tbyers
 
Sally godfreyheatherrarick
Sally godfreyheatherrarickSally godfreyheatherrarick
Sally godfreyheatherrarick
 
Homayoon.dezfuli
Homayoon.dezfuliHomayoon.dezfuli
Homayoon.dezfuli
 
Thomas.mc vittie
Thomas.mc vittieThomas.mc vittie
Thomas.mc vittie
 
Dezfuli.homayoon
Dezfuli.homayoonDezfuli.homayoon
Dezfuli.homayoon
 
Bilardo2 15-2012
Bilardo2 15-2012Bilardo2 15-2012
Bilardo2 15-2012
 
K.pagel.beene
K.pagel.beeneK.pagel.beene
K.pagel.beene
 
Chen.tim
Chen.timChen.tim
Chen.tim
 
Mike.ryschkewitsch
Mike.ryschkewitschMike.ryschkewitsch
Mike.ryschkewitsch
 

Destacado

David.fuller
David.fullerDavid.fuller
David.fullerNASAPMC
 
Norm smith
Norm smithNorm smith
Norm smithNASAPMC
 
Gill.paul
Gill.paulGill.paul
Gill.paulNASAPMC
 
J brennan kmc_laurin_aprince
J brennan kmc_laurin_aprinceJ brennan kmc_laurin_aprince
J brennan kmc_laurin_aprinceNASAPMC
 
Gerst international
Gerst internationalGerst international
Gerst internationalNASAPMC
 
Baniszewski john
Baniszewski johnBaniszewski john
Baniszewski johnNASAPMC
 
James.smith
James.smithJames.smith
James.smithNASAPMC
 

Destacado (7)

David.fuller
David.fullerDavid.fuller
David.fuller
 
Norm smith
Norm smithNorm smith
Norm smith
 
Gill.paul
Gill.paulGill.paul
Gill.paul
 
J brennan kmc_laurin_aprince
J brennan kmc_laurin_aprinceJ brennan kmc_laurin_aprince
J brennan kmc_laurin_aprince
 
Gerst international
Gerst internationalGerst international
Gerst international
 
Baniszewski john
Baniszewski johnBaniszewski john
Baniszewski john
 
James.smith
James.smithJames.smith
James.smith
 

Similar a Dezfuli youngblood

Ashley.edwards
Ashley.edwardsAshley.edwards
Ashley.edwardsNASAPMC
 
Risk-Management-05012023-025512pm.ppt
Risk-Management-05012023-025512pm.pptRisk-Management-05012023-025512pm.ppt
Risk-Management-05012023-025512pm.pptYasirShaikh34
 
Homayoon.dezfuli
Homayoon.dezfuliHomayoon.dezfuli
Homayoon.dezfuliNASAPMC
 
R I S K M A N A G E M E N T
R I S K  M A N A G E M E N TR I S K  M A N A G E M E N T
R I S K M A N A G E M E N Tmadhav shirur
 
Perform qualitative risk analysis
Perform qualitative risk analysis Perform qualitative risk analysis
Perform qualitative risk analysis Shereef Sabri
 
Peoject Risk Management(quantative&qualitive)
Peoject Risk Management(quantative&qualitive)Peoject Risk Management(quantative&qualitive)
Peoject Risk Management(quantative&qualitive)Evgeni Tsonev
 
Microsoft InfoSec for cloud and mobile
Microsoft InfoSec for cloud and mobileMicrosoft InfoSec for cloud and mobile
Microsoft InfoSec for cloud and mobileVijayananda Mohire
 
Cynthia.calhoun
Cynthia.calhounCynthia.calhoun
Cynthia.calhounNASAPMC
 
Risk.panel handout
Risk.panel handoutRisk.panel handout
Risk.panel handoutNASAPMC
 
Ranga's 51 hot topics to remember for Risk Management Professional (RMP) exam
Ranga's 51 hot topics to remember for Risk Management Professional (RMP) examRanga's 51 hot topics to remember for Risk Management Professional (RMP) exam
Ranga's 51 hot topics to remember for Risk Management Professional (RMP) examrrangraj
 
Repeatable Risk Identification - Paper
Repeatable Risk Identification - PaperRepeatable Risk Identification - Paper
Repeatable Risk Identification - PaperDaniel Ackermann
 
Dezfuli.homayoon
Dezfuli.homayoonDezfuli.homayoon
Dezfuli.homayoonNASAPMC
 
Critical role of_risk_assessment_in_international_projects_en
Critical role of_risk_assessment_in_international_projects_enCritical role of_risk_assessment_in_international_projects_en
Critical role of_risk_assessment_in_international_projects_enVyacheslav Guzovsky
 
Risk management overview
Risk management overviewRisk management overview
Risk management overviewHossam Maghrabi
 
PROJECT RISK MANAGEMENT.pdf
PROJECT RISK MANAGEMENT.pdfPROJECT RISK MANAGEMENT.pdf
PROJECT RISK MANAGEMENT.pdfMUST
 

Similar a Dezfuli youngblood (20)

Ashley.edwards
Ashley.edwardsAshley.edwards
Ashley.edwards
 
Risk-Management-05012023-025512pm.ppt
Risk-Management-05012023-025512pm.pptRisk-Management-05012023-025512pm.ppt
Risk-Management-05012023-025512pm.ppt
 
Homayoon.dezfuli
Homayoon.dezfuliHomayoon.dezfuli
Homayoon.dezfuli
 
R I S K M A N A G E M E N T
R I S K  M A N A G E M E N TR I S K  M A N A G E M E N T
R I S K M A N A G E M E N T
 
Perform qualitative risk analysis
Perform qualitative risk analysis Perform qualitative risk analysis
Perform qualitative risk analysis
 
11 project risk management
11 project risk management11 project risk management
11 project risk management
 
Peoject Risk Management(quantative&qualitive)
Peoject Risk Management(quantative&qualitive)Peoject Risk Management(quantative&qualitive)
Peoject Risk Management(quantative&qualitive)
 
11 risk management
11 risk management11 risk management
11 risk management
 
Project Risk Management
Project Risk ManagementProject Risk Management
Project Risk Management
 
Microsoft InfoSec for cloud and mobile
Microsoft InfoSec for cloud and mobileMicrosoft InfoSec for cloud and mobile
Microsoft InfoSec for cloud and mobile
 
Cynthia.calhoun
Cynthia.calhounCynthia.calhoun
Cynthia.calhoun
 
Risk.panel handout
Risk.panel handoutRisk.panel handout
Risk.panel handout
 
Ranga's 51 hot topics to remember for Risk Management Professional (RMP) exam
Ranga's 51 hot topics to remember for Risk Management Professional (RMP) examRanga's 51 hot topics to remember for Risk Management Professional (RMP) exam
Ranga's 51 hot topics to remember for Risk Management Professional (RMP) exam
 
Presentation qrm shc
Presentation qrm shcPresentation qrm shc
Presentation qrm shc
 
Repeatable Risk Identification - Paper
Repeatable Risk Identification - PaperRepeatable Risk Identification - Paper
Repeatable Risk Identification - Paper
 
Dezfuli.homayoon
Dezfuli.homayoonDezfuli.homayoon
Dezfuli.homayoon
 
Critical role of_risk_assessment_in_international_projects_en
Critical role of_risk_assessment_in_international_projects_enCritical role of_risk_assessment_in_international_projects_en
Critical role of_risk_assessment_in_international_projects_en
 
Risk management overview
Risk management overviewRisk management overview
Risk management overview
 
PROJECT RISK MANAGEMENT.pdf
PROJECT RISK MANAGEMENT.pdfPROJECT RISK MANAGEMENT.pdf
PROJECT RISK MANAGEMENT.pdf
 
Session 18 4th edition PMP
Session 18 4th edition PMPSession 18 4th edition PMP
Session 18 4th edition PMP
 

Más de NASAPMC

Bejmuk bo
Bejmuk boBejmuk bo
Bejmuk boNASAPMC
 
Baniszewski john
Baniszewski johnBaniszewski john
Baniszewski johnNASAPMC
 
Yew manson
Yew mansonYew manson
Yew mansonNASAPMC
 
Wood frank
Wood frankWood frank
Wood frankNASAPMC
 
Wood frank
Wood frankWood frank
Wood frankNASAPMC
 
Wessen randi (cd)
Wessen randi (cd)Wessen randi (cd)
Wessen randi (cd)NASAPMC
 
Vellinga joe
Vellinga joeVellinga joe
Vellinga joeNASAPMC
 
Trahan stuart
Trahan stuartTrahan stuart
Trahan stuartNASAPMC
 
Stock gahm
Stock gahmStock gahm
Stock gahmNASAPMC
 
Smalley sandra
Smalley sandraSmalley sandra
Smalley sandraNASAPMC
 
Seftas krage
Seftas krageSeftas krage
Seftas krageNASAPMC
 
Sampietro marco
Sampietro marcoSampietro marco
Sampietro marcoNASAPMC
 
Rudolphi mike
Rudolphi mikeRudolphi mike
Rudolphi mikeNASAPMC
 
Roberts karlene
Roberts karleneRoberts karlene
Roberts karleneNASAPMC
 
Rackley mike
Rackley mikeRackley mike
Rackley mikeNASAPMC
 
Paradis william
Paradis williamParadis william
Paradis williamNASAPMC
 
Osterkamp jeff
Osterkamp jeffOsterkamp jeff
Osterkamp jeffNASAPMC
 
O'keefe william
O'keefe williamO'keefe william
O'keefe williamNASAPMC
 
Muller ralf
Muller ralfMuller ralf
Muller ralfNASAPMC
 
Mulenburg jerry
Mulenburg jerryMulenburg jerry
Mulenburg jerryNASAPMC
 

Más de NASAPMC (20)

Bejmuk bo
Bejmuk boBejmuk bo
Bejmuk bo
 
Baniszewski john
Baniszewski johnBaniszewski john
Baniszewski john
 
Yew manson
Yew mansonYew manson
Yew manson
 
Wood frank
Wood frankWood frank
Wood frank
 
Wood frank
Wood frankWood frank
Wood frank
 
Wessen randi (cd)
Wessen randi (cd)Wessen randi (cd)
Wessen randi (cd)
 
Vellinga joe
Vellinga joeVellinga joe
Vellinga joe
 
Trahan stuart
Trahan stuartTrahan stuart
Trahan stuart
 
Stock gahm
Stock gahmStock gahm
Stock gahm
 
Smalley sandra
Smalley sandraSmalley sandra
Smalley sandra
 
Seftas krage
Seftas krageSeftas krage
Seftas krage
 
Sampietro marco
Sampietro marcoSampietro marco
Sampietro marco
 
Rudolphi mike
Rudolphi mikeRudolphi mike
Rudolphi mike
 
Roberts karlene
Roberts karleneRoberts karlene
Roberts karlene
 
Rackley mike
Rackley mikeRackley mike
Rackley mike
 
Paradis william
Paradis williamParadis william
Paradis william
 
Osterkamp jeff
Osterkamp jeffOsterkamp jeff
Osterkamp jeff
 
O'keefe william
O'keefe williamO'keefe william
O'keefe william
 
Muller ralf
Muller ralfMuller ralf
Muller ralf
 
Mulenburg jerry
Mulenburg jerryMulenburg jerry
Mulenburg jerry
 

Último

Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintMahmoud Rabie
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024D Cloud Solutions
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Commit University
 
AI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarAI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarPrecisely
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaborationbruanjhuli
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxUdaiappa Ramachandran
 
Designing A Time bound resource download URL
Designing A Time bound resource download URLDesigning A Time bound resource download URL
Designing A Time bound resource download URLRuncy Oommen
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Websitedgelyza
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding TeamAdam Moalla
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-pyJamie (Taka) Wang
 
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.YounusS2
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureEric D. Schabell
 
Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Brian Pichman
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsSeth Reyes
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostMatt Ray
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...DianaGray10
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDELiveplex
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxGDSC PJATK
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdfPedro Manuel
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IES VE
 

Último (20)

Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership Blueprint
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)
 
AI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarAI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity Webinar
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptx
 
Designing A Time bound resource download URL
Designing A Time bound resource download URLDesigning A Time bound resource download URL
Designing A Time bound resource download URL
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Website
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-py
 
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability Adventure
 
Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and Hazards
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptx
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdf
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
 

Dezfuli youngblood

  • 1. Integration of Technical Risk Management with Decision Analysis Presented at NASA Project Management Challenge 2007 Conference Galveston, Texas February 6-7, 2007 Homayoon Dezfuli, Ph.D., NASA HQ (hdezfuli@nasa.gov) Robert Youngblood, Ph.D., ISL, Inc. (ryoungblood@islinc.com)
  • 2. Acknowledgement • Some of the materials in this presentation are based on methodology developed by the Office of Safety and Mission assurance in support of the following activities: – Revision of NASA requirements for system safety (NPR 8715.3A dated September 12, 2006) – Revision of NASA System Engineering Handbook (to be released June 2007) – Development of a new NASA standard entitled “Risk-informed Management of Safety and Mission Success” (planned for release before the end of fiscal year 2007) 2
  • 3. Why Integrate Technical Risk Management with Decision Analysis? Excerpt from 2006 Third Quarterly Aerospace Safety Advisory Panel (ASAP) Report “With regards to risk assessments that are being made to support launch decisions, it appears that a series of fragmented, non standardized tools and methodologies are in use. The result is that risk recommendations to senior management concerning individual hazards effecting launch are sometimes made in isolation without consideration of overall launch risk. For example, the most recent Shuttle launch focused heavily on two of the 569 potentially catastrophic hazards currently known to exist, without any assessment of the overall likelihood of such a catastrophic failure. A lack of confidence in the technical basis for the assessments also appears to sometimes exist and variations in risk matrix definitions among programs has been observed. Lastly, only limited guidance is available concerning agency policies on what risks should be accepted under what conditions. The ASAP recommends that a comprehensive risk assessment, communication and acceptance process be implemented to ensure that overall launch risk is considered in an integrated and consistent manner. The process should be sound, mature, consistently implemented to yield high confidence and consistent results that are generally accepted by the majority of the community.” 3
  • 4. Key Points • Decision analysis can be a powerful tool for aligning decisions with program objectives, given the current state of knowledge and the decision-maker’s preferences • Modern risk analysis methodology can significantly improve on the one-at-a-time management of risk issues criticized by the ASAP report • Therefore: – Perform Technical Risk Management within a decision analysis framework – Use modern risk analysis methodology, including probabilistic risk assessment (PRA) – Retain the track, control, and accountability strengths of the Continuous Risk Management (CRM) Process 4
  • 5. Outline • Background • State of Practice of Risk Management (RM) at NASA • An Example of How Risks are Handled in Decision Processes • Need for a More Rigorous Approach to Inform Risk Management Decisions • An Analytical Framework to Inform Risk Management Decisions • The Role of Probabilistic Risk Assessment (PRA) in Risk Management • PRA Methodology Synopsis • Summary 5
  • 6. Background • NASA is embarking on a comprehensive space exploration program • The goals of the exploration program are “safe, sustained, affordable human and robotic exploration of the Moon, Mars, and beyond ... for less than one percent of the federal budget” • Meeting these goals requires development of a constellation of new systems • The design and development of these systems will involve many decisions that require weighting/trading various competing programmatic and technical considerations against one another 6
  • 7. Programmatic and Technical Considerations that Should be Factored into Our Decisions • Affordability – Meet budget constraints related to design and development cost, technology development cost, operation cost, and facility cost • Mission Technical Objectives and Performance – Accomplish technical objectives in a sustainable manner and on schedule – Enhance effectiveness and performance • Safety – Protect the health of public and workforce, the environment and mission assets • Stakeholder Expectations – Meet needs of intra-agency and extra-agency stakeholders 7
  • 8. The Role of Risk Management • Purpose of risk management is to promote program success – Incorporating risk-informed decisionmaking in the design and formulation of the program baseline – Proactive identification and control of departures from the program baseline • An effective and proactive RM process should – Provide input to determine the preferred decision alternative in light of programmatic objectives – Assess risk associated with implementation of the selected alternative – Assist in setting resource priorities (including prioritization of work to resolve uncertainties if warranted) – Plan, track, and control risk during the implementation of the selected alternative – Iterate with previous steps in light of new information 8
  • 9. State of Practice of Risk Management at NASA 9
  • 10. Continuous Risk Management (CRM) Process • Identify – Identify program risk “issues” Identify rol • Analyze – Estimate the likelihood and Anal Cont consequence components of the risk Communicate and yze issues Document Tra • Plan – Plan the Track and Control actions ck n Pla • Track – Track and compile the necessary risk data, measuring how the CRM process is progressing Current emphasis on • Control – Determine the appropriate Control action, execute the decision linked • “management” of individual to that action, and verify its effectiveness “risks,” given a decision already made somewhere else • Communicate and Document – communicating and documenting all risk • monitoring and accountability for information throughout each program action items associated with phase “risks” 10
  • 11. Risk Matrix Paradigm Iso-risk contours LIKELIHOOD These two points have the same expected risk • Risk “issues” are mapped onto the matrix individually – Interaction between risks is not considered – Total risk from all “issues” is not evaluated – Highly subjective; uncertainties are not formally accounted for – Unsuitable for combining risks to obtain aggregate risk • Risk tolerance boundaries are defined based on iso-risk contours – Expected consequences (probability times consequences) do not adequately inform decisions relating to safety • Limited ability to support risk-trade studies 11
  • 12. An Example of How Risks are Handled in Decision Processes 12
  • 13. STS-121 (OV-103 Discovery) Contingency Planning Decision Alternatives 1. Crew Rescue: – Success of contingency (rescue) flight (STS-300) – Success of Contingency Shuttle Crew Support (CSCS) capability (on-orbit safe haven) 2. Repair/Entry: – Successful application of patching materials (STA-54) to repair damaged heat shield – Vehicle integrity during reentry (no downstream heating damage) Source of Figure: Shuttle Program, NASA-JSC – Successful approach and landing (operations) 13
  • 14. Pros and Cons Associated with Decision Alternative 1 (Data Source: Shuttle Program) STS-300 CSCS OBSERVATIONS Adverse consequences of Interest are: • Loss of crew • Loss of vehicle (OV-103) • Evacuation of station It appears that Low (L), Medium (M), and High (H) are expressions of degree of belief in the truth of the proposition 14
  • 15. Pros and Cons Associated with Decision Alternative 2 (Data Source: Shuttle Program) Materials OBSERVATIONS • Additional adverse consequence of interest: Public death or injury • Acknowledgment of uncertainties (imperfect models and a limited state of knowledge) Vehicle Integrity Operations 15
  • 16. Which Decision Alternative is More Desirable? Source of Matrices: Shuttle Program, NASA-JSC COMMENTS • The shuttle program uses the above matrix to assign a level of risk to a hazard based on its likelihood and consequence severity • Catastrophic: hazard could result in a mishap causing fatal injury to personnel and/or loss of one or more major elements of the flight vehicle or ground facility • Infrequent: Could happen in the life of the program. Controls have significant limitations or uncertainties • The matrix is used here to analyze decisions • The definition of “catastrophic” does not discriminate between human safety and asset safety (defined as an inclusive consequence severity level) 16
  • 17. Need for a More Rigorous Approach to Inform Risk Management Decisions • Characteristics of our Decision Situations – They are complex: Need a systematic framework to sort things out – Must deal with uncertainty: Need to assess uncertainty analytically, know when it is necessary to reduce it – Must deal with multiple Objectives: Need to employ analytic decision techniques to handle competing priorities (apples vs. oranges) 17
  • 18. An Analytic Framework to Inform Risk Management Decisions • Model consequences of decision alternatives in terms of impact on fundamental objectives – Use performance Measures (PMs) as metrics to characterize performance of the decision alternatives with respect to a particular fundamental objective. Examples: • PM for crew safety is the probability of loss of crew • PM for space asset safety is the probability of loss of space vehicle • PM for public safety is the probability of public death or injury • PM for station occupancy is the probability of evacuation – Inclusion of uncertainties is critical in evaluating PMs • Compare the consequences of decision alternatives on the PMs • Collectively consider all PMs and associated uncertainties to inform risk management decisions (deliberation has an important role here) 18
  • 19. Decision Analysis Formulation of Objectives Hierarchy and Performance Measures (PMs) Input from Technical and Programmatic Processes Proposing and or Identifying Decision Alternatives Risk Analysis of Decision Alternatives, Performing Trade Studies, & Quantification of Performance Index Deliberation and Recommendation on Decision Alternative Decision Making and Implementation of Decision Alternative Tracking and Controlling Performance Deviations 19
  • 20. Integration of CRM Process with Decision Analysis Probabilistic Risk Decision Analysis Assessment Plays a Major Role in this Formulation of Objectives Hierarchy and Activity Performance Measures (PMs) See Chart 21 Input from Technical and Programmatic Processes Proposing and or Identifying Decision Alternatives Continuous Risk Management (CRM) Process Risk Analysis of Decision Alternatives, Performing Trade Studies, & Quantification of Performance Index See Chart 24 Identify rol Analy Cont Communicate, Deliberation and Deliberate, and ze Recommendation on Document Decision Alternative See Chart 26 Tra ck Decision Making and n Pla Implementation of Decision Alternative Tracking and Controlling Performance Deviations See Chart 27 20
  • 21. Formulation of Objective Hierarchy and Performance Measures (PMs) Mission Success Objectives Technical Objectives Other Hierarchy Affordability Safety and Performance Stakeholders Support Enhance Provide Protect Protect Realize Other Meet Budget Meet Achieve Mission Protect Effectiveness/ Extensibility/ Workforce & Mission and Stakeholders Constraints Schedules Critical Functions Environment Performance Supportability Public Health Public Assets Expectations Design/ Loss of Astronauts Loss of Schedule Mass/ Cargo Commercial Earth Public Development Mission Death or Flight Slippage Capacity Extensibility Contamination Support Performance Cost Overrun Function x Injury Systems Measures (PMs) Representative PMs are Loss of Planetary Loss of Science Operation …….. …….. Reliability/ Public Death Shown Support Contamination Public Community Cost Overrun Availability or Injury Capability x Property Support Focus of Technical Risk Assessment Economics and Schedule Models Models Quantifying Performance Measures (PMs) Stakeholder Model-Based Models Models to Assess Life Cycle Cost and Analysis of Models Quantifying Capability Metrics Relating to Mission Models Quantifying Metrics Relating to Frequency Schedule Performance Performance Requirements (Mass, Thrust, Cargo Capacity, …) or Probability of Failure to Meet High-Level Safety Measures Objectives (e.g., PRA) Models Quantifying Metrics Relating to Probability of Loss Mission Critical System / Function (e.g., PRA) Decision Alternative 21
  • 22. PMs of Interest to the Decision Situation Discussed Earlier Mission Success Objectives Technical Objectives Other Hierarchy Affordability Safety and Performance Stakeholders Support Protect Protect Meet Achieve Mission Workforce & Mission and Schedules Critical Functions Public Health Public Assets Loss of Astronauts Schedule Loss of Mission Death or Slippage Vehicles(s) Function x Injury Performance Measures (PMs) Public Death Station or Injury Evacuation Amenable to PRA 22
  • 23. PMs of Interest to NASA's Exploration Systems Architecture Study (ESAS) In ESAS study, the PMs were referred to as “Figures of Merit (FOMs)” Source: ESAS Report 23
  • 24. Risk Analysis of Decision Alternatives Examples of Decisions Architecture A vs. Architecture B vs. Architecture C Technology A vs. Technology B Making changes to existing systems Extending the life of existing systems Changing requirements Responding to operational occurrences in real time Deliberation and Prioritization Contingency Plan A vs. Contingency Plan B Ranking / Selection of Launch or No Launch Preferred Alternative Decision Alternatives For Analysis Identify Qualitative Risk and PM Analyze Techniques Results Available Techniques Spectrum of Scoping & Is the Determination of Preliminary Ranking / Yes Methods To Be Risk & PM Comparison Used Results Robust? No Identify Quantitative Cost- Analyze No Techniques Beneficial to Reduce Uncertainty? Risk Analysis Techniques Yes PRA Iteration Techniques Additional Uncertainty Reduction If Necessary Per Stakeholders 24
  • 25. Implications of Uncertainty in Comparing Alternatives PDF(PM(Alt 1)) & PDF(PM(Alt2)) PDF(PM(Alt1)) & PDF(PM(Alt2)) 3 PDF(Performance PDF(Performance 2.5 1 0.8 Measure) Measure 2 Alternative 1 0.6 Alternative 1 1.5 Alternative 2 0.4 Alternative 2 1 0.2 0.5 0 0 0 1 2 3 4 0 1 2 3 4 Good Bad Good Performance Measure Bad Performance Measure PDF (PM (Alt X)) : Probability Density Function of Performance Measure associated with Decision Alternative X Alt 2 is clearly better than Alt 1 Alt 1 is slightly better on average, but uncertainty in performance implies Assuming that the uncertainty some probability that Alt 2 is actually distributions reflect the actual state better than Alt 1 of knowledge, it is very unlikely that uncertainty reduction will improve Uncertainty reduction will improve the decision the decision 25
  • 26. Deliberation and RM Planning Need for Need to Risk & TPM Present Preliminary No No Additional Refine / Adjust Results to Results Uncertainty Decision Stakeholders Reduction? Alternatives? Yes Yes Analysis • Deliberation Selection – To make use of collective wisdom to Furnish Recommendation to Decision-Maker promote selection of the best Capture Basis Decision Making and Implementation of alternative for actual Develop / Update Risk Decision Alternative Management Plan implementation Refine Metrics and Develop Monitoring Thresholds • RM Planning – Identified and analyzed risks are managed in one of four possible actions (Mitigate, Research, Watch, or Accept) – A portfolio of observables and thresholds needs to be identified • Measurable (or calculable) parameters • provides timely indication of performance issues – RM Protocols are determined – Responsibility for Tracking activities is assigned 26
  • 27. Monitoring Performance Deviations Decision Making and Implementation of Decision Alternative Performance Monitoring and Feedback / Control • Detection of significant Track Control deviations from program intent Trend Performance Based on PMs and Intervene if Performance in a timely fashion, without Thresholds or Other Decision Rules Thresholds Exceeded over-burdening the program • Mitigation/Control of risk when Identify Iterate Continuous a performance threshold is Metrics & Thresholds from “Plan” Risk Management Process exceeded – A perceived insignificant risk “issue” becomes significant – Emergence of new risk “issues” 27
  • 28. The Role of Probabilistic Risk Assessment (PRA) in Risk Management 28
  • 29. PRA can be a Powerful Tool for Risk Management • Quantifies goal-related performance measures – Probability of Loss of Crew (LOC) versus Reliability Analysis of sub-systems – PRA metrics are integral risk metrics as opposed to limited metrics such as sub-system reliability • Captures dependences and other relationships between sub- systems • Works within a scenario-based concept of risk that best informs decision-making – Identifies contributing elements (initiating events, pivotal events, basic events) – Quantifies the risk significance of contributing elements, helping focus on where improvements will be effective – Provides a means of re-allocating analytical priorities according to where the dominant risk contributors appear to be coming from – Provides a framework for a monitoring / trending program to detect risk-significant adverse trends in performance 29
  • 30. PRA can be a Powerful Tool for Risk Management (cont.) • Quantifies uncertainty and ranks contributors to uncertainty • Supports trade studies by quantifying – The most goal-related metrics – System interfaces and dependencies – Responses to system and function challenges – Effects of varying performance levels of different systems 30
  • 31. PRA Methodology Synopsis Understanding of Consequence of Interest Master Logic Diagram (Hierarchical Structure) Event Sequence Diagram (Inductive Logic) (Performance Measures (PMs)) to Decision- maker End State: ES1 IE A B End State: OK End State: ES2 End State: ES2 End State: ES2 C D E End State: ES1 End State: ES2 Modeling of Pivotal Events Using Techniques such as Modeling of Basic Events Using Various Techniques Event Tree (Inductive Logic) Fault Tree (Some Examples are Shown Below) End Not A IE A B C D E State 1 Logic Gate Q(km ) = α Q 1: OK Basic Event ⎛ m - 1⎞ k t ⎜ ⎟ 2: ES1 ⎝ k - 1⎠ 3: ES2 4: ES2 5: ES2 Prf ( t ) = 1 − e − λt 6: ES2 Link to another fault tree Probabilistic Modeling of Basic Events Communicating Risk Results and Insights to Model Integration and Quantification 30 50 Decision-maker 60 25 40 50 20 100 End State: ES2 40 30 15 30 20 10 20 80 Reporting PMs and associated uncertainties 5 10 End State: ES1 10 60 Ranking of risk scenarios 0.01 0.02 0.03 0.04 0.02 0.04 0.06 0.08 0.02 0.04 0.06 0.08 Quantification of PMs Ranking of risk significance contributors (e.g., 40 and propagation of hardware failure, human errors, etc.) Examples: Probability that the hardware x fails when needed 20 epistemic uncertainties Insights into how various systems interact Probability of nozzle burn-through 0.01 0.02 0.03 0.04 0.05 Tabulation of key modeling assumptions Probability of common cause failure of two redundant devices Identification of key contributors to uncertainty The uncertainty in occurrence frequency of an event Proposing candidate risk reduction strategies is characterized by a probability distribution 31
  • 32. Integrated Nature of PRA Addresses Some ASAP Concerns • Some of the pros and cons of the example discussed earlier would have been modeled explicitly in an integrated risk analysis • Performance uncertainties are included explicitly • Treating the issues as consistently as possible, within an integrated framework making use of all available information, is an improvement over mapping each issue separately into a 5x5 and trying to trade them off against each other 32
  • 33. Summary • The proposed RM approach is based on an analytic- deliberative decision-making methodology • Embeds current Continuous Risk Management (CRM) process in a broader decision analysis framework • It is analytical because – It promotes analyses of the consequences of decision options in terms of PMs relating to program fundamental objectives – It promotes the use of analytical methods rather than judgment, wherever the methods are practical – It promotes systematic incorporation of decision maker’s preferences (values) into decision-alternative ranking process • It is deliberative because – Allows the consideration of elements that have not been captured by the formal analysis – Provides an opportunity to scrutinize the modeling assumptions of the analysis and the relevant uncertainties 33
  • 34. Acronyms • Alt. Decision Alternative • Cat. Catastrophic • CDRA Carbon Dioxide Removal Assembly • ASAP Aerospace Safety Advisory Panel • Crit. Critical • CRM Continuous Risk Management • CSCS Contingency Shuttle Crew Support • ECLSS Environmental Control and Life Support Systems • ESAS Exploration Systems Architecture Study • EVA Extravehicular Activity • FOM Figure of Merit • HAC Heading Alignment Circle • IE Initiating Event • ISS International Space Station • LiOH Lithium Hydroxide • LOC Loss of Crew • Marg. Marginal • PDF Probability Density Function • PRA Probabilistic Risk Assessment • PM Performance Measure • RM Risk Management • STA-54 Shuttle Tile Ablator 54 • STS Space Transportation System 34