Risk Management: Achieving Higher Maturity & Capability Levels through the LEGO Approach

www.eng.it
26°International Workshop on Software
Measurement (IWSM) and 11th International
Conference on Software Process and Product
Measurement (MENSURA)
Berlin (Germany) - October 5-7, 2016
Luigi Buglione
Alain Abran
Christiane Gresse von Wangenheim
Fergal McCaffery
Jean C.R. Hauck
Achieving Higher Maturity &
Capability Levels through the LEGO
Approach
Risk Management

www.eng.it2 IWSM-MENSURA 2016 – October 6, 2016
© 2016 Buglione; Abran, Gresse von Wangenheim, McCaffery, Hauck
Goals of the presentation
1. Discuss the impact an organization can suffer or
achieve from the way risk is managed
2. Look at the ‘big picture’ in order to convert Risks into
Critical Success Factors (CSFs) when dealing with risky
events looking at best practices from several frameworks on
Risk Management
3. Present a LEGO (Living EnGineering prOcess) example
with the Risk Management process
Risk Mgmt and LEGO

ETS - GELOG At a glance
www.etsmtl.ca

DKIT At a glance
Dundalk Institute of
Technology is a 90 acre
campus situated between
Dublin and Belfast (each
approximately 50 miles
away).
The Institute consists of 4
Schools:
1. Business & Humanities
2. Informatics & Creative
Arts
3. Engineering
4. Health & ScienceThe Regulated Software Research Group is part of
LERO (the Irish Software Engineering Research
Centre) at the School of Informatics & Creative
Media

UFSC At a glance
Federal University of Santa Catarina
Florianópolis/Brazil [http://www.ufsc.br]
• 25,737 Undergraduate students
• 8,543 Graduate students
• 34,280 Students
INCoD an institute for excellence in research, validation and dissemination
to support digital convergence. [http://www.incod.ufsc.br]
The Software Quality Group focuses on scientific research, development and
transfer of SE models, methods & tools. [http://www.gqs.ufsc.br]
[http://www.youtube.com/watch?v=V6E1Z5DEuvk]

Engineering At a glance
www.eng.it
ISSRE 2014 – Naples (Italy), Nov 5, 2014

Risk Mgmt and LEGO Let’s Social...ize!
If you want to share
comments/notes/pics…
 @IWSMMensura
 @lbu_measure
 #LEGO
 #MCM
 #Risk
 #RiskManagement
…

Agenda
• Introduction
– A couple of examples about (non) Risk Management…
– Some questions…
• MCMs (Maturity & Capability Models) – Representations & Dimensions
– Why do we need to choose a MCM?
– Coverage & classification of MCMs
• MCMs & Risk Management in Horizontal MCMs (H-MCMs)
– CMMI-DEV/SVC and ISO 15504-2
– Other Sources
• LEGO and Risk Management
– The LEGO approach
– Applying LEGO to Risk Management Elements of Interest (EoI)
– Suggested Improvements
• Conclusions & Prospects
• Q & A
Risk Mgmt and LEGO

Example: latest earthquake in Italy (Sept 2016)Introduction
• 6.2 Richter scale
• 290+ people died
• 2000+ people without home right now
• Did somebody consider such risk in the
past within Italy? How was risk
managed? Did the Government invest
over this past few years in reducing the
chances of such events happening?
Amatrice

Example: Apple ‘Antenna Gate’ (2010)Introduction
• At the iPhone 4 launch (June 2010) [https://en.wikipedia.org/wiki/IPhone_4#Antenna]
• Placed in the wrong place, the signal was lower and the iPhone less performant
• The ‘AntennaGate’ was estimated to impact for 20% of Apple sales for iPhone 4
(http://fortune.com/2010/09/08/antennagate-cost-apple-20-of-sales/)
• Did (Apple) they managed such risk during the Design phase? How? How much?

Some (important) questions...Introduction
What is risk and what is a damage?
E.g...what are the differences between CMMI and
SPICE manage risks?
Are there further frameworks helping to better deal with
risks? Do we have a risk catalogue?
How much value could we achieve converting risks
into a CSF?

Agenda
• Introduction
– Other Sources
• Q & A
Risk Mgmt and LEGO

Why do we need choosing a MCMs?MCMs

Representations - StagedMCMs
• ML: 5
• PA: 24
• N.min PA : ML1 (0)
• N.max PA : ML3 (13)
ML Focus Id. PA Title
5 Optimizing OPM Organizational Performance Management
CAR Causal Analysis & Resolution
4 Predictable OPP Organizational Process Performance
QPM Quantitative Project Management
3 Defined RD Requirement Development
TS Technical Solution
PI Product Integration
VAL Validation
VER Verification
OPD Organizational Process Definition
OPF Organizational Process Focus
OT Organizational Training
IPM Integrated Project Management
RSKM Risk Management
DAR Decision Analysis & Resolution
2 Managed REQM Requirement Management
PP Project Planning
PMC Project Monitoring & Control
SAM Supplier Agreement Management
MA Measurement & Analysis
PPQA Process & Product Quality Assurance

Representations - ContinuousMCMs
• PA categories: 4
• PA: 24  22
• N.min PA per Category : Process Management (5)
• N.max PA per Category: Project Management (7)
Process Categories
Maturity Levels
Process
Management
Project
Management
Engineering Support
Optimizing OPM CAR
Predictable OPP QPM
Defined OPF
OPD
OT
IPM
RKSM
RD
TS
PI
VER
VAL
DAR
Managed PP
PMC
SAM
REQM CM
MA
PPQA
Initial Ad-hoc processes

Representations – Continuous (example)MCMs
Special cause
(GP.2.2 @ OT)
Common cause (GP.2.9 @
+PA)
• Source: SQI Appraisall Assistant - http://goo.gl/i6IvI

MCMs Classifying MCMs by Dimension
• Horizontal: MMs going through the whole supply chain
 SwEng: ISO 15504, CMMI, FAA i-CMM, …
• Vertical: MMs focusing on a single perspective/group of processes
 Test Mgmt: TMM, TPI, …
 Project Mgmt: PM-MM, OPM3, …
 Requirement Mgmt: ....
• Diagonal: MMs focused on Organizational/Support processes
 People CMM, TSP, PSP, …Risk Management
Source:BuglioneL.,AnEcologicalViewonProcessImprovement:SomeThoughtsfor
ImprovingProcessAppraisals,4WCSQ,4thWorldCongressonSoftwareQuality,
WashingtonD.C.(USA),15-18September2008

Agenda
• Introduction
– Other Sources
• Q & A
Risk Mgmt and LEGO

CMMI-DEV and ISO 15504 – Risk Mgmt ref’sMCMs and Risk Mgmt
Model CMMI-DEV/SVC ISO 15504-12207
Domain Sw-SE Sw-SE
PRM (source) CMMI-DEV v1.3 ISO 12207
PRM (# Processes) 22 47
Process Categories RSKM (Risk Management) – ML3
(Staged representation)
MAN.5 (Risk Management)
Risk Mgmt-related
process(es)
SCAMPI v1.3 ISO 15504-2
ISO 15504-5
PAM ext. Appraisals PP-SP-2.2 (Identify Project Risks)
PMC-SP-1.3 (Monitor Project Risks)
ACQ.1, ACQ.3, ACQ.4, OPE.1, ENG.1, ENG.2,
SUP.10, MAN.3, MAN.5, PIM.3, PA2.1, PA4.1,
GP5.1.4, GP5.2.2. related BP (Base Practices)
PAM Risk-related
issues
Sw-SE Sw-SE

MCMs and Risk Mgmt
Model/ Framework
Repr.
Type
ML (#)
Architect-
Type
Comments/Notes
Project Risk Maturity
Model (PRMM)
Staged 4 [1-4] Level-based • 6 perspectives
IACCM CMM Staged 5 [1-5] Level-based • 9 dimensions (#7: Risk Management)
MMGRseg Continuous 5 [1-5] Level-based • Aligned with ISO/IEC 27005 [32]; 43 Control
Objectives into 6 groups; Final Risk Scorecard
MPS RMMM Staged 6 [1-6] Matrix-based • 6 drivers for assessing on an ordinal scale business
risks
RIMS RMM for Enterprise
Risk Management (ERM)
Staged 6 [0-5] Matrix-based • 7 process attributes; for each one, a series of Key
Drivers defined
IS RMM Staged 5 [1-5] Level-based • 9 control elements, each one with a variable
number of components
INCOSE RMM Staged 4 [1-4] Matrix-based • 5 Drivers
Risk Analysis (WBS) + RBS --- --- WBS -based • Creation of a Risk Breakdown Structure according
to the project WBS and quantification of risks by
each WBS task (calculation)
Choosing Risk Mgmt MCMs - Results

Agenda
• Introduction
– Other Sources
• Q & A
Risk Mgmt and LEGO

LEGO and SvcMgmt The LEGO Approach
1. MCM Repository 2. Process
Architecture
4. Appraisal Method3. Mappings &
Comparisons
1.
Identify goals
2.
Query
MCM
repository
3.
Include
new
elements
4.
Adapt
& Adopt
Source: Buglione L., Gresse von Wangenheim C., Hauck J.C.R., Mc Caffery F., The
LEGO Maturity & Capability Model Approach, Proceedings of 5WCSQ, 5th World
Congress on Software Quality, Shanghai (China), Oct 31- Nov 4 2011

Applying LEGO to Risk MgmtExperiencing LEGO...
The LEGO steps & related activities & outcomes:
1. Identify Goals
 Improve the internal Risk Management (RM) capability in order to generate more value to
our organization over time (product+service)
 Assume the target BPM (Business Process Model) to improve is generically the ISO 15504
MAN.5 process
2. Query the MCM repository
 Filter the list of available KM-based MCMs from the MCM repository
 Next table (EoI – Element of Interest) is a filter of the elements by each of the KM MCMs
considered
3. Include new elements into the target BPM
 Next table (Suggested Improvements) lists the possible EoI matched with the requested
MCMs (both SPs and GPs)
4. Adapt & Adopt
 Map each practice of the improved process to the related internal QMS process(es)
 Validate the mapping results before using them in the daily activities

Step 2 - EoI: Elements of Interest (1/4)
Model/ Framework Elements of Interest (EoI)
Project Risk Maturity
Model (PRMM)
 Six (6) perspectives (Stakeholders; Risk Identification; Risk Analysis; Risk
Responses; Project Management; Culture)
 Paid attention to:
o The ‘Culture’ perspective is interesting because it deals with people attitude
towards risk
o The ‘Stakeholders’ analysis can allow to catch all possible threats and
vulnerabilities in terms of missing items to be discussed and analyzed for
possible contingencies to the project plan. The PRMM process considers their
engagement for initiating the risk management process
o ‘Risk Response’ is what in other models/frameworks could be the list of
‘countermeasures’ in a ‘Risk Catalogue’
IACCM CMM  Quantitative approach (from SixSigma practices) with 9 dimensions (1. leadership;
2. customer/supplier experience; 3. execution and delivery; 4. solution
requirements management; 5. financial; 6.information systems/knowledge
management; 7. risk management; 8. strategy; 9. people development)
 Interesting the eventual inclusion of
o ‘Solution Requirements management’
o ‘IS/Knowledge Management’,
o ‘People development’, as in the SEI’s People-CMM
LEGO and Risk Mgmt

MMGRseg  Alignment with security issues (ISO 27005 [32])
 Refinement of the maturity levels into three stages (immaturity, maturity,
excellence)
 6 Control Objectives (CO) – processes - each one with a series of practices
o CD1 Context Definition; AA1 Risk Analysis/Assessment; RT1 Risk Treatment;
RA1 Risk Acceptance; RC1 Risk Communication; MA1 Monitoring & Critical
Analysis
 Paid attention to:
o CD1.9 (Collect and Store information); AA1.7 (Avoid Rework); AA1.8 (Revise
the process of risk estimation); RT1.4 (Define how to measure the
effectiveness of controls); RT1.5 (Calculate Residual Risks); RC1.x (all
practices); MA1.3 (Standardize the Monitoring and Critical Analysis activity)
 Assessment representation with Kiviat graphs, possible to use also a questionnaire
(as in the old Sw-CMM) or also a NPLF ordinal scale using the typical MCM
appraisal approach
MPS RMMM  ML grow with a larger environment to control (the larger the environment, the
higher the ML)
 This MCM is about Police Security and cross a series of organizational structures
that should be in place, according to their org model
 Two dimensions in the matrix-grid: Maturity Level by Maturity Elements
 Ordinal scale (No, Minimal, Partial, Yes, Significant; Substantial, Full) for rating
each crossed cell in the matrix
LEGO and Risk Mgmt

RIMS RMM  7 process attributes (Adoption of ERM-based discipline; ERM process management;
Risk appetite management; Root-cause discipline; Uncovering risk; Performance
Management; Business Resiliency and Sustainability), for each one, a series of Key
Drivers defined
 In each process attribute, there is a definition for matching a certain level (from
Non-Existent till Level 5)
 Particular attention could be devoted to those aspects:
o PA#4 (Root-Cause Discipline)  historicize data, classify risk, understanding
the why’s
o PA#5 (Uncovering Risks)  formalizing risk indicators/measures;
transforming risks into opportunities (CSF’s)
o PA#7 (Business Resiliency and Sustainability)  understanding of
consequences of action or inaction
IS RMM  9 control elements (Participants; Technologies; Information; Work Practices;
Products & Services; Customers ; Infrastructure; Environment ; Strategies)
 Based on ISO 31000 Risk Management Process [31], refining the process activities
into ‘Control Objectives’: EC (Establishment of the Context); AP (Risk Assessment);
TR (Risk Treatment); CR (Communication); SR (Monitoring & Review)
 To pay attention eventually to:
o EC.3 (Define a normalized method for the definition of the context)
o EC.4 (Define a method of appreciation of the risks)
o EC.7 (Define a plan of communication)
o EC.9 (Define the level of tolerance or acceptance of the risks)
o AP.6 + TR.6 + CR.3 + SR.4 (Collect and Store information about…)
o SR.1 (Monitor Risk Management Indicators)
LEGO and Risk Mgmt

INCOSE RMM  5 Drivers (Definition; Culture; Process; Experience; Application)
 Checklist (matrix-based) crossing Levels from 1 (Ad-hoc) to 4
(Managed) with the drivers, asin Crosby’s Quality Management Maturity
Grid (QMMG) [2]
 To pay attention eventually to:
o Definition  towards a proactive use of risk management
o Culture + Experience  learn from experiences, knowledge management
for risk management
o Application  use of quali-quantitative tools helping to deal with risks as an
opportunity when planning and estimate a new activity/project
LEGO and Risk Mgmt

Step 3 - Suggestions for Improvement (1/2)
ISO 15504 MAN.5 process Suggested Improvements
BP 01 – Establish Risk Management
scope
 Add practices/notes for collecting information about the Context for the project to be
analysed (scope management)
 Fundamental a proper definition of events and related risks in a Risk Catalogue
 Add practices about the need to consider the right stakeholders for eliciting
requirements and consequently potential risks form multiple viewpoints. It can help to
better define the scope for the project and its related risks
BP 02 - Define Risk Management
strategy
 Add practices/notes about the strategic need to be resilient as a way to ‘genetically’
manage risks in a proactive way. Define a method for evaluating risks for a proper
(proactive) management.
 Communication needs to be part of a risk strategy: people not aware about what is a
risk couldn’t work for excellence neither for obtain good results (wouldn’t be a lean
organization, at least!)
 Culture and Experience from teams is fundamental to avoid and learn by experience,
sharing information by a ‘Risk Catalogue’ (as well as in IT Service Management
models, ITSM personnel use a ‘Service Catalogue’)
BP 03 – Identify risks  Add practices/notes about the need for a ‘risk catalogue’, querying it for any risk
analysis in order to find yet classified/managed risks, with possible countermeasures.
 Any uncovered risk should be recorded as a new item into the risk catalogue,
updating the organization risk history as a basis for any further improvement
LEGO and Risk Mgmt
In the following tables, there is a list of ‘suggested improvements’
to the target process (in this example MAN.5 from ISO 15504) that
could be added in its next revision by BP (Base Practice), kept from
the EoI previously analysed and listed.

Step 3 - Suggestions for Improvement (2/2)
ISO 15504 MAN.5 process Suggested Improvements
BP 04 - Analyze risks  Add practices/notes about the opportunity to have a yet-ready list of possible
countermeasures from a Risk Catalogue, properly updated over time from the whole
organization’s teams
BP 05 – Define and perform risk
treatment actions
 Add practices for specifying how to measure the effectiveness of controls and
calculate residual risks.
 Another fundamental issue will be the definition of thresholds and criteria based on
historical data for their dynamic revision over time, choosing the proper updating
frequency for any kind/family of risk issues.
BP 06 - Monitor risks  Add in order to standardize the monitoring of risks along time.
 Need to formalize risk indicators/measures and transforming risks into opportunities
(CSF’s).
BP 07 - Take preventive or corrective
actions
 Add practices/notes about the need for RCA (Root-Cause Analysis) as the basic TQM
technique to use for determining the best choice from your own historical
project/organizational data.
 Communication is not only part of the strategy but – as an action – also the closing
step for a corrective/preventive action, checking that the target audience will have
properly received and acted against the requested action.
 Tools could help in making easier the identification of recurring risk patterns and
suggest possible countermeasures
LEGO and Risk Mgmt

Agenda
• Introduction
– Other Sources
• Q & A
Risk Mgmt and LEGO

Conclusions & Future Works
• Risks as threats or opportunities?
 A risk should be known, analyzed and managed: having a ‘risk catalogue’ (as a service catalogue)
can help organizations to manage a threat and possibly convert it into an improvement opportunity
 Contigencies should be evaluated but not spent directly into a Gantt chart if not still happened
 Risk Management is not part of Project Management, but it’s a separated, supporting process
 Possibly risks should be measured, not only evaluated
 Look at Value as the final goal to achieve in order to really improve our activities
• Models and Methods
 Many models, taxonomies and frameworks can be valid for managing risks
 The value when better managing risks can lead to a lower TCO for projects
 E.g. ISO 31000 is not the solely source to consider, but also CMMI/SPICE risk-related process could
be considered
• LEGO’ (Living EnGineering prOcess) approach
• http://slideshare.re/nssLR8 [5WCSQ, Shangai, Nov 2011]
• Choose and integrate the ‘pieces of the puzzle’ you need for your goals the target is your QMS,
not the model(s) you are using
 Next Steps
 Identify further ‘silver bullets’ for leveraging the joint view of products and services, also from a
business viewpoint
 Hybridize more models and techniques between the two communities for benchmarking purposes
All models are wrong. Some models are useful.
(George Box, Mathematician, 1919-2013)
Risk Mgmt and LEGO

Lessons Learned...
URL:www.dilbert.com
Risk Mgmt and LEGO

Q & A
Danke für Ihre Aufmerksamkeit!
Thanks for your attention!
Risk Mgmt and LEGO

Our Contact DataRisk Mgmt and LEGO
Luigi
Buglione
Engineering Ing. Inf. /ETS
luigi.buglione@eng.it
Fergal
McCaffery
DKIT
fergal.mccaffery@dkit.ie
C. Gresse von
Wangenheim
UFSC
gresse@gmail.com
Alain
Abran
ETS
alain.abran@etsmtl.ca
Jean Carlo R.
Hauck
UFSC
jeanhauck@gmail.com

Risk Management: Achieving Higher Maturity & Capability Levels through the LEGO Approach

Recomendados

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente (10)

Destacado

Destacado (19)

Similar a Risk Management: Achieving Higher Maturity & Capability Levels through the LEGO Approach

Similar a Risk Management: Achieving Higher Maturity & Capability Levels through the LEGO Approach (20)

Más de Luigi Buglione

Más de Luigi Buglione (20)

Último

Último (20)

Risk Management: Achieving Higher Maturity & Capability Levels through the LEGO Approach