Your company has had a really, really bad day. Bad Days happen. It is when, not if. In a typical scenario, the top executives come in yelling and screaming and fire whoever they think may have had some part in the Bad Day’s events. After most of the yelling has stopped, then the executives want to know what happened. There is no doubt that autopsies can be messy and emit unpleasant odors, but they are a necessary evil and effective learning tool. But there is a right way and a wrong way to conduct a postmortem analysis of your incident. The obvious objective is to learn what happened so that it doesn’t happen again. The focus should be on the root cause and conducting an efficient analysis of the problem, but there is so much more that can be learned. Using the Incident Command System (ICS), this session provides an overview of the same postmortem process used by Fire Departments across the country.

1. © 2013 Blackrock 3 Partners LLC 1
A Good Post Mortem . . . .
Includes a Pulse Check!
2
2013 Events…so far
2
You are part of Critical Infrastructure!
3
Postmortem Typical
Worst Case (the Usual
Case)
 Yelling & screaming
 Fighting & biting
 Firing
 Everyone hides
 Blame finding
 Many resumes
updated
 Stock option date
reviewed
 Nothing is fixed
Best Case (Utopia)
 Process oriented root
cause(s) analysis
 Team effort
 Focus on positive
 Focus on prevention
 Mistakes kept in
perspective
 Completed by non-
involved personnel

2. © 2013 Blackrock 3 Partners LLC 2
4
EVENT
A Typical Post Mortem Goes Like This
 Failure
 Operations
 Process
 Software
 Hardware
 Response
 Responsibilities
5
Why the Pulse Check?
Cause: Factors that produce an effect
Reason: An explanation or justification
Y
6
Looking for “Some Guy”
 Failure
 Operations
 Process
 Software
 Hardware
 Response
 Responsibilities

3. © 2013 Blackrock 3 Partners LLC 3
7
AAR
 Marketing/Sales want products faster than
Engineering/Operations can design/build/
implement them
 Investors want faster growth than
management can deliver
 Early technology architecture/equipment
decisions never get replaced
 R&D teams love the newest technology
 Operations teams hate the newest
technology
8
It’s a Matter of Perspective
 A wake-up call
 A near miss for a larger, more
devastating incident
 An opportunity to stop and look at the
organization – because you never have
time to stop and look because the
business is running so fast
 You don't know what you don't know
 Don’t let a good crisis go to waste!
9
When done right
The process allows
you to look at risk
differently
Positive Failure!

4. © 2013 Blackrock 3 Partners LLC 4
10
TECH Possible vs. Likely
Operations
Copyright 2013 BlackRock 3 Partners LLC
Operations
Normal
Technological
Environm
ental
H
um
an
Not Normal
C
rim
inal
11
TALENT
Training
Accountability
Leadership
Empowerment
Notification
Trust/Temptation
Experts or Saviors
12
IMS
Incident
Commander
Unified Command
Applications
App 1 App 2
Database
DBA - 1 DBA - 2
Continued
Operations
Customer Service
Executive Liaison
Plans
Disaster Recovery

5. © 2013 Blackrock 3 Partners LLC 5
13
Change – Failure – AAR
 Maybe we shouldn’t talk about failure
and death together!
Failure
AAR
Change
After Action Review (AAR)
Post Incident Evaluation (PIE)
Post Incident Analysis (PIA)
Incident Review (IR)
Reason- Cause – Analysis (RCA)
14
EVENT
Operations Process
Software Hardware
Response
Responsibilities
Sales
Marketing
Growth
Technology
Environment
Communication
Human
After Action Review
15
AAR
Tradeoff between different groups:
 Marketing
 Sales
 Engineering
 Operations
 Finance
 Management Team
 Board
 Investors

6. © 2013 Blackrock 3 Partners LLC 6
16
Establishing a Culture
 Things WILL go wrong
 There should be a process to address them
 Incident Management System (IMS)
– Handling of an occurrence
– Handling of an issue
– Having a really bad day
 RCA with a new perspective
 Retrospective and prospective
 The key is to understand the reason
17
www.blackrock3.com
Chris Hawley –
Chris@blackrock3.com
Rob Schnepp –
Rob@blackrock3.com
Ron Vidal –
Ron@blackrock3.com
San Francisco & Baltimore
Blackrock 3 Partners LLC