Does counting the number of intrusions a firewall blocked in a month really justify the capital spend on security projects? What kind of operational data demonstrates cybersecurity leaders’ long-term budgetary needs for their programs and at the same time shows the progress they’ve made over the years? Learn how a duo of cybersecurity professionals used thought leadership and a goals-based approach to build the case for past capital and future spend — a system that won them both dollars and trust with peers and their boards.

1. Security Metrics
The Kickstarter to Measuring What Matters
Jack D. Nichelson
Director of Infrastructure & Engineering
Medical Mutual
Jason Middaugh
Director of Infrastructure& Security
Cliffs Natural Resources
June 6, 2017

2. ACKNOWLEDGEMENTS
 Steve Holt, CIO - Chart Industries
 Craig Shular, CEO - GrafTech
 David Hilmer, VP & CIO - Graftech
 Erick Asmussen, VP & CFO - Graftech
 John Kocsis, Dir. IS Ops - Swagelok
 Chris Clymer Dir. Info Sec - MRK
 Ed Pollock CISO – STERIS
 John Kory – Dir. Infrastructure - CBIZ
“When the student is ready the master will appear”

3. JACK NICHELSON
 Director of Infrastructure & Security for Chart Industries.
 Executive MBA from Baldwin-Wallace University
 Recognized as one of the “People Who Made a Difference
in Security” by the SANS Institute and Received the
CSO50 award for connecting security initiatives to
business value.
 Adviser for Baldwin Wallace’s, State winner Collegiate
Cyber Defense Competition (CCDC) team.
I defend my companies competitive advantage
by helping solve business problems through
technology to work faster and safer.
“Solving Problems, is my Passion”

4. JASON MIDDAUGH
 Director of Infrastructure & Security Services at Cliffs
Natural Resources
 Holds a bachelor’s degree in information
systems/operations management from the University
of Toledo, an executive master’s degree in business
administration from The Ohio State University, and
holds several IT based certifications (CISSP, MCSE,
VCP, CCNA, CCDA).
 Chair member of the North East Ohio Cyber
Consortium. Selected and currently attending the FBI
Citizens Academy.
 Skiing enthusiast

5. SUMMARY - WHY
Warning: This presentation is not about our favorite security based
metrics to track and report on; it is about the following issue:
Key initiatives we will discuss today and points that will lead you to
the metrics that you should measure in your organization:
 Focus on a goal/outcome
 Should be easy to measure and collect
 Help gain you trust across the organization
 Motivate your team
 Create no surprise moments
 Flexibility to readjust and restart at any point
“A problem well stated is a problem half-solved.” - Charles Kettering
Lots of people are spending time tracking bad metrics, so you need
you know yourself, your stockholders, and customers pains in order
to set the right goals and then use metrics to track your progress to
the results.

6. GUT CHECK – TAKE OWNERSHIP
If I just had a better team, I would do better. "Wrong"
If I am a better leader, my team will be better. That is what I had to learn as a
leader and step up to make happen.
 Be Proactive – Focus on what you can influence
 Begin with the end in mind – Define practical outcomes
 Create a Problem Statement – A goal without a plan is just a wish
 Put first thing first – Plan weekly, act daily
 Chart Performance & Adjust – Shine a light on the problem
“There are no bad teams, only bad leaders.” - Jocko Willink

7. GOOD ADVICE
The most important person for you to manage effectively is yourself. To grow
personally and professionally you need to know yourself before you can help
others.
“Think about how you can simplify security –
make it easy – and focus on the basics.” - Dave Kennedy
Recommendations:
 Take a step back and read “REWORK”
 Remove complexity – Start small
 Start at the epicenter, on what won’t change
 Focus on fewer problems that provide bigger returns
 Build an audience
 Keep score & publish it (Good or Bad)

8. KNOW YOUR STAKEHOLDERS
To make stuff that matters, you have to know what matters so work on
solving the right problems.
Effective managers take the time to identify stakeholders and know their pain points.
 Security is about a lot more than just you
 You are taking actions to protect assets in the
stewardship of others
 You are making choices which will impact the ways
those around you conduct their business
“No one cares what you know until you show them
how much you care”

9. CUSTOMER SERVICE
We often focus on the problem and forget about the customer. They will
forget the problem you solved before they forget how you made them feel.
“The day people stop bringing you their problems is the day you have
stopped leading them” - Colin Powell
 Security is a support role…your job is to help others
safely do the things that make your organization
productive
 You cannot do this job without help
 Your employees are not subjects for you to dictate
rules to…they are your customers
 If you treat them well, they will be your “army of human
sensors”, bringing you all kinds of useful intel, and
helping to enforce policies you've developed to protect
them

10. JUST SAY MAYBE
Effective leadership requires collaboration and empathy for the other person.
It’s OK to be uncomfortable with the results
 Security has often been the Department of “No”
 Taking a hard stance as a “cyber policeman” can
seem to work…until you become perceived as an
obstacle
 If you are an obstacle, process will begin to be routed
around you

11. JASON'S STORY

12. FIRST STEPS – GETTING STARTED
 Know your audience and speak their language
 Leverage existing tools that you already have
 Anticipate what others will ask or what conclusions
they will draw from the metrics you are presenting

13. PAST PROJECTS AND SPEND
 IT Executives and Board members want to see that they getting
results from previous initiatives
 Use these metrics to create a “Wow” factor

14. CURRENTLY MEASURED METRICS
 Your goal should be to eventually get to at least 13 months of
revolving data
 Play the Game of Thrones, or in this case the Game of Met-tricks

15. PROCESS – GET ON BOARD
 Define the problem
 Where is the pain?
 Find your stakeholders
 What is working/what isn’t?
 Identify your resources
 What is readily available?
 Tools, process, people
 Don’t dwell on nice-to-haves
 Build a rough draft
 Won’t be perfect the first time
 Make sure its easy to repeat
 Review with stakeholders
 Above and below
 Have them contribute
 Get them to buy-in
 Rinse & repeat
 Run the process
 See what works and what doesn’t
 Adjust where needed

16. METRICS FOR YOUR FUTURE
 Use your metrics to help justify
additional capital
 Help predict when you are
going to run out or need more
 Show capability gaps that you
want to fill
 Justify additional resources

17. HOW & WHERE DO I GET STARTED?
 Kick things off in your next team meeting
 Don’t worry about the past
 Don’t get hung up on 100% accuracy
You don’t have to hit the bulls-eye on every metric, shoot for the inner-rings

18. SUMMARY
 Know your audience - Speak their language
 Think Past – Leverage metrics to justify and put
historic spend at ease
 Think Present – Collect and analyze your data;
do not be afraid to burn it down and start again
 Think Future – Continually set the stage and
constantly communicate when and where you
are going to need additional resources with
metrics
 Get Started! – You do not have to hit the
bullseye

19. JACK'S JOURNEY

20. BE PROACTIVE
Change starts from within, so you have to make the decision to focus
on the things you can influence rather than reacting to the things
outside of your control.
Manage Yourself:
 Where and how are you spending your time & energy throughout the day?
 Make a list of the things that concern you and things you can Influence.
Ask yourself these 3 questions every day:
 Did I do my best to spend my time on things I can influence?
 Did I do my best to set and communicate clear goals?
 Did I do my best to make progress toward goal achievement?
“The 1st metric you need to track is yourself”

21. CONCERN VS. INFLUENCE
Hackers Organized Crime State Sponsored
Higher Difficulty
~10% of incidents
Security Risks
• Advanced Persistent Threat
• Zero Day Attacks
• The Insider Threat
• BYOD
• Mobile Malware
• The “Cloud”
Lower Difficulty
~90% of incidents
• Missing Patches
• Lost & Stolen Devices
• Local Admin Right’s
• Phishing
• Poor Passwords
• MalwareVerizon's 2013 Data Breach Investigations Report (DBIR)

22. BEGIN WITH THE END IN MIND
If your ladder is not leaning against the right wall, every step you take gets
you to the wrong place faster.
“Try Not to Become a Success. Rather Become a Person of Value.”
First, do you know what “good” looks like?
 Break down the area you have influence over into functional parts
that you and the stockholders can score and rank.
 Now that you have an agreed upon heatmap of your current
state, set short term and long term goals.

23. PROBLEM STATEMENT
The Problem Statement significantly clarifies the current situation by
specifically identifying the problem and its severity, likelihood, and impact. It
also serves as a great communication tool, helping to get buy-in and support
from others.
“A problem well stated is a problem half-solved.” — Charles Kettering
Build & Execute plans to drive for results & share
successes
 Invest more time in project planning and due diligence; time spent
defining the problem is NEVER time wasted.
 Write a Project Charter, clearly state the scope, objectives,
participants, and success measurements.
 Create a Work Breakdown Structure to graphically represent the
project scope, broken down in successive chunks with defined
deliverables.

24. PUT FIRST THINGS FIRST
Focus on the important, not just the urgent. The urgent are not that important,
and the important are never urgent.
“Effectiveness requires the integrity to act on your priorities”
Tips for taking back control of your time:
 Stop saying Yes, When you want to say No.
 Scheduled your own time with purpose & defend it!
 Don’t be afraid to close your email and turn off your phone

25. CHART PERFORMANCE & ADJUST
Gemba (現場) is a Japanese term referring to the place where value is created.
The idea of Gemba is that the problems are visible, and the best improvement
ideas will come from going to the Gemba.
“Good security is not something you have, it’s something you do” - Wendy Nather

26. SUMMARY – TAKE OWNERSHIP
If I just had a better team, I would do better. "Wrong"
If I am a better leader, my team will be better. That is what I had to learn as a
leader and step up to make happen.
 Be Proactive – Focus on what you can influence
 Begin with the end in mind – Define practical outcomes
 Create a Problem Statement – A goal without a plan is just a wish
 Put first thing first – Plan weekly, act daily
 Chart Performance & Adjust – Shine a light on the problem
“There are no bad teams, only bad leaders.” - Jocko Willink

27. SUMMARY - WHY
Warning: This presentation is not about our favorite security based
metrics to track and report on; it is about the following issue:
Key initiatives we will discuss today and points that will lead you to
the metrics that you should measure in your organization:
 Focus on a goal/outcome
 Should be easy to measure and collect
 Help gain you trust across the organization
 Motivate your team
 Create no surprise moments
 Flexibility to readjust and restart at any point
“A problem well stated is a problem half-solved.” - Charles Kettering
Lots of people are spending time tracking bad metrics, so you need
you know yourself, your stockholders, and customers pains in order
to set the right goals and then use metrics to track your progress to
the results.

28. BOOK REFERENCES
Work Smarter and More Easily by "Sharpening Your Axe"”
 The Five Dysfunction of a Team – Patrick Lencioni
 Leading Change – John Kotter
 The 7 Habits of Highly Effective People – Dr. Covey
 The 1 Minute Manager – Ken Blanchard
 Extreme Ownership – Jocko Willink
 The Phoenix Project – Gene Kim
 What got you Here won’t get you There – Gooldsmith
 Leaders Eat Last – Simon Sinek
 The Ideal Team Player – Patrick Lencioni
 Death by Meeting – Patrick Lencioni

29. THANK YOU
Jason.Middaugh@CliffsNR.com Jack@Nichelson.net
Jack D. Nichelson
Director of Infrastructure & Engineering
Medical Mutual
Jason Middaugh
Director of Infrastructure & Security
Cliffs Natural Resources

30. NETWORK
 No time like the present to put your soft skills to work
 Say hi to your neighbor…how can you help each other?
Jack@Nichelson.net

31. APPENDIX
Jack@Nichelson.net

32. MULTI-LAYERED DEFENSE

33. HOW TO BUILD A SQDC BOARD
 Key Performance Indicators – Good data can tell a story
 Predictive Analysis – Your board should help prevent future issues
 Keep the data fresh and useful, address items as quick as possible
using LEAN tools and once addressed remove them from the board.

34. GEMBA BOARD: SECURITY “We measure things that matter”
Example Metrics:
 # of systems not monitored & tracked in inventory by
Location or LoB
 # Top Vulnerabilities by Location or LoB
 # of Legacy Systems by Location or LoB
 # of Users with Local Admin & Accounts with Domain
Admin
 # of Total Security Incidences by Location or LoB
 # of Past Due Security Awareness Training by Location or
LoB
Security - The current security posture at a glance

35. GEMBA BOARD: QUALITY
Example Metrics:
 # of Servers & Workstation missing OS & App patches
(30 day SLA)
 # of infections/Re-Images tickets (3 day SLA)
 # of Security Event tickets (5 day SLA)
 # of Security Request tickets (15 days SAL)
 Cause Mapping Analysis to find root cause of problems
Quality – Results for SLA goals
of events & requests

36. GEMBA BOARD: DELIVERY
Delivery – Active Projects & Audits at a glance
Example Metrics:
 Active Projects Status
 Active Audit Status
 Remediation Progress by Location or LoB
 On-Site Awareness Training by Location

37. GEMBA BOARD: COST
Cost – P&L at a glance
Example Metrics:
 Operating budget spending plan (OPEX & CAPEX)
 ROIC Qualitatively Rating of Perceived Value
 Support Agreements Costs & Renew dates
 Consultant Support Agreements Costs & Renew dates
 Running total of cost savings

38. GEMBA BOARD: PEOPLE
People – Skills matrix at a glance
Example Metrics:
 Skills Matrix of everyone in Security
 Training and development plans
 On-Call & Vacation Schedules
 Awards

39. VISUALIZATION TECHNIQUES: THE HEATMAP
Impact
Low No threat to core business function impact
Medium
Threat to core business function impact, but
has not occurred yet. i.e. ERP system is down
but have not yet missed orders
High
Immediate impact to core business functions.
i.e. products cannot be shipped, or core IP is
lost.
Likelihood
Low Happens once every 10 years, or less
Medium Happens once every 1 to 10 years
High Happens once or more a year
• Develop “Likelihood” to fit your org
• Develop “Impact” to fit your org”
• Score potential risks “high”,
“medium”, or “low” for each
• Map results to the heatmap

40. VISUALIZATION TECHNIQUES: RISK REGISTER

41. VISUALIZATION TECHNIQUES: THE SCORECARD

42. VISUALIZATION TECHNIQUES: THE SCORECARD
 Captures day-to-day operations in security
 One-page roll-up that can be presented to CIO, or used internally
 “Operations” section captures work being done: creating firewall rules,
patching systems, conducting awareness events
 The “Risk” section captures visibility into risk at the organization.
Number of scans, open vulnerabilities
 To the far right is the legend explaining the thresholds for each item