SlideShare una empresa de Scribd logo
1 de 36
Descargar para leer sin conexión
Raffael Marty, CEO
Creating Your Own Threat Intel
Through Hunting & Visualization
Tenerife, Spain
February, 2016
Creating Your Own Threat Intel
Through Hunting & Visualization
Raffael Marty, CEO
Security. Analytics. Insight.3
Contents
HUNTING
AKA INTERNAL THREAT INTELLIGENCE
THREAT INTELLIGENCE
A PROCESS AND INFRASTRUCTURE VIEW
1
2 VISUALIZATION
A THREAT INTELLIGENCE GOLD MINE
3
Threat Intelligence
Security. Analytics. Insight.5
• Products / Tools
• Firewall - Blocks traffic based on pre-defined rules
• Web Application Firewall - Monitors for signs of known malicious activity in Web traffic
• Intrusion Prevention System - Looks for ‘signs’ of known attacks in traffic and protocol violations
• Anti Virus - Looks for ‘signs’ of known attacks on the end system
• Malware Sandbox - Runs new binaries and monitors their behavior for malicious signs
• Security Information Management - Uses pre-defined rules to correlate signs from different data
streams to augment intelligence
• Vulnerability Scanning - Searches for known vulnerabilities and vulnerable software
• Rely on pattern matching and signatures based knowledge from the past
• Reactive -> always behind
• Unknown and new threats -> won’t be detected
• ‘Imperfect’ patterns and rules -> cause a lot of false positives
We Are Monitoring - What is Going Wrong?
Defense Has Been Relying
On Past Knowledge
Security. Analytics. Insight.6
Event Funnel - How We Used To Do It
data
rule-based

correlation
prioritization
simple

statistics
attack
candidates
• What rules do you write?
• Do the vendor provided rules work for you?
• How do you define a priority 10 event?
• High false positive rate!
• Unless alerts are VERY focussed
• High false negative rate!
• Do you know what you don’t know?
Security. Analytics. Insight.7
Then Came Threat Intelligence
• How many hits do you really get?
• You are missing most attacks
IOCs
• How do you match
these efficiently
against a real-time
stream?
• How do you de-
duplicate and
normalize these
feeds?
attack
candidates
70–90%
OF MALWARE SAMPLES ARE UNIQUE TO AN ORGANIZATION.
Security. Analytics. Insight.8
Removing the Event Funnel - Hello Data Lake
any 

data
Big Data Lake
Rules
• Storing more, and more diverse data
• Kafka and “dynamic parsing”
• Enabling large-scale processing
• Spark, SparkStreaming, Storm, Parquet
• Using “standard” data access (SQL, REST)
• Plug in any other tool!
context
IOCs
This per-se is not new …
Security. Analytics. Insight.9
Adding Interactive - Analyst Driven Exploration
any 

data
Big Data Lake
Rules
context
IOCs
… but first we get the human in the loop …
Hunting
• interactive visualization
• analyst driven
• machine assisted
Security. Analytics. Insight.10
Hunting Creates Internal Threat Intelligence
any 

data
Big Data Lake
Rules
context
IOCs
… then, let’s rethink our rules … Novel, Advanced Attacks
internal TI
Security. Analytics. Insight.11
Hunting Creates Internal Threat Intelligence
any 

data
Big Data Lake
Rules
context
IOCs
… then, let’s rethink our rules … patterns anyone?
internal TI
Novel, Advanced Attacks
Low False Positive Alerts
Patterns
Security. Analytics. Insight.12
Buzzword Bingo
any 

data
Big Data Lake
Rules
context
IOCs
… and finally, we are buzzword compliant …
behavioral monitoring

scoring
anomaly detection
machine learning
artificial intelligence
“models”
data science
internal TI
Patterns
Security. Analytics. Insight.13
How Does All That Architecture Stuff Matter?
In the following we’ll explore how this all matters …
… but first, let’s see how visualization plays a key role here.
14Visualization
Security. Analytics. Insight.15
“How Can We See, 

Not To Confirm -
But To Learn”
- Edward Tufte
Security. Analytics. Insight.16
Why Visualization?dport
time
Security. Analytics. Insight.17
SELECT count(distinct protocol) FROM flows;
SELECT count(distinct port) FROM flows;
SELECT count(distinct src_network) FROM flows;
SELECT count(distinct dest_network) FROM flows;
SELECT port, count(*) FROM flows GROUP BY port;
SELECT protocol,
count(CASE WHEN flows < 200 THEN 1 END) AS [<200],
count(CASE WHEN flows>= 201 AND flows < 300 THEN 1 END) 

AS [201 - 300],
count(CASE WHEN flows>= 301 AND flows < 350 THEN 1 END) 

AS [301 - 350],
count(CASE WHEN flows>= 351 THEN 1 END) AS [>351]
FROM flows GROUP BY protocol;
SELECT port, count(distinct src_network) FROM flows GROUP BY port;
SELECT src_network, count(distinct dest_network) FROM flows GROUP
BY port;
SELECT src_network, count(distinct dest_network) AS dn, sum(flows)
FROM flows GROUP BY port, dn;
SELECT port, protocol, count(*) FROM flows GROUP BY port, protocol;
SELECT sum(flows), dest_network FROM flows GROUP BY dest_network;
…
One Graph Summarizes Dozens of Queries
port dest_network
protocol src_network flows
Security. Analytics. Insight.18
Visualization To …
Present / Communicate Discover / Explore
Security. Analytics. Insight.19
We will have a look at a couple components from earlier:
• Context
• Data Science
• Clustering
• Seriation - Data Science Gone Wrong
• Time-series Analysis
Analytics Components
Security. Analytics. Insight.20
Did You Know?
Users accessing Sharepoint
servers
User
Sharepoint Server
data processing visualization
This graph of users accessing
sharepoint servers, does not
immediately reveal any interesting
patterns.
Security. Analytics. Insight.21
Did You Know - How Context Tells a Story
Using HR data as context
Remote User
San Francisco Office User
Sharepoint Server
data processing visualization
HR data
Using color to add context to the
graph helps immediately identify
outliers and potential problems.
Security. Analytics. Insight.22
• Simple stuff works!
• dc(dest), dc(d_port)
• What is normal?
• Use data science / data mining to prepare
data. Then visualize the output for human
analyst.
Data Science in Security - Words of Caution
Security. Analytics. Insight.23
Challenges With Clustering Network Traffic
The graph shows an abstract
space with colors being
machine identified clusters.
Hard Questions:
• What are these clusters?
• Do Web servers cluster?
• What are good clusters?
• What’s anomalous?
Security. Analytics. Insight.24
Data Science That Works
threshold
outliers have different magnitudes
Security. Analytics. Insight.25
Approximate Curve
fitting a curve distance to curve
Security. Analytics. Insight.26
Data Mining Applied
better 

threshold
27Hunting
Security. Analytics. Insight.28
Hunting - Ready, Fire, Aim
• Analysts are your best and most expensive resource
• They need the right tools and data
• Speed (see earlier architecture)
• Interaction (visual!)
• Machine-assisted insight
Examples
• Exploring DNS traffic
• High business impact machine analysis
• Lateral movement
Security. Analytics. Insight.29
HBI Metric Analysis
Visually learn, Test, Automate
Security. Analytics. Insight.30
HBI Metric Analysis - If you like Black Backgrounds
Security. Analytics. Insight.31
We have tried many thing:
• Social Network Analysis
• Seasonality detection
• Entropy over time
• Frequent pattern mining
• Clustering
All kinds of challenges.
Simple works!
Let’s Get Mathematical
U−matrix
4.28e−05
0.0461
0.0921
Security. Analytics. Insight.32
Simple - Data Abstraction
Security. Analytics. Insight.33
Lateral Movement - Cross Network Communications
Challenges
• Scale
• You will find one of everything
• Defining white-lists and
keeping them up to date (i.e.,
network and asset hygiene)
VPN
DMZ
Office
GIA
Unknown
Internet
AWS
Security. Analytics. Insight.34
http://secviz.org
List: secviz.org/mailinglist
Twitter: @secviz
Share, discuss, challenge, and learn about security visualization.
Security Visualization Community
Security. Analytics. Insight.35
BlackHat Workshop
Visual Analytics
Delivering Actionable Security
Intelligence
July 30,31 & August 1,2 - Las Vegas, USA
big data | analytics | visualization
http://secviz.org
Security. Analytics. Insight.36
After some exploration …
raffael.marty@pixlcloud.com
http://slideshare.net/zrlram
http://secviz.org and @secviz
Further resources:

Más contenido relacionado

La actualidad más candente

Threat Hunting
Threat HuntingThreat Hunting
Threat HuntingSplunk
 
Threat hunting for Beginners
Threat hunting for BeginnersThreat hunting for Beginners
Threat hunting for BeginnersSKMohamedKasim
 
Cyber Threat Intelligence
Cyber Threat IntelligenceCyber Threat Intelligence
Cyber Threat Intelligencemohamed nasri
 
Adversary Emulation and the C2 Matrix
Adversary Emulation and the C2 MatrixAdversary Emulation and the C2 Matrix
Adversary Emulation and the C2 MatrixJorge Orchilles
 
Threat Hunting Workshop
Threat Hunting WorkshopThreat Hunting Workshop
Threat Hunting WorkshopSplunk
 
Cyber Threat Hunting: Identify and Hunt Down Intruders
Cyber Threat Hunting: Identify and Hunt Down IntrudersCyber Threat Hunting: Identify and Hunt Down Intruders
Cyber Threat Hunting: Identify and Hunt Down IntrudersInfosec
 
OSINT- Leveraging data into intelligence
OSINT- Leveraging data into intelligenceOSINT- Leveraging data into intelligence
OSINT- Leveraging data into intelligenceDeep Shankar Yadav
 
Delivering Security Insights with Data Analytics and Visualization
Delivering Security Insights with Data Analytics and VisualizationDelivering Security Insights with Data Analytics and Visualization
Delivering Security Insights with Data Analytics and VisualizationRaffael Marty
 
Intelligence Failures of Lincolns Top Spies: What CTI Analysts Can Learn Fro...
 Intelligence Failures of Lincolns Top Spies: What CTI Analysts Can Learn Fro... Intelligence Failures of Lincolns Top Spies: What CTI Analysts Can Learn Fro...
Intelligence Failures of Lincolns Top Spies: What CTI Analysts Can Learn Fro...MITRE ATT&CK
 
Threat Hunting Report
Threat Hunting Report Threat Hunting Report
Threat Hunting Report Morane Decriem
 
BSidesLV 2016 - Powershell - Hunting on the Endpoint - Gerritz
BSidesLV 2016 - Powershell - Hunting on the Endpoint - GerritzBSidesLV 2016 - Powershell - Hunting on the Endpoint - Gerritz
BSidesLV 2016 - Powershell - Hunting on the Endpoint - GerritzChristopher Gerritz
 
Fantastic Red Team Attacks and How to Find Them
Fantastic Red Team Attacks and How to Find ThemFantastic Red Team Attacks and How to Find Them
Fantastic Red Team Attacks and How to Find ThemRoss Wolf
 
What is ATT&CK coverage, anyway? Breadth and depth analysis with Atomic Red Team
What is ATT&CK coverage, anyway? Breadth and depth analysis with Atomic Red TeamWhat is ATT&CK coverage, anyway? Breadth and depth analysis with Atomic Red Team
What is ATT&CK coverage, anyway? Breadth and depth analysis with Atomic Red TeamMITRE ATT&CK
 
Security Strategy and Tactic with Cyber Threat Intelligence (CTI)
Security Strategy and Tactic with Cyber Threat Intelligence (CTI)Security Strategy and Tactic with Cyber Threat Intelligence (CTI)
Security Strategy and Tactic with Cyber Threat Intelligence (CTI)Priyanka Aash
 
How MITRE ATT&CK helps security operations
How MITRE ATT&CK helps security operationsHow MITRE ATT&CK helps security operations
How MITRE ATT&CK helps security operationsSergey Soldatov
 
MITRE ATT&CKcon 2018: Hunters ATT&CKing with the Data, Roberto Rodriguez, Spe...
MITRE ATT&CKcon 2018: Hunters ATT&CKing with the Data, Roberto Rodriguez, Spe...MITRE ATT&CKcon 2018: Hunters ATT&CKing with the Data, Roberto Rodriguez, Spe...
MITRE ATT&CKcon 2018: Hunters ATT&CKing with the Data, Roberto Rodriguez, Spe...MITRE - ATT&CKcon
 
Bsides 2019 - Intelligent Threat Hunting
Bsides 2019 - Intelligent Threat HuntingBsides 2019 - Intelligent Threat Hunting
Bsides 2019 - Intelligent Threat HuntingDhruv Majumdar
 
MITRE ATT&CKcon 2018: Helping Your Non-Security Executives Understand ATT&CK ...
MITRE ATT&CKcon 2018: Helping Your Non-Security Executives Understand ATT&CK ...MITRE ATT&CKcon 2018: Helping Your Non-Security Executives Understand ATT&CK ...
MITRE ATT&CKcon 2018: Helping Your Non-Security Executives Understand ATT&CK ...MITRE - ATT&CKcon
 
Using IOCs to Design and Control Threat Activities During a Red Team Engagement
Using IOCs to Design and Control Threat Activities During a Red Team EngagementUsing IOCs to Design and Control Threat Activities During a Red Team Engagement
Using IOCs to Design and Control Threat Activities During a Red Team EngagementJoe Vest
 

La actualidad más candente (20)

Threat Hunting
Threat HuntingThreat Hunting
Threat Hunting
 
Threat hunting for Beginners
Threat hunting for BeginnersThreat hunting for Beginners
Threat hunting for Beginners
 
Cyber Threat Intelligence
Cyber Threat IntelligenceCyber Threat Intelligence
Cyber Threat Intelligence
 
Adversary Emulation and the C2 Matrix
Adversary Emulation and the C2 MatrixAdversary Emulation and the C2 Matrix
Adversary Emulation and the C2 Matrix
 
Threat Hunting Workshop
Threat Hunting WorkshopThreat Hunting Workshop
Threat Hunting Workshop
 
Cyber Threat Hunting: Identify and Hunt Down Intruders
Cyber Threat Hunting: Identify and Hunt Down IntrudersCyber Threat Hunting: Identify and Hunt Down Intruders
Cyber Threat Hunting: Identify and Hunt Down Intruders
 
OSINT- Leveraging data into intelligence
OSINT- Leveraging data into intelligenceOSINT- Leveraging data into intelligence
OSINT- Leveraging data into intelligence
 
Delivering Security Insights with Data Analytics and Visualization
Delivering Security Insights with Data Analytics and VisualizationDelivering Security Insights with Data Analytics and Visualization
Delivering Security Insights with Data Analytics and Visualization
 
Intelligence Failures of Lincolns Top Spies: What CTI Analysts Can Learn Fro...
 Intelligence Failures of Lincolns Top Spies: What CTI Analysts Can Learn Fro... Intelligence Failures of Lincolns Top Spies: What CTI Analysts Can Learn Fro...
Intelligence Failures of Lincolns Top Spies: What CTI Analysts Can Learn Fro...
 
Threat Hunting Report
Threat Hunting Report Threat Hunting Report
Threat Hunting Report
 
BSidesLV 2016 - Powershell - Hunting on the Endpoint - Gerritz
BSidesLV 2016 - Powershell - Hunting on the Endpoint - GerritzBSidesLV 2016 - Powershell - Hunting on the Endpoint - Gerritz
BSidesLV 2016 - Powershell - Hunting on the Endpoint - Gerritz
 
Fantastic Red Team Attacks and How to Find Them
Fantastic Red Team Attacks and How to Find ThemFantastic Red Team Attacks and How to Find Them
Fantastic Red Team Attacks and How to Find Them
 
What is ATT&CK coverage, anyway? Breadth and depth analysis with Atomic Red Team
What is ATT&CK coverage, anyway? Breadth and depth analysis with Atomic Red TeamWhat is ATT&CK coverage, anyway? Breadth and depth analysis with Atomic Red Team
What is ATT&CK coverage, anyway? Breadth and depth analysis with Atomic Red Team
 
Security Strategy and Tactic with Cyber Threat Intelligence (CTI)
Security Strategy and Tactic with Cyber Threat Intelligence (CTI)Security Strategy and Tactic with Cyber Threat Intelligence (CTI)
Security Strategy and Tactic with Cyber Threat Intelligence (CTI)
 
How MITRE ATT&CK helps security operations
How MITRE ATT&CK helps security operationsHow MITRE ATT&CK helps security operations
How MITRE ATT&CK helps security operations
 
Global Cyber Threat Intelligence
Global Cyber Threat IntelligenceGlobal Cyber Threat Intelligence
Global Cyber Threat Intelligence
 
MITRE ATT&CKcon 2018: Hunters ATT&CKing with the Data, Roberto Rodriguez, Spe...
MITRE ATT&CKcon 2018: Hunters ATT&CKing with the Data, Roberto Rodriguez, Spe...MITRE ATT&CKcon 2018: Hunters ATT&CKing with the Data, Roberto Rodriguez, Spe...
MITRE ATT&CKcon 2018: Hunters ATT&CKing with the Data, Roberto Rodriguez, Spe...
 
Bsides 2019 - Intelligent Threat Hunting
Bsides 2019 - Intelligent Threat HuntingBsides 2019 - Intelligent Threat Hunting
Bsides 2019 - Intelligent Threat Hunting
 
MITRE ATT&CKcon 2018: Helping Your Non-Security Executives Understand ATT&CK ...
MITRE ATT&CKcon 2018: Helping Your Non-Security Executives Understand ATT&CK ...MITRE ATT&CKcon 2018: Helping Your Non-Security Executives Understand ATT&CK ...
MITRE ATT&CKcon 2018: Helping Your Non-Security Executives Understand ATT&CK ...
 
Using IOCs to Design and Control Threat Activities During a Red Team Engagement
Using IOCs to Design and Control Threat Activities During a Red Team EngagementUsing IOCs to Design and Control Threat Activities During a Red Team Engagement
Using IOCs to Design and Control Threat Activities During a Red Team Engagement
 

Similar a Creating Your Own Threat Intel Through Hunting & Visualization

Protecting Financial Networks from Cyber Crime
Protecting Financial Networks from Cyber CrimeProtecting Financial Networks from Cyber Crime
Protecting Financial Networks from Cyber CrimeLancope, Inc.
 
RIoT (Raiding Internet of Things) by Jacob Holcomb
RIoT  (Raiding Internet of Things)  by Jacob HolcombRIoT  (Raiding Internet of Things)  by Jacob Holcomb
RIoT (Raiding Internet of Things) by Jacob HolcombPriyanka Aash
 
Splunk Enterpise for Information Security Hands-On
Splunk Enterpise for Information Security Hands-OnSplunk Enterpise for Information Security Hands-On
Splunk Enterpise for Information Security Hands-OnSplunk
 
Splunk Enterprise for InfoSec Hands-On Breakout Session
Splunk Enterprise for InfoSec Hands-On Breakout SessionSplunk Enterprise for InfoSec Hands-On Breakout Session
Splunk Enterprise for InfoSec Hands-On Breakout SessionSplunk
 
Understanding Intrusion Detection & Prevention Systems (1).pptx
Understanding Intrusion Detection & Prevention Systems (1).pptxUnderstanding Intrusion Detection & Prevention Systems (1).pptx
Understanding Intrusion Detection & Prevention Systems (1).pptxRineri1
 
Using Splunk for Information Security
Using Splunk for Information SecurityUsing Splunk for Information Security
Using Splunk for Information SecuritySplunk
 
Using Splunk for Information Security
Using Splunk for Information SecurityUsing Splunk for Information Security
Using Splunk for Information SecurityShannon Cuthbertson
 
Visualization in the Age of Big Data
Visualization in the Age of Big DataVisualization in the Age of Big Data
Visualization in the Age of Big DataRaffael Marty
 
Discover advanced threats with threat intelligence - Jeremy Li
Discover advanced threats with threat intelligence - Jeremy LiDiscover advanced threats with threat intelligence - Jeremy Li
Discover advanced threats with threat intelligence - Jeremy LiJeremy Li
 
AI on Spark for Malware Analysis and Anomalous Threat Detection
AI on Spark for Malware Analysis and Anomalous Threat DetectionAI on Spark for Malware Analysis and Anomalous Threat Detection
AI on Spark for Malware Analysis and Anomalous Threat DetectionDatabricks
 
Novetta Cyber Analytics
Novetta Cyber AnalyticsNovetta Cyber Analytics
Novetta Cyber AnalyticsNovetta
 
Sumo Logic Cert Jam - Security Analytics
Sumo Logic Cert Jam - Security AnalyticsSumo Logic Cert Jam - Security Analytics
Sumo Logic Cert Jam - Security AnalyticsSumo Logic
 
CyberCrime in the Cloud and How to defend Yourself
CyberCrime in the Cloud and How to defend Yourself CyberCrime in the Cloud and How to defend Yourself
CyberCrime in the Cloud and How to defend Yourself Alert Logic
 
Security Certification: Security Analytics using Sumo Logic - Oct 2018
Security Certification: Security Analytics using Sumo Logic - Oct 2018Security Certification: Security Analytics using Sumo Logic - Oct 2018
Security Certification: Security Analytics using Sumo Logic - Oct 2018Sumo Logic
 
SplunkSummit 2015 - Splunk User Behavioral Analytics
SplunkSummit 2015 - Splunk User Behavioral AnalyticsSplunkSummit 2015 - Splunk User Behavioral Analytics
SplunkSummit 2015 - Splunk User Behavioral AnalyticsSplunk
 
The Golden Rules - Detecting more with RSA Security Analytics
The Golden Rules  - Detecting more with RSA Security AnalyticsThe Golden Rules  - Detecting more with RSA Security Analytics
The Golden Rules - Detecting more with RSA Security AnalyticsDemetrio Milea
 
The Heatmap
 - Why is Security Visualization so Hard?
The Heatmap
 - Why is Security Visualization so Hard?The Heatmap
 - Why is Security Visualization so Hard?
The Heatmap
 - Why is Security Visualization so Hard?Raffael Marty
 
MMIX Peering Forum and MMNOG 2020: Packet Analysis for Network Security
MMIX Peering Forum and MMNOG 2020: Packet Analysis for Network SecurityMMIX Peering Forum and MMNOG 2020: Packet Analysis for Network Security
MMIX Peering Forum and MMNOG 2020: Packet Analysis for Network SecurityAPNIC
 
Security From The Big Data and Analytics Perspective
Security From The Big Data and Analytics PerspectiveSecurity From The Big Data and Analytics Perspective
Security From The Big Data and Analytics PerspectiveAll Things Open
 

Similar a Creating Your Own Threat Intel Through Hunting & Visualization (20)

Protecting Financial Networks from Cyber Crime
Protecting Financial Networks from Cyber CrimeProtecting Financial Networks from Cyber Crime
Protecting Financial Networks from Cyber Crime
 
RIoT (Raiding Internet of Things) by Jacob Holcomb
RIoT  (Raiding Internet of Things)  by Jacob HolcombRIoT  (Raiding Internet of Things)  by Jacob Holcomb
RIoT (Raiding Internet of Things) by Jacob Holcomb
 
Splunk Enterpise for Information Security Hands-On
Splunk Enterpise for Information Security Hands-OnSplunk Enterpise for Information Security Hands-On
Splunk Enterpise for Information Security Hands-On
 
Cybersecurity - Jim Butterworth
Cybersecurity - Jim ButterworthCybersecurity - Jim Butterworth
Cybersecurity - Jim Butterworth
 
Splunk Enterprise for InfoSec Hands-On Breakout Session
Splunk Enterprise for InfoSec Hands-On Breakout SessionSplunk Enterprise for InfoSec Hands-On Breakout Session
Splunk Enterprise for InfoSec Hands-On Breakout Session
 
Understanding Intrusion Detection & Prevention Systems (1).pptx
Understanding Intrusion Detection & Prevention Systems (1).pptxUnderstanding Intrusion Detection & Prevention Systems (1).pptx
Understanding Intrusion Detection & Prevention Systems (1).pptx
 
Using Splunk for Information Security
Using Splunk for Information SecurityUsing Splunk for Information Security
Using Splunk for Information Security
 
Using Splunk for Information Security
Using Splunk for Information SecurityUsing Splunk for Information Security
Using Splunk for Information Security
 
Visualization in the Age of Big Data
Visualization in the Age of Big DataVisualization in the Age of Big Data
Visualization in the Age of Big Data
 
Discover advanced threats with threat intelligence - Jeremy Li
Discover advanced threats with threat intelligence - Jeremy LiDiscover advanced threats with threat intelligence - Jeremy Li
Discover advanced threats with threat intelligence - Jeremy Li
 
AI on Spark for Malware Analysis and Anomalous Threat Detection
AI on Spark for Malware Analysis and Anomalous Threat DetectionAI on Spark for Malware Analysis and Anomalous Threat Detection
AI on Spark for Malware Analysis and Anomalous Threat Detection
 
Novetta Cyber Analytics
Novetta Cyber AnalyticsNovetta Cyber Analytics
Novetta Cyber Analytics
 
Sumo Logic Cert Jam - Security Analytics
Sumo Logic Cert Jam - Security AnalyticsSumo Logic Cert Jam - Security Analytics
Sumo Logic Cert Jam - Security Analytics
 
CyberCrime in the Cloud and How to defend Yourself
CyberCrime in the Cloud and How to defend Yourself CyberCrime in the Cloud and How to defend Yourself
CyberCrime in the Cloud and How to defend Yourself
 
Security Certification: Security Analytics using Sumo Logic - Oct 2018
Security Certification: Security Analytics using Sumo Logic - Oct 2018Security Certification: Security Analytics using Sumo Logic - Oct 2018
Security Certification: Security Analytics using Sumo Logic - Oct 2018
 
SplunkSummit 2015 - Splunk User Behavioral Analytics
SplunkSummit 2015 - Splunk User Behavioral AnalyticsSplunkSummit 2015 - Splunk User Behavioral Analytics
SplunkSummit 2015 - Splunk User Behavioral Analytics
 
The Golden Rules - Detecting more with RSA Security Analytics
The Golden Rules  - Detecting more with RSA Security AnalyticsThe Golden Rules  - Detecting more with RSA Security Analytics
The Golden Rules - Detecting more with RSA Security Analytics
 
The Heatmap
 - Why is Security Visualization so Hard?
The Heatmap
 - Why is Security Visualization so Hard?The Heatmap
 - Why is Security Visualization so Hard?
The Heatmap
 - Why is Security Visualization so Hard?
 
MMIX Peering Forum and MMNOG 2020: Packet Analysis for Network Security
MMIX Peering Forum and MMNOG 2020: Packet Analysis for Network SecurityMMIX Peering Forum and MMNOG 2020: Packet Analysis for Network Security
MMIX Peering Forum and MMNOG 2020: Packet Analysis for Network Security
 
Security From The Big Data and Analytics Perspective
Security From The Big Data and Analytics PerspectiveSecurity From The Big Data and Analytics Perspective
Security From The Big Data and Analytics Perspective
 

Más de Raffael Marty

Exploring the Defender's Advantage
Exploring the Defender's AdvantageExploring the Defender's Advantage
Exploring the Defender's AdvantageRaffael Marty
 
Extended Detection and Response (XDR) An Overhyped Product Category With Ulti...
Extended Detection and Response (XDR)An Overhyped Product Category With Ulti...Extended Detection and Response (XDR)An Overhyped Product Category With Ulti...
Extended Detection and Response (XDR) An Overhyped Product Category With Ulti...Raffael Marty
 
How To Drive Value with Security Data
How To Drive Value with Security DataHow To Drive Value with Security Data
How To Drive Value with Security DataRaffael Marty
 
Cyber Security Beyond 2020 – Will We Learn From Our Mistakes?
Cyber Security Beyond 2020 – Will We Learn From Our Mistakes?Cyber Security Beyond 2020 – Will We Learn From Our Mistakes?
Cyber Security Beyond 2020 – Will We Learn From Our Mistakes?Raffael Marty
 
Artificial Intelligence – Time Bomb or The Promised Land?
Artificial Intelligence – Time Bomb or The Promised Land?Artificial Intelligence – Time Bomb or The Promised Land?
Artificial Intelligence – Time Bomb or The Promised Land?Raffael Marty
 
Understanding the "Intelligence" in AI
Understanding the "Intelligence" in AIUnderstanding the "Intelligence" in AI
Understanding the "Intelligence" in AIRaffael Marty
 
AI & ML in Cyber Security - Why Algorithms are Dangerous
AI & ML in Cyber Security - Why Algorithms are DangerousAI & ML in Cyber Security - Why Algorithms are Dangerous
AI & ML in Cyber Security - Why Algorithms are DangerousRaffael Marty
 
AI & ML in Cyber Security - Why Algorithms Are Dangerous
AI & ML in Cyber Security - Why Algorithms Are DangerousAI & ML in Cyber Security - Why Algorithms Are Dangerous
AI & ML in Cyber Security - Why Algorithms Are DangerousRaffael Marty
 
AI & ML in Cyber Security - Welcome Back to 1999 - Security Hasn't Changed
AI & ML in Cyber Security - Welcome Back to 1999 - Security Hasn't ChangedAI & ML in Cyber Security - Welcome Back to 1999 - Security Hasn't Changed
AI & ML in Cyber Security - Welcome Back to 1999 - Security Hasn't ChangedRaffael Marty
 
Security Insights at Scale
Security Insights at ScaleSecurity Insights at Scale
Security Insights at ScaleRaffael Marty
 
Creating Your Own Threat Intel Through Hunting & Visualization
Creating Your Own Threat Intel Through Hunting & VisualizationCreating Your Own Threat Intel Through Hunting & Visualization
Creating Your Own Threat Intel Through Hunting & VisualizationRaffael Marty
 
Big Data Visualization
Big Data VisualizationBig Data Visualization
Big Data VisualizationRaffael Marty
 
Workshop: Big Data Visualization for Security
Workshop: Big Data Visualization for SecurityWorkshop: Big Data Visualization for Security
Workshop: Big Data Visualization for SecurityRaffael Marty
 
Visualization for Security
Visualization for SecurityVisualization for Security
Visualization for SecurityRaffael Marty
 
The Heatmap
 - Why is Security Visualization so Hard?
The Heatmap
 - Why is Security Visualization so Hard?The Heatmap
 - Why is Security Visualization so Hard?
The Heatmap
 - Why is Security Visualization so Hard?Raffael Marty
 
DAVIX - Data Analysis and Visualization Linux
DAVIX - Data Analysis and Visualization LinuxDAVIX - Data Analysis and Visualization Linux
DAVIX - Data Analysis and Visualization LinuxRaffael Marty
 
Cloud - Security - Big Data
Cloud - Security - Big DataCloud - Security - Big Data
Cloud - Security - Big DataRaffael Marty
 
Cyber Security – How Visual Analytics Unlock Insight
Cyber Security – How Visual Analytics Unlock InsightCyber Security – How Visual Analytics Unlock Insight
Cyber Security – How Visual Analytics Unlock InsightRaffael Marty
 

Más de Raffael Marty (20)

Exploring the Defender's Advantage
Exploring the Defender's AdvantageExploring the Defender's Advantage
Exploring the Defender's Advantage
 
Extended Detection and Response (XDR) An Overhyped Product Category With Ulti...
Extended Detection and Response (XDR)An Overhyped Product Category With Ulti...Extended Detection and Response (XDR)An Overhyped Product Category With Ulti...
Extended Detection and Response (XDR) An Overhyped Product Category With Ulti...
 
How To Drive Value with Security Data
How To Drive Value with Security DataHow To Drive Value with Security Data
How To Drive Value with Security Data
 
Cyber Security Beyond 2020 – Will We Learn From Our Mistakes?
Cyber Security Beyond 2020 – Will We Learn From Our Mistakes?Cyber Security Beyond 2020 – Will We Learn From Our Mistakes?
Cyber Security Beyond 2020 – Will We Learn From Our Mistakes?
 
Artificial Intelligence – Time Bomb or The Promised Land?
Artificial Intelligence – Time Bomb or The Promised Land?Artificial Intelligence – Time Bomb or The Promised Land?
Artificial Intelligence – Time Bomb or The Promised Land?
 
Understanding the "Intelligence" in AI
Understanding the "Intelligence" in AIUnderstanding the "Intelligence" in AI
Understanding the "Intelligence" in AI
 
Security Chat 5.0
Security Chat 5.0Security Chat 5.0
Security Chat 5.0
 
AI & ML in Cyber Security - Why Algorithms are Dangerous
AI & ML in Cyber Security - Why Algorithms are DangerousAI & ML in Cyber Security - Why Algorithms are Dangerous
AI & ML in Cyber Security - Why Algorithms are Dangerous
 
AI & ML in Cyber Security - Why Algorithms Are Dangerous
AI & ML in Cyber Security - Why Algorithms Are DangerousAI & ML in Cyber Security - Why Algorithms Are Dangerous
AI & ML in Cyber Security - Why Algorithms Are Dangerous
 
AI & ML in Cyber Security - Welcome Back to 1999 - Security Hasn't Changed
AI & ML in Cyber Security - Welcome Back to 1999 - Security Hasn't ChangedAI & ML in Cyber Security - Welcome Back to 1999 - Security Hasn't Changed
AI & ML in Cyber Security - Welcome Back to 1999 - Security Hasn't Changed
 
Security Insights at Scale
Security Insights at ScaleSecurity Insights at Scale
Security Insights at Scale
 
Creating Your Own Threat Intel Through Hunting & Visualization
Creating Your Own Threat Intel Through Hunting & VisualizationCreating Your Own Threat Intel Through Hunting & Visualization
Creating Your Own Threat Intel Through Hunting & Visualization
 
Big Data Visualization
Big Data VisualizationBig Data Visualization
Big Data Visualization
 
Workshop: Big Data Visualization for Security
Workshop: Big Data Visualization for SecurityWorkshop: Big Data Visualization for Security
Workshop: Big Data Visualization for Security
 
Visualization for Security
Visualization for SecurityVisualization for Security
Visualization for Security
 
The Heatmap
 - Why is Security Visualization so Hard?
The Heatmap
 - Why is Security Visualization so Hard?The Heatmap
 - Why is Security Visualization so Hard?
The Heatmap
 - Why is Security Visualization so Hard?
 
DAVIX - Data Analysis and Visualization Linux
DAVIX - Data Analysis and Visualization LinuxDAVIX - Data Analysis and Visualization Linux
DAVIX - Data Analysis and Visualization Linux
 
Cloud - Security - Big Data
Cloud - Security - Big DataCloud - Security - Big Data
Cloud - Security - Big Data
 
Cyber Security – How Visual Analytics Unlock Insight
Cyber Security – How Visual Analytics Unlock InsightCyber Security – How Visual Analytics Unlock Insight
Cyber Security – How Visual Analytics Unlock Insight
 
AfterGlow
AfterGlowAfterGlow
AfterGlow
 

Último

IP addressing and IPv6, presented by Paul Wilson at IETF 119
IP addressing and IPv6, presented by Paul Wilson at IETF 119IP addressing and IPv6, presented by Paul Wilson at IETF 119
IP addressing and IPv6, presented by Paul Wilson at IETF 119APNIC
 
『澳洲文凭』买拉筹伯大学毕业证书成绩单办理澳洲LTU文凭学位证书
『澳洲文凭』买拉筹伯大学毕业证书成绩单办理澳洲LTU文凭学位证书『澳洲文凭』买拉筹伯大学毕业证书成绩单办理澳洲LTU文凭学位证书
『澳洲文凭』买拉筹伯大学毕业证书成绩单办理澳洲LTU文凭学位证书rnrncn29
 
PHP-based rendering of TYPO3 Documentation
PHP-based rendering of TYPO3 DocumentationPHP-based rendering of TYPO3 Documentation
PHP-based rendering of TYPO3 DocumentationLinaWolf1
 
办理多伦多大学毕业证成绩单|购买加拿大UTSG文凭证书
办理多伦多大学毕业证成绩单|购买加拿大UTSG文凭证书办理多伦多大学毕业证成绩单|购买加拿大UTSG文凭证书
办理多伦多大学毕业证成绩单|购买加拿大UTSG文凭证书zdzoqco
 
『澳洲文凭』买詹姆士库克大学毕业证书成绩单办理澳洲JCU文凭学位证书
『澳洲文凭』买詹姆士库克大学毕业证书成绩单办理澳洲JCU文凭学位证书『澳洲文凭』买詹姆士库克大学毕业证书成绩单办理澳洲JCU文凭学位证书
『澳洲文凭』买詹姆士库克大学毕业证书成绩单办理澳洲JCU文凭学位证书rnrncn29
 
Film cover research (1).pptxsdasdasdasdasdasa
Film cover research (1).pptxsdasdasdasdasdasaFilm cover research (1).pptxsdasdasdasdasdasa
Film cover research (1).pptxsdasdasdasdasdasa494f574xmv
 
SCM Symposium PPT Format Customer loyalty is predi
SCM Symposium PPT Format Customer loyalty is prediSCM Symposium PPT Format Customer loyalty is predi
SCM Symposium PPT Format Customer loyalty is predieusebiomeyer
 
Unidad 4 – Redes de ordenadores (en inglés).pptx
Unidad 4 – Redes de ordenadores (en inglés).pptxUnidad 4 – Redes de ordenadores (en inglés).pptx
Unidad 4 – Redes de ordenadores (en inglés).pptxmibuzondetrabajo
 
Top 10 Interactive Website Design Trends in 2024.pptx
Top 10 Interactive Website Design Trends in 2024.pptxTop 10 Interactive Website Design Trends in 2024.pptx
Top 10 Interactive Website Design Trends in 2024.pptxDyna Gilbert
 
TRENDS Enabling and inhibiting dimensions.pptx
TRENDS Enabling and inhibiting dimensions.pptxTRENDS Enabling and inhibiting dimensions.pptx
TRENDS Enabling and inhibiting dimensions.pptxAndrieCagasanAkio
 
Company Snapshot Theme for Business by Slidesgo.pptx
Company Snapshot Theme for Business by Slidesgo.pptxCompany Snapshot Theme for Business by Slidesgo.pptx
Company Snapshot Theme for Business by Slidesgo.pptxMario
 

Último (11)

IP addressing and IPv6, presented by Paul Wilson at IETF 119
IP addressing and IPv6, presented by Paul Wilson at IETF 119IP addressing and IPv6, presented by Paul Wilson at IETF 119
IP addressing and IPv6, presented by Paul Wilson at IETF 119
 
『澳洲文凭』买拉筹伯大学毕业证书成绩单办理澳洲LTU文凭学位证书
『澳洲文凭』买拉筹伯大学毕业证书成绩单办理澳洲LTU文凭学位证书『澳洲文凭』买拉筹伯大学毕业证书成绩单办理澳洲LTU文凭学位证书
『澳洲文凭』买拉筹伯大学毕业证书成绩单办理澳洲LTU文凭学位证书
 
PHP-based rendering of TYPO3 Documentation
PHP-based rendering of TYPO3 DocumentationPHP-based rendering of TYPO3 Documentation
PHP-based rendering of TYPO3 Documentation
 
办理多伦多大学毕业证成绩单|购买加拿大UTSG文凭证书
办理多伦多大学毕业证成绩单|购买加拿大UTSG文凭证书办理多伦多大学毕业证成绩单|购买加拿大UTSG文凭证书
办理多伦多大学毕业证成绩单|购买加拿大UTSG文凭证书
 
『澳洲文凭』买詹姆士库克大学毕业证书成绩单办理澳洲JCU文凭学位证书
『澳洲文凭』买詹姆士库克大学毕业证书成绩单办理澳洲JCU文凭学位证书『澳洲文凭』买詹姆士库克大学毕业证书成绩单办理澳洲JCU文凭学位证书
『澳洲文凭』买詹姆士库克大学毕业证书成绩单办理澳洲JCU文凭学位证书
 
Film cover research (1).pptxsdasdasdasdasdasa
Film cover research (1).pptxsdasdasdasdasdasaFilm cover research (1).pptxsdasdasdasdasdasa
Film cover research (1).pptxsdasdasdasdasdasa
 
SCM Symposium PPT Format Customer loyalty is predi
SCM Symposium PPT Format Customer loyalty is prediSCM Symposium PPT Format Customer loyalty is predi
SCM Symposium PPT Format Customer loyalty is predi
 
Unidad 4 – Redes de ordenadores (en inglés).pptx
Unidad 4 – Redes de ordenadores (en inglés).pptxUnidad 4 – Redes de ordenadores (en inglés).pptx
Unidad 4 – Redes de ordenadores (en inglés).pptx
 
Top 10 Interactive Website Design Trends in 2024.pptx
Top 10 Interactive Website Design Trends in 2024.pptxTop 10 Interactive Website Design Trends in 2024.pptx
Top 10 Interactive Website Design Trends in 2024.pptx
 
TRENDS Enabling and inhibiting dimensions.pptx
TRENDS Enabling and inhibiting dimensions.pptxTRENDS Enabling and inhibiting dimensions.pptx
TRENDS Enabling and inhibiting dimensions.pptx
 
Company Snapshot Theme for Business by Slidesgo.pptx
Company Snapshot Theme for Business by Slidesgo.pptxCompany Snapshot Theme for Business by Slidesgo.pptx
Company Snapshot Theme for Business by Slidesgo.pptx
 

Creating Your Own Threat Intel Through Hunting & Visualization

  • 1. Raffael Marty, CEO Creating Your Own Threat Intel Through Hunting & Visualization Tenerife, Spain February, 2016
  • 2. Creating Your Own Threat Intel Through Hunting & Visualization Raffael Marty, CEO
  • 3. Security. Analytics. Insight.3 Contents HUNTING AKA INTERNAL THREAT INTELLIGENCE THREAT INTELLIGENCE A PROCESS AND INFRASTRUCTURE VIEW 1 2 VISUALIZATION A THREAT INTELLIGENCE GOLD MINE 3
  • 5. Security. Analytics. Insight.5 • Products / Tools • Firewall - Blocks traffic based on pre-defined rules • Web Application Firewall - Monitors for signs of known malicious activity in Web traffic • Intrusion Prevention System - Looks for ‘signs’ of known attacks in traffic and protocol violations • Anti Virus - Looks for ‘signs’ of known attacks on the end system • Malware Sandbox - Runs new binaries and monitors their behavior for malicious signs • Security Information Management - Uses pre-defined rules to correlate signs from different data streams to augment intelligence • Vulnerability Scanning - Searches for known vulnerabilities and vulnerable software • Rely on pattern matching and signatures based knowledge from the past • Reactive -> always behind • Unknown and new threats -> won’t be detected • ‘Imperfect’ patterns and rules -> cause a lot of false positives We Are Monitoring - What is Going Wrong? Defense Has Been Relying On Past Knowledge
  • 6. Security. Analytics. Insight.6 Event Funnel - How We Used To Do It data rule-based
 correlation prioritization simple
 statistics attack candidates • What rules do you write? • Do the vendor provided rules work for you? • How do you define a priority 10 event? • High false positive rate! • Unless alerts are VERY focussed • High false negative rate! • Do you know what you don’t know?
  • 7. Security. Analytics. Insight.7 Then Came Threat Intelligence • How many hits do you really get? • You are missing most attacks IOCs • How do you match these efficiently against a real-time stream? • How do you de- duplicate and normalize these feeds? attack candidates 70–90% OF MALWARE SAMPLES ARE UNIQUE TO AN ORGANIZATION.
  • 8. Security. Analytics. Insight.8 Removing the Event Funnel - Hello Data Lake any 
 data Big Data Lake Rules • Storing more, and more diverse data • Kafka and “dynamic parsing” • Enabling large-scale processing • Spark, SparkStreaming, Storm, Parquet • Using “standard” data access (SQL, REST) • Plug in any other tool! context IOCs This per-se is not new …
  • 9. Security. Analytics. Insight.9 Adding Interactive - Analyst Driven Exploration any 
 data Big Data Lake Rules context IOCs … but first we get the human in the loop … Hunting • interactive visualization • analyst driven • machine assisted
  • 10. Security. Analytics. Insight.10 Hunting Creates Internal Threat Intelligence any 
 data Big Data Lake Rules context IOCs … then, let’s rethink our rules … Novel, Advanced Attacks internal TI
  • 11. Security. Analytics. Insight.11 Hunting Creates Internal Threat Intelligence any 
 data Big Data Lake Rules context IOCs … then, let’s rethink our rules … patterns anyone? internal TI Novel, Advanced Attacks Low False Positive Alerts Patterns
  • 12. Security. Analytics. Insight.12 Buzzword Bingo any 
 data Big Data Lake Rules context IOCs … and finally, we are buzzword compliant … behavioral monitoring
 scoring anomaly detection machine learning artificial intelligence “models” data science internal TI Patterns
  • 13. Security. Analytics. Insight.13 How Does All That Architecture Stuff Matter? In the following we’ll explore how this all matters … … but first, let’s see how visualization plays a key role here.
  • 15. Security. Analytics. Insight.15 “How Can We See, 
 Not To Confirm - But To Learn” - Edward Tufte
  • 16. Security. Analytics. Insight.16 Why Visualization?dport time
  • 17. Security. Analytics. Insight.17 SELECT count(distinct protocol) FROM flows; SELECT count(distinct port) FROM flows; SELECT count(distinct src_network) FROM flows; SELECT count(distinct dest_network) FROM flows; SELECT port, count(*) FROM flows GROUP BY port; SELECT protocol, count(CASE WHEN flows < 200 THEN 1 END) AS [<200], count(CASE WHEN flows>= 201 AND flows < 300 THEN 1 END) 
 AS [201 - 300], count(CASE WHEN flows>= 301 AND flows < 350 THEN 1 END) 
 AS [301 - 350], count(CASE WHEN flows>= 351 THEN 1 END) AS [>351] FROM flows GROUP BY protocol; SELECT port, count(distinct src_network) FROM flows GROUP BY port; SELECT src_network, count(distinct dest_network) FROM flows GROUP BY port; SELECT src_network, count(distinct dest_network) AS dn, sum(flows) FROM flows GROUP BY port, dn; SELECT port, protocol, count(*) FROM flows GROUP BY port, protocol; SELECT sum(flows), dest_network FROM flows GROUP BY dest_network; … One Graph Summarizes Dozens of Queries port dest_network protocol src_network flows
  • 18. Security. Analytics. Insight.18 Visualization To … Present / Communicate Discover / Explore
  • 19. Security. Analytics. Insight.19 We will have a look at a couple components from earlier: • Context • Data Science • Clustering • Seriation - Data Science Gone Wrong • Time-series Analysis Analytics Components
  • 20. Security. Analytics. Insight.20 Did You Know? Users accessing Sharepoint servers User Sharepoint Server data processing visualization This graph of users accessing sharepoint servers, does not immediately reveal any interesting patterns.
  • 21. Security. Analytics. Insight.21 Did You Know - How Context Tells a Story Using HR data as context Remote User San Francisco Office User Sharepoint Server data processing visualization HR data Using color to add context to the graph helps immediately identify outliers and potential problems.
  • 22. Security. Analytics. Insight.22 • Simple stuff works! • dc(dest), dc(d_port) • What is normal? • Use data science / data mining to prepare data. Then visualize the output for human analyst. Data Science in Security - Words of Caution
  • 23. Security. Analytics. Insight.23 Challenges With Clustering Network Traffic The graph shows an abstract space with colors being machine identified clusters. Hard Questions: • What are these clusters? • Do Web servers cluster? • What are good clusters? • What’s anomalous?
  • 24. Security. Analytics. Insight.24 Data Science That Works threshold outliers have different magnitudes
  • 25. Security. Analytics. Insight.25 Approximate Curve fitting a curve distance to curve
  • 26. Security. Analytics. Insight.26 Data Mining Applied better 
 threshold
  • 28. Security. Analytics. Insight.28 Hunting - Ready, Fire, Aim • Analysts are your best and most expensive resource • They need the right tools and data • Speed (see earlier architecture) • Interaction (visual!) • Machine-assisted insight Examples • Exploring DNS traffic • High business impact machine analysis • Lateral movement
  • 29. Security. Analytics. Insight.29 HBI Metric Analysis Visually learn, Test, Automate
  • 30. Security. Analytics. Insight.30 HBI Metric Analysis - If you like Black Backgrounds
  • 31. Security. Analytics. Insight.31 We have tried many thing: • Social Network Analysis • Seasonality detection • Entropy over time • Frequent pattern mining • Clustering All kinds of challenges. Simple works! Let’s Get Mathematical U−matrix 4.28e−05 0.0461 0.0921
  • 33. Security. Analytics. Insight.33 Lateral Movement - Cross Network Communications Challenges • Scale • You will find one of everything • Defining white-lists and keeping them up to date (i.e., network and asset hygiene) VPN DMZ Office GIA Unknown Internet AWS
  • 34. Security. Analytics. Insight.34 http://secviz.org List: secviz.org/mailinglist Twitter: @secviz Share, discuss, challenge, and learn about security visualization. Security Visualization Community
  • 35. Security. Analytics. Insight.35 BlackHat Workshop Visual Analytics Delivering Actionable Security Intelligence July 30,31 & August 1,2 - Las Vegas, USA big data | analytics | visualization http://secviz.org
  • 36. Security. Analytics. Insight.36 After some exploration … raffael.marty@pixlcloud.com http://slideshare.net/zrlram http://secviz.org and @secviz Further resources: