IT data visualization for - Perimeter Threat - Insider Threat More on security visualization at http://secviz.org

1. IT Data Visualization
Raffael Marty, GCIA, CISSP
Chief Security Strategist @ Splunk>
SUMIT, Michigan - October ‘08

2. Raffael Marty
• Chief Security Strategist @ Splunk>
• Looked at logs/IT data for over 10 years
- IBM Research
- Conference boards / committees
• Presenting around the world on SecViz
• Passion for Visualization
Applied Security Visualization
- http://secviz.org Paperback: 552 pages
Publisher: Addison Wesley (August, 2008)
- http://afterglow.sourceforge.net
ISBN: 0321510100

3. Agenda
• IT Data Visualization
- Security Visualization Dichotomy
- Research Dichotomy
Visualization is a more effective
• IT Data Management way of IT data management and
analysis.
- A shifted crime landscape
• Perimeter Threat
• Insider Threat
• Security Visualization Community
3

4. Visualization Questions
• Who analyzes logs?
• Who uses visualization for log analysis?
• Who has used DAVIX?
• Have you heard of SecViz.org?
• What tools are you using for log analysis?
4

5. IT Data Visualization
Applied Security Visualization, Chapter 3

6. What is Visualization?
Generate a picture from IT data
A picture is worth a thousand log records.
Explore and Inspire
Discover
Answer a Pose a New Increase Communicate Support
Question Question Efficiency Information Decisions
6

7. Information Visualization Process
Capture Process Visualize
7

8. The 1st Dichotomy
Security Visualization
• security data • types of data
• networking protocols • perception
two domains
• routing protocols (the Internet) • optics
• security impact • color theory
Security & Visualization
• security policy • depth cue theory
• jargon • interaction theory
• use-cases • types of graphs
• are the end-users • human computer interaction
8

9. The Failure - New Graphs
9

10. The Right Thing - Reuse Graphs
10

11. The Failure - The Wrong Graph
11

12. The Right Thing - Adequate Graphs
12

13. The Failure - The Wrong Integration
/usr/share/man/man5/launchd.plist.5
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
• Using proprietary data format <plist version="1.0">
<dict>
<key>_name</key>
• Provide parsers for various data formats <dict>
<key>_isColumn</key>
<string>YES</string>
<key>_isOutlineColumn</key>
• does not scale <string>YES</string>
<key>_order</key>
<string>0</string>
</dict>
• is probably buggy / incomplete <key>bsd_name</key>
<dict>
<key>_order</key>
<string>62</string>
• Use wrong data access paradigm </dict>
<key>detachable_drive</key>
<dict>
• complex configuration <key>_order</key>
<string>59</string>
</dict>
e.g., needs an SSH connection <key>device_manufacturer</key>
<dict>
<key>_order</key>
<string>41</string>
</dict>
<key>device_model</key>
<dict>
<key>_order</key>
<string>42</string>
</dict>
<key>device_revision</key>
13

14. The Right Thing - KISS
/usr/share/man/man5/launchd.plist.5
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
• Keep It Simple Stupid <plist version="1.0">
<dict>
<key>_name</key>
<dict>
• Use CSV input <key>_isColumn</key>
<string>YES</string>
<key>_isOutlineColumn</key>
<string>YES</string>
• Use files as input <key>_order</key>
<string>0</string>
</dict>
<key>bsd_name</key>
# Using node sizes:
• Offload to other tools <dict>
<key>_order</key>
<string>62</string> size.source=1;
</dict>
• parsers <key>detachable_drive</key>
<dict>
size.target=200
<key>_order</key>
<string>59</string>
maxNodeSize=0.2
• data conversions </dict>
<key>device_manufacturer</key>
<dict>
<key>_order</key>
<string>41</string>
</dict>
<key>device_model</key>
<dict>
<key>_order</key>
<string>42</string>
</dict>
<key>device_revision</key>
14

15. The Failure - Unnecessary Ink
15

16. The Right Thing - Apply Good Visualization Practices
• Don't use graphics to decorate a few numbers
• Reduce data ink ratio
• Visualization principles
16

17. The 2nd Dichotomy
Some comments are based on paper reviews from
RAID 2007/08, VizSec 2007/08
Industry Academia
• don’t understand the real impact • don’t know what’s been done in industry
• get the 70% solution • don’t understand the use-cases
two worlds
• don’t think big • don’t understand the environments /
data / domain
• no time/money for real research
Industry & Academia
• can’t scale
•
•
work on simulated data
construct their own problems
• work based off of a few • use overly complicated, impractical
customer’s input solutions
• use graphs / visualization where it is not
needed
17

18. The Way Forward
• Building a secviz discipline
• Bridging the gap Security Visualization
• Learning the “other” discipline
• More academia / industry collaboration
SecViz
18

19. My Focus Areas
• Use-case oriented visualization
• IT data management
• Perimeter Threat
• Governance Risk Compliance (GRC)
• Insider Threat
• IT data visualization
• SecViz.Org
• DAVIX
19

20. IT Data Management

21. A Shifted Crime Landscape
• Crimes are moving up the stack
• Insider crime Application Layer
• Large-scale spread of many small attacks Transport Layer
Questions are not known in advance!
Network Layer
• Are you prepared? Have the data when you need it!
Link Layer
• Are you monitoring enough?
Physical Layer
21

22. What Is IT Data?
/var/log/messags multi-line files
Logs /opt/log/*
/etc/syslog.conf entire files
Configurations /etc/hosts
1.3.6.1.2.1.25.3.3.1.2.2 multi-line structures
Traps & Alerts iso. org. dod. internet. mgmt. mib-2. host. hrDevice.
hrProcessorTable. hrProcessorEntry. hrProcessorLoad
ps multi-line table format
Scripts & Code netstat
File system changes hooks into the OS
Change Events Windows Registry
The IT Search Company

23. Perimeter Threat
Applied Security Visualization, Chapter 6

24. Sparklines
• "Data-intense, design-simple, word-sized graphics". Edward Tufte (2006). Beautiful Evidence. Graphics Press.
Average } Standard Deviation
• Examples: • Java Script Implementation:
- stock price over a day http://omnipotent.net/jquery.sparkline/
- access to port 80 over the last week
24

25. Port
Sparklines
Source IP Destination IP
25

26. Insider Threat
Applied Security Visualization, Chapter 8

27. Three Types of Insider Threats
Information
Fraud
Leak
Sabotage
27

28. Example - Insider Threat Visualization
• More and other data sources than for • The questions are not known in advance!
the traditional security use-cases • Visualization provokes questions and
• Insiders often have legitimate access helps find answers
to machines and data. You need to log • Dynamic nature of fraud
more than the exceptions • Problem for static algorithms
• Insider crimes are often executed on • Bandits quickly adapt to fixed threshold-
the application layer. You need based detection systems
transaction data and chatty • Looking for any unusual patterns
application logs
28

29. User Activity
Color indicates
failed logins High ratio of failed logins
29

30. 30

31. Security Visualization
Community

32. SecViz - Security Visualization
This is a place to share, discuss, challenge, and learn about
security visualization.

33. V
D X
Data Analysis and Visualization Linux
davix.secviz.org

34. Tools
Capture Processing Visualization
- Network tools - Shell tools - Network Traffic
‣ Argus ‣ awk, grep, sed ‣ EtherApe
- Graphic preprocessing ‣ InetVis
‣ Snort
‣ tnv
‣ Wireshark ‣ Afterglow
- Generic
- Logging ‣ LGL
‣ Afterglow
‣ syslog-ng - Date enrichment
‣ Treemap
- Fetching data ‣ geoiplookup
‣ Mondrian
‣ wget ‣ whois/gwhois
‣ R Project
‣ ftp
‣ scp * Non-concluding list of tools

35. Thank You!
raffy @ splunk . com