This document summarizes how Digital River, a global e-commerce company, uses Splunk for real-time security monitoring and business analytics. It discusses how Splunk logs their web application firewall events, correlates security data, and provides global views of attacks. Splunk also replaces many of their databases by directly ingesting data and provides real-time monitoring of traffic, sales, and response times. Digital River plans to expand Splunk usage to additional log and business data sources for centralized reporting, network planning, and executive dashboards.
2. Digital River
Founded in 1994
Global leader in e-commerce
Cloud commerce, marketing and payment
solutions
1400 global employees
Builds and manages online businesses for
more than 40,000 software publishers,
consumer technology manufacturers,
distributors, online retailers and affiliates
Clients include Microsoft, EA, Adobe,
Autodesk, Nuance, Logictech, nVidia,
Trend Micro, Kaspersky, Capcom, Square Enix
2
3. About Me
Computer science background – 15+ years IT and misc development
Application Security Analyst at Digital River for 2 years
– Static and dynamic analysis of our applications
– Manage our web application firewalls and security policies
– Help push discovered defects through the development process
– Security research
Systems Administrator at Digital River for 1.5 years
- Monitor and assist with troubleshooting issues
Have become one of the two Splunk owners in the organization
Photographer, inventor, security, 3D Printer and Unix enthusiast. Using Linux
since ’93, Slackware 1.0. Irony, Digital River acquired ftp.cdrom.com with
Simtel
3
4. Splunk for Real-time Monitoring of Attacks
Web Application Firewall (WAF)
– System can only store 300,000 events (3 days worth of events)
– Needed at least a year's retention of data
– Filtering capability very limited
Splunk natural fit for logging these security events
– Splunk tracked security events across our data centers
– Not only logging security events for historical purpose but also real-time
dashboards
– Work-flow action to understand exact request attacker used and how web
application responded
4
6. Correlated Searches
Currently correlating searches based on IP Address
Investigate IDS events and WAF events from same IP address or
multiple attacks based on destination addresses
Just scratching the surface
6
Most security tools have limited search capabilities. Splunk makes any search you
can imagine possible.
8. Replacing Databases for Real-time Monitoring
Currently use many databases for logging
– Biggest use is for tracking global site page hits
– Millions of rows per day across 20+ databases with 70+ million rows each
Pulled page hit data into Splunk with Splunk DB Connect
– Showed Developer, Engineering and Ops Teams
– Suggested to log application directly to Splunk
– RESULT: real-time views of all traffic across all data centers
8
Onboarding new data into Splunk was easy with Splunk DB Connect
9. Global View of Malicious Attacks
• Global view of malicious
attacks and where they are
coming from
9
• Heat map and alerts allow for
drill down to events for
investigation
10. Help visualize the Business with
Real-Time Sales Data
By generating events on the final check out page from the e-commerce
system, we can see sales as they happen across our global
infrastructure
Real-time trending items – what are people placing in their cart and
purchasing
10
12. Real-Time monitoring of response times
• Big push for better operational monitoring of external processing
pipelines
• External entity had problem with monitoring health of their environment
• Creative solutions like watching data over the wire and generating events
to Splunk with our Web Application Firewalls.
12
14. Splunk for Central Reporting
Splunk to be main aggregation point for divergent data
1414
SolarWinds
SCOM
vCOPS for
Vmware
Synergetic
Interoperability
Security Events
Legacy
Monitoring
Logs
…everything else
15. App by Security Architect Matt Kirby, LogCompactor
http://splunk-base.splunk.com/apps/69654/logcompactor
15
16. Future
• Database audit logs going into Splunk for reporting and trends
• Use Cisco app for telecom/networking planning
• Monitoring site deployments, 200+ per day
- Visualizing operational change, performance and incidents in one view
• Dashboards for executives/business data
• Automatic validation of malicious attacks
- Kick of a security scan or other analysis to confirm or reject malicious attempts
16
With Splunk it’s easy to bring together many disparate data sources.
Did you use Splunk for your data center move at all? (John Bauer): It’s currently in progress. Yes, basically – oh so, yes, I’ll tell you about the whole top down view. We’re basically planning on having Splunk be the main aggregation point for all these divergent systems that kind of compete in the same space. So we use (Scum), we use SolarWinds, we use vCOPS for VMware, we use Nagios internally in some places. So what I’ve done with like SolarWinds and vCOPS is now I trap alerts to Splunk so, even though I don’t have like the fully enriched highly finely grained metric data, I at least have, when those other external systems notice events I get traps about them so I can – right now I’m not using that data but it’s basically – Splunk will be the central like reporting and like a stash-board spot to correlate all those things – all these events coming in from all these different systems. And actually, I have a white-board screen that has a lot more points but I could basically put together like a screen shot for you.