Splunk and Cisco UCS Breakout Session
•
3 recomendaciones•2,345 vistas
Splunk and Cisco UCS Breakout Session
Recomendados
Recomendados
Más contenido relacionado
La actualidad más candente
La actualidad más candente (20)
Destacado
Similar a Splunk and Cisco UCS Breakout Session
Similar a Splunk and Cisco UCS Breakout Session (20)
Más de Splunk
Más de Splunk (20)
Último
Último (20)
Splunk and Cisco UCS Breakout Session
- 1. Robert Novak, Cisco Big Data Partner CSE March 2016 Splunk in the Cisco UCS Ecosystem How Cisco and its customers deploy, use, and scale Splunk environments with the Cisco Unified Computing System
- 2. Who am I and why am I here? Today: Consulting Systems Engineer for Cisco’s Americas Partner Organization Focused on big data and analytics UNIX Sysadmin for ~20 years (retired) Full stack: servers, storage, network, coffee 149 to 149k person companies Sun, Nortel, 3PAR, Ebay, Trulia, Disney, etc “Big Data” herder since 2003 Hadoop admin (certifiable) since 2009 Cisco UCS C-Series admin since 2011 (early adopter!) Charter Cisco Champion, VMware vExpert since 2013 Blogger at rsts11.com and Cisco Blogs Tweeter at @gallifreyan and @rsts11
- 3. Agenda • Hardware still matters! • How Cisco uses Splunk internally • How some of our customers use Splunk on UCS • Cisco integrations with Splunk • The Unified Computing System advantage • Learn More
- 4. Hardware Still Matters A quick glance at infrastructure for Splunk
- 5. Why does hardware still matter? 5 • Splunk will run on almost anything (even my laptop) • Standalone servers have lower admin overhead • Build up your clusters and you have to keep them consistent • Grow your data sources (and uses) and you have to add servers • Cluster constipation is bad, mmmkay?
- 6. Why does hardware still matter? 6 • Cisco customer big data pools tend to grow 2-3x/year • Cisco customer IT staff doesn’t grow as fast • The Cisco Unified Computing System (UCS) provides scalable, repeatable, predictable, and manageable deployments across dozens to thousands of servers for any application deployment • Pallet to production in hours, not days or weeks • Deep engineering integration between Cisco and Splunk with tested and proven configurations More on this later…
- 7. How Cisco Uses Splunk Part 1 Operational Analytics at Enterprise Scale within Cisco IT
- 8. Big Data at a Big Customer: Cisco 8 • 10s of thousands of employees, contractors, devices • 100s of offices, business apps, audiences • Lots of data in lots of places • No one tool (not even Splunk) can do everything for everyone all the time • High volume, low value, low shelf life • Lancope, Hadoop feed into Splunk • Low to moderate volume, high value, (any) shelf life • Splunk on its own, sometimes with fronting dashboards • Additional visualizations with Platfora, Tableau, etc
- 9. A closer look at Splunk within Cisco 9 • Customer for 7+ years, strategic partner for 3+ years • Geographically disparate data collection and analysis • Over 70 business applications/use cases across the company • Around 20 teams using Splunk including Cisco IT and CSIRT • Nearly 10x growth in search volume from 2014-2016
- 10. 10 Indexers 16 Search Heads thanks to search head clustering in Splunk 6.3 47 Search Heads 20 Indexers Daily Indexing ~ 2TB 2014 2014 2015 2015 2015 Cisco’s IT Operations Evolving with Splunk Daily Indexing 300G 2010
- 11. Splunk Searches – Daily Average 1. Interactive Searches = 55K+ 2. Scheduled Searches = 45K+ 3. Total Searches = 100K+ 4. Number of Users = 180+
- 14. How Cisco Uses Splunk Part 2 Security Analytics at Enterprise Scale: Cisco’s Computer Security Incident Response Team (CSIRT)
- 15. About CSIRT • Cisco Computer Security Incident Response Team (CSIRT) • CSIRT = Security Monitoring and Incident Response • Architecture, Engineering, Research, and Investigations • Enterprise global threat and 24x7 incident response
- 16. CSIRT Environments Recent Snapshot 300 locations in 90 countries 400 buildings 1500+ labs 100,000+ employees on network 50-300 malware-related cases opened in a typical week 650,000+ ip devices on network 130,000 windows hosts 50,000 Linux hosts 40,000 routers 2-3 million highly tuned ids events per day 10+ billion netflow records per day
- 17. Deploying Splunk as SIEM • SIEM: Security Information and Event Management platform – Easy to index any type of machine data from any source – Over 60 users doing investigations, correlations, reporting, advanced threat detection – All the data + flexible searches and reporting = empowered and effective team – 2TB/day and searches take less than a minute. 7 global data centers with 350TB stored data – Flashback Malware contained to a fraction of the environment – Replaced older pre-big-data SIEM Previous solution didn’t scale effectively Queries in the minutes (or worse) rather than seconds with Splunk Diverse functionality across the same aggregate data
- 18. Looking at our customers Successful deployments with Cisco UCS and Splunk
- 19. Threat Management at Govt Agency 19 • Agency wanted to manage and monitor all relevant alert data • Needed visibility across multiple security platforms • Centralized on scalable appliance model through a partner • Splunk Enterprise with Enterprise Security[1] premium app • By deploying on Cisco UCS with proven Cisco Validated Design, partner was able to deliver easily upgraded and expanded deployment with predictable results [1] Splunk won Best SIEM Solution (Enterprise Security) and Best Fraud Prevention Solution (Splunk Enterprise) awards from SC Magazine this month (Splunk press release)
- 20. Fraud prevention for Online School 20 • Leading online university needed to track student activity • Federal agencies have stringent requirements for loan qualifications and fulfillment • Deployed Splunk on UCS for student activity tracking • Blocking millions in fraudulent loan claims • Saving over 75% on auditing and compliance expenses • Saving over $1M/year on data processing • Deployed and expanded other analytics (security operations, IT operations, and application deployment) • Splunk on UCS grows beyond initial use cases and teams at most of our customers
- 21. 21 • Leading worldwide financial services company used Splunk for IT Operations analytics • When an electronic payment platform deployment came up, Splunk was enlisted to support rollout and monitoring in ridiculously short time frame • Speed and scalability led to use cases for security and fraud detection/prevention, marketing optimization, customer engagement and offers, and more • Customer continues to grow their Splunk environment (over 10x in first year, and still growing!) IT Ops & beyond for financial services
- 22. 22 • Customer needed quick updates, secure services, and high availability • Deployed Splunk Enterprise on UCS to replace older hardware and software platforms that didn’t scale well • Splunk and UCS delivered a more robust security posture with faster investigation and resolution of security events • High performance security analytics solution enables hospital to identify attack patterns and unauthorized actions that would otherwise go undetected. • Reduced space/power/cooling by 75% • Server deployment time reduced from 7 days to 1 day. See Cisco’s case study at cisco.com and Splunk’s case study at splunk.com Secure Healthcare at Union Hospital
- 23. Got Cisco? There’s an app for that… (or a technology add-on, at least)
- 24. CiscoSecuritySuiteApp Splunk & Cisco Integrations Security Identity Services Engine (pxGrid) Sourcefire (including AMP) ASA/PIX/FWSM Firewalls Web Security Appliance (WSA) Email Security Appliance (ESA) IPS Cloud Web Security (CWS) AnyConnect OpenDNS, ThreatGrid (in development) Data Center/ Insieme Cisco UCS Nexus 9K Application Centric Infrastructure (ACI - APIC) UCS Integrated Infrastructures Optimized for Splunk Enterprise High Performance High Capacity Enterprise Networking Switching and Routing Catalyst Switches Nexus (1000V, 2000, 3000, 4000, 5000, 6000, 7000, 9000) Meraki Wireless NGN Routers (CRS, ASR, ISR) Open SDN Network Controller APIC EM Collaboration Call Manager • Inaugural SIEM & Threat Defense Partner • Inaugural pxGrid partner • Inaugural member of new Cisco Security Technical Alliances program • Inaugural ACI Partner • Inaugural Data Analytics Partner • Cisco Cloud Security for VMDC 1.0 Design Guide (link) • Cisco UCS Integrated Infrastructure for Splunk Enterprise (Distributed Deployment, High Capacity) (link) CiscoNetworksApp
- 25. Splunk App for Cisco UCS NEW AND IMPROVED as of May 28, 2016 Aggregates, monitors, trends and analyzes all relevant data from Cisco UCS Manager instances Enables proactive capacity and performance monitoring/ management, fault trending, power and cooling, and more Works with other Splunk add-ons and data sources (including Enterprise Security and PCI Compliance add-ons) to aggregate and correlate data across your enterprise 25 Application s Operating Systems Hypervisors UCS server, storage, network
- 26. Splunk on Cisco UCS
- 27. What is Cisco’s Unified Computing System (UCS)? Unified Management: UCS Manager uses policy-based configuration to ensure consistent deployments Unified Fabric: Integrated 10/40 Gigabit Ethernet and Storage Networking (FCoE/iSCSI) Service Profiles: Maintain consistency across batches of servers and multiple applications. Deploy and expand in record time. Performance: Built with 10GbE and 40GbE at the core, repeatable configurations and performance, and over 100 benchmark records
- 28. Why Splunk on Cisco UCS? Time to Deployment: Spin up a mutually validated, pre-tested environment in hours rather than days or weeks Total Cost of Ownership: Integrated networking and management reduce customer cost and effort to migrate, deploy, and expand Time to Grow: Expand servers and network capacity quickly and consistently
- 29. Cisco UCS + Splunk = Better Together Seamless Scalability Facilitates Rapid Growth – Scale Splunk from a single server to distributed/clustered deployment – Grow your clusters efficiently and consistently – Runs on the same UCS C-Series servers as other big data platforms Split Second Response Times – Exceptional performance for “needle-in-a-haystack” searches – Consistent performance as simultaneous users increase Simplified Repeatable Deployments – Four pre-tested UCS Integrated Infrastructures – Capacity or performance optimization – NEW! Cisco Validated Design (CVD) with HA and Archiving
- 30. 250 GB indexed per day 4 months retention 250 GB indexed per day 1 month retention Single Server Cisco UCS Reference Architectures UP to 4TB indexed per day 3 months Retention Up to 4TB indexed per day 1 year Retention Clustered Deployment Retention optimized Performance optimized
- 31. Cisco Validated Design (CVD) for Splunk • Developed by Cisco and Splunk engineers in Spring 2016 • 250+ page guide to design and deployment, pallet to production • Based on UCS C-Series (C220, C240, C3160) servers and Splunk Enterprise software • Includes high availability & data archiving • Download for free at cisco.com/go/bigdata_design
- 32. Splunk on UCS : Performance Benchmark Test bed Topology
- 33. Cisco UCS Benchmark Results (Splunk Enterprise 6.2 vs 6.3)
- 34. Learn more about Splunk and Cisco UCS
- 35. SplunkBase app resources: splunkbase.splunk.com Cisco’s Big Data Design Hub: cisco.com/go/bigdata_design features Cisco Validated Designs (CVDs) and other architectural docs Big Data Applications Hub: cisco.com/go/bigdata features reference architectures, solution briefs, infrastructure, automation, etc. Reach Out! Already using Splunk? Talk to your Splunk team about Cisco UCS! Already using Cisco UCS? Talk to your Cisco team about Splunk! Learn More About Splunk on Cisco UCS!
- 36. Cisco’s CSIRT engineers applied their experiences during the CSIRT deployment to a new O’Reilly book now available bitly.com/infosecplaybook “they wrote the book …” 36
- 37. Thank you.
Notas del editor
- Cisco does servers? – quick answer Cisco does big data? – almost-as-quick answer What’s with Cisco and Splunk? – lead into next segment
- Cisco does servers? – quick answer Cisco does big data? – almost-as-quick answer What’s with Cisco and Splunk? – lead into next segment
- Cisco does servers? – quick answer Cisco does big data? – almost-as-quick answer What’s with Cisco and Splunk? – lead into next segment
- Cisco does servers? – quick answer Cisco does big data? – almost-as-quick answer What’s with Cisco and Splunk? – lead into next segment
- 10
- Snapshot from earlier this year but could have really been taken anytime. This shows the growth trend mentioned earlier Over 500 unique users per month
- A look at our pre 6.2 environment Initially a search head pool was deployed for each client team that was integrated. Which was fine in the beginning. 47 SHs and 12 SHPs Painpoints: An administration nightmare Resource availability – Lots of compute dedicated overall for search heads but not it’s not always available where needed
- Current 6.2 based setup was built side by side with the existing pre 6.2 environment SHPs Migrated each client team over one by one If we had kept out heads down and didn’t know of the new features we would have continued down the same path that lead to headaches And with that, back to Robert, TY!
- The Computer Security Incident Response Team (CSIRT) reduces the risk of loss as a result of security incidents for Cisco-owned business. CSIRT regularly engages in proactive threat assessment, mitigation planning, incident trending with analysis, security architecture, incident detection and response. CSIRT is our internal protection team within Cisco that protects the integrity of Cisco’s information, resources, ecommerce environment, TAC, etc.
- This is the scope of what the Cisco CSIRT team is protecting. We are BIG…and Splunk works well for us because it scales, is flexible and adaptable!!
- Easy to index any type of machine data coming in from anywhere 60 users 7x24 around the globe investigating and reporting Massive amounts of data combined with flexible searches empowered the team at Cisco 2TB a day and searches take less than a 1 minute. 25% cost – multi-purpose tool, its not a dedicated niche or point product, Cisco saves money, does SIEM’s + Much more (Swiss Army Knife – does many things). “We moved to Splunk from traditional SIEM as Splunk is designed and engineered for “big data” use cases. Our previous SIEM was not and simply could not scale to the data volumes we have” Former Director, Cisco Computer Security Incident Response Team Cisco’s footprint of security monitoring spans across our 7 global data centers with searches taking less than a minute. A search could be anything from known exploits to uncommon error messages.
- Cisco does servers? – quick answer Cisco does big data? – almost-as-quick answer What’s with Cisco and Splunk? – lead into next segment
- Cisco does servers? – quick answer Cisco does big data? – almost-as-quick answer What’s with Cisco and Splunk? – lead into next segment
- Cisco does servers? – quick answer Cisco does big data? – almost-as-quick answer What’s with Cisco and Splunk? – lead into next segment
- Cisco does servers? – quick answer Cisco does big data? – almost-as-quick answer What’s with Cisco and Splunk? – lead into next segment
- The key value proposition for Splunk App for Cisco UCS is that it aggregates, monitors, trends and analyzes all data from all UCS managers in one central location. We also explored the importance of correlation. With this app our customers are getting the Cisco UCS data into Splunk where they can analyze it along side other types of data to create various reports, of performance metrics, security, alerts and more Also, UCS has massive scalability and it generates lots of data. Therefore it requires analytics and monitoring solution that can scale to match. And Splunk uniquely fulfills this requirement. Finally our Splunk App for UCS is now certified by Cisco. Few months ago, they passed Cisco’s rigorous Interoperability and Verification Testing and now they can put a stamp UCS validated or Cisco Compatible. What does this mean for you, if you’re a Cisco shop you can now deploy the App without worry of interoperability or doing your own verification testing! Recap: UCS and most management tools generate point in time information. UCS’s massive scalability generates lots of data. Combine that with Converged Infrastructure logs, OS, vSphere, Nexus, etc. and try to correlate. Bottom Line: It requires an analytics and monitoring solution that can scale. Splunk is effectively the “Sherlock Holmes” of data analysis.
- Why is UCS so valuable in big data deployments? Most people talk about the hardware capabilities and even though we hold countless benchmark records, its not as important as policy based configurations and management. When you talk big data, you can get started with any hardware. If you have 10-12 machines, who cares what brand they are or if they are white box. If something goes wrong, you deal with it. But what happens when the environment grows? What happens when you have dozens or hundreds of servers? How do you manage the firmware revisions? What about the specific components which cause driver conflicts, or stack interoperability issues? Unified Management: You don’t manage the endpoints, you manage the UCS-M and it manages the endpoints: Chassis, Switches, Blades, Adapters Unified Fabric – FCoE!! Less infrastructure to manage as you don’t have Fibre Channel and Ethernet switches! Just like we combined the Voice and Data network 12+ years ago. Less gear, less expensive and easier to manage! Is there a potential for a use case in your organization where a few servers are needed to access the SAN? For example, backups? Maybe a need for Isilon? Did you plan for this change? Don’t worry. With UCS, use software to turn on the feature and you gain access to all the data on the SAN w/o having to crack open a box or run a cable or install a SAN switch. Do you suddenly need VLAN partitioning for firewalls, security etc? Many customers have a requirement for both an internal and external network on the same system. In the past, it was another cable, another card, and a reconfiguration of the OS. Instead, with Cisco, you just add this in UCS Manager.
- Cisco and Splunk provides an infrastructure solution which improves the time to deployment, reduces operations costs, while providing the capability to expand over time. TCO in terms of people supporting the environment, time to deployment, MTTR, etc. One large media company had to rethink their strategy once they realized OPEX was sapping their profitability. They started out with white box and grew to 1400 units. How many engineers do you think it took to support it? Would you believe 40! The next environment was UCS and they grew to 700 servers before that engineer came to work for Cisco. Those 700 servers were managed by 1 engineer!! Do you think your company can afford $4-$6m in people costs per year to manage a large big data infrastructure? You can buy a lot of technology for that! Let’s reiterate or summarize the key benefits of UCS and Splunk together. While the data volume grows tremendously, Splunk maintains control with reliable and repeatable performance.
- As mentioned, Splunk can be deployed on a single server to get started, but as organizations realize the value the value they can extract from data they have an insatiable appetite for more insights. They are sifting through massive volumes of data for fraud detection, digging through multiple data sources to identify the extent of a breach. As a result, we have customers indexing a TB of data a day keen to expand their data sources or change retention periods. As Splunk environments grow, infrastructure can easily become a growth inhibitor. - Because Splunk software can address a variety of use cases, you can start from virtually any line of business or department and grow the implementation to fit your needs. Customers that experience the most compelling ROI are the ones that realize that analyzing machine data provides insights for every part of the organization. For example, simply pulling in web data allows your: IT Operations & Applications teams to receive real-time information on how web infrastructure updates are working in production and resolve issues before they impact the customer Marketing team to gain insight into usage trends on your website, allowing them to deliver the most impactful campaigns Security team to identify the fingerprints of fraud and stop fraudulent activity before it impacts your company and customers Business Analysts to identify and understand issues such as shopping cart abandonment Why others (HP/Dell) cant compete against UCS: Central management, integrated networking (UCS 10 Gig built in) designed and documented so you don’t outgrow your network.
- Why start from scratch?! Use a Reference Architecture that our two companies have verified are tried, true and rock solid! These are the four reference architecture bundles that are available. For Example: If you’re looking more into transaction history you go with Higher Retention; but if you’re a Card Swiping company looking for fraud, then Performance is more important. Also to note: Customers are clamoring for Cisco Validate Designs (CVD). They are coming soon and the rough ETA is Late Spring 2016. (Over 220 pages!)
- We also have 3rd party validation with Cisco UCS – well known for high performance systems. Today, I’d like to share with you an early preview of Cisco’s ongoing benchmark tests. 1 – First off – data indexing results show that with 16 cores you can get 4x the throughput of 6.2 2 – Running searches with 4 cores show that you can get 6x or more the speed of 6.2 3 – A typical indexing + search workload shows that you can get 3x the search performance and twice the data indexing with 6.3 Bottom line –Splunk Enterprise is ready to put your available CPU power to work to get more done, faster Watch for more results and a full report from Cisco on the benchmark tests. System – single 36 core UCS Server. Note that there are 8 concurrent searches for the search results.
- Here is Splunk on Cisco Marketplace and the link for the reference architecture. Marketplace Includes: - Big Data/UCS Solutions - Splunk for Cisco Security Environments - Splunk for Cisco Security Suite - Cisco ACI for Splunk Enterprise - Splunk for Cisco Networks Reference Architecture Highlights - Comprehensive Integrated Infrastructure - Real-Time Operational Intelligence - Powerful Search, Analysis, and Visualization - Built on Cisco UCS Advantages - Architectural Scalability Come by the Cisco table to pick up the Reference Architecture Brief and enter yourself to win a Go Pro camera! Appreciate you hanging in there today! Thank you and let me not hold you back for cocktails.
- Cisco’s three lead engineers behind the Splunk deployment wrote the book, Crafting the Infosec (information security) Playbook. - Available end of May or early June. http://www.amazon.com/Crafting-InfoSec-Playbook-Security-Monitoring/dp/1491949406 by Jeff Bollinger (Author), Brandon Enright (Author), Matthew Valites (Author) Tim O’Reilly http://en.wikipedia.org/wiki/O%27Reilly_Media