This document summarizes a presentation about using Splunk for IT operations. It discusses how Splunk can map machine data to entities, applications, services and visualize relationships. It then walks through a scenario where an issue is reported, and Splunk is used to investigate an alert related to long database queries, create a ticket, and set up an alert to proactively notify teams of such issues.
33. The 6th Annual Splunk Worldwide Users’ Conference
September 21-24, 2015 The MGM Grand Hotel, Las Vegas
• 50+ Customer Speakers
• 50+ Splunk Speakers
• 35+ Apps in Splunk Apps Showcase
• 65 Technology Partners
• 4,000+ IT & Business Professionals
• 2 Keynote Sessions
• 3 days of technical content (150+ Sessions)
• 3 days of Splunk University
– Get Splunk Certified
– Get CPE credits for CISSP, CAP, SSCP, etc.
– Save thousands on Splunk education!
33
Register at: conf.splunk.com
Both IT and business professionals can analyze machine data to get real-time visibility and operational intelligence.
With our platform for machine data, organizations can meaningfully improve their performance in a wide range of areas
e.g. meet service levels, reduce costs, mitigate security risks, maintain compliance and gain insights.
Here's how using Splunk and your machine data can drive significant benefits for your organization.
Search and investigation. Using Splunk, organizations identify and resolve issues up to 70% faster and reduce costly escalations by up to 90%. Splunk is one place to find and fix problems, and investigate incidents across all your IT systems and infrastructure.
Proactive monitoring. Monitor IT systems in real time to identify issues, problems and attacks before they impact your customers, services and revenue. Splunk keeps watch of specific patterns, trends and thresholds in your machine data so you don't have to. Trigger notifications in real-time via email or RSS, execute a script to take remedial actions, send an SNMP trap to your system management console or generate a service desk ticket.
Operational visibility. See the whole picture, track performance and make better decisions. Visualize usage trends to better plan for capacity; spot SLA infractions, track how you are being measured by the business. Do all of this using your existing machine data without spending millions of dollars instrumenting your IT infrastructure.
Real-time business insight. Make better-informed business decisions by understanding trends, patterns and gaining Operational Intelligence from your machine data. See the success of new online services by channel or demographic, reconcile 3rd-party service provider fees against actual use, find your heaviest users and heaviest abusers, and more. Because machine data captures every behavior, the possibilities are game changing. You'll find the lead times to get to this intelligence dramatically less than other solutions - measured in minutes/hours instead of months.
Splunk is the leading platform for machine data analytics with over 7,000 organizations using Splunk – for data volumes ranging from tens of GBs to tens of TBs to over 100 TBs of data PER DAY.
Splunk software reliably collects and indexes all the streaming data from IT systems, technology devices and the Internet of Things in real-time - tens of thousands of sources in unpredictable formats and types. Splunk software is optimized for real-time, low latency and interactivity.
Organizations use Splunk software and their data the following ways:
1. Find and fix problems dramatically faster
2. Automatically monitor to identify issues, problems and attacks
3. Gain end-to-end visibility to track and deliver on IT KPIs and make better-informed IT decisions
4. Gain real-time insight from operational data to make better-informed business decisions
This is described as Operational Intelligence: visibility, insights and intelligence from operational data.
Splunk Cloud is currently only available in the United States and Canada.
The CIM allows you to normalize your data to match a common standard, using the same field names and event tags for equivalent events from different sources or vendors.
The CIM acts as a search-time schema ("schema-on-the-fly") to allow you to define relationships in the event data while leaving the raw machine data intact.
Once you have normalized the data from multiple different source types, you can develop reports, correlation searches, and dashboards to present a unified view of a data domain.
You can display your normalized data in the dashboards provided by other Splunk-developed applications such as the Splunk App for Enterprise Security and the Splunk App for PCI Compliance.
ITOps – Heterogonous environments – Who has one type of Server, Storage, Switch, Firewall?
- Database
- Select
Splunk TA and SA Map to CIM
Where does Splunk Fit with CIM – Schema on the Fly Map field names and event tags for equivalent events from different sources or vendors.
Splunk APP - A Splunk App is a prebuilt collection of dashboards, panels and UI elements powered by saved searches and packaged for a specific technology or use case to make Splunk immediately useful and relevant to different roles.
Splunk Add-on – Capture/Index Data Identify relative events, field extractions, tags, CIM Compliancy
Why do they work – Come prepackaged with inputs, props, transforms to standardize the obtaining the data, indexing of data, Search Time Extractions, saved searches, macros
Where do you put them – They tell you where to put them, NIX addon goes on Forwarder, Indexer, Searchhead, Deployment Server
Splunk 6.1 is our latest version of Splunk software – the industry-leading machine data platform.
Lets recap what Splunk Enterprise 6.1 delivers:
Enabling the Mission-critical Enterprise
Continuous availability of mission-critical machine data with expanded insights from new sources
Multi-site Clustering: Delivers continuous availability for Spunk Enterprise deployments that span sites, countries or continents by replicating raw and indexed data in a clustered configuration
Search Affinity: Provides a performance increase when using multi-site clustering by routing search and analytics requests to the nearest cluster, increasing performance and decreasing network usage.
zLinux Forwarder: Allows for application and platform data from IBM mainframes to be easily collected and indexed by Splunk Enterprise.
Data Preview with Structured Inputs: Enables previewing of massive data files to verify alignment of fields and headers before indexing improving data quality and the time it takes to discover critical insights.
Delivering Enhanced Interactive Analytics
Easier to build dashboards and more interactive visualizations.
Enhanced Dashboard Editor: Provides the ability to build advanced dashboards through the UI and without requiring advanced XML coding.
Chart Overlay: Improves data analysis by providing the ability to overlay one chart on top of another.
Contextual Drilldown: Enables more detailed insights when clicking on a dashboard panel without leaving the context of the dashboard itself.
Pan and Zoom Controls: Enables more focused analytics by providing the ability to selecting a range of interest on a chart and zooming in for deeper analysis.
Embedding Operational Intelligence
Extends Operational Intelligence to common business applications.
Embedded Reports: Enable any report, table to be embedded in third-party business applications such as salesforce.com, WordPress, Wiki Microsoft® SharePoint, and more.
Custom Alerts: Deliver alerts with embedded machine data context reducing mean-time-to-resolution (MTTR), and provide ability to customize alert templates.
Splunk 6.1 represents a significant milestone in our mission to make machine data accessible, usable and valuable by everyone.
Find out more at www.splunk.com/6
https://54.146.150.218
Here is the raw data – date and other fields have been extracted
Eventypes –
Common Information Model Fields –
Src / Dest src_ip/dest_ip etc
Splunk Workflow – Event Actions to redirect User to another dashboard or action
Discuss CMDB Configuration Management Database –
What is it?
A repository that acts as a data warehouse for information technology (IT) organizations. Its contents are intended to hold a collection of IT assets that are commonly referred to as configuration items (CI), as well as descriptive relationships between such assets.
Can you Splunk it?
Oh yeah and use its details to enhance Splunk Data collection
Again CMDB Details
If we understand Host/Entity to Application to Services then we can search/visualize/report/alert on the time series events based on this detail right?
This is a customized for the items important for this NOC
Entities/Hosts -> Applications ->Services
We can evaluate the individual components that make up a Service from Host components Network/Storage/Compute
Why is this important?
MTTR
Capacity Planning
Everyone on the Same Page
Blame Games
Highlight Different Visualizations
GeoIP – Convienant when you looking for a correlation – sometime a link/pop goes down?
Logically break out visualizations to represent a flow
Highlight the common issues which could occur
Web – time_taken/ Response codes
Websphere – Java – time_taken/JVM Heap
Database – time_taken / active queries
Break out of details by query User CPU Memory
All the same data just pivoting on it in a different way allows for better visibility into what is happening
Workflow again – Standard activity that is repeatable
-
And finally, I would like to encourage all of you to attend our user conference in September.
The energy level and passion that our customers bring to this event is simply electrifying.
Combined with inspirational keynotes and 150+ breakout session across all areas of operational intelligence,
It is simply the best forum to bring our Splunk community together, to learn about new and advanced Splunk offerings, and most of all to learn from one another.