The document discusses a Splunk user group meeting about using Telegraf to monitor metrics. The agenda includes an introduction to Telegraf architecture, how to connect Telegraf with Splunk, deploying Telegraf, and analyzing metrics with Splunk. Attendees are encouraged to join the Slack channel and ask questions during the session, and the slides and recording will later be posted online.

1. Splunk
Mumbai User
Group
 Join splunk_mumbai_usergroup on Slack
 Use _mumbai_usergroup for Q&A during
session.
 Please keep your line muted .
 Questions/doubts to be entered in conversation.
 Slides, Recording and Feedback form will be
posted on the Event Page after the session.
https://usergroups.splunk.com/mumbai-splunk-user-group/
1

2. Agenda 1. Introduction
2. Why Metrics?
3. Telegraf Architecture
4. Connecting Telegraf with Splunk
5. Deployability
6. Metric Analytics
7. Q&A
2

3. • 2+ years of Splunk experience
• Senior Analyst at Avotrix
• Enterprise Security, ITSI, Phantom & UBA
• Web Developer
• Creating Blogs, Youtube Videos & many more
About me !
3

4. Introduction to Mumbai User Group
4

5. Splunk Metrics via
Telegraf
5

6. 6

7. Why Metrics ?
7

8. Ref: https://www.splunk.com/en_us/resources/videos/splunk-metric-store.html
8

9. 2000x
Splunk now handles metrics in its native, lightweight format which directly contributes to providing 2000x
performance increases over traditional log queries. 9

10. Logs vs Metrics
• Unstructured data
• Text based
• Scaling can be costly
• Needle in the haystack
• Proactive monitoring, alerting
• Great for anomaly detection trending
• Structured data
• Numeric based
• Cost Efficient Scaling
• Best way to observe a process/device
• Reactive
• Great for forensics analysis
10

11. Metric Data Format
metric_type, _dims, host, index, sourcetype and source are the by default internal fields and are not directly writable
Ref: https://conf.splunk.com/files/2019/slides/FN2268.pdf
11

12. Telegraf Architecture
12

13. 13

14. Telegraf Architecture
write metrics
to various
destinations
create
aggregate
metrics (e.g.
mean, min,
max, quantiles,
etc.)
transform,
decorate,
and/or filter
metrics
collect metrics
from the
system,
services, or
3rd party APIs
INPUT PROCESSORS OUTPUT
AGGREGATORS
14

15. Connecting Telegraf with Splunk
15

16. Deployability
16

17. Standalone
Deployment
•NO additional Splunk
components required
•Very small memory
and processor resource
requirements
•Talks directly to the
HEC
•Allows for centralized
management of
metrics collectors from
other tools (Ansible,
Puppet, etc.)
Sidecar
Deployment
Telegraf is installed
alongside a universal
or heavy forwarder
Splunk is configured to
read the file that
Telegraf outputs
Allows Splunk admins
to administer System
in real-time
Splunk has a monitor
the output file that
Telegraf generates
Splunk
Application
Deployment
Telegraf is installed on
a Universal or Heavy
forwarder by a
deployment server
Uses the Splunk
forwarder’s already
configured outputs to
ingest the data from
Telegraf
Scripted input controls
Telegraf’s configuration
file
Splunk starts Telegraf
and ensures it
continues to run
17

18. Metric Analytics
18

19. 1. Analytics workspace to quickly
visualize, aggregate, and analyze
any indexed metric
2. Support for multiple dimensions
allows easy grouping and
filtering
3. Easy export your workspace
content to XML dashboard or
a new dashboard in the
Dashboards app (beta)
4. Enhanced Alerting by using
chart data and trigger when
search results meet
specific conditions.
19

20. Operating system monitoring with telegraf
The Splunk application for OS monitoring with Telegraf leverages the Influxdata Telegraf agent to provide key layer
Operating System monitoring for Windows and Linux, ingested in the high performance Splunk metric store.
Ref: https://splunkbase.splunk.com/app/4271/
20

21. Q&A
21

22. 22