In this training webinar, Samantha Wang will walk you through the basics of Telegraf. Telegraf is the open source server agent which is used to collect metrics from your stacks, sensors and systems. It is InfluxDB’s native data collector that supports nearly 300 inputs and outputs. Learn how to send data from a variety of systems, apps, databases and services in the appropriate format to InfluxDB. Discover tips and tricks on how to write your own plugins. The know-how learned here can be applied to a multitude of use cases and sectors. This one-hour session will include the training and time for live Q&A. Join this training as Samantha Wang dives into: Types of Telegraf plugins (i.e. input, output, aggregator and processor) Specific plugins including Execd input plugins and the Starlark processor plugin How to install and start using Telegraf

1. Samantha Wang
Product Manager, InfluxData
Getting Started: Intro
to Telegraf

2. © 2021 InfluxData. All rights reserved.
2
2
– What is Telegraf
– Plugin Ecosystem
– Configuration Basics
– Extending the Ecosystem
Agenda

3. © 2021 InfluxData. All rights reserved.
3
InfluxDB

4. © 2021 InfluxData. All rights reserved.
4
What is InfluxDB?
Characteristics
of the data
– Time-stamped
– Generated in regular
(metric) and
irregular (event) time
periods
– Huge volumes
– Real time and time
sensitive

5. © 2021 InfluxData. All rights reserved.
5
Visualization
Alerts
Triggers
Metrics
Logs
Traces
Events
The platform of choice for all metrics & event workloads

6. © 2021 InfluxData. All rights reserved.
6
InfluxData Platform Architecture

7. © 2021 InfluxData. All rights reserved.
7
InfluxData products and offerings
* Availability TBD

8. © 2021 InfluxData. All rights reserved.
8
Telegraf

9. © 2021 InfluxData. All rights reserved.
9
9
Current Data Acquisition - All Methods
Agent-based Push (aka Telegraf)
Agentless Pull (aka Scrapers)
Client Libraries
Native/Ecosystem
● 200+ Telegraf plugins
● Regular cadence of releases
● Why use it?
○ No code
○ Large community
○ Lightweight but powerful
○ Customizable
● Prometheus scraper (OSS only)
● Flux prometheus.from
● Flux csv.from(url)
● Why use them?
○ Get data in quickly
○ Doesn’t require agent downloads on
monitoring device
● 7 Libraries: Python, C#, Java, GO,
Javascript/Node.js, Ruby, PHP
● Handles batching, chunking, setting right
headers, etc.
● Why use them?
○ Easy way to get started
○ Need libraries when building custom
applications
● Source system speaks line protocol
● Examples: JMeter, NiFi, Vector, FluentD
● Influx CLI CSV Import
● Why use them?
○ Know what you want to monitor,
quick and easy integration

10. © 2021 InfluxData. All rights reserved.
10
Telegraf provides the benefits of…
● Low/No code
● Robust scheduler
● High-speed ingestion
● Full-streaming support
● Metric routing
● Flexible parsing, formatting, serializing
● Customizable and Extensible (ExecD
plugins, Starlark)
Instead of…
● Writing long data scraping scripts
● Worrying about unreliable data
collection
● Trouble scaling your data collection
● Resulting in messy data
● Having a lot of unnecessary data in
your database

11. © 2021 InfluxData. All rights reserved.
11
11
Telegraf:
Agent for Collecting Metrics & Events
Plugin-driven server agent for
collecting and reporting metrics
• Written in Go
• Single Binary, No external
dependencies
• Minimal memory footprint
• Optimized for writing to InfluxDB
• Optimized for streaming data
Input plugins to source a variety of
metrics
• directly from the system it’s running on
• pull metrics from third party APIs
• listen for metrics via a statsd and
Kafka consumer services

12. © 2021 InfluxData. All rights reserved.
12
12
Telegraf:
Agent for Collecting Metrics & Events
Output plugins to send metrics to a
variety of other datastores, services,
and message queues
• InfluxDB, Graphite, OpenTSDB,
Datadog, and many others
• Writes to multiple outputs at the
same time
Metrics routing and conversion
Aggregation stage for InfluxDB

13. © 2021 InfluxData. All rights reserved.
13
13
Telegraf Agent
– Core of Telegraf functionality providing the following
– Robust scheduler
– Adjustments for clock-drift
– Adjustments for job scheduling issues that may occur
– In-memory metric buffers
– Metric tracking with flow back-pressure in plugins like Kafka
– Full-streaming support
– Metric routing: name & field pass & drop
– Flexible parsing, formatting, serializing

14. © 2021 InfluxData. All rights reserved.
14
Plugins! Plugins! Plugins!

15. © 2021 InfluxData. All rights reserved.
15
15
Telegraf Plugin Ecosystem
Input Plugins
● collect metrics from system,
services, or third party APIs
Output Plugins
● write data to various destinations
Processors
● transform, decorate, and/or filter
metrics
Aggregators
● create aggregate metrics (e.g.
mean, min, max, quantiles, etc.)

16. © 2021 InfluxData. All rights reserved.
16
16
Telegraf Plugin Types
170+ 40+ 15+ 5+
Input Output Processors Aggregators

17. © 2021 InfluxData. All rights reserved.
17
17
Input Plugins
activemq
aerospike
amqp_consumer
apache
apcupsd
aurora
azure_storage_que
ue
bcache
beanstalkd
bind
bond
burrow
cassandra
ceph
cgroup
chrony
cisco_telemetry_mdt
clickhouse
cloud_pubsub
cloud_pubsub_push
cloudwatch
conntrack
consul
couchbase
couchdb
cpu
dcos
disk
diskio
disque
dmcache
dns_query
docker
docker_log
dovecot
ecs
elasticsearch
ethtool
eventhub_consumer
exec
execd
fail2ban
fibaro
file
filecount
filestat
fireboard
fluentd
github
gnmi
graylog
haproxy
hddtemp
http
http_listener_v2
http_response
httpjson
icinga2
infiniband
influxdb

18. © 2021 InfluxData. All rights reserved.
18
18
Input Plugins (2)
influxdb_listener
influxdb_v2_listener
intel_rdt
internal
interrupts
ipmi_sensor
ipset
iptables
ipvs
jenkins
jolokia
jolokia2
jti_openconfig_telemetr
y
kafka_consumer
kafka_consumer_legacy
kapacitor
kernel
kernel_vmstat
kibana
kinesis_consumer
kube_inventory
kubernetes
lanz
leofs
linux_sysctl_fs
logparser
logstash
lustre2
mailchimp
marklogic
mcrouter
mem
memcached
mesos
minecraft
modbus
mongodb
monit
mqtt_consumer
multifile
mysql
nats
nats_consumer
neptune_apex
net
net_response
nginx
nginx_plus
nginx_plus_api
nginx_sts
nginx_upstream_check
nginx_vts
nsd
nsq
nsq_consumer
nstat
ntpq
nvidia_smi
opcua
openldap

19. © 2021 InfluxData. All rights reserved.
19
19
Input Plugins (3)
openntpd
opensmtpd
openweathermap
passenger
pf
pgbouncer
phpfpm
ping
postfix
postgresql
postgresql_extensibl
e
powerdns
powerdns_recursor
processes
procstat
prometheus
proxmox
puppetagent
rabbitmq
raindrops
ras
redfish
redis
rethinkdb
riak
salesforce
sensors
sflow
smart
snmp
snmp_legacy
snmp_trap
socket_listener
solr
sqlserver
stackdriver
statsd
suricata
swap
synproxy
syslog
sysstat
system
systemd_units
tail
tcp_listener
teamspeak
temp
tengine
tomcat
trig
twemproxy
udp_listener
unbound
uwsgi
varnish
vsphere
webhooks
win_eventlog
win_perf_counte
rs
win_services
wireguard
wireless
x509_cert
zfs
zipkin
zookeeper

20. © 2021 InfluxData. All rights reserved.
20
20
Processor Plugins
clone
converter
date
dedup
defaults
enum
execd
filepath
ifname
override
parser
pivot
port_name
printer
regex
rename
reverse_dns
s2geo
starlark
strings
tag_limit
template
topk
unpivot

21. © 2021 InfluxData. All rights reserved.
21
21
Aggregator Plugins
basicstats
final
histogram
merge
minmax
valuecounter

22. © 2021 InfluxData. All rights reserved.
22
22
Output Plugins
amon
amqp
application_insights
azure_monitor
cloud_pubsub
cloudwatch
cratedb
datadog
discard
dynatrace
elasticsearch
exec
execd
file
graphite
graylog
health
http
influxdb
influxdb_v2
instrumental
kafka
kinesis
librato
logzio
mqtt
nats
newrelic
nsq
opentsdb
prometheus_client
riemann
riemann_legacy
socket_writer
stackdriver
sumologic
syslog
timestream
warp10
wavefront
yandex_cloud_monito
ring

23. © 2021 InfluxData. All rights reserved.
23
23
Telegraf
CPU
Mem
Disk
Docker
Kubernetes
/metrics
Kafka
MySQL
Process
- transform
- decorate
- filter
Aggregate
- mean
- min,max
- count
- variance
- stddev
InfluxDB
File
Kafka
CloudWatch
CloudWatch
Input
Output
One Telegraf, Multiple Plugins

24. © 2021 InfluxData. All rights reserved.
24
24
Can’t find the plugin you need?
Telegraf is 100% open source
with a strong community of
contributors
It’s easy to write your own
Telegraf plugin!
1) Follow the contribution
guide for Go
2) Write your plugin in any
language and run it externally
with ExecD

25. © 2021 InfluxData. All rights reserved.
25
25
Popular Plugins
Out-of-the-box Custom
Kubernetes (kubelet) HTTP/socket listener
Kube_inventory (apiserver) HTTP (formatted endpoints)
Kafka (consumer) Prometheus (/metrics)
SNMP Exec/Execd
AMQP (mq metadata) StatsD
Redis
Nginx
HAproxy
Jolokia2

26. © 2021 InfluxData. All rights reserved.
26
26
Parsing
JSON
CSV
Graphite
Prometheus
CollectD
Dropwizard
Form URL-encoded
Grok

27. © 2021 InfluxData. All rights reserved.
27
Getting Started with
Telegraf

28. © 2021 InfluxData. All rights reserved.
28
28
Download Telegraf

29. © 2021 InfluxData. All rights reserved.
29
Configuration Basics

30. © 2021 InfluxData. All rights reserved.
30
30
Telegraf Configuration Snippets
Include many plugins per
Telegraf
- multiple inputs, outputs,
processors, aggregators

31. © 2021 InfluxData. All rights reserved.
31
31
The Telegraf config file
needs to be specified
for Telegraf agent to
operate properly.
It contains setup for the
agent, global tags, and
enabled outputs
(through commenting
out or removing
unnecessary lines)
What’s in a configuration file?

32. © 2021 InfluxData. All rights reserved.
32
32
Set and Test Your Configuration
$ telegraf --input-filter cpu:system --output-filter influxdb --processor-filter
starlark --aggregator-filter valuecounter config > telegraf.conf
WRITE DESIRED CONFIG FILE
RUN CONFIG FILE TEST
$ telegraf --config telegraf.conf --test
VERIFY YOUR CONFIG WORKS

33. © 2021 InfluxData. All rights reserved.
33
Extending the Ecosystem

34. © 2021 InfluxData. All rights reserved.
34
34
● Work with the open-source community or submit upgrades or
enhancements for existing plugins
● Use ExecD to write a plugin in Go or a language of your choice
You can extend Telegraf by:

35. © 2021 InfluxData. All rights reserved.
35
35
● Plugin runs in its own process
● Requires line protocol
● Avoid the need for review by Telegraf team
● Supports the same API as an internal plugin
● Can use for non-GO plugins
● Can use for licensed software plugins
● Can use for any type of plugin (input, output, processor,
aggregator)
External plugins via ExecD

36. © 2021 InfluxData. All rights reserved.
36
36
Execd
Plugin
Arch.

37. © 2021 InfluxData. All rights reserved.
37
37
[[processors.execd]]
command = ["python", "./processors/forecasting.py"]
Sample ExecD Config: Input, Output, Processor

38. © 2021 InfluxData. All rights reserved.
38
38
Customer Quotes
“Telegraf is like a swiss army
knife for connecting various
MQTT sources and OPC UA
sources.”
— Fr. Ant. Niedermayr
“Our next-generation
pipeline takes advantage of
Kafka and the Telegraf
streaming service to create a
more robust data topology.
Essentially this allows us to
explicitly implement the four
R’s: routability, retention,
resilience, and redundancy.”
— Wayfair

39. © 2021 InfluxData. All rights reserved.
39
Telegraf Community

40. © 2021 InfluxData. All rights reserved.
40
Get involved with Telegraf
• Telegraf GitHub:
github.com/influxdata/telegraf
• Community Slack:
influxdata.com/slack
#telegraf
#telegraf-dev
• Community Website:
community.influxdata.com

41. © 2021 InfluxData. All rights reserved.
41
41
References
Getting started with Telegraf
Telegraf plugins
Telegraf GitHub Page
External Plugins Guide

42. Thank You

43. © 2021 InfluxData. All rights reserved.
43
43
Input Plugins
activemq
aerospike
amqp_consumer
apache
apcupsd
aurora
azure_storage_queu
e
bcache
beanstalkd
bind
bond
burrow
cassandra
ceph
cgroup
chrony
cisco_telemetry_mdt
clickhouse
cloud_pubsub
cloud_pubsub_push
cloudwatch
conntrack
consul
couchbase
couchdb
cpu
dcos
disk
diskio
disque
dmcache
dns_query
docker
docker_log
dovecot
ecs
elasticsearch
ethtool
eventhub_consumer
exec
execd
fail2ban
fibaro
file
filecount
filestat
fireboard
fluentd
github
gnmi
graylog
haproxy
hddtemp
http
http_listener_v2
http_response
httpjson
icinga2
infiniband
influxdb

44. © 2021 InfluxData. All rights reserved.
44
44
Input Plugins (2)
influxdb_listener
influxdb_v2_listener
intel_rdt
internal
interrupts
ipmi_sensor
ipset
iptables
ipvs
jenkins
jolokia
jolokia2
jti_openconfig_telemetry
kafka_consumer
kafka_consumer_legacy
kapacitor
kernel
kernel_vmstat
kibana
kinesis_consumer
kube_inventory
kubernetes
lanz
leofs
linux_sysctl_fs
logparser
logstash
lustre2
mailchimp
marklogic
mcrouter
mem
memcached
mesos
minecraft
modbus
mongodb
monit
mqtt_consumer
multifile
mysql
nats
nats_consumer
neptune_apex
net
net_response
nginx
nginx_plus
nginx_plus_api
nginx_sts
nginx_upstream_check
nginx_vts
nsd
nsq
nsq_consumer
nstat
ntpq
nvidia_smi
opcua
openldap

45. © 2021 InfluxData. All rights reserved.
45
45
Input Plugins (3)
openntpd
opensmtpd
openweathermap
passenger
pf
pgbouncer
phpfpm
ping
postfix
postgresql
postgresql_extensibl
e
powerdns
powerdns_recursor
processes
procstat
prometheus
proxmox
puppetagent
rabbitmq
raindrops
ras
redfish
redis
rethinkdb
riak
salesforce
sensors
sflow
smart
snmp
snmp_legacy
snmp_trap
socket_listener
solr
sqlserver
stackdriver
statsd
suricata
swap
synproxy
syslog
sysstat
system
systemd_units
tail
tcp_listener
teamspeak
temp
tengine
tomcat
trig
twemproxy
udp_listener
unbound
uwsgi
varnish
vsphere
webhooks
win_eventlog
win_perf_counte
rs
win_services
wireguard
wireless
x509_cert
zfs
zipkin
zookeeper

46. © 2021 InfluxData. All rights reserved.
46
46
Processor Plugins
clone
converter
date
dedup
defaults
enum
execd
filepath
ifname
override
parser
pivot
port_name
printer
regex
rename
reverse_dns
s2geo
starlark
strings
tag_limit
template
topk
unpivot

47. © 2021 InfluxData. All rights reserved.
47
47
Aggregator Plugins
basicstats
final
histogram
merge
minmax
valuecounter

48. © 2021 InfluxData. All rights reserved.
48
48
Output Plugins
amon
amqp
application_insights
azure_monitor
cloud_pubsub
cloudwatch
cratedb
datadog
discard
dynatrace
elasticsearch
exec
execd
file
graphite
graylog
health
http
influxdb
influxdb_v2
instrumental
kafka
kinesis
librato
logzio
mqtt
nats
newrelic
nsq
opentsdb
prometheus_client
riemann
riemann_legacy
socket_writer
stackdriver
sumologic
syslog
timestream
warp10
wavefront
yandex_cloud_monitor
ing