Más contenido relacionado La actualidad más candente (20) Similar a How to Build a Telegraf Plugin by Noah Crowley (20) How to Build a Telegraf Plugin by Noah Crowley2. © 2018 InfluxData. All rights reserved.
What we’ll be covering
✔ Telegraf overview
✔ Examples of Telegraf plugins
✔ Telegraf plugin architecture
✔ How to a write a Telegraf plugin
5. © 2018 InfluxData. All rights reserved.
Telegraf Overview
• Telegraf is an open source, plugin-driven agent for collecting
and reporting metrics
• “Push” or “Pull”
• Large collection of plugins: 192 (as of 3/13/2019)
• Inputs: 149
• Outputs: 30
• Aggregators: 9
• Processors: 4
6. © 2018 InfluxData. All rights reserved.
Telegraf Overview
• Can act as:
• Agent
• Collector
• Part of an Ingest Pipeline
7. © 2018 InfluxData. All rights reserved.
Telegraf Plugins
• Outputs
• Allows you to send metrics to a variety of datastores
• Plugins for both InfluxDB 1.x and 2.0
• Supports Kafka, MQTT, NSQ, OpenMetrics, and more
• Aggregators and Processors allow you to manipulate data as it
flows through Telegraf
• Transform tags, convert types, calculate histograms
8. © 2018 InfluxData. All rights reserved.
Telegraf Input Plugins
• Metrics ingestion from host system: CPU, I/O, Network…
• Common Applications like NGINX, Postgres, Redis…
• Third Party APIs: Mailchimp, Cloudwatch, Google Analytics…
• General-Purpose Protocols: HTTP, Socket, MQTT…
• …and more!
9. © 2018 InfluxData. All rights reserved.
Telegraf Benefits
• Minimal memory footprint
• Tagging of metrics
• Batching of metrics to reduce the number of atomic writes
• Easy contribution of functionality thanks to Go
• Static Binary
• Strong Community Contributions
11. © 2018 InfluxData. All rights reserved.
statsd
• Listens for incoming data
• Acts as a statsd instance
• Outputs to any configured output
12. © 2018 InfluxData. All rights reserved.
postgresql
• Collects performance data
• Uses data from built-in views:
• pg_stat_database
• pg_stat_bgwriter
13. © 2018 InfluxData. All rights reserved.
apache
• Connects over HTTP
• Reads data from:
• /server-status?auto
14. © 2018 InfluxData. All rights reserved.
win_perf_counters
• Collects performance data from
Windows machines
15. © 2018 InfluxData. All rights reserved.
Input Plugin Considerations
• Where does the data exist?
• How can it be collected?
• What shape is the data?
16. © 2018 InfluxData. All rights reserved.
prometheus_client
• Output plugin
• Exposes metrics in Prometheus
(now OpenMetrics) format.
17. © 2018 InfluxData. All rights reserved.
mqtt
• Output half of the MQTT plugins
• Allows you to write messages to
an MQTT broker.
18. © 2018 InfluxData. All rights reserved.
Output Plugin Considerations
• How should the data be shaped?
• How is the data output?
• Where is the data output to?
20. © 2018 InfluxData. All rights reserved.
Plugin Architecture
• Built to be extensible from the beginning
• Make use of interfaces in Go
• Each plugin type has an interface
22. © 2018 InfluxData. All rights reserved.
package simple
// simple.go
import (
"github.com/influxdata/telegraf"
"github.com/influxdata/telegraf/plugins/inputs"
)
type Simple struct {
Ok bool
}
func (s *Simple) Description() string {
return "a demo plugin"
}
func (s *Simple) SampleConfig() string {
return `
## Indicate if everything is fine
ok = true
`
}
func (s *Simple) Gather(acc telegraf.Accumulator) error {
if s.Ok {
acc.AddFields("state", map[string]interface{}{"value":
"pretty good"}, nil)
} else {
acc.AddFields("state", map[string]interface{}{"value":
"not great"}, nil)
}
return nil
}
func init() {
inputs.Add("simple", func() telegraf.Input { return
&Simple{} })
}
24. © 2018 InfluxData. All rights reserved.
Getting started…
• Recommend using Go 1.11
• Understanding of Git & Pull Requests
• What is our plugin going to do?
• Generate data using “sine” and “cosine” trigonometric
functions
• Read the CONTRIBUTING.md file to ensure you can build & test
Telegraf before continuing
25. © 2018 InfluxData. All rights reserved.
Getting started…
• Set up your workspace, get the code, and create a branch:
• Create the required directories and files:
• Add boilerplate…
$ go get github.com/influxdata/telegraf
$ cd $GOPATH/github.com/influxdata/telegraf
$ git checkout -b trig-demo
$ cd plugins/inputs
$ mkdir trig
$ touch trig/trig.go
26. © 2018 InfluxData. All rights reserved.
package trig
import (
"github.com/influxdata/telegraf"
"github.com/influxdata/telegraf/plugins/inputs"
)
type Trig struct {
}
func (s *Trig) SampleConfig() string {
return ""
}
func (s *Trig) Description() string {
return ""
}
func (s *Trig) Gather(acc telegraf.Accumulator) error {
return nil
}
func init() {
inputs.Add("trig", func() telegraf.Input { return &Trig{} })
}
trig.go
27. © 2018 InfluxData. All rights reserved.
Import your plugin
• Add the following to telegraf/plugins/inputs/all/all.go
• This will your plugin into the main Telegraf package and ensure
that it can run.
_ "github.com/influxdata/telegraf/plugins/inputs/trig"
28. © 2018 InfluxData. All rights reserved.
Add configuration variables
• Each plugin has a struct
• There must be a property on the struct to hold any
configuration variables
• Any other variables should be unexported
• We’ll add two variables:
• “x” will hold the state of the plugin between collection intervals
• “Amplitude” will hold the amplitude value for the sin wave
29. © 2018 InfluxData. All rights reserved.
package trig
import (
"github.com/influxdata/telegraf"
"github.com/influxdata/telegraf/plugins/inputs"
)
type Trig struct {
x float64
Amplitude float64
}
func (s *Trig) SampleConfig() string {
return ""
}
func (s *Trig) Description() string {
return ""
}
func (s *Trig) Gather(acc telegraf.Accumulator) error {
trig.go
30. © 2018 InfluxData. All rights reserved.
Add sample configuration
• Telegraf can dynamically construct configuration files
• Aggregates sample configs for all plugins
• The CLI allows you to provide arguments to filter which plugins
are included
• In 2.0, InfluxDB will use this info to generate Telegraf configs
31. © 2018 InfluxData. All rights reserved.
package trig
import (
"github.com/influxdata/telegraf"
"github.com/influxdata/telegraf/plugins/inputs"
)
type Trig struct {
x float64
Amplitude float64
}
var TrigConfig = `
## Set the amplitude
amplitude = 10.0
`
func (s *Trig) SampleConfig() string {
return TrigConfig
}
trig.go
32. © 2018 InfluxData. All rights reserved.
Add a simple description
• This description is included above the plugin configuration and
should briefly describe what your plugin does
33. © 2018 InfluxData. All rights reserved.
type Trig struct {
x float64
Amplitude float64
}
var TrigConfig = `
## Set the amplitude
amplitude = 10.0
`
func (s *Trig) SampleConfig() string {
return TrigConfig
}
func (s *Trig) Description() string {
return "Inserts sine and cosine waves for demonstration
purposes"
}
func (s *Trig) Gather(acc telegraf.Accumulator) error {
trig.go
34. © 2018 InfluxData. All rights reserved.
Test your config!
• First, build Telegraf
• Generate a config:
$ make telegraf
$ telegraf -sample-config -input-filter trig
-output-filter influxdb -debug
35. © 2018 InfluxData. All rights reserved.
...
###############################################################################
# INPUT PLUGINS #
###############################################################################
# Inserts sine and cosine waves for demonstration purposes
[[inputs.trig]]
## Set the amplitude
amplitude = 10.0
36. © 2018 InfluxData. All rights reserved.
The “Gather” function
• The heart of the plugin, it is run every time telegraf collects
metrics
• This is where you’ll do the majority of the work
• We’ll generate sine and cosine values
• At the end of the function, we add any data we’ve collected
and to the Telegraf “Accumulator” by
calling .AddFields(measurement, tags, fields).
• This defines a new point in InfluxDB with the timestamp being
the collection time.
37. © 2018 InfluxData. All rights reserved.
return "Inserts sine and cosine waves for demonstration
purposes"
}
func (s *Trig) Gather(acc telegraf.Accumulator) error {
sinner := math.Sin((s.x*math.Pi)/5.0) * s.Amplitude
cosinner := math.Cos((s.x*math.Pi)/5.0) * s.Amplitude
fields := make(map[string]interface{})
fields["sine"] = sinner
fields["cosine"] = cosinner
tags := make(map[string]string)
s.x += 1.0
acc.AddFields("trig", fields, tags)
return nil
}
trig.go
38. © 2018 InfluxData. All rights reserved.
Add initial state
• In our boilerplate, we added an init() function
• Called when the plugin is first initialized
• Makes the plugin available to the Telegraf agent
• We can define starting state as part of this function
• We need to initialize our “x” variable
39. © 2018 InfluxData. All rights reserved.
cosinner := math.Cos((s.x*math.Pi)/5.0) * s.Amplitude
fields := make(map[string]interface{})
fields["sine"] = sinner
fields["cosine"] = cosinner
tags := make(map[string]string)
s.x += 1.0
acc.AddFields("trig", fields, tags)
return nil
}
func init() {
inputs.Add("trig", func() telegraf.Input { return &Trig{x: 0.0} })
}
trig.go
40. © 2018 InfluxData. All rights reserved.
Compile and test
• Once again, build Telegraf:
• Generate a config and write it to a file:
$ make telegraf
$ ./telegraf -sample-config -input-filter
trig -output-filter influxdb -debug >>
telegraf.conf.test
• Run Telegraf with the new config:
$ ./telegraf -config telegraf.conf.test
-debug
41. © 2018 InfluxData. All rights reserved.
$ ./telegraf -config telegraf.conf.test -debug
2019-03-14T06:32:12Z I! Starting Telegraf
2019-03-14T06:32:12Z I! Loaded inputs: trig
2019-03-14T06:32:12Z I! Loaded aggregators:
2019-03-14T06:32:12Z I! Loaded processors:
2019-03-14T06:32:12Z I! Loaded outputs: influxdb
2019-03-14T06:32:12Z I! Tags enabled: host=noah-mbp.local
2019-03-14T06:32:12Z I! [agent] Config: Interval:10s, Quiet:false,
Hostname:"noah-mbp.local", Flush Interval:10s
2019-03-14T06:32:12Z D! [agent] Connecting outputs
2019-03-14T06:32:12Z D! [agent] Attempting connection to output: influxdb
2019-03-14T06:32:12Z D! [agent] Successfully connected to output: influxdb
2019-03-14T06:32:12Z D! [agent] Starting service inputs
2019-03-14T06:32:30Z D! [outputs.influxdb] wrote batch of 1 metrics in
23.857525ms
2019-03-14T06:32:30Z D! [outputs.influxdb] buffer fullness: 1 / 10000 metrics.
43. © 2018 InfluxData. All rights reserved.
Final steps before submitting
• Add a few things to your plugin…
• a README.md
• a LICENSE
• A sample of the input/output format
• Tests!
44. © 2018 InfluxData. All rights reserved.
package trig
import (
"math"
"testing"
"github.com/influxdata/telegraf/testutil"
)
func TestTrig(t *testing.T) {
s := &Trig{
Amplitude: 10.0,
}
for i := 0.0; i < 10.0; i++ {
var acc testutil.Accumulator
sine := math.Sin((i*math.Pi)/5.0) * s.Amplitude
cosine := math.Cos((i*math.Pi)/5.0) * s.Amplitude
s.Gather(&acc)
fields := make(map[string]interface{})
fields["sine"] = sine
fields["cosine"] = cosine
acc.AssertContainsFields(t, "trig", fields)
}
}
trig_test.go
45. © 2018 InfluxData. All rights reserved.
Final steps before submitting
• Add a few things to your plugin…
• a README.md
• a LICENSE
• A sample of the input/output format
• Tests!
• …and submit!
• github.com/influxdata/telegraf/pulls