Más contenido relacionado La actualidad más candente (20) Similar a Alan Pope, Sebastian Spaink [InfluxData] | Data Collection 101 | InfluxDays NA 2021 (20) Alan Pope, Sebastian Spaink [InfluxData] | Data Collection 101 | InfluxDays NA 20211. Alan Pope | Engineering Manager,
InfluxData
Sebastian Spaink | Software Engineer,
InfluxData
Data Collection 101
2. © 2021 InfluxData Inc. All Rights Reserved.
© 2021 InfluxData Inc. All Rights Reserved.
Talk outline
• Introduction to Telegraf
• What’s new with Telegraf?
• New parser demos
• Tiger bot and more!
• Final words
3. © 2021 InfluxData Inc. All Rights Reserved.
© 2021 InfluxData Inc. All Rights Reserved.
Writing to InfluxDB
Several ways to write and query InfluxDB v2
• HTTP API
• Client library
• Telegraf
4. © 2021 InfluxData Inc. All Rights Reserved.
© 2021 InfluxData Inc. All Rights Reserved.
Telegraf
The plugin-driven server agent for collecting & reporting metrics
• Over 300 plugins
• Current release: v1.20.2
• Get artifacts here:
https:/
/github.com/influxdata/telegraf/releases/tag/v1.20.2
5. © 2021 InfluxData Inc. All Rights Reserved.
© 2021 InfluxData Inc. All Rights Reserved.
How does Telegraf work?
• Component called “agent” controlling main functionality
• Extended functionality with plugins
• Configured using TOML
6. © 2021 InfluxData Inc. All Rights Reserved.
© 2021 InfluxData Inc. All Rights Reserved.
The Crew & Community
7. © 2021 InfluxData Inc. All Rights Reserved.
© 2021 InfluxData Inc. All Rights Reserved.
New input plugins
• OpenTelemetry
• Receives traces, metrics and logs from OpenTelemetry clients and agents via
gRPC
• https://github.com/influxdata/telegraf/tree/master/plugins/inputs/opentelemetry
• Elasticsearch Query
• Queries endpoints to obtain metrics from data stored in an Elasticsearch cluster
• https://github.com/influxdata/telegraf/tree/master/plugins/inputs/elasticsearch_query
8. © 2021 InfluxData Inc. All Rights Reserved.
© 2021 InfluxData Inc. All Rights Reserved.
New output plugins
• Generic SQL Output Plugin
• Saves Telegraf metric data to an SQL database
• Multiple supported SQL databases
• https://github.com/influxdata/telegraf/tree/master/plugins/outputs/sql
• Azure Data Explorer Output Plugin
• Azure Data Explorer is a distributed, columnar store, purpose-built for any type of
logs, metrics and time series data
• https://github.com/influxdata/telegraf/tree/master/plugins/outputs/azure_data_explorer
9. © 2021 InfluxData Inc. All Rights Reserved.
© 2021 InfluxData Inc. All Rights Reserved.
New Parser Demo Time!
• json_v2 parser
• A new and improved JSON parser
• Still uses gjson path expressions
• xpath parser
• The first XML parser
• Uses xpath path expressions
• ISS (International space station) current location
• http://api.open-notify.org/iss-now.json
10. © 2021 InfluxData Inc. All Rights Reserved.
© 2021 InfluxData Inc. All Rights Reserved.
ISS location JSON
{
"iss_position": {
"longitude": "116.2486",
"latitude": "-37.0576"
},
"timestamp": 1633645318,
"message": "success"
}
11. © 2021 InfluxData Inc. All Rights Reserved.
© 2021 InfluxData Inc. All Rights Reserved.
What is GJSON ?
● GJSON Path: a text syntax that describes a search pattern to retrieve values from JSON
● Easily expressed as a series of components separated by a . character
● Given: {"example": { "test": 1 }}
● GJSON Path: example.test
● Returned value: 1
• Source: https://github.com/tidwall/gjson
• Documentation: https://github.com/tidwall/gjson/blob/master/SYNTAX.md
• Try it yourself: https://gjson.dev/
12. © 2021 InfluxData Inc. All Rights Reserved.
© 2021 InfluxData Inc. All Rights Reserved.
json_v2 config (v1.20)
[[inputs.http]]
interval = "1m"
urls = ["http://api.open-notify.org/iss-now.json"]
data_format = "json_v2"
[[inputs.http.json_v2]]
[[inputs.http.json_v2.object]]
path = "iss_position"
timestamp_path="timestamp"
timestamp_format="unix"
[inputs.http.json_v2.object.fields]
longitude = "float"
latitude = "float"
13. © 2021 InfluxData Inc. All Rights Reserved.
© 2021 InfluxData Inc. All Rights Reserved.
json_v2 config option #2 (v1.21)
[[inputs.http]]
interval = "1m"
urls = ["http://api.open-notify.org/iss-now.json"]
data_format = "json_v2"
[[inputs.http.json_v2]]
[[inputs.http.json_v2.object]]
path = "@this"
timestamp_path="timestamp"
timestamp_format="unix"
[[inputs.http.json_v2.object.field]]
path = "iss_position.longitude"
type = "float"
[[inputs.http.json_v2.object.field]]
path = "iss_position.latitude"
type = "float"
14. © 2021 InfluxData Inc. All Rights Reserved.
© 2021 InfluxData Inc. All Rights Reserved.
ISS JSON converted to XML
<?xml version="1.0" encoding="UTF-8" ?>
<root>
<iss_position>
<longitude>116.2486</longitude>
<latitude>-37.0576</latitude>
</iss_position>
<timestamp>1633645318</timestamp>
<message>success</message>
</root>
15. © 2021 InfluxData Inc. All Rights Reserved.
© 2021 InfluxData Inc. All Rights Reserved.
What is XPath?
Recommended blog to learn more ⇒
Scan QR code
XPath: path expressions to select nodes or node-sets in an XML document
● Given:
<root>
<example>
1
</example>
</root>
● GJSON Path: /root/example
● Returned value: 1
16. © 2021 InfluxData Inc. All Rights Reserved.
© 2021 InfluxData Inc. All Rights Reserved.
xml config (v1.20)
[[inputs.file]]
files = ["input.xml"]
data_format = "xml"
[[inputs.file.xpath]]
[inputs.file.xpath.fields]
longitude =
"number(/root/iss_position/longitude)"
latitude = "number(/root/iss_position/latitude)"
17. © 2021 InfluxData Inc. All Rights Reserved.
© 2021 InfluxData Inc. All Rights Reserved.
18. © 2021 InfluxData Inc. All Rights Reserved.
© 2021 InfluxData Inc. All Rights Reserved.
19. © 2021 InfluxData Inc. All Rights Reserved.
© 2021 InfluxData Inc. All Rights Reserved.
Community Shoutout
• Thomas Casteleyn (hipska)
• Paweł Żak (zak-pawel)
• Sven Rebhan (srebhan)
… (your name here?)
20. © 2021 InfluxData Inc. All Rights Reserved.
© 2021 InfluxData Inc. All Rights Reserved.
Telegraf Developer Improvements
• conventional commit messages
• e.g. feat: or fix:
• linter
• Using: https://github.com/golangci/golangci-lint
• improved CI times
• 40 minutes -> 12 minutes
21. © 2021 InfluxData Inc. All Rights Reserved.
© 2021 InfluxData Inc. All Rights Reserved.
Telegraf Tiger Bot
• Updates to telegraf tiger bot
• Checks CLA
• Manages labels
• Shares artifacts
22. © 2021 InfluxData Inc. All Rights Reserved.
© 2021 InfluxData Inc. All Rights Reserved.
Telegraf Bot shares artifacts
23. © 2021 InfluxData Inc. All Rights Reserved.
© 2021 InfluxData Inc. All Rights Reserved.
Get involved!
• Slack:
https:/
/influxdata.com/slack
• Forum:
https:/
/community.influxdata.com/c/6
• Issues:
https:/
/github.com/influxdata/telegraf/issues