More Related Content Similar to Virtual training intro to InfluxDB - June 2021 (20) More from InfluxData (20) Virtual training intro to InfluxDB - June 20212. © 2021 InfluxData. All rights reserved.
2
Agenda
1. What is InfluxDB
2. Basic Concepts
3. Demo
4. © 2021 InfluxData. All rights reserved.
4
A massive community of open source & cloud developers
• Self-starters
• Production teams
• Expect OSS & usage-based
models
• Industrial IoT
• End-user & consumer IoT
• Real-time analytics
• Networks & transactions
by cloud
by
language
& tool
Builders of Apps From Technical Ecosystems Use Case Focused
5. © 2021 InfluxData. All rights reserved.
5
Get started quickly
with more tools
• Rest API
• OSS integrations
• Cloud delivery
and less code.
A POWERFUL API & TOOL SET FOR BUILDING REAL-TIME APPS
Collect
using hundreds of
integrations &
OSS tools
Write/Query
in multiple
languages built for
real-time data
Abstract
using client
libraries for your
preferred language
Manage
applications &
account via the
developer console
INFLUXDATA API & TOOLS
DEVELOPER APPLICATIONS
Io
T
Transactions Analytics
6. © 2021 InfluxData. All rights reserved.
6
Run apps in
production with
absolute confidence
on the only
purpose-built time
series engine.
High Speed Ingest
via both batch &
streaming
Flexible Schema
learns & adapts
as it goes
High & Low
Fidelity
retention &
storage
Managed Functions
hosted in
the cloud
A HIGH-PERFORMANCE ENGINE TO HANDLE REAL-TIME
DATA WORKLOADS
INFLUXDATA API & TOOLS
INFLUXDATA REAL-TIME ENGINE
TIME SERIES
DATABASE
7. © 2021 InfluxData. All rights reserved.
7
INFLUXDB ENTERPRISE
Self Managed, High
Availability & Secure
priced per node
INFLUXDB
Open source
time series database.
High performing,
Schemaless, Smart
extraction of data (raw,
sliding, aggregates)
INFLUXDB EDGE
Extend your applications to
the IoT edge for real-time
insights in connected,
disconnected, or hybrid
environments.
INFLUXDB CLOUD
Elastic Serverless
Time Series as a Service
pay per use
INFLUXDB PLATFORM
PRODUCT OFFERINGS
8. © 2021 InfluxData. All rights reserved.
8
InfluxDB
Time Series
Platform
EMPOWERS DEVELOPERS
TO BUILD IOT,
ANALYTICS, &
MONITORING SOFTWARE.
– Designed for time series analysis
– Easy to share, easy to extend
– Multi data source
– Open Source (MIT license)
– Easy to get started, powerful to
scale
9. © 2021 InfluxData. All rights reserved.
9
Powerful APIs, Tools, & Ecosystem
Developers using the InfluxDB platform build their applications
with less effort, less code, and less configuration.
Unrestricted developer
productivity
Client and server
libraries in everything
from React, JavaScript,
Go, Python and more.
Get a head-start with
templates
Inspiring templates for
every use case. Easy to
create and share — find
what’s right for you.
Vibrant open source
community
A diverse and highly
motivated community,
making contributions in
code, & documentation.
Powerful & active
ecosystem
200+ Telegraf plugins,
integrations with
Grafana or data sources
like Google Bigtable.
10. © 2021 InfluxData. All rights reserved.
10
Current Data Acquisition - All Methods
Agent-based Push (aka Telegraf)
Agentless Pull (aka Scrapers)
Client Libraries
Native/Ecosystem
● 200+ Telegraf plugins
● Regular cadence of releases
● Why use it?
○ No code
○ Large community
○ Lightweight but powerful
○ Customizable
● Prometheus scraper (OSS only)
● Flux prometheus.from
● Flux csv.from(url)
● Why use them?
○ Get data in quickly
○ Doesn’t require agent downloads on
monitoring device
● 11 Libraries: Python, C#, Java, GO,
Javascript/Node.js, Ruby, PHP, Scala, Swift,
Kotlin, Arduino
● Handles batching, chunking, setting right
headers, etc.
● Why use them?
○ Easy way to get started
○ Need libraries when building custom
applications
● Source system speaks line protocol
● Examples: JMeter, NiFi, Vector, FluentD
● Influx CLI CSV Import
● Why use them?
○ Know what you want to monitor,
quick and easy integration
12. © 2021 InfluxData. All rights reserved.
12
Get InfluxDB
InfluxDB Cloud
https:/
/cloud2.influxdata.com/signup
Download OSS
https:/
/portal.influxdata.com/downloads/
14. © 2021 InfluxData. All rights reserved.
14
Concepts: Data Model
● Measurement
○ A name to a group of data at a high level
● Tag set
○ A set of key-value pairs to group data at a low level (values are strings)
● Field set
○ A set of key-value pairs to represent data (values are numerical & strings)
● Timestamp
○ Time of the data with nanosecond precision
● Series
○ A unique combination of measure+tags
15. © 2021 InfluxData. All rights reserved.
15
Data model vs ingestion & storage
Measurement
Tag set
Field set
Timestamp
Ingestion Storage
Series
Buckets
(set of series)
16. © 2021 InfluxData. All rights reserved.
16
Concepts: Time Series Database Schema
Data Ingestion Format
• Points are written to InfluxDB using the Line Protocol, which
follows the following format:
<measurement>[,<tag-key>=<tag-value>] [<field-key>=<field-value>]
[unix-nano-timestamp]
Reference: https://docs.influxdata.com/influxdb/v2.0/reference/syntax/line-protocol/
Tag Set
hostname=server02, us_west=az
Measurement
cpu_load
Field Set
temp=24.5, volts=7
Timestamp
1234567890000000
17. © 2021 InfluxData. All rights reserved.
17
Bucket physical view
● Columnar Data Stores
● Best for column selection and aggregation thanks to
- Disk and memory locality
- Cache locality
_time
1492...1
1492...0
_m
obs
obs
host
ovenB
ovenA
num
1
1
line
west
west
temp
301
125
humidity
23
75
_m=_measurment tags fields
18. © 2021 InfluxData. All rights reserved.
18
Schema Considerations
• Tags are indexed. Fields are not.
• Field Types are immutable
• Store data in tags if you plan to use them with GROUP BY()
• Store data in fields if you plan to use them with Flux function
• Store data in fields if you need them to be something other than a string
• The best schema depends primarily on what queries you want to run. At a high level:
– You can't GROUP BY fields, only tags
– You can't do math on tags or across measurements
– Queries that filter by tags are fast
– Queries that filter by fields are slower
– Be aware of series cardinality . . . It matters
Concepts: InfluxDB Schema Best Practices
19. © 2021 InfluxData. All rights reserved.
19
Series Cardinality
• The number of unique database, measurement, and tag set combinations in an InfluxDB
instance.
• For example, assume that an InfluxDB instance has a single database and one
measurement.
– The single measurement has two tag keys: email and status.
– If there are three different emails, and each email address is associated with two different statuses then the
series cardinality for the measurement is 6: (3 * 2 = 6)
Concepts: InfluxDB Schema Best Practices
email status
lorr@influxdata.com start
lorr@influxdata.com finish
marv@influxdata.com start
marv@influxdata.com finish
cliff@influxdata.com start
cliff@influxdata.com finish
Reference:
https://docs.influxdata.com/influxdb/v2.0/write-data/best-practices/resolve-high-cardinality/
20. © 2021 InfluxData. All rights reserved.
20
Series Cardinality Example
• Now imagine that a measurement has two tag keys:
– containerId and processId
– Each tag has 10,000 possibilities
– In this series cardinality for the measurement is 100,000,000 (10,000 x 10,000)
– Further let’s assume that each series takes up 1k in memory, that equates to 102.4GB of
RAM. . . you begin to see the problem
• We call this “Run-away series cardinality”
• Read more about this
– https://www.influxdata.com/blog/tldr-influxdb-tech-tips-monitoring-tasks-and-finding-th
e-source-of-runaway-cardinality/
– Index and Time-Structured Merge Tree (TSM) Overview in InfluxDB 1.5
Concepts: InfluxDB Schema Best Practices
21. © 2021 InfluxData. All rights reserved.
21
Introducing Flux
A functional data scripting language designed for querying, analyzing, and
acting on data. It is intended to be usable, readable, flexible, composable,
testable, contributable, and shareable.
“
I don't want to have to write query
code in one language and processing
code in another language.
– Paul Dix, CTO InfluxData
23. © 2021 InfluxData. All rights reserved.
23
What’s Next
– Build an IoT application workshop | August 5, 2020
○ https://www.influxdata.com/resources/iot-center-workshop/
– Get your cloud account
○ And try InfluxDB Templates
– Check out
○ https://www.influxdata.com/products/influxdb-templates/gallery/
– Performance Tips
– Index and Time-Structured Merge Tree (TSM) Overview
– Integrations
○ https://docs.influxdata.com/telegraf/v1.18/
○ https://www.influxdata.com/products/integrations/