Time series data is everywhere -- connected IoT devices, application monitoring & observability platforms, and more. What makes time series datastreams challenging is that they often have orders of magnitude more data than other workloads, with millions of time series datapoints being quite common. Given its ability to ingest high volumes of data, Kafka is a natural part of any data architecture handling large volumes of time series telemetry, specifically as an intermediate buffer before that data is persisted in InfluxDB for processing, analysis, and use in other applications. In this session, we will show you how you can stream time series data to your IoT application using Kafka queues and InfluxDB, drawing upon deployments done at Hulu and Wayfair that allow both to ingest 1 million metrics per second. Once this session is complete, you’ll be able to connect a Kafka queue to an InfluxDB instance as the beginning of your own time series data pipeline.
Gen AI in Business - Global Trends Report 2024.pdf
Stream processing IoT time series data with Kafka & InfluxDB | Al Sargent, InfluxData
1. Al Sargent
Sr. Director, Product
InfluxData
April 2021
Stream processing IoT time
series data with Kafka &
InfluxDB
Mirek Malecha
Product Manager
Bonitoo
April 2021
12. Delivered an Industrial IoT data historian
that
• provides automation and control of
their oil drilling platforms
• provides edge and centralized insight
into emerging patterns
by storing and analyzing all the sensor data
derived from energy production sensors
Internet of Things (IoT)
IoT monitoring