Presenter: Paul Dix, CEO, InfluxDB

1. Working with time series
data with InfluxDB
Paul Dix
@pauldix
paul@influxdb.com

2. What is time series
data?

3. Stock trades and quotes

4. Metrics

5. Analytics

6. Events

7. Sensor data

8. Two kinds of time series
data…

9. Regular time series
t0 t1 t2 t3 t4 t6 t7
Samples at regular intervals

10. Irregular time series
t0 t1 t2 t3 t4 t6 t7
Events whenever they come in

11. Inducing a regular time series
from an irregular one
query: select count(customer_id) from events
where time > now() - 1h
group by time(1m), customer_id

12. Data that you ask
questions about over time

13. InfluxDB is an open
source distributed time
series database
* still working on the distributed part

14. Why would you want a
database for time series
data?

15. Scale

16. Example from DevOps
• 2,000 servers, VMs, containers, or sensor units
• 200 measurements per server/unit
• every 10 seconds
• = 3,456,000,000 distinct points per day

17. Sharding Data
usually requires application level code

18. Data retention
application level code and sharding

19. Rollups and
aggregation

20. InfluxDB features

21. SQL style query
language

22. Retention policies
automatically managed data retention

23. Continuous queries
for rollups and aggregation

24. HTTP API - 2 endpoints

25. HTTP API - 2 endpoints
/write?db=mydb&rp=fooWrite: HTTP POST

26. HTTP API - 2 endpoints
/write?db=mydb&rp=foo
/query?db=mydb&rp=foo&q=
Write: HTTP POST
Read: HTTP GET

27. InfluxDB Schema
• Measurements (e.g. cpu, temperature, event,
memory)

28. InfluxDB Schema
• Measurements (e.g. cpu, temperature, event,
memory)
• Tags (e.g. region=uswest, host=serverA,
sensor=23)

29. InfluxDB Schema
• Measurements (e.g. cpu, temperature, event,
memory)
• Tags (e.g. region=uswest, host=serverA,
sensor=23)
• Fields (e.g. value=23.2, info=‘this is some extra
stuff`, present=true)

30. InfluxDB Schema
• Measurements (e.g. cpu, temperature, event,
memory)
• Tags (e.g. region=uswest, host=serverA,
sensor=23)
• Fields (e.g. value=23.2, info=‘this is some extra
stuff`, present=true)
• Timestamp (nano-second epoch)

31. All data is indexed by
measurement, tagset,
and time

32. Influx CLI
$ ./influx
Connected to http://localhost:8086 version 0.9
InfluxDB shell 0.9
>

33. Create a database
CREATE DATABASE foo

34. Create a retention policy
CREATE RETENTION POLICY <rp-name> ON <db-name>
DURATION <duration> REPLICATION <n> [DEFAULT]

35. Create a retention policy
CREATE RETENTION POLICY <rp-name> ON <db-name>
DURATION <duration> REPLICATION <n> [DEFAULT]
CREATE RETENTION POLICY high_precision ON mydb
DURATION 7d REPLICATION 3 DEFAULT

36. Create a retention policy
CREATE RETENTION POLICY <rp-name> ON <db-name>
DURATION <duration> REPLICATION <n> [DEFAULT]
CREATE RETENTION POLICY high_precision ON mydb
DURATION 7d REPLICATION 3 DEFAULT
Writes will go into this RP unless
otherwise specified

37. Discovery

38. Inverted index
of measurements and tags

39. Discovery
SHOW MEASUREMENTs

40. Discovery
SHOW MEASUREMENTs
SHOW MEASUREMENTS where host = 'serverA'

41. Discovery
SHOW MEASUREMENTs
SHOW MEASUREMENTS where host = 'serverA'
SHOW TAG KEYS

42. Discovery
SHOW MEASUREMENTs
SHOW MEASUREMENTS where host = 'serverA'
SHOW TAG KEYS
SHOW TAG KEYS from CPU

43. Discovery
SHOW MEASUREMENTs
SHOW MEASUREMENTS where host = 'serverA'
SHOW TAG KEYS
SHOW TAG KEYS from CPU
SHOW TAG VALUES from CPU WITH KEY = 'region'

44. Discovery
SHOW MEASUREMENTs
SHOW MEASUREMENTS where host = 'serverA'
SHOW TAG KEYS
SHOW TAG KEYS from CPU
SHOW TAG VALUES from CPU WITH KEY = 'region'
SHOW SERIES

45. Discovery
SHOW MEASUREMENTs
SHOW MEASUREMENTS where host = 'serverA'
SHOW TAG KEYS
SHOW TAG KEYS from CPU
SHOW TAG VALUES from CPU WITH KEY = 'region'
SHOW SERIES
SHOW SERIES where service = 'redis'

46. Queries

47. SQL-ish
select * from some_series
where time > now() - 1h

48. Aggregates
select percentile(90, value) from cpu
where time > now() - 1d
group by time(10m)

49. Aggregates
select percentile(90, value) from cpu
where time > now() - 1d
group by time(10m), region
Group by a tag

50. Where against Regex (field)
select value from some_log_series
where value =~ /.*ERROR.*/ and
time > "2014-03-01" and time < "2014-03-03"

51. Where against Regex (tag)
select value from some_log_series
where host =~ /.*asdf.*/ and
time > "2014-03-01" and time < “2014-03-03"
group by host

52. Functions
min
max
percentile
first
last
stddev
mean
count
sum
median
distinct
count(distinct)
more soon: difference, histogram, moving_average

53. Continuous queries
CREATE CONTINUOUS QUERY "10m_event_count"
ON mydb
BEGIN
SELECT count(value)
INTO "6_months".events
FROM events
GROUP BY time(10m)
END;

54. Other tools

55. Telegraf
data collection

56. Chronograf

57. Grafana

58. More coming
• Compression
• Clustering
• Custom functions

59. Thank you!
Paul Dix
@pauldix
paul@influxdb.com