SlideShare una empresa de Scribd logo
1 de 17
Descargar para leer sin conexión
Upgrading to InfluxDB 0.10:
Things to Consider
Sean Beckett
Director of Support & Professional Services
• New Storage Engine, TSM
• New Continuous Query syntax
• New Clustering
What's New?
0.8 —> 0.10
• New Schema
• New API and endpoints
• No MERGEs
• Shard Spaces become Retention Policies
• Updated Query syntax
• Administrative queries replace Admin UI
• Clustering in InfluxDB 0.10 is
not backwards compatible
• Continuous Query syntax and
configuration are different
Brief Mentions
https://github.com/influxdata/influxdb/blob/master/importer/README.md
https://docs.influxdata.com/influxdb/v0.9/concepts/08_vs_09/
https://docs.influxdata.com/influxdb/v0.10/concepts/09_vs_010/
TSM: Time Structured Merge-Tree
• Purpose-built
• Compression by design
• No "empty" space
• Read only what is needed
TSM: Best Practices - measurements
• Measurements are logical containers for series
• Measurements contain series with related field sets
• No JOINs or MERGEs across measurements
TSM: Best Practices - tags
• Tags are indexed
• Tags in the WHERE clause are performant
• GROUP BY operates on tags
• Tag Cardinality matters!
TSM: Best Practices - fields
• Fields are not indexed
• Fields in the WHERE clause are much less performant
• Functions operate only on fields
• Field cardinality is irrelevant*
• Multiple fields per point is more performant
TSM: Best Practices - timestamps
• Timestamp precision is controlled at write time
• Lower precision uses less disk, faster writes and reads
Migration Tools
InfluxDB 0.8.9 - Export (endpoint)
InfluxDB 0.9+ - Import via CLI (./influx)
InfluxDB 0.10+ - TSM conversion tool (./influx_tsm)
Example Migration #1
Conditions:
• Downtime is irrelevant
• Converting from 0.9
• Not clustered
Example Migration #1
1. Stop traffic to InfluxDB 0.9
2. Restart the process and wait for the WAL to flush
3. Stop InfluxDB
4. Install 0.10.1 on top of 0.9
5. Run ./influx_tsm to convert shards
influx_tsm -parallel /var/lib/influxdb/data
6. Once complete, launch InfluxDB 0.10.1
7. Resume traffic to InfluxDB
Example Migration #2
Conditions:
• Downtime is unacceptable
• Converting from 0.9
• Not clustered
Example Migration #2
1. Install 0.10.1 on top of 0.9 and restart
2. Copy "cold" B1/BZ1 shards to /tmp
3. Run ./influx_tsm to convert cold shards
influx_tsm -backup /tmp/influxdb_backup /tmp/data
4. Remove cold B1/BZ1 shards and replace with
converted TSM shards
5. Wait until current "hot" shards become "cold", repeat
steps 2-4 on newly cold shards
Example Migration #3
Conditions:
• Downtime is acceptable
• Converting from 0.8
• Not clustered
Example Migration #3
1. Stop traffic to InfluxDB 0.8.x
2. Install InfluxDB 0.8.9 and restart
3. Run the Export tool
curl -o export.gz --compressed http://localhost:8086/ 
export/<db>/<shard_space>
4. Launch InfluxDB 0.10.1 on a new server
5. Use the CLI to import the exported data
influx -import -path=export.gz -compressed > failures
6. Update clients and dashboards for new API, etc.
7. Resume traffic to InfluxDB 0.10.1
The Hard Stuff
• Fixing failed points in 0.8 import
• Migrating 0.8 queries where a column is treated as
both a tag and a field
• Controlling series cardinality
• Updating the clients to avoid downtime
• Updating the dashboards to avoid downtime
• Controlling for performance changes
Questions
docs.influxdata.com
influxdb@googlegroups.com
support@influxdb.com

Más contenido relacionado

La actualidad más candente

La actualidad más candente (20)

Stack Exchange Infrastructure - LISA 14
Stack Exchange Infrastructure - LISA 14Stack Exchange Infrastructure - LISA 14
Stack Exchange Infrastructure - LISA 14
 
Printing with AventX 10.0 is Faster than Ever!
Printing with AventX 10.0 is Faster than Ever!Printing with AventX 10.0 is Faster than Ever!
Printing with AventX 10.0 is Faster than Ever!
 
AB testing
AB testingAB testing
AB testing
 
Logical Replication in PostgreSQL - FLOSSUK 2016
Logical Replication in PostgreSQL - FLOSSUK 2016Logical Replication in PostgreSQL - FLOSSUK 2016
Logical Replication in PostgreSQL - FLOSSUK 2016
 
Native Clients, more the merrier with GFProxy!
Native Clients, more the merrier with GFProxy!Native Clients, more the merrier with GFProxy!
Native Clients, more the merrier with GFProxy!
 
Netcentric: customer experience
Netcentric: customer experienceNetcentric: customer experience
Netcentric: customer experience
 
How to optimize CloudLinux OS limits
How to optimize CloudLinux OS limitsHow to optimize CloudLinux OS limits
How to optimize CloudLinux OS limits
 
How to Add Swap on Ubuntu
How to Add Swap on UbuntuHow to Add Swap on Ubuntu
How to Add Swap on Ubuntu
 
Seastore: Next Generation Backing Store for Ceph
Seastore: Next Generation Backing Store for CephSeastore: Next Generation Backing Store for Ceph
Seastore: Next Generation Backing Store for Ceph
 
Postman tests in jenkins
Postman tests in jenkinsPostman tests in jenkins
Postman tests in jenkins
 
Benchmarking, Load Testing, and Preventing Terrible Disasters
Benchmarking, Load Testing, and Preventing Terrible DisastersBenchmarking, Load Testing, and Preventing Terrible Disasters
Benchmarking, Load Testing, and Preventing Terrible Disasters
 
Basics of Node.js
Basics of Node.jsBasics of Node.js
Basics of Node.js
 
Server Side Rendering, caching and performance
Server Side Rendering, caching and performanceServer Side Rendering, caching and performance
Server Side Rendering, caching and performance
 
Beyond 1000 bosh Deployments
Beyond 1000 bosh DeploymentsBeyond 1000 bosh Deployments
Beyond 1000 bosh Deployments
 
Extreme HTTP Performance Tuning: 1.2M API req/s on a 4 vCPU EC2 Instance
Extreme HTTP Performance Tuning: 1.2M API req/s on a 4 vCPU EC2 InstanceExtreme HTTP Performance Tuning: 1.2M API req/s on a 4 vCPU EC2 Instance
Extreme HTTP Performance Tuning: 1.2M API req/s on a 4 vCPU EC2 Instance
 
Scylla Summit 2018: Rebuilding the Ceph Distributed Storage Solution with Sea...
Scylla Summit 2018: Rebuilding the Ceph Distributed Storage Solution with Sea...Scylla Summit 2018: Rebuilding the Ceph Distributed Storage Solution with Sea...
Scylla Summit 2018: Rebuilding the Ceph Distributed Storage Solution with Sea...
 
Improving Development and Deployment with Docker
Improving Development and Deployment with DockerImproving Development and Deployment with Docker
Improving Development and Deployment with Docker
 
SVC / Storwize analysis cost effective storage planning (use case)
SVC / Storwize analysis cost effective storage planning (use case)SVC / Storwize analysis cost effective storage planning (use case)
SVC / Storwize analysis cost effective storage planning (use case)
 
Scaling application servers for efficiency
Scaling application servers for efficiencyScaling application servers for efficiency
Scaling application servers for efficiency
 
Storage Services
Storage ServicesStorage Services
Storage Services
 

Similar a InfluxDB: Upgrade to 0.10 considerations

Service-Level Objective for Serverless Applications
Service-Level Objective for Serverless ApplicationsService-Level Objective for Serverless Applications
Service-Level Objective for Serverless Applications
alekn
 
Sa106 – practical solutions for connections administrators
Sa106 – practical solutions for connections administratorsSa106 – practical solutions for connections administrators
Sa106 – practical solutions for connections administrators
Sharon James
 

Similar a InfluxDB: Upgrade to 0.10 considerations (20)

Hive acid-updates-strata-sjc-feb-2015
Hive acid-updates-strata-sjc-feb-2015Hive acid-updates-strata-sjc-feb-2015
Hive acid-updates-strata-sjc-feb-2015
 
Best And Worst Practices Deploying IBM Connections
Best And Worst Practices Deploying IBM ConnectionsBest And Worst Practices Deploying IBM Connections
Best And Worst Practices Deploying IBM Connections
 
Overview and Demonstration of Dimensions CM 14.2 (FUG presentation track 2)
Overview and Demonstration of Dimensions CM 14.2 (FUG presentation track 2)Overview and Demonstration of Dimensions CM 14.2 (FUG presentation track 2)
Overview and Demonstration of Dimensions CM 14.2 (FUG presentation track 2)
 
MongoDB Days UK: Using MongoDB to Build a Fast and Scalable Content Repositor...
MongoDB Days UK: Using MongoDB to Build a Fast and Scalable Content Repositor...MongoDB Days UK: Using MongoDB to Build a Fast and Scalable Content Repositor...
MongoDB Days UK: Using MongoDB to Build a Fast and Scalable Content Repositor...
 
Service-Level Objective for Serverless Applications
Service-Level Objective for Serverless ApplicationsService-Level Objective for Serverless Applications
Service-Level Objective for Serverless Applications
 
Journey towards serverless infrastructure
Journey towards serverless infrastructureJourney towards serverless infrastructure
Journey towards serverless infrastructure
 
Sa106 – practical solutions for connections administrators
Sa106 – practical solutions for connections administratorsSa106 – practical solutions for connections administrators
Sa106 – practical solutions for connections administrators
 
Domino Server Health - Monitoring and Managing
 Domino Server Health - Monitoring and Managing Domino Server Health - Monitoring and Managing
Domino Server Health - Monitoring and Managing
 
Practical solutions for connections administrators
Practical solutions for connections administratorsPractical solutions for connections administrators
Practical solutions for connections administrators
 
Hive Does ACID
Hive Does ACIDHive Does ACID
Hive Does ACID
 
Writing Services with ZF2
Writing Services with ZF2Writing Services with ZF2
Writing Services with ZF2
 
IBM WebSphere MQ for z/OS V8 - Latest Features Deep Dive
IBM WebSphere MQ for z/OS V8 - Latest Features Deep DiveIBM WebSphere MQ for z/OS V8 - Latest Features Deep Dive
IBM WebSphere MQ for z/OS V8 - Latest Features Deep Dive
 
(ATS4-PLAT01) Core Architecture Changes in AEP 9.0 and their Impact on Admini...
(ATS4-PLAT01) Core Architecture Changes in AEP 9.0 and their Impact on Admini...(ATS4-PLAT01) Core Architecture Changes in AEP 9.0 and their Impact on Admini...
(ATS4-PLAT01) Core Architecture Changes in AEP 9.0 and their Impact on Admini...
 
(ATS4-PLAT08) Server Pool Management
(ATS4-PLAT08) Server Pool Management(ATS4-PLAT08) Server Pool Management
(ATS4-PLAT08) Server Pool Management
 
ICONUK 2018 - IBM Notes V10 Performance Boost
ICONUK 2018 - IBM Notes V10 Performance BoostICONUK 2018 - IBM Notes V10 Performance Boost
ICONUK 2018 - IBM Notes V10 Performance Boost
 
Serverless Computing Model
Serverless Computing ModelServerless Computing Model
Serverless Computing Model
 
OSDC 2018 | Three years running containers with Kubernetes in Production by T...
OSDC 2018 | Three years running containers with Kubernetes in Production by T...OSDC 2018 | Three years running containers with Kubernetes in Production by T...
OSDC 2018 | Three years running containers with Kubernetes in Production by T...
 
Introducing Scylla Open Source 4.0
Introducing Scylla Open Source 4.0Introducing Scylla Open Source 4.0
Introducing Scylla Open Source 4.0
 
Introduction to Kubernetes
Introduction to KubernetesIntroduction to Kubernetes
Introduction to Kubernetes
 
20140128 webinar-get-more-out-of-mysql-with-tokudb-140319063324-phpapp02
20140128 webinar-get-more-out-of-mysql-with-tokudb-140319063324-phpapp0220140128 webinar-get-more-out-of-mysql-with-tokudb-140319063324-phpapp02
20140128 webinar-get-more-out-of-mysql-with-tokudb-140319063324-phpapp02
 

Último

Computer science Sql cheat sheet.pdf.pdf
Computer science Sql cheat sheet.pdf.pdfComputer science Sql cheat sheet.pdf.pdf
Computer science Sql cheat sheet.pdf.pdf
SayantanBiswas37
 
Top profile Call Girls In Hapur [ 7014168258 ] Call Me For Genuine Models We ...
Top profile Call Girls In Hapur [ 7014168258 ] Call Me For Genuine Models We ...Top profile Call Girls In Hapur [ 7014168258 ] Call Me For Genuine Models We ...
Top profile Call Girls In Hapur [ 7014168258 ] Call Me For Genuine Models We ...
nirzagarg
 
怎样办理圣地亚哥州立大学毕业证(SDSU毕业证书)成绩单学校原版复制
怎样办理圣地亚哥州立大学毕业证(SDSU毕业证书)成绩单学校原版复制怎样办理圣地亚哥州立大学毕业证(SDSU毕业证书)成绩单学校原版复制
怎样办理圣地亚哥州立大学毕业证(SDSU毕业证书)成绩单学校原版复制
vexqp
 
In Riyadh ((+919101817206)) Cytotec kit @ Abortion Pills Saudi Arabia
In Riyadh ((+919101817206)) Cytotec kit @ Abortion Pills Saudi ArabiaIn Riyadh ((+919101817206)) Cytotec kit @ Abortion Pills Saudi Arabia
In Riyadh ((+919101817206)) Cytotec kit @ Abortion Pills Saudi Arabia
ahmedjiabur940
 
Top profile Call Girls In Bihar Sharif [ 7014168258 ] Call Me For Genuine Mod...
Top profile Call Girls In Bihar Sharif [ 7014168258 ] Call Me For Genuine Mod...Top profile Call Girls In Bihar Sharif [ 7014168258 ] Call Me For Genuine Mod...
Top profile Call Girls In Bihar Sharif [ 7014168258 ] Call Me For Genuine Mod...
nirzagarg
 
Sealdah % High Class Call Girls Kolkata - 450+ Call Girl Cash Payment 8005736...
Sealdah % High Class Call Girls Kolkata - 450+ Call Girl Cash Payment 8005736...Sealdah % High Class Call Girls Kolkata - 450+ Call Girl Cash Payment 8005736...
Sealdah % High Class Call Girls Kolkata - 450+ Call Girl Cash Payment 8005736...
HyderabadDolls
 
Top profile Call Girls In dimapur [ 7014168258 ] Call Me For Genuine Models W...
Top profile Call Girls In dimapur [ 7014168258 ] Call Me For Genuine Models W...Top profile Call Girls In dimapur [ 7014168258 ] Call Me For Genuine Models W...
Top profile Call Girls In dimapur [ 7014168258 ] Call Me For Genuine Models W...
gajnagarg
 
Top profile Call Girls In Begusarai [ 7014168258 ] Call Me For Genuine Models...
Top profile Call Girls In Begusarai [ 7014168258 ] Call Me For Genuine Models...Top profile Call Girls In Begusarai [ 7014168258 ] Call Me For Genuine Models...
Top profile Call Girls In Begusarai [ 7014168258 ] Call Me For Genuine Models...
nirzagarg
 
Lecture_2_Deep_Learning_Overview-newone1
Lecture_2_Deep_Learning_Overview-newone1Lecture_2_Deep_Learning_Overview-newone1
Lecture_2_Deep_Learning_Overview-newone1
ranjankumarbehera14
 
Jodhpur Park | Call Girls in Kolkata Phone No 8005736733 Elite Escort Service...
Jodhpur Park | Call Girls in Kolkata Phone No 8005736733 Elite Escort Service...Jodhpur Park | Call Girls in Kolkata Phone No 8005736733 Elite Escort Service...
Jodhpur Park | Call Girls in Kolkata Phone No 8005736733 Elite Escort Service...
HyderabadDolls
 
+97470301568>>weed for sale in qatar ,weed for sale in dubai,weed for sale in...
+97470301568>>weed for sale in qatar ,weed for sale in dubai,weed for sale in...+97470301568>>weed for sale in qatar ,weed for sale in dubai,weed for sale in...
+97470301568>>weed for sale in qatar ,weed for sale in dubai,weed for sale in...
Health
 
Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
ZurliaSoop
 

Último (20)

Computer science Sql cheat sheet.pdf.pdf
Computer science Sql cheat sheet.pdf.pdfComputer science Sql cheat sheet.pdf.pdf
Computer science Sql cheat sheet.pdf.pdf
 
Top profile Call Girls In Hapur [ 7014168258 ] Call Me For Genuine Models We ...
Top profile Call Girls In Hapur [ 7014168258 ] Call Me For Genuine Models We ...Top profile Call Girls In Hapur [ 7014168258 ] Call Me For Genuine Models We ...
Top profile Call Girls In Hapur [ 7014168258 ] Call Me For Genuine Models We ...
 
Nirala Nagar / Cheap Call Girls In Lucknow Phone No 9548273370 Elite Escort S...
Nirala Nagar / Cheap Call Girls In Lucknow Phone No 9548273370 Elite Escort S...Nirala Nagar / Cheap Call Girls In Lucknow Phone No 9548273370 Elite Escort S...
Nirala Nagar / Cheap Call Girls In Lucknow Phone No 9548273370 Elite Escort S...
 
怎样办理圣地亚哥州立大学毕业证(SDSU毕业证书)成绩单学校原版复制
怎样办理圣地亚哥州立大学毕业证(SDSU毕业证书)成绩单学校原版复制怎样办理圣地亚哥州立大学毕业证(SDSU毕业证书)成绩单学校原版复制
怎样办理圣地亚哥州立大学毕业证(SDSU毕业证书)成绩单学校原版复制
 
RESEARCH-FINAL-DEFENSE-PPT-TEMPLATE.pptx
RESEARCH-FINAL-DEFENSE-PPT-TEMPLATE.pptxRESEARCH-FINAL-DEFENSE-PPT-TEMPLATE.pptx
RESEARCH-FINAL-DEFENSE-PPT-TEMPLATE.pptx
 
In Riyadh ((+919101817206)) Cytotec kit @ Abortion Pills Saudi Arabia
In Riyadh ((+919101817206)) Cytotec kit @ Abortion Pills Saudi ArabiaIn Riyadh ((+919101817206)) Cytotec kit @ Abortion Pills Saudi Arabia
In Riyadh ((+919101817206)) Cytotec kit @ Abortion Pills Saudi Arabia
 
20240412-SmartCityIndex-2024-Full-Report.pdf
20240412-SmartCityIndex-2024-Full-Report.pdf20240412-SmartCityIndex-2024-Full-Report.pdf
20240412-SmartCityIndex-2024-Full-Report.pdf
 
Top profile Call Girls In Bihar Sharif [ 7014168258 ] Call Me For Genuine Mod...
Top profile Call Girls In Bihar Sharif [ 7014168258 ] Call Me For Genuine Mod...Top profile Call Girls In Bihar Sharif [ 7014168258 ] Call Me For Genuine Mod...
Top profile Call Girls In Bihar Sharif [ 7014168258 ] Call Me For Genuine Mod...
 
Sealdah % High Class Call Girls Kolkata - 450+ Call Girl Cash Payment 8005736...
Sealdah % High Class Call Girls Kolkata - 450+ Call Girl Cash Payment 8005736...Sealdah % High Class Call Girls Kolkata - 450+ Call Girl Cash Payment 8005736...
Sealdah % High Class Call Girls Kolkata - 450+ Call Girl Cash Payment 8005736...
 
Predicting HDB Resale Prices - Conducting Linear Regression Analysis With Orange
Predicting HDB Resale Prices - Conducting Linear Regression Analysis With OrangePredicting HDB Resale Prices - Conducting Linear Regression Analysis With Orange
Predicting HDB Resale Prices - Conducting Linear Regression Analysis With Orange
 
Digital Advertising Lecture for Advanced Digital & Social Media Strategy at U...
Digital Advertising Lecture for Advanced Digital & Social Media Strategy at U...Digital Advertising Lecture for Advanced Digital & Social Media Strategy at U...
Digital Advertising Lecture for Advanced Digital & Social Media Strategy at U...
 
Top Call Girls in Balaghat 9332606886Call Girls Advance Cash On Delivery Ser...
Top Call Girls in Balaghat  9332606886Call Girls Advance Cash On Delivery Ser...Top Call Girls in Balaghat  9332606886Call Girls Advance Cash On Delivery Ser...
Top Call Girls in Balaghat 9332606886Call Girls Advance Cash On Delivery Ser...
 
Fun all Day Call Girls in Jaipur 9332606886 High Profile Call Girls You Ca...
Fun all Day Call Girls in Jaipur   9332606886  High Profile Call Girls You Ca...Fun all Day Call Girls in Jaipur   9332606886  High Profile Call Girls You Ca...
Fun all Day Call Girls in Jaipur 9332606886 High Profile Call Girls You Ca...
 
Top profile Call Girls In dimapur [ 7014168258 ] Call Me For Genuine Models W...
Top profile Call Girls In dimapur [ 7014168258 ] Call Me For Genuine Models W...Top profile Call Girls In dimapur [ 7014168258 ] Call Me For Genuine Models W...
Top profile Call Girls In dimapur [ 7014168258 ] Call Me For Genuine Models W...
 
High Profile Call Girls Service in Jalore { 9332606886 } VVIP NISHA Call Girl...
High Profile Call Girls Service in Jalore { 9332606886 } VVIP NISHA Call Girl...High Profile Call Girls Service in Jalore { 9332606886 } VVIP NISHA Call Girl...
High Profile Call Girls Service in Jalore { 9332606886 } VVIP NISHA Call Girl...
 
Top profile Call Girls In Begusarai [ 7014168258 ] Call Me For Genuine Models...
Top profile Call Girls In Begusarai [ 7014168258 ] Call Me For Genuine Models...Top profile Call Girls In Begusarai [ 7014168258 ] Call Me For Genuine Models...
Top profile Call Girls In Begusarai [ 7014168258 ] Call Me For Genuine Models...
 
Lecture_2_Deep_Learning_Overview-newone1
Lecture_2_Deep_Learning_Overview-newone1Lecture_2_Deep_Learning_Overview-newone1
Lecture_2_Deep_Learning_Overview-newone1
 
Jodhpur Park | Call Girls in Kolkata Phone No 8005736733 Elite Escort Service...
Jodhpur Park | Call Girls in Kolkata Phone No 8005736733 Elite Escort Service...Jodhpur Park | Call Girls in Kolkata Phone No 8005736733 Elite Escort Service...
Jodhpur Park | Call Girls in Kolkata Phone No 8005736733 Elite Escort Service...
 
+97470301568>>weed for sale in qatar ,weed for sale in dubai,weed for sale in...
+97470301568>>weed for sale in qatar ,weed for sale in dubai,weed for sale in...+97470301568>>weed for sale in qatar ,weed for sale in dubai,weed for sale in...
+97470301568>>weed for sale in qatar ,weed for sale in dubai,weed for sale in...
 
Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
 

InfluxDB: Upgrade to 0.10 considerations

  • 1. Upgrading to InfluxDB 0.10: Things to Consider Sean Beckett Director of Support & Professional Services
  • 2. • New Storage Engine, TSM • New Continuous Query syntax • New Clustering What's New? 0.8 —> 0.10 • New Schema • New API and endpoints • No MERGEs • Shard Spaces become Retention Policies • Updated Query syntax • Administrative queries replace Admin UI
  • 3. • Clustering in InfluxDB 0.10 is not backwards compatible • Continuous Query syntax and configuration are different Brief Mentions https://github.com/influxdata/influxdb/blob/master/importer/README.md https://docs.influxdata.com/influxdb/v0.9/concepts/08_vs_09/ https://docs.influxdata.com/influxdb/v0.10/concepts/09_vs_010/
  • 4. TSM: Time Structured Merge-Tree • Purpose-built • Compression by design • No "empty" space • Read only what is needed
  • 5. TSM: Best Practices - measurements • Measurements are logical containers for series • Measurements contain series with related field sets • No JOINs or MERGEs across measurements
  • 6. TSM: Best Practices - tags • Tags are indexed • Tags in the WHERE clause are performant • GROUP BY operates on tags • Tag Cardinality matters!
  • 7. TSM: Best Practices - fields • Fields are not indexed • Fields in the WHERE clause are much less performant • Functions operate only on fields • Field cardinality is irrelevant* • Multiple fields per point is more performant
  • 8. TSM: Best Practices - timestamps • Timestamp precision is controlled at write time • Lower precision uses less disk, faster writes and reads
  • 9. Migration Tools InfluxDB 0.8.9 - Export (endpoint) InfluxDB 0.9+ - Import via CLI (./influx) InfluxDB 0.10+ - TSM conversion tool (./influx_tsm)
  • 10. Example Migration #1 Conditions: • Downtime is irrelevant • Converting from 0.9 • Not clustered
  • 11. Example Migration #1 1. Stop traffic to InfluxDB 0.9 2. Restart the process and wait for the WAL to flush 3. Stop InfluxDB 4. Install 0.10.1 on top of 0.9 5. Run ./influx_tsm to convert shards influx_tsm -parallel /var/lib/influxdb/data 6. Once complete, launch InfluxDB 0.10.1 7. Resume traffic to InfluxDB
  • 12. Example Migration #2 Conditions: • Downtime is unacceptable • Converting from 0.9 • Not clustered
  • 13. Example Migration #2 1. Install 0.10.1 on top of 0.9 and restart 2. Copy "cold" B1/BZ1 shards to /tmp 3. Run ./influx_tsm to convert cold shards influx_tsm -backup /tmp/influxdb_backup /tmp/data 4. Remove cold B1/BZ1 shards and replace with converted TSM shards 5. Wait until current "hot" shards become "cold", repeat steps 2-4 on newly cold shards
  • 14. Example Migration #3 Conditions: • Downtime is acceptable • Converting from 0.8 • Not clustered
  • 15. Example Migration #3 1. Stop traffic to InfluxDB 0.8.x 2. Install InfluxDB 0.8.9 and restart 3. Run the Export tool curl -o export.gz --compressed http://localhost:8086/ export/<db>/<shard_space> 4. Launch InfluxDB 0.10.1 on a new server 5. Use the CLI to import the exported data influx -import -path=export.gz -compressed > failures 6. Update clients and dashboards for new API, etc. 7. Resume traffic to InfluxDB 0.10.1
  • 16. The Hard Stuff • Fixing failed points in 0.8 import • Migrating 0.8 queries where a column is treated as both a tag and a field • Controlling series cardinality • Updating the clients to avoid downtime • Updating the dashboards to avoid downtime • Controlling for performance changes