Enviar búsqueda
Cargar
Metrics: where and how
•
Descargar como PPTX, PDF
•
8 recomendaciones
•
2,230 vistas
Vsevolod Polyakov
Seguir
Graphite tuning story from Kyiv Devops Day 2016
Leer menos
Leer más
Software
Denunciar
Compartir
Denunciar
Compartir
1 de 65
Descargar ahora
Recomendados
PySpark Programming | PySpark Concepts with Hands-On | PySpark Training | Edu...
PySpark Programming | PySpark Concepts with Hands-On | PySpark Training | Edu...
Edureka!
Lesson planning
Lesson planning
Nazia Ashraf
Apache Arrow: Open Source Standard Becomes an Enterprise Necessity
Apache Arrow: Open Source Standard Becomes an Enterprise Necessity
Wes McKinney
Tools and techniques of blended learning
Tools and techniques of blended learning
Forum of Blended Learning
Observing The Observation: Towards A Reflective Classroom Peer Observation
Observing The Observation: Towards A Reflective Classroom Peer Observation
Abdeslam Badre, PhD
Brain Based Learning
Brain Based Learning
hrush189
Enhancing Students larning and negative attitudes
Enhancing Students larning and negative attitudes
Omar Noursi
Treasure Dataを支える技術 - MessagePack編
Treasure Dataを支える技術 - MessagePack編
Taro L. Saito
Recomendados
PySpark Programming | PySpark Concepts with Hands-On | PySpark Training | Edu...
PySpark Programming | PySpark Concepts with Hands-On | PySpark Training | Edu...
Edureka!
Lesson planning
Lesson planning
Nazia Ashraf
Apache Arrow: Open Source Standard Becomes an Enterprise Necessity
Apache Arrow: Open Source Standard Becomes an Enterprise Necessity
Wes McKinney
Tools and techniques of blended learning
Tools and techniques of blended learning
Forum of Blended Learning
Observing The Observation: Towards A Reflective Classroom Peer Observation
Observing The Observation: Towards A Reflective Classroom Peer Observation
Abdeslam Badre, PhD
Brain Based Learning
Brain Based Learning
hrush189
Enhancing Students larning and negative attitudes
Enhancing Students larning and negative attitudes
Omar Noursi
Treasure Dataを支える技術 - MessagePack編
Treasure Dataを支える技術 - MessagePack編
Taro L. Saito
Language And Reading
Language And Reading
Heather Weiler
Active learning
Active learning
ahmedabbas1121
Accelerating Data Processing in Spark SQL with Pandas UDFs
Accelerating Data Processing in Spark SQL with Pandas UDFs
Databricks
HBase Vs Cassandra Vs MongoDB - Choosing the right NoSQL database
HBase Vs Cassandra Vs MongoDB - Choosing the right NoSQL database
Edureka!
Brain based learning
Brain based learning
praveenraj265
Intro to Apache Spark
Intro to Apache Spark
Robert Sanders
Student Engagement
Student Engagement
Rajeev Ranjan
Reading models
Reading models
Jesullyna Manuel
ADDIE Model for Instructional Design by Operational Excellence Consulting
ADDIE Model for Instructional Design by Operational Excellence Consulting
Operational Excellence Consulting
Data Source API in Spark
Data Source API in Spark
Databricks
Reading in Kindergarten
Reading in Kindergarten
Hertiki Marsaid
Lesson Plan PowerPoint Presentation
Lesson Plan PowerPoint Presentation
yseauy
Technology Action Plan I
Technology Action Plan I
evabmccarty
Apache Spark Overview
Apache Spark Overview
Vadim Y. Bichutskiy
Scoring Rubric: Oral presentation
Scoring Rubric: Oral presentation
Lee Kerk
Tuning Apache Spark for Large-Scale Workloads Gaoxiang Liu and Sital Kedia
Tuning Apache Spark for Large-Scale Workloads Gaoxiang Liu and Sital Kedia
Databricks
Spark streaming , Spark SQL
Spark streaming , Spark SQL
Yousun Jeong
Effective classroom strategies ppt
Effective classroom strategies ppt
laraib asif
classroom observation
classroom observation
Lakhyajit Nath
Benefits of Cooperative Learning in the Classroom
Benefits of Cooperative Learning in the Classroom
Dr. Aries Cobb
Путь мониторинга, DevOps club в Grammarly
Путь мониторинга, DevOps club в Grammarly
Vsevolod Polyakov
Monitoring base, golang meetup, kyiv
Monitoring base, golang meetup, kyiv
Vsevolod Polyakov
Más contenido relacionado
La actualidad más candente
Language And Reading
Language And Reading
Heather Weiler
Active learning
Active learning
ahmedabbas1121
Accelerating Data Processing in Spark SQL with Pandas UDFs
Accelerating Data Processing in Spark SQL with Pandas UDFs
Databricks
HBase Vs Cassandra Vs MongoDB - Choosing the right NoSQL database
HBase Vs Cassandra Vs MongoDB - Choosing the right NoSQL database
Edureka!
Brain based learning
Brain based learning
praveenraj265
Intro to Apache Spark
Intro to Apache Spark
Robert Sanders
Student Engagement
Student Engagement
Rajeev Ranjan
Reading models
Reading models
Jesullyna Manuel
ADDIE Model for Instructional Design by Operational Excellence Consulting
ADDIE Model for Instructional Design by Operational Excellence Consulting
Operational Excellence Consulting
Data Source API in Spark
Data Source API in Spark
Databricks
Reading in Kindergarten
Reading in Kindergarten
Hertiki Marsaid
Lesson Plan PowerPoint Presentation
Lesson Plan PowerPoint Presentation
yseauy
Technology Action Plan I
Technology Action Plan I
evabmccarty
Apache Spark Overview
Apache Spark Overview
Vadim Y. Bichutskiy
Scoring Rubric: Oral presentation
Scoring Rubric: Oral presentation
Lee Kerk
Tuning Apache Spark for Large-Scale Workloads Gaoxiang Liu and Sital Kedia
Tuning Apache Spark for Large-Scale Workloads Gaoxiang Liu and Sital Kedia
Databricks
Spark streaming , Spark SQL
Spark streaming , Spark SQL
Yousun Jeong
Effective classroom strategies ppt
Effective classroom strategies ppt
laraib asif
classroom observation
classroom observation
Lakhyajit Nath
Benefits of Cooperative Learning in the Classroom
Benefits of Cooperative Learning in the Classroom
Dr. Aries Cobb
La actualidad más candente
(20)
Language And Reading
Language And Reading
Active learning
Active learning
Accelerating Data Processing in Spark SQL with Pandas UDFs
Accelerating Data Processing in Spark SQL with Pandas UDFs
HBase Vs Cassandra Vs MongoDB - Choosing the right NoSQL database
HBase Vs Cassandra Vs MongoDB - Choosing the right NoSQL database
Brain based learning
Brain based learning
Intro to Apache Spark
Intro to Apache Spark
Student Engagement
Student Engagement
Reading models
Reading models
ADDIE Model for Instructional Design by Operational Excellence Consulting
ADDIE Model for Instructional Design by Operational Excellence Consulting
Data Source API in Spark
Data Source API in Spark
Reading in Kindergarten
Reading in Kindergarten
Lesson Plan PowerPoint Presentation
Lesson Plan PowerPoint Presentation
Technology Action Plan I
Technology Action Plan I
Apache Spark Overview
Apache Spark Overview
Scoring Rubric: Oral presentation
Scoring Rubric: Oral presentation
Tuning Apache Spark for Large-Scale Workloads Gaoxiang Liu and Sital Kedia
Tuning Apache Spark for Large-Scale Workloads Gaoxiang Liu and Sital Kedia
Spark streaming , Spark SQL
Spark streaming , Spark SQL
Effective classroom strategies ppt
Effective classroom strategies ppt
classroom observation
classroom observation
Benefits of Cooperative Learning in the Classroom
Benefits of Cooperative Learning in the Classroom
Destacado
Путь мониторинга, DevOps club в Grammarly
Путь мониторинга, DevOps club в Grammarly
Vsevolod Polyakov
Monitoring base, golang meetup, kyiv
Monitoring base, golang meetup, kyiv
Vsevolod Polyakov
Путь мониторинга: модульность, гибкость, devops
Путь мониторинга: модульность, гибкость, devops
Vsevolod Polyakov
Мониторинг. Опять, rootconf 2016
Мониторинг. Опять, rootconf 2016
Vsevolod Polyakov
Chef wtf
Chef wtf
Vsevolod Polyakov
Путь мониторинга 2.0 всё стало другим / Всеволод Поляков (Grammarly)
Путь мониторинга 2.0 всё стало другим / Всеволод Поляков (Grammarly)
Ontico
Federated Graphite in Docker - Denver Docker Meetup
Federated Graphite in Docker - Denver Docker Meetup
Phil Zimmerman
Infrastructure as code might be literally impossible part 2
Infrastructure as code might be literally impossible part 2
ice799
DevOps в реальном времени
DevOps в реальном времени
Andriy Samilyak
DevOps in realtime
DevOps in realtime
Andriy Samilyak
Сергей Кибиткин - Meet Magento Ukraine - Что вы никогда не сделаете в Magento
Сергей Кибиткин - Meet Magento Ukraine - Что вы никогда не сделаете в Magento
Atwix
Александр Смага, Юрий Муратов - Meet Magento Ukraine - Технический обзор OroCRM
Александр Смага, Юрий Муратов - Meet Magento Ukraine - Технический обзор OroCRM
Atwix
Itea dev ops_course_topic2
Itea dev ops_course_topic2
Vad Tymoshyk
Андрей Самиляк - Meet Magento Ukraine - Как мы играли в DevOps и как получилс...
Андрей Самиляк - Meet Magento Ukraine - Как мы играли в DevOps и как получилс...
Atwix
Functional web with elixir and elm in phoenix
Functional web with elixir and elm in phoenix
Izzet Mustafaiev
Roman Valchuk "Introducing to DevOps technologies"
Roman Valchuk "Introducing to DevOps technologies"
Vadym Muliavka
Final melnyk 2
Final melnyk 2
Roman Melnyk
#nostaging - Software Circus - Amsterdam, 2-9-2016
#nostaging - Software Circus - Amsterdam, 2-9-2016
Pavel Chunyayev
Continuous integration with Docker and Ansible
Continuous integration with Docker and Ansible
Dmytro Slupytskyi
Implementing DevOps In Practice
Implementing DevOps In Practice
Zoltán Németh
Destacado
(20)
Путь мониторинга, DevOps club в Grammarly
Путь мониторинга, DevOps club в Grammarly
Monitoring base, golang meetup, kyiv
Monitoring base, golang meetup, kyiv
Путь мониторинга: модульность, гибкость, devops
Путь мониторинга: модульность, гибкость, devops
Мониторинг. Опять, rootconf 2016
Мониторинг. Опять, rootconf 2016
Chef wtf
Chef wtf
Путь мониторинга 2.0 всё стало другим / Всеволод Поляков (Grammarly)
Путь мониторинга 2.0 всё стало другим / Всеволод Поляков (Grammarly)
Federated Graphite in Docker - Denver Docker Meetup
Federated Graphite in Docker - Denver Docker Meetup
Infrastructure as code might be literally impossible part 2
Infrastructure as code might be literally impossible part 2
DevOps в реальном времени
DevOps в реальном времени
DevOps in realtime
DevOps in realtime
Сергей Кибиткин - Meet Magento Ukraine - Что вы никогда не сделаете в Magento
Сергей Кибиткин - Meet Magento Ukraine - Что вы никогда не сделаете в Magento
Александр Смага, Юрий Муратов - Meet Magento Ukraine - Технический обзор OroCRM
Александр Смага, Юрий Муратов - Meet Magento Ukraine - Технический обзор OroCRM
Itea dev ops_course_topic2
Itea dev ops_course_topic2
Андрей Самиляк - Meet Magento Ukraine - Как мы играли в DevOps и как получилс...
Андрей Самиляк - Meet Magento Ukraine - Как мы играли в DevOps и как получилс...
Functional web with elixir and elm in phoenix
Functional web with elixir and elm in phoenix
Roman Valchuk "Introducing to DevOps technologies"
Roman Valchuk "Introducing to DevOps technologies"
Final melnyk 2
Final melnyk 2
#nostaging - Software Circus - Amsterdam, 2-9-2016
#nostaging - Software Circus - Amsterdam, 2-9-2016
Continuous integration with Docker and Ansible
Continuous integration with Docker and Ansible
Implementing DevOps In Practice
Implementing DevOps In Practice
Similar a Metrics: where and how
"Metrics: Where and How", Vsevolod Polyakov
"Metrics: Where and How", Vsevolod Polyakov
Yulia Shcherbachova
Managing terabytes: When Postgres gets big
Managing terabytes: When Postgres gets big
Selena Deckelmann
Managing terabytes: When PostgreSQL gets big
Managing terabytes: When PostgreSQL gets big
Selena Deckelmann
Am I reading GC logs Correctly?
Am I reading GC logs Correctly?
Tier1 App
Vaex pygrunn
Vaex pygrunn
Maarten Breddels
JDD2015: On-heap cache vs Off-heap cache - Radek Grębski
JDD2015: On-heap cache vs Off-heap cache - Radek Grębski
PROIDEA
On heap cache vs off-heap cache
On heap cache vs off-heap cache
rgrebski
Pick diamonds from garbage
Pick diamonds from garbage
Tier1 App
Chicago-Java-User-Group-Meetup-Some-Garbage-Talk-2015-01-14
Chicago-Java-User-Group-Meetup-Some-Garbage-Talk-2015-01-14
Jayesh Thakrar
Hadoop performance optimization tips
Hadoop performance optimization tips
Subhas Kumar Ghosh
Java Garbage Collectors – Moving to Java7 Garbage First (G1) Collector
Java Garbage Collectors – Moving to Java7 Garbage First (G1) Collector
Gurpreet Sachdeva
InfluxDB IOx Tech Talks: Intro to the InfluxDB IOx Read Buffer - A Read-Optim...
InfluxDB IOx Tech Talks: Intro to the InfluxDB IOx Read Buffer - A Read-Optim...
InfluxData
Adaptive Linear Solvers and Eigensolvers
Adaptive Linear Solvers and Eigensolvers
inside-BigData.com
Ensuring High Availability for Real-time Analytics featuring Boxed Ice / Serv...
Ensuring High Availability for Real-time Analytics featuring Boxed Ice / Serv...
MongoDB
Couchbase live 2016
Couchbase live 2016
Pierre Mavro
Top 5 Mistakes When Writing Spark Applications
Top 5 Mistakes When Writing Spark Applications
Spark Summit
Top 5 mistakes when writing Spark applications
Top 5 mistakes when writing Spark applications
hadooparchbook
Tweaking perfomance on high-load projects_Думанский Дмитрий
Tweaking perfomance on high-load projects_Думанский Дмитрий
GeeksLab Odessa
Top 5 mistakes when writing Spark applications
Top 5 mistakes when writing Spark applications
markgrover
Jvm & Garbage collection tuning for low latencies application
Jvm & Garbage collection tuning for low latencies application
Quentin Ambard
Similar a Metrics: where and how
(20)
"Metrics: Where and How", Vsevolod Polyakov
"Metrics: Where and How", Vsevolod Polyakov
Managing terabytes: When Postgres gets big
Managing terabytes: When Postgres gets big
Managing terabytes: When PostgreSQL gets big
Managing terabytes: When PostgreSQL gets big
Am I reading GC logs Correctly?
Am I reading GC logs Correctly?
Vaex pygrunn
Vaex pygrunn
JDD2015: On-heap cache vs Off-heap cache - Radek Grębski
JDD2015: On-heap cache vs Off-heap cache - Radek Grębski
On heap cache vs off-heap cache
On heap cache vs off-heap cache
Pick diamonds from garbage
Pick diamonds from garbage
Chicago-Java-User-Group-Meetup-Some-Garbage-Talk-2015-01-14
Chicago-Java-User-Group-Meetup-Some-Garbage-Talk-2015-01-14
Hadoop performance optimization tips
Hadoop performance optimization tips
Java Garbage Collectors – Moving to Java7 Garbage First (G1) Collector
Java Garbage Collectors – Moving to Java7 Garbage First (G1) Collector
InfluxDB IOx Tech Talks: Intro to the InfluxDB IOx Read Buffer - A Read-Optim...
InfluxDB IOx Tech Talks: Intro to the InfluxDB IOx Read Buffer - A Read-Optim...
Adaptive Linear Solvers and Eigensolvers
Adaptive Linear Solvers and Eigensolvers
Ensuring High Availability for Real-time Analytics featuring Boxed Ice / Serv...
Ensuring High Availability for Real-time Analytics featuring Boxed Ice / Serv...
Couchbase live 2016
Couchbase live 2016
Top 5 Mistakes When Writing Spark Applications
Top 5 Mistakes When Writing Spark Applications
Top 5 mistakes when writing Spark applications
Top 5 mistakes when writing Spark applications
Tweaking perfomance on high-load projects_Думанский Дмитрий
Tweaking perfomance on high-load projects_Думанский Дмитрий
Top 5 mistakes when writing Spark applications
Top 5 mistakes when writing Spark applications
Jvm & Garbage collection tuning for low latencies application
Jvm & Garbage collection tuning for low latencies application
Último
%in kaalfontein+277-882-255-28 abortion pills for sale in kaalfontein
%in kaalfontein+277-882-255-28 abortion pills for sale in kaalfontein
masabamasaba
%+27788225528 love spells in new york Psychic Readings, Attraction spells,Bri...
%+27788225528 love spells in new york Psychic Readings, Attraction spells,Bri...
masabamasaba
Architecture decision records - How not to get lost in the past
Architecture decision records - How not to get lost in the past
Papp Krisztián
VTU technical seminar 8Th Sem on Scikit-learn
VTU technical seminar 8Th Sem on Scikit-learn
AmarnathKambale
%in Midrand+277-882-255-28 abortion pills for sale in midrand
%in Midrand+277-882-255-28 abortion pills for sale in midrand
masabamasaba
WSO2Con2024 - From Code To Cloud: Fast Track Your Cloud Native Journey with C...
WSO2Con2024 - From Code To Cloud: Fast Track Your Cloud Native Journey with C...
WSO2
%in Benoni+277-882-255-28 abortion pills for sale in Benoni
%in Benoni+277-882-255-28 abortion pills for sale in Benoni
masabamasaba
WSO2CON2024 - It's time to go Platformless
WSO2CON2024 - It's time to go Platformless
WSO2
%+27788225528 love spells in Toronto Psychic Readings, Attraction spells,Brin...
%+27788225528 love spells in Toronto Psychic Readings, Attraction spells,Brin...
masabamasaba
%in tembisa+277-882-255-28 abortion pills for sale in tembisa
%in tembisa+277-882-255-28 abortion pills for sale in tembisa
masabamasaba
%in Stilfontein+277-882-255-28 abortion pills for sale in Stilfontein
%in Stilfontein+277-882-255-28 abortion pills for sale in Stilfontein
masabamasaba
%+27788225528 love spells in Atlanta Psychic Readings, Attraction spells,Brin...
%+27788225528 love spells in Atlanta Psychic Readings, Attraction spells,Brin...
masabamasaba
%+27788225528 love spells in Knoxville Psychic Readings, Attraction spells,Br...
%+27788225528 love spells in Knoxville Psychic Readings, Attraction spells,Br...
masabamasaba
%+27788225528 love spells in Boston Psychic Readings, Attraction spells,Bring...
%+27788225528 love spells in Boston Psychic Readings, Attraction spells,Bring...
masabamasaba
%in Bahrain+277-882-255-28 abortion pills for sale in Bahrain
%in Bahrain+277-882-255-28 abortion pills for sale in Bahrain
masabamasaba
WSO2CON 2024 - API Management Usage at La Poste and Its Impact on Business an...
WSO2CON 2024 - API Management Usage at La Poste and Its Impact on Business an...
WSO2
Artyushina_Guest lecture_YorkU CS May 2024.pptx
Artyushina_Guest lecture_YorkU CS May 2024.pptx
AnnaArtyushina1
%+27788225528 love spells in Colorado Springs Psychic Readings, Attraction sp...
%+27788225528 love spells in Colorado Springs Psychic Readings, Attraction sp...
masabamasaba
%in Soweto+277-882-255-28 abortion pills for sale in soweto
%in Soweto+277-882-255-28 abortion pills for sale in soweto
masabamasaba
Large-scale Logging Made Easy: Meetup at Deutsche Bank 2024
Large-scale Logging Made Easy: Meetup at Deutsche Bank 2024
VictoriaMetrics
Último
(20)
%in kaalfontein+277-882-255-28 abortion pills for sale in kaalfontein
%in kaalfontein+277-882-255-28 abortion pills for sale in kaalfontein
%+27788225528 love spells in new york Psychic Readings, Attraction spells,Bri...
%+27788225528 love spells in new york Psychic Readings, Attraction spells,Bri...
Architecture decision records - How not to get lost in the past
Architecture decision records - How not to get lost in the past
VTU technical seminar 8Th Sem on Scikit-learn
VTU technical seminar 8Th Sem on Scikit-learn
%in Midrand+277-882-255-28 abortion pills for sale in midrand
%in Midrand+277-882-255-28 abortion pills for sale in midrand
WSO2Con2024 - From Code To Cloud: Fast Track Your Cloud Native Journey with C...
WSO2Con2024 - From Code To Cloud: Fast Track Your Cloud Native Journey with C...
%in Benoni+277-882-255-28 abortion pills for sale in Benoni
%in Benoni+277-882-255-28 abortion pills for sale in Benoni
WSO2CON2024 - It's time to go Platformless
WSO2CON2024 - It's time to go Platformless
%+27788225528 love spells in Toronto Psychic Readings, Attraction spells,Brin...
%+27788225528 love spells in Toronto Psychic Readings, Attraction spells,Brin...
%in tembisa+277-882-255-28 abortion pills for sale in tembisa
%in tembisa+277-882-255-28 abortion pills for sale in tembisa
%in Stilfontein+277-882-255-28 abortion pills for sale in Stilfontein
%in Stilfontein+277-882-255-28 abortion pills for sale in Stilfontein
%+27788225528 love spells in Atlanta Psychic Readings, Attraction spells,Brin...
%+27788225528 love spells in Atlanta Psychic Readings, Attraction spells,Brin...
%+27788225528 love spells in Knoxville Psychic Readings, Attraction spells,Br...
%+27788225528 love spells in Knoxville Psychic Readings, Attraction spells,Br...
%+27788225528 love spells in Boston Psychic Readings, Attraction spells,Bring...
%+27788225528 love spells in Boston Psychic Readings, Attraction spells,Bring...
%in Bahrain+277-882-255-28 abortion pills for sale in Bahrain
%in Bahrain+277-882-255-28 abortion pills for sale in Bahrain
WSO2CON 2024 - API Management Usage at La Poste and Its Impact on Business an...
WSO2CON 2024 - API Management Usage at La Poste and Its Impact on Business an...
Artyushina_Guest lecture_YorkU CS May 2024.pptx
Artyushina_Guest lecture_YorkU CS May 2024.pptx
%+27788225528 love spells in Colorado Springs Psychic Readings, Attraction sp...
%+27788225528 love spells in Colorado Springs Psychic Readings, Attraction sp...
%in Soweto+277-882-255-28 abortion pills for sale in soweto
%in Soweto+277-882-255-28 abortion pills for sale in soweto
Large-scale Logging Made Easy: Meetup at Deutsche Bank 2024
Large-scale Logging Made Easy: Meetup at Deutsche Bank 2024
Metrics: where and how
1.
Metrics: where and
how graphite-oriented story
2.
• Vsevolod Polyakov •
Platform Engineer at Grammarly
3.
Graphite All whisper-based systems
4.
Default graphite architecture
5.
what? • RRD-like (gram.ly/gfsx) •
so.it.is.my.metric → /so/it/is/my/metric.wsp • Fixed retention (by namepattern) • Fixed size (actually no)
6.
Retention and size •
1s:1d → 1 036 828 bytes • 10s:10d → 1 036 828 bytes • 1s:365d → 378 432 028 bytes (1 TB ~ 3 000) • 10s:365d → 37 843 228 bytes (1 TB ~ 30 000) whisper calc
7.
Retention and size •
10s:30d,1m:120d,10m:365d → 4 564 864 bytes • 240 864 metrics in 1 TB • aggregation: average, sum, min, max, and last. • can be assign per metric
8.
How • terraform (https://www.terraform.io/) •
docker (https://www.docker.com/) • ansible (https://www.ansible.com/) • rocker (https://github.com/grammarly/rocker) • rocker-compose (https://github.com/grammarly/rocker-compose)
9.
Default graphite architecture
10.
carbon-cache.py • single-core • many
options in config file • default link
11.
architecture carbon-cache.py
12.
Start load testing •
m4.xlarge instance (4 CPU, 16 GB ram, 256 GB disk EBS gp2) • retentions = 1s:1d • MAX_CACHE_SIZE, MAX_UPDATES_PER_SECOND, MAX_CR • defaults • almost 1.5h to get limit :(
13.
carbon-cache.py cache size
→ 75k reqs
14.
15.
16.
results • 75 000
reqs max • 60 000 reqs flagman speed • IO :(
17.
Try to tune! •
WHISPER_SPARSE_CREATE = true (don’t allocate space on creation) non-linear IO load. • CACHE_WRITE_STRATEGY = sorted (default)
18.
cache size 1k
→ 195k reqs
19.
results • 120 000
reqs flagman speed • cache flush problem :(
20.
Try to tune! •
CACHE_WRITE_STRATEGY = max will give a strong flush preference to frequently updated metrics and will also reduce random file-io.
21.
from 1k to
150k
22.
results • 90 000
reqs flagman speed • cache flush problem :(
23.
Try to tune! •
CACHE_WRITE_STRATEGY = naive just flush. Better with random IO.
24.
from 45k to
135k
25.
results • 120 000
reqs flagman speed • still CPU
26.
sorted max naive
27.
• Maybe it’s
IO EBS limitation? → 512 GB disk. • No.
28.
go-carbon • multi-core single
daemon • written in golang • not many options to tune :( link
29.
Start load testing •
m4.xlarge instance (4 CPU, 16 GB ram, 256 GB disk EBS gp2) • retentions = 1s:1d • max-size = 0 • max-updates-per-second = 0 • almost 1h to get limit :(
30.
1k → 130k
reqs ~3k/min
31.
32.
results • 120 000
reqs flagman speed • but it’s without sparse. • try to implement
33.
try to tune! remaining
:= whisper.Size() - whisper.MetadataSize() whisper.file.Seek(int64(remaining-1), 0) whisper.file.Write([]byte{0}) chunkSize := 16384 zeros := make([]byte, chunkSize) for remaining > chunkSize { // if _, err = whisper.file.Write(zeros); err != nil { // return nil, err // } remaining -= chunkSize } if _, err = whisper.file.Write(zeros[:remaining]); err != nil { return nil, err }
34.
180 000 reqs
!
35.
36.
try to tune! •
max update operation = 1500
37.
results • TLDR 210
000 - 240 000 reqs flagman speed • 31 000 000 cache size!
38.
39.
try to tune! •
max update operation = 0 • input-buffer = 400 000
40.
results • 270 000
reqs flagman speed • 10-20 million req cache size!
41.
42.
try to tune! •
vm.dirty_background_ratio=40 • vm.dirty_ratio=60
43.
300 000 reqs
44.
results • 300 000
reqs flagman speed • 180k+ reqs ±without cache
45.
Re:Lays
46.
Default graphite architecture
47.
arch forward
48.
arch namedregexp
49.
arch hash
50.
arch hash replicafactor:
2
51.
carbon-relay.py • twisted based •
native
52.
Start load testing •
c4.xlarge instance (4 CPU, 7.5 GB ram) • ~1 Gb lan • default parameters • hashing • 10 connections
53.
WTF!
54.
carbon-relay-ng • golang-based • web-panel •
live-updates • aggregators • spooling link
55.
<150 000 reqs
56.
carbon-c-relay • written in
C • advanced cluster management
57.
from 100 000
to 1 600 000 reqs
58.
1 400 000
flagman speed. Or not?
59.
So… go-carbon + carbon-c-relay
= ♡
60.
BTW. influx, 130k
reqs on cluster
61.
influx
62.
openTSDB single instance +
hbase cluster = upto 150k reqs
63.
ALSO • zipper: • https://github.com/grobian/carbonserver •
https://github.com/grobian/carbonwriter • https://github.com/dgryski/carbonzipper • https://github.com/dgryski/carbonapi • https://github.com/dgryski/carbonmem • https://github.com/jssjr/carbonate
64.
plans • Cyanite, retest •
newTS • openTSDB tuninig • zipper tuning
65.
feel free to
ask • Vsevolod Polyakov • ctrlok@gmail.com • skype: ctrlok1987 • github.com/ctrlok • twitter.com/ctrlok • slack: HangOps • Gitter: dev_ua/devops • skype: DevOps from Ukraine
Notas del editor
последние 2,5 года работаю в граммарли разрабатываем крутые штуки, надо писать много метрик, бла бла бла
очень простой, все знают. Можно сказать стандарт индустрии для метрик
простота быстро читать, быстро писать
отключил логи, у амазона странные EBS
если кеш не сбрасывается то это плохо
скорость записи — большими кусками создаются файлы
линейная скорость чтения
отключил логи, у амазона странные EBS
Descargar ahora