Se ha denunciado esta presentación.
Se está descargando tu SlideShare. ×

ログ収集プラットフォーム開発におけるElasticsearchの運用

Anuncio
Anuncio
Anuncio
Anuncio
Anuncio
Anuncio
Anuncio
Anuncio
Anuncio
Anuncio
Anuncio
Anuncio
Cargando en…3
×

Eche un vistazo a continuación

1 de 45 Anuncio

ログ収集プラットフォーム開発におけるElasticsearchの運用

Descargar para leer sin conexión

齋藤 智之 (LINE株式会社 開発3室)
ログ収集プラットフォーム開発におけるElasticsearchの運用
LINE Developer Meetup in Tokyo #27 -Elastic-の発表資料です
https://line.connpass.com/event/76226/

齋藤 智之 (LINE株式会社 開発3室)
ログ収集プラットフォーム開発におけるElasticsearchの運用
LINE Developer Meetup in Tokyo #27 -Elastic-の発表資料です
https://line.connpass.com/event/76226/

Anuncio
Anuncio

Más Contenido Relacionado

Presentaciones para usted (20)

Similares a ログ収集プラットフォーム開発におけるElasticsearchの運用 (20)

Anuncio

Más de LINE Corporation (20)

Más reciente (20)

Anuncio

ログ収集プラットフォーム開発におけるElasticsearchの運用

  1. 1. Elasticsearch( , tomoyuki.saito@linecorp.com)
  2. 2. l l Site Reliability Engineer l l LINE l l Elasticsearch
  3. 3. • Project • Elasticsearch Issues Agenda
  4. 4. • • • Elasticsearch Clusters Project
  5. 5. LINE
  6. 6. talk-server: OBS: Auth: LINE
  7. 7. talk-server 9TB/day200 /day 40 /s
  8. 8. • • • Elasticsearch Clusters Project
  9. 9. Front-end Kafka l Apache Kafka: l l :
  10. 10. Sink l Kafka consumer application l l Elasticsearch: JSON bulk indexing
  11. 11. l Protocol Buffers message l Table( unit) l message : l Kafka topic: l : l Kafka message RequestLog { string method_name = 1; string url = 2; …}
  12. 12. l Custom logback appender (Logback: Java ) l protobuf message l Kafka l Kafka l library l logback Logging SDK
  13. 13. l Kafka, Elasticsearch, Hadoop l Sink l Logging l
  14. 14. • • • Elasticsearch Clusters Project
  15. 15. Elasticsearch Clusters 100+3 160+ docs 18+TB
  16. 16. 3,000+ 3,700+ docs 400+TB 17,000+
  17. 17. • • • Elasticsearch Clusters Project
  18. 18. • • Index template • Shard sizing • Indexing performance Issues
  19. 19. • • Index template • Shard sizing • Indexing performance Issues
  20. 20. 1Gbps
  21. 21. 1Gbps 10Gbps
  22. 22. 1Gbps 10Gbps
  23. 23. Performance degradation : Multiple Elasticsearch Clusters or a Monster Cluster?
  24. 24. -
  25. 25. • • Index template • Shard sizing • Indexing performance Issues
  26. 26. l "template" index l l number_of_shards l number_of_replicas l l type Index template { "template": "talk.request_log-*", "settings": { "number_of_shards": 32, "number_of_replicas": 1, }, "mappings": { "_default_": { "properties": { "method_name": { "type": "keyword", }, "ip": { "type": "ip", }, … …}
  27. 27. Template { "template": "talk.request_log-*", "settings": { "number_of_shards": 32, "number_of_replicas": 1, }, "mappings": { "_default_": { "properties": { "method_name": { "type": "keyword", }, "ip": { "type": "ip", }, … …} message RequestLog { string method_name = 1; string ip = 2 [(EsMapping.type)="ip"]; …} Protobuf elasticsearch: number_of_shards: 32 number_of_replication: 1
  28. 28. • • Index template • Shard sizing • Indexing performance Issues
  29. 29. l l resource overhead l l Indexing Shard sizing How many shards should I have in my Elasticsearch cluster? “How many shards should I have?” and “How large should my shards be?”
  30. 30. indexing 50GB/shard index rolling period Shard sizing : How many shards should I have in my Elasticsearch cluster?
  31. 31. à Rollover Index API size-base rolling? Shard sizing - : And the big one said "Rollover" — Managing Elasticsearch time-based indices efficiently
  32. 32. • • Index template • Shard sizing • Indexing performance Issues
  33. 33. 2.2x speed up 25 K docs/s 55 K docs/s ES Indexing
  34. 34. l Elasticsearch l Version: 5.6.1 l 40 CPU cores, 256 GB memory, 8 TB HDD x 12ea JBOD, 10Gbps network l disk write throughput: 200 MB/s l l auto-generated document id l processors l shard 1 à 2)
  35. 35. l document id l Index API: l document l auto-generated id document id l Index API: l 1.3x speed up auto-generated document id : [5.6]Tune for indexing speed#Use auto-generated ids, [5.6.1]InternalEngine#planIndexingAsPrimary()
  36. 36. l _nodes/os l os.available_processors: 40 l os.allocated_processors: 32 l processor l elastic/elasticsearch#20874 l processors: 38 bulk request l 1.3x speed up processors : [5.6]Thread Pool#Processors setting
  37. 37. indexing throttling Disk segment merge in-flight merge indexing throttling
  38. 38. l disk l 1.4x speed up l Elasticsearch l CPU usage: 50% ~ 80% l Disk write: 125 MB/s x 2 shard
  39. 39. l 2.2x speed up l auto document id l processors l CPU l l document mapping parameter l disk (SSD? RAID?) Indexing performance
  40. 40. • • Index template • Shard sizing • Indexing performance Issues
  41. 41. LINE Elasticsearch Petabyte
  42. 42. THANK YOU

×