3. ;
Introduction
Logstash
Kibana
Conclusion
Logging
ā¢ Recording of eventsRecording of events
ā¢ Voice of your systems and applicationsVoice of your systems and applications
ā¢ It tells you almost everythingIt tells you almost everything
ā¢ It is a source of knowledgeIt is a source of knowledge
Julien Pivotto Logstash and friends
5. ;
Introduction
Logstash
Kibana
Conclusion
Logging is useful
ā¢ Understanding outagesUnderstanding outages
ā¢ not only when itās wrongnot only when itās wrong
ā¢ you can extract metricsyou can extract metrics
ā¢ no logs means somethingno logs means something
ā¢ it tells you what, why, who, whenit tells you what, why, who, when
Julien Pivotto Logstash and friends
6. ;
Introduction
Logstash
Kibana
Conclusion
Logging in the wild
ā¢ SyslogSyslog
ā¢ |tee /var/log/myapp.log|tee /var/log/myapp.log
ā¢ Cron + MAILTO=Cron + MAILTO=
ā¢ &>/dev/null&>/dev/null
Julien Pivotto Logstash and friends
7. ;
Introduction
Logstash
Kibana
Conclusion
Logging in the past
ā¢ Logging to ļ¬les on each serverLogging to ļ¬les on each server
ā¢ Using syslog protocolUsing syslog protocol
ā¢ DecentralizedDecentralized
ā¢ Reading requires SSH accessReading requires SSH access
ā¢ Not developer friendlyNot developer friendly
Julien Pivotto Logstash and friends
8. ;
Introduction
Logstash
Kibana
Conclusion
The tools nowadays
ā¢ Jenkins, Icinga, Graphite, ForemanJenkins, Icinga, Graphite, Foreman
ā¢ Nice web interfacesNice web interfaces
ā¢ CentralizedCentralized
ā¢ Easy to useEasy to use
Julien Pivotto Logstash and friends
10. ;
Introduction
Logstash
Kibana
Conclusion
3 separate tools
ā¢ Elasticsearch, distributed search & analytics engineElasticsearch, distributed search & analytics engine
ā¢ Logstash, logs managementLogstash, logs management
ā¢ Kibana, very nice webui to ES and LogstashKibana, very nice webui to ES and Logstash
Julien Pivotto Logstash and friends
12. ;
Introduction
Logstash
Kibana
Conclusion
Missions
Inputs
Filters
Output
Shipping the logs
ā¢ Some applications can only write to ļ¬lesSome applications can only write to ļ¬les
ā¢ But you need them on the main logstash serverBut you need them on the main logstash server
ā¢ Logstash can act as a daemon to ship the logsLogstash can act as a daemon to ship the logs
ā¢ Destinations can be syslog, redis,. . .Destinations can be syslog, redis,. . .
ā¢ Then you can act on your logsThen you can act on your logs
Julien Pivotto Logstash and friends
13. ;
Introduction
Logstash
Kibana
Conclusion
Missions
Inputs
Filters
Output
Collecting the logs
ā¢ You can plug logstash to a lot of data sourcesYou can plug logstash to a lot of data sources
ā¢ It can be passive or activeIt can be passive or active
ā¢ Listening on a UDP port vs checking mailsListening on a UDP port vs checking mails
ā¢ All your logs are managed by one applicationAll your logs are managed by one application
ā¢ It creates ļ¬elds from the logsIt creates ļ¬elds from the logs
Julien Pivotto Logstash and friends
16. ;
Introduction
Logstash
Kibana
Conclusion
Missions
Inputs
Filters
Output
What is an "event"?
ā¢ Several ļ¬eldsSeveral ļ¬elds
ā¢ Several tagsSeveral tags
ā¢ A type (syslog message, irc message,. . . )A type (syslog message, irc message,. . . )
ā¢ A @message ļ¬eldA @message ļ¬eld
ā¢ A timestampA timestamp
Julien Pivotto Logstash and friends
19. ;
Introduction
Logstash
Kibana
Conclusion
Missions
Inputs
Filters
Output
UDP and TCP input
ā¢ Compatible with rsyslog protocolCompatible with rsyslog protocol
ā¢ Each syslog talks with logstash directlyEach syslog talks with logstash directly
ā¢ Allow you to use the syslog toolchains: logger, rsyslogAllow you to use the syslog toolchains: logger, rsyslog
ā¢ UDP is shoot and forgetUDP is shoot and forget
Julien Pivotto Logstash and friends
21. ;
Introduction
Logstash
Kibana
Conclusion
Missions
Inputs
Filters
Output
UDP and TCP input
Rsyslog conļ¬guration
*.* @logstash.example.com:5544
ā¢ IIn /etc/rsyslog.conf
ā¢ TThat line will forward all the logs to logstash
ā¢ LLogstash will make useful ļ¬elds out of it: priority, severity,
program. . .
Julien Pivotto Logstash and friends
22. ;
Introduction
Logstash
Kibana
Conclusion
Missions
Inputs
Filters
Output
File
ā¢ Enable you to use logstash with every applicationEnable you to use logstash with every application
ā¢ Useful to ship the logsUseful to ship the logs
ā¢ Acts as a tail -n 0 -FActs as a tail -n 0 -F
ā¢ It works even if you use logrotateIt works even if you use logrotate
Julien Pivotto Logstash and friends
33. ;
Introduction
Logstash
Kibana
Conclusion
Kibana
ā¢ Kibana is a web interface for Logstash/ESKibana is a web interface for Logstash/ES
ā¢ Kibana 1 was written in PHPKibana 1 was written in PHP
ā¢ Kibana 2 was written in RubyKibana 2 was written in Ruby
ā¢ Kibana 3 is written in AngularJSKibana 3 is written in AngularJS
Julien Pivotto Logstash and friends
34. ;
Introduction
Logstash
Kibana
Conclusion
Kibana 3
ā¢ Everything happens in the browserEverything happens in the browser
ā¢ The browser is connected to ElasticsearchThe browser is connected to Elasticsearch
ā¢ You can save dashboards into ESYou can save dashboards into ES
ā¢ You can write/template dashboards to ļ¬lesYou can write/template dashboards to ļ¬les
Julien Pivotto Logstash and friends
36. ;
Introduction
Logstash
Kibana
Conclusion
Kibana queries
Example of a kibana query
@fields.syslog_program:"httpd" AND @fields.http_host:"test.example.com" AND @fields.response:"404"
ā¢ LLucene query syntax
ā¢ SSimple and eļ¬ective
ā¢ PPoint & click web interface
Julien Pivotto Logstash and friends
41. ;
Introduction
Logstash
Kibana
Conclusion
Conclusion
ā¢ Logstash is a small daemonLogstash is a small daemon
ā¢ Simple to package & deploy (jar ļ¬le)Simple to package & deploy (jar ļ¬le)
ā¢ Scalable thanks to ElasticsearchScalable thanks to Elasticsearch
ā¢ Developer friendly thanks to KibanaDeveloper friendly thanks to Kibana
Julien Pivotto Logstash and friends