Recomendados
Más contenido relacionado
La actualidad más candente
La actualidad más candente (14)
Destacado
Destacado (20)
Similar a Poznań JUG: Listening to the sounds of your application
Similar a Poznań JUG: Listening to the sounds of your application (20)
Último
Último (20)
Poznań JUG: Listening to the sounds of your application
- 1. Listen to the sounds of your application
- 2. Maciej Biłas @maciejb softwaremill.com jbison.com !2
- 3. Presented at J-Day Lbn.sc Originally authored by: Maciej Biłas Krzysztof Ciesielski
- 4. Agenda • Monitoring, huh? • Introducing Graphite • Log analysis – the whys • Logstash architecture & use cases • Exploring logs with Kibana
- 15. Graphite
- 16. echo "local.random.diceroll 4 `date +%s`" | nc graphite-server.your.org 2003
- 17. successful.login.attempt 1 1384471287 successful.login.attempt 1 1384471297 successful.login.attempt = 1
- 18. Yammer metrics private final Meter successfulLogins = metrics.meter(name(LoginHandler.class, "successful")); ! public void login(String user, String password) { if (canLogin(user,password)) { successfulLogins.mark(); // ... } else { // ... } }
- 21. Aggregation app00.webservice.requestTime app01.webservice.requestTime app02.webservice.requestTime …
- 23. You can also check out • collectd https://collectd.org • StatsD https://github.com/etsy/statsd/ • Riemman http://riemann.io/ • Twitter’s Ostrich https://github.com/twitter/ostrich • Ganglia http://ganglia.sourceforge.net/ • Dashboards: http://shopify.github.io/dashing/ https://github.com/obfuscurity/descartes https://github.com/obfuscurity/dusk
- 24. Log aggregation?
- 25. Event Sourcing Tracing and storing all the events Current app state: replaying the stream Great for data mining and analysis
- 26. Log aggregation • Less invasive way to build an “event stream” • Logs are data with plenty of value
- 28. Logstash • Open source • Written in JRuby • Gathers logs from various inputs • Parses and extracts metadata • Writes to various outputs
- 29. Architecture
- 31. Grok Thin layer on top of a regular expression 2013-11-10 19:41:25.321 [main] INFO o.a.camel.impl.DefaultCamelContext - Apache Camel 2.11.1 (CamelContext: camel-1) started in 4.450 seconds time 2013-11-10 19:41:25.321 thread main loglevel INFO source o.a.camel.impl.DefaultCamelContext message Apache Camel 2.11.1 (CamelContext: camel-1) started in 4.450 seconds
- 32. Grok 2013-11-10 19:41:25.321 [main] INFO o.a.camel.impl.DefaultCamelContext - Apache Camel 2.11.1 (CamelContext: camel-1) started in 4.450 seconds grok { patterns_dir => “./some-dir“ match => ["message", "%{TIMESTAMP_ISO8601:time} [%{DATA:thread}] %{LOGLEVEL:loglevel} %{DATA:source} - % {MULTILINE_GREEDYDATA:message}"] }
- 34. Kibana
- 37. Regular events
- 38. How often is a feature used?
- 39. Plotting multiple event types
- 41. References • Graphite http://graphite.readthedocs.org/en/latest/ • Yammer Metrics http://metrics.codahale.com/ • Logstash http://logstash.net/ • Surfing the event stream by Sam Newman at Geecon http://www.slideshare.net/spnewman/surfing-the-event-stream • Lessons from Building and Scaling LinkedIn by Jay Kreps http://www.infoq.com/presentations/linkedin-architecture-stack • Code as Craft http://codeascraft.com/