The document discusses using the ELK stack for load testing and monitoring. It provides an overview of the ELK stack components - Elasticsearch, Logstash, Kibana. It describes how to configure Filebeat and Logstash for log collection and parsing. It also discusses using Kibana for visualizing and analyzing load testing results stored in Elasticsearch. Alternative solutions like the jmeter-elasticsearch-backend-listener plugin are presented. Examples of Kibana visualizations and dashboards for metrics like response times, errors are shown. Contact information is provided to find out more about using the ELK stack for performance monitoring.
General Principles of Intellectual Property: Concepts of Intellectual Proper...
QA Fest 2018. Александр Венгер. ELK stack. Применение в нагрузочном тестировании и не только
1. ELK Stack. Применение в нагрузочном
тестирование и не только
t WITH PASSION TO QUALITY
Александр Венгер
QA CONFERENCE #1 IN UKRAINE, KYIV 2018
2. Aleksandr Venger
Ciklum, TCoE Performance Unit
- Performance testing of the frontend/backend parts of client-server
applications.
Winner of several TestUaStartups testathones in IOS Team nomination,
Winner and finalist of several Dev Challenges in QA category.
t WITH PASSION TO QUALITY
KYIV 2018
3. Тема доклада
Тема доклада
Тема доклада
WITH PASSION TO QUALITY
KYIV 2018
What is ELK Stack
QA CONFERENCE #1 IN UKRAINE KYIV 2018
4. Тема доклада
Тема доклада
Тема доклада
WITH PASSION TO QUALITY
KYIV 2018
What is ELK Stack
QA CONFERENCE #1 IN UKRAINE KYIV 2018
5. Тема доклада
Тема доклада
Тема доклада
WITH PASSION TO QUALITY
KYIV 2018
What is ELK Stack
QA CONFERENCE #1 IN UKRAINE KYIV 2018
6. Тема доклада
Тема доклада
Тема доклада
WITH PASSION TO QUALITY
KYIV 2018
What is ELK Stack
QA CONFERENCE #1 IN UKRAINE KYIV 2018
Log file (example) -> Kibana visualization (example)
7. Тема доклада
Тема доклада
Тема доклада
WITH PASSION TO QUALITY
KYIV 2018
What is ELK Stack
QA CONFERENCE #1 IN UKRAINE KYIV 2018
[Discover tab screenshot]
8. Тема доклада
Тема доклада
Тема доклада
WITH PASSION TO QUALITY
KYIV 2018
What is ELK Stack
QA CONFERENCE #1 IN UKRAINE KYIV 2018
[Dashboard screenshot]
9. Тема доклада
Тема доклада
Тема доклада
WITH PASSION TO QUALITY
KYIV 2018
FileBeat tune-up
QA CONFERENCE #1 IN UKRAINE KYIV 2018
• Input
• Output
10. Тема доклада
Тема доклада
Тема доклада
WITH PASSION TO QUALITY
KYIV 2018
FileBeat tune-up
QA CONFERENCE #1 IN UKRAINE KYIV 2018
#=========================== Filebeat inputs =============================
filebeat.inputs:
- type: log
# Change to true to enable this input configuration.
enabled: true
# Paths that should be crawled and fetched. Glob based paths.
paths:
- ..logs*
#- c:programdataelasticsearchlogs*
#----------------------------- Logstash output --------------------------------
output.logstash:
# The Logstash hosts
hosts: ["localhost:5044"]
11. Тема доклада
Тема доклада
Тема доклада
WITH PASSION TO QUALITY
KYIV 2018
LogStash tune-up
QA CONFERENCE #1 IN UKRAINE KYIV 2018
• Input
• Filter
• Output
12. Тема доклада
Тема доклада
Тема доклада
WITH PASSION TO QUALITY
KYIV 2018
LogStash tune-up
QA CONFERENCE #1 IN UKRAINE KYIV 2018
• Input
• Filter
• Output
13. Тема доклада
Тема доклада
Тема доклада
WITH PASSION TO QUALITY
KYIV 2018
Getting started
QA CONFERENCE #1 IN UKRAINE KYIV 2018
1.Filebeat.inputs.paths
2.Logstash pipeline
3.jmeter.save.saveservice.autoflush=true
4.Observe results
14. Тема доклада
Тема доклада
Тема доклада
WITH PASSION TO QUALITY
KYIV 2018
Alternative solution
QA CONFERENCE #1 IN UKRAINE KYIV 2018
15. Тема доклада
Тема доклада
Тема доклада
WITH PASSION TO QUALITY
KYIV 2018
jmeter-elasticsearch-backend-listener
QA CONFERENCE #1 IN UKRAINE KYIV 2018
16. Тема доклада
Тема доклада
Тема доклада
WITH PASSION TO QUALITY
KYIV 2018
jmeter-elasticsearch-backend-listener
QA CONFERENCE #1 IN UKRAINE KYIV 2018
Features
• ElasticSearch low-level REST client
• Bulk requests
• Filters
• Verbose, semi-verbose, error only, and quiet mode
• Send custom metrics/values to your ElasticSearch
17. Тема доклада
Тема доклада
Тема доклада
WITH PASSION TO QUALITY
KYIV 2018
jmeter-elasticsearch-backend-listener
QA CONFERENCE #1 IN UKRAINE KYIV 2018
[[Video появления результатов из джиметра в кибане]]
18. Тема доклада
Тема доклада
Тема доклада
WITH PASSION TO QUALITY
KYIV 2018
Kibana tabs
QA CONFERENCE #1 IN UKRAINE KYIV 2018
• Discover
• Visualize
• Dashboard
19. Тема доклада
Тема доклада
Тема доклада
WITH PASSION TO QUALITY
KYIV 2018
Kibana visualizations examples [[video]]
QA CONFERENCE #1 IN UKRAINE KYIV 2018
• Number of users
• Response time
• Error count
20. Тема доклада
Тема доклада
Тема доклада
WITH PASSION TO QUALITY
KYIV 2018
Ready to go Data
QA CONFERENCE #1 IN UKRAINE KYIV 2018
• Apache
• Docker
• Kubernetes
• MySQL
• Netflow
• Nginx
• Redis
• System logs and metrics
21. Тема доклада
Тема доклада
Тема доклада
WITH PASSION TO QUALITY
KYIV 2018
Contacts
QA CONFERENCE #1 IN UKRAINE KYIV 2018
https://github.com/a-venger/Jmeter-ELK_Stack
https://www.facebook.com/venger.aa
https://www.linkedin.com/in/vengers/
Features
ElasticSearch low-level REST client
Using the low-level client makes the plugin compatible with any ElasticSearch version
Bulk requests
By making bulk requests, there are practically no impacts on the performance of the tests themselves.
Filters
Only send the samples you want by using Filters! Simply type them as follows in the appropriate field : filter1;filter2;filter3 or sampleLabel_must_contain_this.
Verbose, semi-verbose, error only, and quiet mode
debug : Send request/response information of all samplers (headers, body, etc.)
info : Sends all samplers to the ElasticSearch engine, but only sends the headers, body info for the failed samplers.
quiet : Only sends the response time, bytes, and other metrics
error : Only sends the failing samplers to the ElasticSearch engine (Along with their headers and body information).
Send custom metrics/values to your ElasticSearch! Simply add a new field in the backend listener's configuration. *** This functionnality is sadly limited only to variables loaded from a CSV with the jp@gc plugin, or variables iniated in the Test Plan element. This is due to the way JMeter and its backend listener work ***
Features
ElasticSearch low-level REST client
Using the low-level client makes the plugin compatible with any ElasticSearch version
Bulk requests
By making bulk requests, there are practically no impacts on the performance of the tests themselves.
Filters
Only send the samples you want by using Filters! Simply type them as follows in the appropriate field : filter1;filter2;filter3 or sampleLabel_must_contain_this.
Verbose, semi-verbose, error only, and quiet mode
debug : Send request/response information of all samplers (headers, body, etc.)
info : Sends all samplers to the ElasticSearch engine, but only sends the headers, body info for the failed samplers.
quiet : Only sends the response time, bytes, and other metrics
error : Only sends the failing samplers to the ElasticSearch engine (Along with their headers and body information).
Send custom metrics/values to your ElasticSearch! Simply add a new field in the backend listener's configuration. *** This functionnality is sadly limited only to variables loaded from a CSV with the jp@gc plugin, or variables iniated in the Test Plan element. This is due to the way JMeter and its backend listener work ***
Features
ElasticSearch low-level REST client
Using the low-level client makes the plugin compatible with any ElasticSearch version
Bulk requests
By making bulk requests, there are practically no impacts on the performance of the tests themselves.
Filters
Only send the samples you want by using Filters! Simply type them as follows in the appropriate field : filter1;filter2;filter3 or sampleLabel_must_contain_this.
Verbose, semi-verbose, error only, and quiet mode
debug : Send request/response information of all samplers (headers, body, etc.)
info : Sends all samplers to the ElasticSearch engine, but only sends the headers, body info for the failed samplers.
quiet : Only sends the response time, bytes, and other metrics
error : Only sends the failing samplers to the ElasticSearch engine (Along with their headers and body information).
Send custom metrics/values to your ElasticSearch! Simply add a new field in the backend listener's configuration. *** This functionnality is sadly limited only to variables loaded from a CSV with the jp@gc plugin, or variables iniated in the Test Plan element. This is due to the way JMeter and its backend listener work ***
Apache
Docker
Kubernetes
MySQL
Netflow
Nginx
Redis
System logs and metrics