MoT presentation

1. Real Time Test Data
with Grafana
Speakers: Giannis Papadakis
Software QA Automation Manager
29th November 2018
1

2. Agenda
2
• Introduction to Grafana
• Introduction to Monitoring
• Test Data Visualization
• Demo
• Q&A

3. Goal
3
● Represent Test Data as Time Series Data
● Build Test Metrics
● Review Tests Runtime
● Alerting based on conditions

4. Real-time data monitoring (RTDM) is a process through which an
administrator can review, evaluate and modify the use of data on
software, a database or a system.
It enables data administrators to review the overall processes and
functions performed on the data in real time, or as it happens,
through graphical charts and bars on a central
interface/dashboard.
RTDM
4

5. Time Series
5
● A time series is a series of data points indexed in time
order
● Time series is a sequence taken at successive equally
spaced points in time
● Thus it is a sequence of discrete-time data
“Wikipedia”

6. Data Point
6
● A data point is a discrete unit of information.
● In a general sense, any single fact is a data point.
● In a statistical or analytical context, a data point is usually
derived from a measurement or research and can be
represented numerically and/or graphically

7. Introduction to Grafana
7
Observability

8. Introduction to Grafana
8

9. Introduction to Grafana
9
Grafana allows you to query, visualize, alert on and understand
your metrics no matter where they are stored and no matter
from where they came from

10. Introduction to Grafana
10
● Advanced graphing
● Powerful query editors
● Dashboards
● Dynamic queries and dashboards
● Multi tenant user / organization support
● Client side & server side rendering of panels

11. Introduction to Grafana
11

12. Application Logs can be time series data
Introduction to Grafana
12

13. Tests can wrap time series data
Introduction to Grafana
13

14. Not really relational data
More efficient at storing timeseries data
Better at querying timeseries data
Introduction to Grafana
14

15. Introduction to Grafana
15

16. InfluxDB Query Editor
16

17. Prometheus Query Editor
17

18. Alert Notifications
18

19. Alert Notifications
19
Supported Types of Alerts:
● Slack
● Email
● Kafka
● PagerDuty

20. Alert Notifications
20

21. Alert Notifications
21
Define Slack Settings and Test it on the fly!!!
● Slack URL
● Channel
● Bot Token

22. Alert Notifications
22

23. Dashboards
23
● Dashboards are composed of Panels arranged on a grid
● Grafana makes it easy to construct the right queries
● Create the perfect Dashboard for your need

24. ● A culture of being data-driven
● Whitebox monitoring
● Application metrics
● Test metrics
● Something you have to build into your system
Introduction to Monitoring
24

25. Monitor symptoms not causes
● Rate
● Errors
● Duration
Introduction to Monitoring
25

26. ● Running integration tests on a daily basis is
inevitable nowadays!
● The results are produced when CI completes
its execution
● It might be “annoying” in certain cases!!!
● It might be “cool” to have a real time data
monitoring platform to produce results runtime
● Also it could be “cool” to build metrics around
your test infrastructure
Test Data Visualization
26

27. Seeding Test Data
Test Data Visualization
27

28. Test Data Visualization
28

29. ● Measurement is the assignment of a number to
a characteristic of an object or event, which
can be compared with other objects or events
“Wikipedia”
Test Data Visualization
29

30. We can collect all the possible test metrics like
● Test case status
● Test case duration
● Test suite status
● Test suite duration
Test Data Visualization
30

31. We can derive some of the metrics
● Test cases count grouped by status on a
specific day
● Test suite duration trend for the week,
month, year etc
Test Data Visualization
31

32. InfluxDB is a time-series database
● sudo docker run -p 8086:8086 -v $PWD:/var/lib/influxdb
influxdb
● curl -i -XPOST http://localhost:8086/query
--data-urlencode "q=CREATE DATABASE selenium"
InfluxDB Setup
32

33. Test Data Visualization
33
Add InfluxDB Java Driver as Maven/Gradle dependency

34. InfluxDB Setup
34
Connect to your Datasource with InfluxDB Driver

35. Test Data Visualization
35
Test Class Metrics as points to InfluxDB
● Tag keys/values holds metadata adding info in measurements (String)
● Field keys contains information of what is measured (String)
● Field values can be of any type and actual holds the measured values

36. Test Data Visualization
36
Test Method Metrics as points to InfluxDB
● Description tag: Group results by description
● Name tag: Group

37. Test Data Visualization
37

38. Test Data Visualization
38
We made it to send results to InfluxDB which is great so far
How to query the results and visualize our test data!?
Grafana is our solution of course!!!

39. Test Data Visualization
39

40. Test Data Visualization
40

41. Test Data Visualization
41

42. Test Data Visualization
42
“Get your facts first,
then you can distort them as you please.”
- Mark Twain

43. ● https://grafana.com/
● https://www.influxdata.com/
● https://confengine.com/selenium-conf-2018/proposal/6086/enh
ancing-tests-with-smart-reports-and-useful-visualisation
● https://github.com/atypon/workshop/tree/mot
Resources
43

44. 44
Q & A

45. Thank you!