Introduction to InfluxDB and PHP Symfony2 and how to monitor your application workflow

1. Measure your app internals with InfluxDB
(time-series database)
Symfony Day 2015 - Reggio Emilia

2. Trust me

3. Why we measure things?

4. Measure - Learn - Build - Measure

5. Measure for a change

6. Measure for discovery

7. Measure for compare

8. What we need?
* A data provider (our application)
* A database that will collects data
* A dashboard for representing information

9. What is a time-series database?
A database optimized for handling time series data:
arrays of numbers indexed by time
(a datetime or a datetime range)
sign elem is 4 -> @10:13 elem is 4
A time series is a sequence of data points, measured typically at
successive points in time spaced at uniform time intervals

10. Time series Databases
InfluxDB
github.com/influxdb

11. InfluxDB
Easy to use
Written in Golang (no external dependencies)
ready for bilion of rows (scalable)
UDP/IP and HTTP network adapters
Several client libraries (PHP, Golang, etc...)
Integrated query web panel
Ready for external dashboards (grafana, chronograf, etc...)

12. Write information in your InfluxDB
curl ­XPOST 'http://localhost:8086/write?db=mydb'
­d 'cpu load=2 1434055562000000000'

13. Read data back from InfluxDB
curl ­G http://localhost:8086/query?pretty=true ­­data­urlencode "db=mydb"
­­data­urlencode "q=SELECT * FROM cpu WHERE host='server01' AND time < now() ­ 1d"

14. For example:
InfluxDB tags
You can mark new points with a tag information
sign @10.32 key=value tag1=value tag2=value
sign @10.32 temperature=21.3 location=home env=test
sign @10.32 cpu=21.3,mem=123.3 location=eu­west­1 env=dev instance=i­a91afcd1

15. Use tags in your queries
Example
SELECT * FROM home_sensors WHERE
location="home" AND
temperature > 19.2 AND
time >= now() ­ 10m
Example for websites:
SELECT * FROM index_page WHERE
env="prod" AND
load_time > 19.2 AND
time >= now() ­ 1d
Just use them like string column values

16. Play from the InfluxDB web console
http://your-domain.tld:8083/

17. GRAFANA DASHBOARD
An open source, feature rich, metrics dashboard and graph editor
for several time series databases (InfluxDB, Graphite, etc.)

18. Annotate your graphs
Thanks to annotations you can use a serie
as an annotation list
Mark a new software release in your graph (Continuous Delivery)
Mark any critic event in all your graphs (Information Point)

19. CHRONOGRAF DASHBOARD
A new data visualization tool for InfluxDB.

20. How to send my application data?
* Send every single point directly in real-time
* Collect data during the application life-cycle and send those
points once per request

21. InfluxDB supports two network adapters
HTTP or UDP/IP
Which one we should have to use?
UDP/IP must be enabled by configuration

22. HTTP
* TCP/IP based
* Data point delivery guaranteed
* Error handling
* ~ 374 Ops/second (for compare with UDP/IP)

23. UDP/IP
* No delivery guarantees
* No error handling
* ~ 22917 Ops/second (for compare with TCP/IP)

24. HTTP
* Sensitive data
* Good for once at a time strategy
(or if we do not care about monitor impact on our application)
UDP/IP
* Good for immediate delivery

25. InfluxDB-PHP-SDK
https://github.com/corley/influxdb-php-sdk
Supports both: HTTP and UDP/IP
Star 46 Fork 13
composer require corley/influxdb-sdk

26. How it works
$client­>mark("serie_name", [
"customer_name" => "walter",
"item_price" => 100.12,
//...
]);
Concise Format (good for immediate delivery)

27. How it works (2)
$client­>mark([
"points" => [
[
"measurement" => "instance",
"fields" => [
"cpu" => 18.12,
"free" => 712423,
],
],
[
"measurement" => "page_load",
"fields" => [
"listeners_speed" => 1.12,
"counted_elements" => 341143,
],
],
]
]);
Extended Format (good for a bulk strategy)

28. InfluxDB Client
UDP/IP + HTTP
$httpOptions = new Options();
$reader = new HttpReader($httpOptions);
$udpOptions = new Options();
$writer = new UdpWriter($udpOptions);
$client = new Client($reader, $writer);
HTTP only
$http = new GuzzleHttpClient();
$options = new Options();
$reader = new HttpReader($http, $options);
$writer = new HttpWriter($http, $options);
$client = new Client($reader, $writer);
Supports Guzzle versions: ~4, ~5, ~6

29. Symfony: Dependency Injection Container
parameters:
influxdb_host: "localhost"
influxdb_port: 4444
services:
influxdb:
class: InfluxDBClient
arguments: ["@influxdb_reader", "@influxdb_writer"]
influxdb_writer:
class: InfluxDBAdapterWriterUdp
arguments: ["@influxdb_udp_options"]
influxdb_reader:
class: InfluxDBAdapterHttpReader
arguments: ["@influxdb_http_options"]

30. Symfony: Dependency Injection Container
#Only options
influxdb_udp_options:
class: InfluxDBOptions
calls:
­ ["setHost", ["%influxdb_host%"]] # InfluxDB host
­ ["setPort", ["%influxdb_udp_port%"]] # UDP/IP Port
­ ["setTags", [{"env": "%kernel.environment%"}]] # tag every point with env
influxdb_http_options:
class: InfluxDBOptions
calls:
­ ["setHost", ["%influxdb_host%"]] # InfluxDB host
­ ["setPort", ["%influxdb_http_port%"]] # HTTP Port
parameters:
influxdb_host: "influxdb.domain.tld"
influxdb_udp_port: 4444
influxdb_http_port: 8086

31. Don't try this at home
class DefaultController extends Controller
{
public function indexAction(Request $request)
{
$client = $this­>container­>get("influxdb");
$client­>mark(...);
}
}
Goals: single-responsibility, reduce coupling, simplifing testing

32. Controllers as a service?
For a monitor? Make sense?

33. With events
class DefaultController extends Controller
{
public function indexAction(Request $request)
{
//...
$dispatcher­>dispatch("my_event_name", $event);
}
}
Event listener
class MyMonitorAwareForMyEventListener
{
public function __construct(Client $client) { ... }
public function onMyEvent(Event $event)
{
$this­>client­>mark(...);
}
}
Deregister this listener in order to stop data collection

34. Compose your listener
monitor_listener:
class: AppBundleListenerMonitorListener
arguments: ["@influxdb", "@debug.stopwatch"]
tags:
­ { name: kernel.event_listener, event: kernel.request, method: startStopwatch }
­ { name: kernel.event_listener, event: kernel.response, method: stopStopwatch }
The stopwatch will help us to collect page timing information

35. A listener example with stopwatch
class MonitorListener
{
public function startStopwatch(GetResponseEvent $event)
{
$routeName = $event­>getRequest()­>attributes­>get("_route");
$this­>stopwatch­>start($routeName);
}
public function stopStopwatch(FilterResponseEvent $event)
{
$routeName = $event­>getRequest()­>attributes­>get("_route");
$events = $this­>stopwatch­>stop($routeName);
foreach ($events­>getPeriods() as $measure) {
$this­>client­>mark($routeName, [
"memory" => $measure­>getMemory(), //...
]);
}
}
}

36. Check your data via InfluxDB

37. Prepare your dashboard with grafana

38. Checkout out the app example
https://github.com/wdalmut/symfonyday2015-example

39. Thanks
github.com/wdalmut