MariaDB features that both developers and operators will find benefit from, especially when we focus on the 10.0 release. This is specific to a Red Hat Summit/DevNation crowd.

2. MariaDB for DevOps
Colin Charles

3. MariaDB for DevOps
Colin Charles,Team MariaDB, SkySQL Ab
colin@mariadb.org | http://mariadb.org/
http://bytebot.net/blog/ | @bytebot on Twitter
DevNation/Red Hat Summit 2014, San Francisco,
California, USA - 15 April 2014

4. whoami
• Work on MariaDB at SkySQL Ab
• Merged with Monty Program Ab, makers of MariaDB
• Formerly MySQL AB (exit: Sun Microsystems)
• Past lives include Fedora Project (FESCO), OpenOffice.org

5. Who are you?
• Developer?
• Operator? (DBA, sysadmin)
• A bit of both?

6. First up… RHEL 6.5
mariadb55.x86_64 : Package that installs mariadb55
mariadb55-mariadb-bench.x86_64 : MariaDB benchmark scripts and data
mariadb55-mariadb-devel.x86_64 : Files for development of MariaDB plugins
mariadb55-mariadb-libs.x86_64 : The shared libraries required for MariaDB/MySQL
: clients
mariadb55-mariadb-server.x86_64 : The MariaDB server and related files
mariadb55-runtime.x86_64 : Package that handles mariadb55 Software Collection.
mariadb55-mariadb.x86_64 : A community developed branch of MySQL
mariadb55-mariadb-test.x86_64 : The test suite distributed with MariaD

7. RHEL 7…
• There’s no “community MySQL” package
• You upgrade to MariaDB 5.5.33a (http://ftp.redhat.com/redhat/rhel/
beta/7/x86_64/os/Packages/)
• OpenShift MariaDB Cartridge: https://github.com/openshift-
cartridges/mariadb-cartridge

8. 5W1H is MariaDB
• Drop-in compatible MySQL replacement
• Community developed, Foundation backed, feature enhanced,
backwards compatible, GPLv2 licensed
• Steady stream of releases in 4 years 2 months: 5.1, 5.2, 5.3, 5.5,
10.0, MariaDB Galera Cluster 5.5, MariaDB with TokuDB 5.5
• Enterprise features open: PAM authentication plugin, threadpool,
audit plugin

10. Microseconds
• TIME, DATETIME, TIMESTAMP, temporal functions, CAST, dynamic
columns
CREATE TABLE microsec(
column_microsec DATETIME(6),
column_millisec TIME(3)
);
SELECT CURTIME(6);
MariaDB 5.3+

11. Microseconds & 5.6
• TIME_TO_SEC(), UNIX_TIMESTAMP() preserve microseconds of the
argument
MariaDB 10.0 MySQL 5.6
SELECT TIME_TO_SEC('10:10:10.12345');
+-------------------------------+
| TIME_TO_SEC('10:10:10.12345') |
+-------------------------------+
| 36610.12345 |
+-------------------------------+
1 row in set (0.01 sec)
SELECT TIME_TO_SEC('10:10:10.12345');
+-------------------------------+
| TIME_TO_SEC('10:10:10.12345') |
+-------------------------------+
| 36610 |
+-------------------------------+
1 row in set (0.00 sec)

12. Virtual Columns
• A column in a table that has its value automatically calculated either
with a pre-calculated/deterministic expression or values of other
fields in the table
• VIRTUAL - computed on the fly when data is queried (like a VIEW)
• PERSISTENT - computed when data is inserted and stored in a table
MariaDB 5.2+

13. Virtual Columns
CREATE TABLE table1 (
a INT NOT NULL,
b VARCHAR(32),
c INT AS (a mod 10) VIRTUAL,
d VARCHAR(5) AS (left(b,5)) PERSISTENT);

14. Virtual columns example
CREATE TABLE product (
-> productname VARCHAR(25),
-> price_eur DOUBLE,
-> xrate DOUBLE,
-> price_usd DOUBLE AS (price_eur*xrate) VIRTUAL);
INSERT INTO product VALUES ('toothpaste', 1.5, 1.39, default);
INSERT into product VALUES ('shaving cream', 3.59, 1.39,
default);

15. Virtual columns example II
select * from product;
+---------------+-----------+-------+-------------------+
| productname | price_eur | xrate | price_usd |
+---------------+-----------+-------+-------------------+
| toothpaste | 1.5 | 1.39 | 2.085 |
| shaving cream | 3.59 | 1.39 | 4.990099999999999 |
+---------------+-----------+-------+-------------------+
2 rows in set (0.00 sec)

16. Virtual column use cases elsewhere
• http://openlife.cc/blogs/2010/october/what-would-you-use-virtual-columns
• http://openlife.cc/blogs/2010/october/mariadb-52-using-mariadb-column-store-and-
virtual-columns-indexing
• http://www.jonathanlevin.co.uk/2012/04/mariadbs-virtual-columns.html
• http://daniel-bartholomew.com/wordpress/2010/09/road-to-mariadb-5-2-virtual-columns/
• http://falseisnotnull.wordpress.com/2012/11/29/observations-about-mariadbs-virtual-
columns/
• https://mariadb.com/kb/en/virtual-columns/
• MariaDB Cookbook (2014) has a chapter dedicated to it

17. PCRE Regular Expressions
• Powerful REGEXP/RLIKE operator
• New operators:
• REGEXP_REPLACE(sub,pattern,replace)
• REGEXP_INSTR(sub,pattern)
• REGEXP_SUBSTR(sub,pattern)
• Works with multi-byte character sets that MariaDB supports, including
East-Asian sets
MariaDB 10.0+

18. REGEXP_REPLACE()
SELECT REGEXP_REPLACE('ab12cd','[0-9]','') AS remove_digits;
+---------------+
| remove_digits |
+---------------+
| abcd |
+---------------+
1 row in set (0.00 sec)

19. REGEXP_INSTR()
SELECT REGEXP_INSTR('abc','b');
+-------------------------+
| REGEXP_INSTR('abc','b') |
+-------------------------+
| 2 |
+-------------------------+
1 row in set (0.00 sec)

20. REGEXP_SUBSTR()
SELECT REGEXP_SUBSTR(
-> 'See https://mariadb.org/en/foundation/ for details',
-> 'https?://[^/]*');
+--------------------------------------------------------------------------------------------+
| REGEXP_SUBSTR(
'See https://mariadb.org/en/foundation/ for details',
'https?://[^/]*') |
+--------------------------------------------------------------------------------------------+
| https://mariadb.org |
+--------------------------------------------------------------------------------------------+
1 row in set (0.00 sec)

21. PCRE features
• Can have character classes, unicode character types, check script names
SELECT 'abc' RLIKE '^[[:ascii:]]+$';
+------------------------------+
| 'abc' RLIKE '^[[:ascii:]]+$' |
+------------------------------+
| 1 |
+------------------------------+
1 row in set (0.00 sec)

22. PCRE script names
SELECT 'ΣΦΩ' RLIKE '^p{Greek}+$';
+--------------------------------+
| 'ΣΦΩ' RLIKE '^p{Greek}+$' |
+--------------------------------+
| 1 |
+--------------------------------+
MariaDB [developers]> SELECT 'ΣΦΩ' RLIKE '^p{Old_Persian}+$';
+--------------------------------------+
| 'ΣΦΩ' RLIKE '^p{Old_Persian}+$' |
+--------------------------------------+
| 0 |
+--------------------------------------+

23. DELETE … RETURNING
• Delete operations that return a result set of the deleted rows to the
client
DELETE post FROM blog INNER JOIN post WHERE blog.id
= post.blog_id;
DELETE from t1 WHERE a=2 RETURNING *;
MariaDB 10.0+

24. GIS
• MariaDB implements a subset of SQL with Geometry Types
• No longer just minimum bounding rectangles (MBR) - shapes
considered
CREATE TABLE geom (g GEOMETRY NOT NULL, SPATIAL
INDEX(g)) ENGINE=MyISAM;
• ST_ prefix - as per OpenGIS requirements
MariaDB 5.3+

25. Sample use cases
• Import OpenStreetMap data into MariaDB: http://www.slideshare.net/
hholzgra/fosdem-2014mariadbgis
• Use the OpenStreetMap dataset: https://mariadb.com/kb/en/
openstreetmap-dataset/
• Screencast: https://blog.mariadb.org/screencast-mariadb-gis-demo/
• node.js example use case for mapping GPX data: https://
blog.mariadb.org/node-js-mariadb-and-gis/ & jQuery usage: https://
blog.mariadb.org/jquery-and-gis-distance-in-mariadb/

26. Dynamic columns
• Allows you to create virtual columns with dynamic content for each row in
table. Store different attributes for each item (like a web store).
• Basically a BLOB with handling functions: COLUMN_CREATE,
COLUMN_ADD, COLUMN_GET, COLUMN_DELETE, COLUMN_EXISTS,
COLUMN_LIST, COLUMN_CHECK, COLUMN_JSON
• In MariaDB 10.0: name support (instead of referring to columns by numbers,
name it), convert all dynamic column content to JSON array, interface with
Cassandra
INSERT INTO tbl SET
dyncol_blob=COLUMN_CREATE("column_name", "value");
MariaDB 5.3+

27. Full-text search via SphinxSE
mysql> INSTALL PLUGIN sphinx SONAME 'ha_sphinx.so';
Query OK, 0 rows affected (0.01 sec)
MariaDB 5.2+

28. What is SphinxSE?
• SphinxSE is just the storage engine that still depends on the Sphinx
daemon
• It doesn’t store any data itself
• Its just a built-in client to allow MariaDB to talk to Sphinx searchd,
run queries, obtain results
• Indexing, searching is performed on Sphinx

29. Sphinx search table
CREATE TABLE t1
(
id INTEGER UNSIGNED NOT NULL,
weight INTEGER NOT NULL,
query VARCHAR(3072) NOT NULL,
group_id INTEGER,
INDEX(query)
) ENGINE=SPHINX CONNECTION="sphinx://localhost:9312/test";
!
SELECT * FROM t1 WHERE query='test it;mode=any';

30. Sphinx search tables
• 1st column: INTEGER UNSIGNED or BIGINT (document ID)
• 2nd column: match weight
• 3rd column: VARCHAR or TEXT (your query)
• Query column needs indexing, no other column needs to be

31. Query Cassandra
• Data is mapped: rowkey, static columns, dynamic columns
• super columns aren’t supported
• No 1-1 direct map for data types
• Write to Cassandra from SQL (SELECT, INSERT, UPDATE, DELETE)
MariaDB 10.0+

32. Cassandra II
pk varchar(36) primary key,
data1 varchar(60),
data2 bigint
) engine=cassandra keyspace='ks1' column_family='cf1'
• Table must have a primary key
• name/type must match Cassandra’s rowkey
• Columns map to Cassandra’s static columns
• name must be same as in Cassandra, datatypes must match, can be subset of CF’s columns

33. Mapping
• Datatype mapping - complete table at KB
• Data mapping is safe - engine will refuse incorrect mappings
• Command mapping: INSERT overwrites rows, UPDATE reads then
writes, DELETE reads then writes

34. Typical use cases
• Web page hits collection, streaming data
• Sensor data
• Reads served with a lookup
• Want an auto-replicated, fault-tolerant table?

35. CONNECT
• Target: ETL for BI or analytics
• Import data from CSV, XML, ODBC, MS Access, etc.
• WHERE conditions pushed to ODBC source
• DROP TABLE just removes the stored definition, not data itself
• “Virtual” tables cannot be indexed
MariaDB 10.0+

36. Engines, etc
• Plan for backups - TokuDB can be cool for your uses as an example
• Galera: study your workload patterns, your application, etc.
• SPIDER (built-in sharding capabilities, partitioning & XA transaction
capable with multiple backends including Oracle)
• its not going to be straightforward to “just start” - need to know
right tables to implement, etc.

37. PAM Authentication
• Authentication using /etc/shadow
• Authentication using LDAP, SSH pass phrases, password expiration,
username mapping, logging every login attempt, etc.
• INSTALL PLUGIN pam SONAME ‘auth_pam.so’;
• CREATE USER foo@host IDENTIFIED via pam
• Remember to configure PAM (/etc/pam.d or /etc/pam.conf)
• http://www.mysqlperformanceblog.com/2013/02/24/using-two-factor-
authentication-with-percona-server/
MariaDB 5.2+

38. Threadpool
• Modified from 5.1 (libevent based), great for CPU bound
loads and short running queries
• Windows (threadpool), Linux (epoll), Solaris (event
ports), FreeBSD/OSX (kevents)
• No minimization of concurrent transactions with dynamic
pool size
• thread_handling=pool-of-threads
• https://mariadb.com/kb/en/thread-pool-in-mariadb-55/
MariaDB 5.5+

39. Non-blocking client library
• start operation, do work in thread, operation processed, result
travels back
• use cases: multiple queries against single server (utilize more
CPUs); queries against multiple servers (SHOW STATUS on many
machines)
• https://mariadb.com/kb/en/about-non-blocking-operation-in-the-
client-library/
• fast node.js driver available: mariasql
MariaDB 5.5+

40. LIMIT ROWS EXAMINED
• The purpose of this optimization is to provide the means to terminate
the execution of SELECTstatements which examine too many rows,
and thus use too many resources.
• SELECT * from t1, t2 LIMIT 10 ROWS EXAMINED 1000;
• https://mariadb.com/kb/en/limit-rows-examined/
MariaDB 5.5+

41. SQL Error Logging Plugin
• Log errors sent to clients in a log file that can be analysed later. Log
file can be rotated (recommended)
• a MYSQL_AUDIT_PLUGIN
install plugin SQL_ERROR_LOG soname 'sql_errlog.so';
MariaDB 5.5+

42. Audit Plugin
• Log server activity - who connects to the server, what queries run,
what tables touched - rotating log file or syslogd
• a MYSQL_AUDIT_PLUGIN
INSTALL PLUGIN server_audit SONAME
‘server_audit.so’;
MariaDB 10.0+

43. Replication made better
• Selective skipping of replication events (session-based or on master
or slave)
• Dynamic control of replication variables (no restarts!)
• Using row-based replication? Annotate the binary log with SQL
statements
• Slaves perform checksums on binary log events
MariaDB 5.3+

44. Replication made better II
• Group commit in the binary log - finally, sync_binlog=1,
innodb_flush_log_at_trx_commit=1 performs
• START TRANSACTION WITH CONSISTENT SNAPSHOT
• mysqldump —single-transaction —master-data - full non-
blocking backup
• Slaves crash-safe (data stored inside transaction tables)
• Multi-source replication - (real-time) analytics, shard provisioning,
backups, etc.

45. New KILL syntax
• HARD | SOFT & USER USERNAME are MariaDB-specific (5.3.2)
• KILL QUERY ID query_id (10.0.5) - kill by query id, rather than thread id
• SOFT ensures things that may leave a table in an inconsistent state
aren’t interrupted (like REPAIR or INDEX creation for MyISAM or Aria)
KILL [HARD | SOFT] [CONNECTION | QUERY] [thread_id |
USER user_name]
MariaDB 5.3+

46. Statistics
• Understand server activity better to understand database loads
• SET GLOBAL userstat=1;
• SHOW CLIENT_STATISTICS; SHOW USER_STATISTICS;
• # of connections, CPU usage, bytes received/sent, row statistics
• SHOW INDEX_STATISTICS; SHOW TABLE_STATISTICS;
• # rows read, changed, indexes
• INFORMATION_SCHEMA.PROCESSLIST has MEMORY_USAGE, EXAMINED_ROWS
(similar with SHOW STATUS output)
MariaDB 5.2+
MariaDB 10.0+

47. EXPLAIN enhanced
• Explain analyser: https://mariadb.org/explain_analyzer/analyze/
• SHOW EXPLAIN for <thread_id>
• EXPLAIN output in the slow query log
• EXPLAIN not just for SELECT but INSERT/UPDATE/DELETE
MariaDB 10.0+

48. Roles
• Bundles users together, with similar privileges - follows the SQL
standard
CREATE ROLE audit_bean_counters;
GRANT SELECT ON accounts.* to audit_bean_counters;
GRANT audit_bean_counters to ceo;
MariaDB 10.0+

49. Connectors
• The MariaDB project provides LGPL connectors (client libraries) for:
• C
• Java
• ODBC
• Embedding a connector? Makes sense to use these LGPL licensed
ones…

50. Optimizer
MariaDB 10 MySQL 5.6
index_merge=on
index_merge_union=on
index_merge_sort_union=on
index_merge_intersection=on
index_merge_sort_intersection=off
engine_condition_pushdown=off
index_condition_pushdown=on
derived_merge=on
derived_with_keys=on
firstmatch=on
loosescan=on
materialization=on
in_to_exists=on
semijoin=on
partial_match_rowid_merge=on
partial_match_table_scan=on
subquery_cache=on
mrr=off
mrr_cost_based=off
mrr_sort_keys=off
outer_join_with_cache=on
semijoin_with_cache=on
join_cache_incremental=on
join_cache_hashed=on
join_cache_bka=on
optimize_join_buffer_size=off
table_elimination=on
extended_keys=on
exists_to_in=off
index_merge=on
index_merge_union=on
index_merge_sort_union=on
index_merge_intersection=on
engine_condition_pushdown=on
index_condition_pushdown=on
mrr=on
mrr_cost_based=on
block_nested_loop=on
batched_key_access=off
materialization=on
semijoin=on
loosescan=on
firstmatch=on
subquery_materialization_cost_based=on
use_index_extensions=on
MariaDB 5.3+

51. https://mariadb.com/kb/en/

52. Q&A
colin@mariadb.org | byte@bytebot.net
http://skysql.com/ | http://mariadb.org/
twitter: @bytebot | url: http://bytebot.net/blog/