M|18 Writing Stored Procedures in the Real World

1. Hacking with MariaDB for
fun and not so much profit.
Piotr Jarmuż, Allegro.pl
Maria DB M|18 - Feb 26/27 2018, New York

2. 2
• Introduction about me and my company
• MariaDB stored procedures current state and history
• The "Wordament" game
• Game solution - Single thread
• Inherent parallelism in game and MariaDB events
• Game solution - Multiple threads
• Scaling out – Another node
• Game solution - Multiple threads on multiple nodes
• Can we wrap it all up in a nice thread API? - Not now, but wait! There is UDF
• Conclusions
Agenda

3. 3
• MariaDB, as any other RDBMS, has stored procedures
• in MySQL core since 5.0 beta in 2005 - loosely based on Sybase Transact SQL
• procedures, functions, triggers and events
• low level "C" UDFs – plugins
• resources – oficial documentation - https://mariadb.com/kb/en/library/stored-
procedures/
• tutorials - https://mariadb.com/kb/en/library/programming-customizing-mariadb/
MariaDB stored procedures current state and history

4. 4
• original from Microsoft
• popular as mobile app
• displays a board 4x4 with random letters
• goal is find as many words as long as possible
• 120 sec for solution
• 3 letter minimum length
• no reuse of board tiles in current run
• great for learning new words :)
Wordament game
CART

5. 5
• data structures
 current word being built - local to the session
 board representing state of the game - temporary table engine memory
 solution table for found words - temporary table engine memory
 dictionary for checking valid words - regular MariaDB table
• algorithm used – depth first search tree with dynamic decision prunning
• unicode support for many languages
$> (echo "start transaction;"; aspell -d ru dump master | aspell -l ru expand |
sed 's/ /n/g' | (sed -r 's/(.*)/U1/g’| sort | uniq -i | sed = |
sed 'N;s/n/t/'";s/'/''/g"|
sed -r "s/(.*)t(.*)/insert into words_ru values (1,'2');/g"; echo "commit;" )) |
mysql -D wordament
Solution - single thread

6. 6
• let’s play!
• mysql> call play (‘xtoe evrc aean ygas’);
Solution - single thread

7. 7
mysql> call play (‘xtoe evrc aean ygas’);
| CARNAGE |
| CAVERNS |
| CORNAGE |
| CRANAGE |
| EXTRAVAGANCE |
+-------------------------+
275 rows in set (1.40 sec)
Solution - single thread

8. 8
• up to 16 independent search trees can be run in parallel
• leveraging MariaDB event scheduler
• in Mysql core since 5.1
• complementary objects events
• DDL create, drop, alter
• schedule, one-time, recurring
• event states ENABLED, DISABLED
Inherent parallelism in game and MariaDB events

9. 9
• refactoring code a bit
• range root search for distributing load
• replicating game state
• expanding data structures - new table gsolution
• auxillary tables and support procedures
• it scales up!
Solution - multiple threads

10. 10
• another node set-up in multi-master replication
• event behavior in replicated environment
• another event state SLAVESIDE_DISABLED
• flip-flopping state - there can only be one
• transfering execution via replication
Scaling out – Another node

11. 11
• no refactoring this time - we are already parallel
• create all events on one node
• selectively enable events on 2 hosts - load distribution
• gsolution table converges results via bidirectional replication
• it scales out!
Solution - multiple threads, multiple nodes

12. 12
Solution - multiple threads, multiple nodes

13. 13
• there is a "thread" infrastructure in MariaDB - a bit awkward
• we would need a nice posix thread like API
• create_thread(), start_thread(), join_thread(), destroy_thread()
• sadly event management are DDL - no-go in stored procedures
• why not write our own udf_execute_immediate with DDL support
• "C" level UDF to the rescue
• back to the game - putting it all together with thread API
Can we wrap it all up in a nice thread API?

14. 14
• MariaDB is inherently parallel environment
• needs a little user support in parallelisation
• scaling up and out - events scheduler + replication and/or sharding
• new exciting features in future
Conclusions

15. Thank you for your attention.
Questions?