Long journey of Ruby Standard library at RubyKaigi 2024
MySQL Proxy
1. MySQL Proxy
Making MySQL more flexible
Jan Kneschke
jan@mysql.com
2. MySQL Proxy
proxyservers forward requests to backends
●
and can
transform, handle or block them
–
released under the GPL
●
see http://forge.mysql.com/wiki/MySQL_Proxy
–
developed as part of the Enterprise Tools since
●
February 2007
3. Design Decisions
goal is to be transparent to the application layer
●
supports all platforms and languages
●
designed to handle thousands of parallel
●
connections (c10k)
uses a embedded scripting language for
●
customizations
4. Transparency
SHOW WARNINGS can be worked around with
●
Query Injection
SELECT USER() shows the connected user
●
(the proxy, not the client) which can be
corrected with resultset rewriting
host auth against the MySQL server
●
5. Latency
early tests via localhost
●
same script run directly and through the proxy
●
latency per mysqlpacket: 0.4ms
●
ping RTT on 1Gbit: 0.1ms
●
6. Load Balancing
load balancing distributes the load across
●
several slaves
Shortest Queue First is default
●
send new connections to the server with the least
–
number of open connections
7. Fail Over
dead host are detected
●
taking out of load balancing for 2min
●
uses custom load balancers to decide how to
●
handle a dead host
hot + standby
–
uses load balancing
–
8. Removing SPoF
one Proxy == Single Point of Failure
●
use external Heartbeat (linuxha.org) or
●
2 LB proxies + 1 Host Selecting Proxy per
●
application server
11. LUA
PiL http://lua.org/manual/5.1/
●
embedded, simple, efficient
●
can do OOlike programming
●
has scalars, tables, metatables and anonymous
●
functions
12. Query Rewriting
Macro Packages (ls, cd, who, ...)
●
tagging queries with SQL_CACHE
●
migrating tablenames and SQL dialects
●
turn EXPLAIN UPDATE|DELETE into
●
equivalent EXPLAIN SELECT
13. Query Profiling
SHOW SESSION STATUS around a Query
●
Exec_time: 6749 us
.. Handler_read_rnd_next = 252
.. Handler_write = 252
.. Select_scan = 1
14. Query Statistics
Normalize Queries to track query usage
●
Count Table and Index usage
●
Optimize Query Cache Usage by injecting
●
SQL_CACHE in cachable queries
see lib/analyzequeries.lua
●
15. Auditing
Diagnostic Auditing
●
track which user+ip run which query or
●
accessed which objects when
assign querycosts
●
log gathered information in a central place
●
see lib/auditing.lua
●
16. Global Transaction ID
Inject a counter in all transactions
●
Answers questions like
●
which slave is most current
–
can I read from this slave, or do I have to read from
–
master
you name it
–
17. Connection Pooling
reusing open connections between proxy and
●
server
reduces concurrency on the MySQL Server
●
external connection pool for PHP
●
18. Statement Routing
split the query stream into reading and writing
●
READs go to the slaves
–
WRITEs and transactions to the master
–
automatic scaleout
●
sharding
●
19. Tokenizer
turns a SQL query into a token stream
●
not a full parser, just a tokenizer for speed
●
reasons
understands KEYWORDS, /*comments*/,
●
“strings”, 123 and `literals`
later we'll add support for SQL modes
●
22. Internals – LUA scripting
proxy.* is the namespace
●
proxy.connection.* is the current
●
connection
proxy.backends[...] are the backends
●
proxy.global.* is the global table
●
proxy.global.config.* is used for the
●
config
23. Internals Scope
Each connection has its own script scope
●
proxy.global.* to share data between
●
connections
use local to make variables local to the
●
function
use package.seeall() to export functions
●
from modules
24. Internals Threading
the global scope and threading don't play nice
●
by default
http://www.cs.princeton.edu/~diego/professional/lua
●
patches lua to apply mutexes around variable
●
access
25. Internals – Script Cache
0.6.0 we reload the script on each connection
●
start
adding a script cache with mtime check
●
lua_pushvalue(L, 1) does the trick
●
26. Roadmap
to be released 0.6.0
●
tokenizer
–
readwrite splitting
–
Query Statistics
–
later
●
parallel Queries
–
proxy initiates connections
–
27. LUA Gotchas
only false and nil are !true, 0 is true
●
to say “not equal” you use ~=
●
there are no shortcuts
●
no a++, no a *= 4, ...
–
no a > b ? a : b (there is “(a > b) and a or b)
–