Recomendados
Recomendados
Más contenido relacionado
La actualidad más candente
La actualidad más candente (20)
Similar a Built-in query caching for all PHP MySQL extensions/APIs
Similar a Built-in query caching for all PHP MySQL extensions/APIs (20)
Más de Ulf Wendel
Último
Último (20)
Built-in query caching for all PHP MySQL extensions/APIs
- 1. MySQL native driver for PHP: The query cache plugin
- 2. mysqlnd_qc: The query cache plugin Ulf Wendel, Andrey Hristov MySQL Connectors Team Sun Microsystems
- 21. Chart compares handler – no more
- 22. f Basic Usage $mysqli = new mysqli($host, $user, $passwd, $db, $port, $socket);; // Cached: SQL hint used $res = $mysqli->query("/*qc=on*/" . "SELECT id, label FROM test"); var_dump($res->fetch_all(MYSQLI_ASSOC)); $res->free(); SQL hint /*qc=on*/ to control caching // Uncached: no SQL hint $res = $mysqli->query("SELECT id, label FROM test"); var_dump($res->fetch_all(MYSQLI_ASSOC)); $res->free();
- 24. ./configure --help | grep mysql
- 25. ./configure --help | grep qc
- 26. make clean && make
- 28. Shorter distance to the app (round trip times)
- 29. Scale by client
- 31. Less sophisticated invalidation (stale data!)
- 33. Often used for storing cached data
- 35. caching of raw data instead of processed data
- 40. Configuration
- 41. How PHP connects to MySQL PHP MySQL Server Library: implements MySQL Client-Server Protocol PHP API for PHP applications MySQL native driver for PHP / MySQL Client Library
- 42. Inside PHP (on the C level!) PHP Extensions Zend Engine PDO ext/mysql ext/mysqli SAPI PDO_MYSQL PDO_XYZ MySQL Client Library (libmysql) or MySQL native driver for PHP (default as of PHP 5.3)
- 43. PHP 5.3.2: mysqlnd plugin interface <?php /* Any PHP MySQL application */ ?> ext/*mysql* ext/*mysql* Plugin MySQL native driver for PHP C plugins operate transparently: no user API changes
- 44. “Object-orientation” in mysqlnd <?php mysqli_query($link, … ); ?> Inside mysqlnd: “objects” of data and function pointer mysqlnd_connection connection_methods host user password ... connection_methods query() store_result() use_result() free_result() ...
- 45. Plugins as “Proxies” or “Decorators” <?php mysqli_query($link, … ); ?> Plugins replace or extend internal mysqlnd objects connection_methods query() store_result() use_result() free_result() ... Plugin query() store_result() use_result() free_result() ... mysqlnd_connection connection_methods host user password ...
- 47. metadata access
- 52. via result set object
- 54. Query cache: proxy-style plugin Cache MySQL <?php /* Any PHP MySQL application */ ?> ext/*mysql* ext/*mysql* mysqlnd mysqlnd Cache miss: record wire data, cache hit: replay Query Cache Plugin
- 55. Query Cache: Miss mysqlnd orig. mysqlnd_conn object mysqlnd_qc object query() Should cache? Yes! Is cached? No! Activate data recorder query() Send query to MySQL Deactivate recorder Cache wire data Decode data Fetch reply
- 56. Query Cache: Hit mysqlnd mysqlnd_qc object query() Should cache? Yes! Is cached? Yes! Fetch data from cache Decode data
- 58. Location: distance to cache
- 64. User can flush cache
- 66. Location: local
- 67. User can flush cache
- 68. Handler maintains storage statistics
- 70. Location: can be local or remote
- 71. User cannot flush cache
- 73. Internal and not exposed to PHP land
- 75. Scope, if using file: single (multiple) machine
- 76. Location: can be local or remote
- 77. User can flush cache
- 80. Can be specialized by user
- 85. Choice of distance to client
- 86. Cache Miss Client 2...100 Client 2...100 Client 2...100 BTW, plan your cache carefully! What if a shared cache entry expires? Client 1 Client 2...n Cache Hit MySQL Client 2...100 Client 2...100 Client 2...100 Client 1 Client 2...n MySQL
- 87. Cache Miss Optimize storage for reuse? Memory consumption versus peak loads Client 1 Client 2 MySQL Client 2...100 Client 2...100 Client 2...100 Client 1 Client 2...n MySQL Cache Miss Cache Hit Client 3..n
- 92. Configuration
- 94. Control which query gets cached
- 95. Control where to store cached data
- 96. Implement your own invalidation strategy
- 97. Maintain your own statistics
- 100. customize build-in default handler
- 102. you must implement all storage handler functions
- 103. Procedural user storage handler void mysqlnd_qc_set_user_handlers( string get_hash_key, string find_query_in_cache, string return_to_cache, string add_query_to_cache_if_not_exists, string query_is_select, string update_cache_stats, string clear_cache ) Yes, this API may be ugly. BUT: Prototype! BUT: See extra presentation for vodoo! mysqlnd_qc_set_user_handlers()
- 104. User storage handler interface (I) function get_hash($host_info, $port, $user, $db, $query) { /* returns string */ return $key; } function find($key) { /* returns boolean */ return $found; } function return_to_cache($key) { /* void – leave empty */ } function add($key, $data, $ttl, $runtime, $store_time, $row_count) { /* returns boolean */ return $added_to_cache; }
- 105. User storage handler interface (II) function query_is_select($query) { /* returns mixed - boolean false if the query shall not be cached boolean true or double 0 to use mysqlnd_qc.ttl default double >= 0 to set TTL different from mysqlnd_qc.ttl */ return $to_cache_or_not; } function update_stats($key, $run_time, $store_time) { /* void – data to update your per query cache statistics *// } function clear_cache() { /* returns boolean */ return $cache_has_been_flushed; }
- 108. See also extra presentation!
- 117. Changing the storage handler bool mysqlnd_qc_change_handler(string handler) Changes the storage handler. Returns false if the current handler cannot be shutdown or the requested handler cannot be initialized. Failing to change the handler should be considered as a fatal error unless the change fails because the requested handler is unknown.
- 118. Procedural user storage handler void mysqlnd_qc_set_user_handlers( string get_hash_key, string find_query_in_cache, string return_to_cache, string add_query_to_cache_if_not_exists, string query_is_select, string update_cache_stats, string clear_cache ) Sets the function names of a user defined storage handler and puts them into use. See also extra presentation!
- 119. Available handlers array mysqlnd_qc_get_handlers() Returns a list of available handler and their versions. The handler “default”, “user”, the class “ myslqnd_qc_handler_default” are always available. “ apc” and “memcache” will be reported if support for those handlers has been enabled at compile time.
- 120. Flushing the cache The function is not supported by the Memcache handler! bool mysqlnd_qc_clear_cache() Returns true if the handler supports the operation, and has flushed the cache.
- 121. Cache info and handler statistics // Run some queries and generate cache hits $res = $mysqli->query("/*qc=1*/" . "SELECT id, label FROM test"); var_dump($res->fetch_all(MYSQLI_ASSOC)); C-based handler cache information and statistics var_dump(mysqlnd_qc_get_cache_info()); array(4) { ["num_entries"]=> int(%d) ["handler"]=> string(7) "default" ["handler_version"]=> string(5) "1.0.0" ["data"]=> array(%d) { ["%s”] => array(2) { [“statistics”] => array(...), [“metadata”] => array(...) } [, ...] }
- 124. [“data”][key][“statistics”] : run/store time statistics
- 125. [“data”][key][“statistics”] : cache hits
- 127. Core statistics php.ini setting: mysqlnd_qc_collect_statistics = 1 array mysqlnd_qc_get_core_statistics() Returns a list 20+ statistics collected by the core of the query cache plugin, if the PHP configuration setting mysqlnd_qc_collect_statistics is set to 1. The statistics are provided by the core and therefore available with all storage handlers and when using user-defined storage handlers. The statistics cover cache accesses, failures, network traffic as well as aggregated run and store times.
- 128. Query back trace php.ini setting: mysqlnd_qc.query_trace = 1 array mysqlnd_qc_get_query_trace_log() Returns a query back trace for every query that has been inspected by the query cache regardless if the query ended up being cached or not. The trace tells you where a query has been issues (see also debug_backtrace()). Together with the back trace you get run and store times and information on if the query has been cached.
- 129. Normalized query back trace mysqlnd_qc.collect_normalized_query_trace = 1 array mysqlnd_qc_get_normalized_query_trace_log() Similar to mysqlnd_qc_get_query_trace_log() but with SQL statements normalized and aggregated by the normalized query string. Normalization refers to replacing actual parameters, for example in “WHERE a > 1”, with questionmarks like “ WHERE a > ?”. “WHERE a > ?” will match any value for “?”, for example “1”, “2”, “'abc'” but not other identifiers.
- 137. EXPERIMENTAL – use default of “0”
- 141. APC handler stores data in the APC user cache
- 142. Users can manipulate the cache entries!
- 146. You should use SQLite as an in-memory storage
- 150. All constants can be changed as compile time!
- 151. Exported PHP classes class mysqlnd_qc_handler_default { public function init() {} public function is_select(...) {} public function get_hash_key(...) {} public function return_to_cache(...) {} public function add_to_cache(...) {} public function find_in_cache(...) {} public function update_cache_stats(...) {} public function get_stats(...) {} public function clear_cache() {} public function shutdown() {} }
- 152. Exported PHP interfaces interface mysqlnd_qc_handler { public function is_select(...) {} public function get_hash_key(...) {} public function return_to_cache(...) {} public function add_to_cache(...) {} public function find_in_cache(...) {} public function update_cache_stats(...) {} public function get_stats(...) {} public function clear_cache() {} }
- 154. this is what you look at ;-)
- 155. Query cache plugin benchmark impressions
- 156. Dig deeper with QC statistics
- 157. Developing user storage handler Further reading
- 158. The End Feedback: ulf.wendel@sun.com The End Feedback: [email_address] , [email_address]