The presentation provides you with the essential tips to ensure that your database runs smoothly if you are pressed for time.
Percona provides an in-depth review of your database and recommends appropriate changes by performing a complete MySQL health check in which we identify inefficiencies, find problems before they occur, and ensure that your MySQL database is in the best condition.
5. WHAT CAN YOU DO IN 5
MINUTES?
The top tips to ensure your database is running smoothly
Company Confidential December 2010
-5-
6. BIG DISCLAIMER!
Company Confidential December 2010
-6-
• Its important to distinguish this up front:
– I am giving you the easy answers, which are not always
the correct answers..
– If you only have 5 minutes to spend on one of these
database topics, you lose a lot of flexibility
– I could spend lots of time talking about everything here,
instead I am focusing on maximizing your benefit.
• This presentation assumes you are running on
Linux/Unix
9. THOU SHALL HAVE BACKUPS!
Company Confidential December 2010
-9-
They can save your bacon!
10. • Probably the most common problem to get called
on is some sort of slowness
• Keys to success
– Identify what is slow
– When its slow
– Can you repeat it
• Dig in
The Slow Down Of Doom!
Company Confidential December 2010
-10-
11. No sense in targeting area’s that do not matter
Target your pain first
Company Confidential December 2010
-11-
12. • Google is your friend to research problems
• But don’t tune via Google
• Trust but verify
Know where to look
Company Confidential December 2010
-12-
13. • Know the tools out in the community
• Have them ready to jump in and fight fires at a
moments notice
• Tools like:
– Percona Toolkit
– Cacti
– Home grown scripts
– etc
Know thy tools
Company Confidential December 2010
-13-
14. • Define what is slow? If its a single page, the
process is drastically different then if its general
slowness.
• Assume General Slowness
– #1 What has changed? A new deployment? More users?
Etc? Sometimes a roll-back of a buggy code release is
the best fix.
• DO NOT ASSUME ITS THE DATABASE!
• Start with the obvious and the quick and move to
more complex
Tuning walk through
Company Confidential December 2010
-14-
15. • 5.1 -vs- 5.0 provides improvements
• 5.5 -vs- 5.1 provides improvements
• 5.6 –vs- 5.5 provides improvements
HUGE Differences in Versions!
Company Confidential December 2010
-15-
16. • Often stock OS repos are out of date!
• Watch out for old versions installed as
dependencies of off the shelf software
• Just because a hosting provider installed it does not
mean its the latest or best
Where did you get it?
Company Confidential December 2010
-16-
17. • Innodb
• Innodb Plugin
• XtraDB
• TokuDB
• MyISAM
• Aria
• Many Many more....
Storage Engines
Company Confidential December 2010
-17-
18. • Old Kernel?
• Multi-core servers, scalability getting better (~32 cores + )
• Fit your hot data into memory!
– Typically more memory is better
• IO is a Major bottleneck
– RAID For your databases is a must
– Battery Backed Controller
– RAID 10 typically better then RAID 5
– SSD's for optimal performance
• Network should not be forgotten
• 32bit binary or OS on 64 bit machine
Hardware & OS
Company Confidential December 2010
-18-
19. • Next I start by looking at the server (Sar, vmstat,
top, iostat -x)... I typically start low and work my
way into the database
– Too much io, maybe not enough memory, missing
indexes, or bad config
– Cpu maxed out could be lots of users, lots of concurrent
queries, or something else.
– Is the problem even MySQL? Often times we see issues
with non-database software
– top process is something else ( i.e. apache, php, cron
job, etc )
– Network is often overlooked
Review OS Metrics
Company Confidential December 2010
-19-
21. • Sar is your friend, every unix/linux server should
setup sar to collect stats in the background
– Sar Can give you historical reports on : CPU, Disk,
Memory, Network, and a lot more.
– Scheduled to collect data via cron
• Iostat is valuable in gauging disk performance
– avwait+svctim = response time (iostat -x)
Systat
Company Confidential December 2010
-21-
28. Percona Toolkit
Company Confidential December 2010
-28-
• Collection of scripts used for monitoring system and
DB performance
• pt-stalk
• pt-diskstats
• pt-pmp
• Pt-Ioprofiler
• pt-mext
29. Cacti
Company Confidential December 2010
-29-
• Great Visual Presentation of Historical Stats
• MySQL, Apache, Sphinx, Memcached, (Entire
Stack)
• Percona Maintains the MySQL Templates
• Can add a lot of value...
Example:
Mysterious Slowdown
30. OMG The Server is Slow!
Company Confidential December 2010
-30-
31. Dig a little deeper
Company Confidential December 2010
-31-
32. Oh Wait it Always Happens
Company Confidential December 2010
-32-
33. Percona Monitoring Plugins- Cacti!
Company Confidential December 2010
-33-
Grab them here:
http://www.percona.com/software/percona-monitoring-plugins/
36. Configuration Options – Top 10
Company Confidential December 2010
-36-
General Configuration Options
• query_cache_size (Disable in many versions)
• log_slow_queries
• long_query_time
• max_connections
• expire_logs_days
MyISAM Specific
• key_buffer_size
37. Configuration Options
Company Confidential December 2010
-37-
Innodb Specific Options
• innodb_buffer_pool_size
• innodb_flush_log_at_trx_commit
• innodb_log_file_size
Plugin/Xtradb/Mysql 5.5 Innodb Options
• innodb_io_capacity
• innodb_read_ahead
• innodb_write_io_threads
• innodb_read_io_threads
38. Online Tools to help you set your my.cnf!
Company Confidential December 2010
-38-
• https://tools.percona.com/wizard
39. Memory and Swap
Company Confidential December 2010
-39-
• Disk IO is bad, Avoiding swapping things that should be
in memory to disk ( do not disable swap however )
• SSD’s/Fast Disks
• Hot Data in Memory!
• read_buffer, read_rnd_buffer, sort_buffer, join_buffer
are all allocated per thread so be careful!
• You want temp tables to be built in memory not on disk
40. Check Current Activity
• Mysql
– “show full processlist”
– “show global status”
– “Show innodb status”
• Mysqladmin
• Slow Query Log
• pt-query-digest
• Mysqldumpslow
• Innotop
• Mytop
• pt-mext
Company Confidential December 2010
-40-
42. Mysqladmin
Company Confidential December 2010
-42-
• Mysqladmin extended -r -i1 will give you mysql stats at
1 second intervals, change the -i to increase
decrease...
• A few stats of note:
43. pt-mext
Company Confidential December 2010
-43-
• Mext can take the output from mysqladmin and put the
status variables side by side, this makes it very
convenient to look for differences
44. Slow Query Log
Company Confidential December 2010
-44-
• The Slow Query log is your friend! As mentioned earlier
queries that take over a certain amount of time are
logged here. (Percona Patches for complete picture )
• Summarize the slow query log with percona toolkit, pt-
query-digest or mysqldumpslow if the percona toolkit is
not available
• Look for not only the longest running query, but also the
one with lots of executions and cumulated time
47. Memcached/Redis/Other Caching
Company Confidential December 2010
-47-
• Review: More memory the better
• Caching frequently used data is an excellent way to
boost performance
• This typically requires changes to your app
• Performance improvement can be dramatic
48. Query Tuning
Company Confidential December 2010
-48-
• Review the slow query log, parse the log with pt-
query-digest.
– If I see something here, I will analyze it
• If your still at a loss, try using tcpdump to dump
SQL statements into pt-query-digest and look for
repeated queries. ( Easier then the general query
log if you need sub second slow queries).
49. Query Tuning
Company Confidential December 2010
-49-
• Use Explain Plan
• Add Indexes where needed, do not over index
– pt-duplicate-key-checker
• Lots of small queries can sometimes be more
impactful than 1 big one.