A short course I had few weeks ago that I wanted to share with you. All the MySQL issues from basics to experts: tuning, ERD, DDL, DML, Backup, Security
11. What to Install
wget mysql_package
yum –y localinstall mysql_package
The Packages:
Devel+Client+Server
Relevant platform (x86/x64)
Relevant operating system
http://top-performance.blogspot.com
26. Tables Manipulation
SHOW TABLES;
CREATE TABLE users (
id int,
INDEX users_id (id)
);
DROP TABLE users;
ALTER TABLE users ADD INDEX IX_id (id);
ALTER TABLE users DROP INDEX IX_id;
http://top-performance.blogspot.com
27. The VP R&D Open Seminar
SQL: DML
http://top-performance.blogspot.com
31. GROUP BY
SELECT gender, count(id)
FROM users
WHERE gender = ‘M’
GROUP BY gender
HAVING count(id) > 10
http://top-performance.blogspot.com
32. UNION
SELECT id
FROM users
WHERE gender = ‘M’
UNION ALL
SELECT id
FROM users
WHERE gender = ‘F’
ORDER BY id;
http://top-performance.blogspot.com
33. UPDATE
UPDATE users
SET gender = ‘M’
WHERE gender IS NULL;
http://top-performance.blogspot.com
34. INSERT
INSERT INTO users (gender)
VALUES (‘M’);
INSERT INTO users (gender)
SELECT gender
FROM users
WHERE gender = ‘M’;
http://top-performance.blogspot.com
35. DELETE
DELETE
FROM users
WHERE gender = ‘M’;
http://top-performance.blogspot.com
36. The VP R&D Open Seminar
MySQL Authentication and Authorization
SECURITY
http://top-performance.blogspot.com
37. Root User
First Time
mysqladmin -u root password NEWPASSWORD
Nth Time
mysqladmin -u root -p'oldpassword' password
newpass
How to enter
Mysql –uuser_name –ppassword –D
database_name
-h host_name/ip -A
http://top-performance.blogspot.com
38. Users Table
mysql> SELECT * FROM mysql.user limit 1G
*************************** 1. row ***************************
Host: localhost
User: root
Password: *4FB5C598BC3FF56E262F863A715F8CB2976A9C69
Select_priv: Y
Insert_priv: Y
Update_priv: Y
Delete_priv: Y
Create_priv: Y
Drop_priv: Y
….
1 row in set (0.00 sec)
http://top-performance.blogspot.com
39. Create a Users and Provide Permissions
-- Create a user w/ password
CREATE USER 'readonly'@'%' IDENTIFIED BY
‘sakexkas';
-- Grant Permissions
GRANT select ON my_db.* to 'readonly'@'%';
GRANT ALL PRIVILEGES ON *.* to
‘master_user’@’%’ WITH GRANT OPTION;
FLUSH PRIVILEGES;
-- Beware of too complex passwords
http://top-performance.blogspot.com
40. The VP R&D Open Seminar
MYSQL AND LINUX ISSUES
http://top-performance.blogspot.com
41. Built in Linux Security Measures
IP TABLES
sudo /etc/init.d/iptables stop
sudo yum –y remove iptables
Configure the relevant rules
SELinux
sudo echo 0 > /selinux/enforce
vi /etc/selinux/config: SELINUX=enforcing SELINUX=disabled
Files Limit (Table Partitioning)
/etc/security/limits.conf
mysql soft nofile 8192
mysql hard nofile 8192
my.cnf:
open_files_limit = 8192
http://top-performance.blogspot.com
47. Show Status (1)
mysql> show slave statusG
*************************** 1. row ***************************
Slave_IO_State: Waiting for master to send event
Master_Host: 10.12.12.167
Master_User: repl
Master_Port: 3306
Connect_Retry: 60
Master_Log_File: master.000094
Read_Master_Log_Pos: 167250867
Relay_Log_File: mysqld-relay-bin.000137
Relay_Log_Pos: 167251010
Relay_Master_Log_File: master.000094
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
Replicate_Do_DB:
Replicate_Ignore_DB:
Replicate_Do_Table:
Replicate_Ignore_Table:
Replicate_Wild_Do_Table:
Replicate_Wild_Ignore_Table: mysql.%,information_schema.%
http://top-performance.blogspot.com
48. Show Status (2)
Last_Errno: 0
Last_Error:
Skip_Counter: 0
Exec_Master_Log_Pos: 167250867
Relay_Log_Space: 167251310
Until_Condition: None
Until_Log_File:
Until_Log_Pos: 0
Master_SSL_Allowed: No
Master_SSL_CA_File:
Master_SSL_CA_Path:
Master_SSL_Cert:
Master_SSL_Cipher:
Master_SSL_Key:
Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
Last_IO_Errno: 0
Last_IO_Error:
Last_SQL_Errno: 0
Last_SQL_Error:
Replicate_Ignore_Server_Ids:
Master_Server_Id: 1
1 row in set (0.02 sec)
http://top-performance.blogspot.com
49. Skip
DON’T DO THAT unless you have to!
The Process:
STOP SLAVE;
SET GLOBAL SQL_SLAVE_SKIP_COUNTER =
1;
START SLAVE;
SHOW SLAVE STATUSG
http://top-performance.blogspot.com
50. How to Implement?
Stop a Master and record position
Copy a Master (instance/disk) to a Slave
Start the Master
Change slave server-id
SET MASTER HOST …
START SLAVE
Verify
Or…
http://top-performance.blogspot.co.il/2012/03/how-to-
setup-mysql-replication-in-11.html
http://top-performance.blogspot.com
61. When Should I Choose NoSQL?
Eventually Consistent
Document Store
Key Value
http://guyharrison.squarespace.com/blog/tag/nosql
61
http://top-performance.blogspot.com
64. Horizontal Sharding
Static Hashing
Complex growth Mod 10 = 0
Mod 10 = 1
Simple
Mod 10 = 2
Mod 10 = 3
Mod 10 = 4
Mod 10 = 5
Mod 10 = 6
Mod 10 = 7
Mod 10 = 8
Mod 10 = 9
64
http://top-performance.blogspot.com
65. Horizontal Sharding
Key locations are defined in a directory
Simple growth
Directory is SPOF
65
http://top-performance.blogspot.com
66. Horizontal Sharding
Static Hashing with Directory Mapping
Simple Growth
Small Directory still SPOF
Mod 1000 = 4
66
http://top-performance.blogspot.com
67. Horizontal Sharding
Each key signed by DB#
generated on creation
Simple growth
New key generation is SPOF
67
http://top-performance.blogspot.com
83. explain
mysql> explain SELECT COUNT(*), post_type FROM wp_posts GROUP
BY post_type;
+--+-------------+--------+-----+-------------+-----------+-------+----+----+-----------+
|id| select_type |table | type|possible_keys|key |key_len|ref |rows| Extra
+--+-------------+--------+-----+-------------+-----------+-------+----+----+-----------+
|1 | SIMPLE |wp_posts|index|NULL |type_status| 140 |NULL|693 | Using index
+--+-------------+--------+-----+-------------+-----------+-------+----+----+-----------+
1 row in set (0.02 sec)
http://top-performance.blogspot.com
85. RAM Usage Optimization
Preferred: SizeOf(database) < SizeOf(RAM)
innodb_buffer_pool_size
50-70% of your RAM
85
http://top-performance.blogspot.com
86. Do We Utilize the Cache?
SHOW STATUS LIKE 'Qcache%';
Qcache_free_blocks 718
Qcache_free_memory 13004008
Qcache_hits 780759
Qcache_inserts 56292
Qcache_lowmem_prunes 0
Qcache_not_cached 3711
Qcache_queries_in_cache 1715
Qcache_total_blocks 4344
86
http://top-performance.blogspot.com
92. What happens when mysql fails to start…
Create my.cnf
touch /etc/my.cnf
Create the error log
vi /etc/my.cnf
[mysqld]
log-error=/var/lib/mysql/log.err
If the my.cnf is already configured
more /etc/my.cnf | grep –i error
Restart mysql
Find the error
tail –n 50 /var/lib/mysql/log.err
In this case we kill all the mysql proceses
ps –aux | grep –i mysql
sudo kill process_id
http://top-performance.blogspot.com
93. The Bottom Line: Grow ∞
Thank you!
and Keep Performing!
Moshe Kaplan
93
http://top-performance.blogspot.com
Notas del editor
The Matrix, The Architect Speach http://www.youtube.com/watch?v=mod6xr-lyBc A better w/ titles: http://www.youtube.com/watch?v=K3OXs_5AD2A Second, Kong foo: http://www.youtube.com/watch?v=j82GKTgVDkw Twitter Source: http://www.slideshare.net/Blaine/scaling-twitter
The world is changing more and more fast You have to minimize NRE You must support unexpected demand You must provide top service (people now leaves cell operator after single incident, rather then 5, 10 years ago Firms a vulnerable – Citi worth 20Bill $ instead of 200 Bill a year ago Break and Mortar bookstores 15 years ago and Amazon…. Will it happen again to banks, insurance, real estate agencies… IS YOUR MARKET THE NEXT FOR PENETRATION – Finance? Real Estate?How to win a rival that is not existing yet? http://www.johnmwillis.com/ibm/cloud-computing-and-the-enterprise/ -
Sharded database – OLTP Little/No reporting OLAP – must be implemented for reporting Loads data from sharded DBs Custom mechanism Any commercial