Доклад Петра Зайцева на HighLoad++ 2014.

1. Architecture and Design of MySQL
Powered Applications
Peter Zaitsev
CEO, Percona
Highload++ 2014
Moscow, Russia
31 Oct 2014

2. About Percona
Open Source
Software for MySQL
Ecosystem
• Percona Server
• Percona XtraDB
Cluster (PXC)
• Percona Xtrabackup
• Percona Toolkit
Solutions to succeed
with MySQL
• Consulting
• Support
• Managed Services
2

3. About Presentation
What's in your
Toolbox ?
What Questions
you should ask ?
Common MySQL
Architecture
Patterns
3

4. LiveJournal Architecture 10 Years Ago
4

5. 10 Years ago we see
Load Balancers
FrontEnd vs BackEnd Web Server Separation
MemcacheD (Caching)
Global Database for Shared Data
Read Write Splitting
Sharding “Database Clusters”
5

6. History Lesson
MySQL is quite Mature
Battle tested Architecture Patterns
Most Problems have proven solutions
6

7. Well… certain things have changed
YOUR TOOLBOX NOW
7

8. Very Powerful Hardware
• 16GB to 512GB
Lots of
Memory
• 1000 IOPS to 100.000 IOPS
• 10ms latency to 0.1ms IO Latency
Very Fast
Storage
• 4 Cores to 32 Cores
Many CPU
Cores
8

9. Reasonable Software
•MySQL
•Linux Kernel
•File Systems
•Etc..
Can use
this
Hardware
Pretty
Well
9

10. Cloud
Dynamic Scalability
Throw Away Server Mentality
Agility
Less involved Operations
10

11. Automation
Development Testing Deployment Operations
11

12. New Languages and Frameworks
More Agile
Less “system” code
More Client Side Development
Less Database Dependence
ORM
12

13. Multiple “Clients”
Desktop Web Browsers
Mobile Web Browsers
“Apps”
API
13

14. Not Only “MySQL”
• Memcache, Redis
• RabbitMQ
• Cassandra, MongoDB
• Hadoop
• Column Store Engines,
Vertica
Right
Tools
for
Right
Job!
14

15. More HA Options
Mature MySQL Replication
Percona XtraDB Cluster, Galera
MySQL Cluster
Proprietary cloud HA
15

16. Ready solutions for Replication Management
MHA
PRM
Continuent Tungsten
16

17. Making Sharding Better
• Clustrix
• MemSQL
• ScaleDB
Not Quite
MySQL
• ScaleArc, ScaleBase, Tesora
• MySQL Proxy, MaxScale, ProxySQL
Proxy
Solutions
• Vitess
• JetPants
• MySQL Fabric
OpenSource
Frameworks
17

18. ARCHITECTURE QUESTIONS
18

19. Basics
Make some Right decisions Early
Your first Architecture will not be perfect
Some poor choices are very expensive to fix
19

20. Consultants are like Lawyers
You can waste a lot of money with
them
Getting right advice on the right topics
will save you a lot of trouble in the end
20

21. Dimensions to consider
Application Scale
High Availability
Team Experience
21

22. Approach to Scaling
Avoid Over-
Engineering
Give yourself
some
runway
Do Some
Capacity
Planning
22

23. High Availability
Real High Availability is Really Expensive
Medium Availability is getting more
affordable
People are often leading cause of
downtime
23

24. Team Experience
Chose Solutions which your Team can Support
Often Developers know little about Databases
Right Development, Operational Process Is a Key
Tools to Support your Process
• Percona Toolkit, Percona Cloud Tools
• Systematically find and review new queries
24

25. Simplicity
• Less complexity lower costs
Use fewer
components when
possible
• Monitor
• Troubleshoot
Each components
needs experience
to
• “Follow the Crowd”
• Unless you have a reason not to
There is safety in
numbers
25

26. World of “Small” Applications
• Really
Majority of
Applications are
Small
• Especially with support of caching
May well run on
single database
instance
• Active intranet site for company with 200K
employees
• Ecommerce site supporting $10M+ in Sales
Such “Small”
Applications can
be significant
26

27. Lets do Some Math
Modern
MySQL
Instance
Handle 100K
QPS
20 Queries
per User
Interaction
5K User
Interactions
Per Second
144M User
Interactions a
Day
(Considering
3x peak to
average)
10M Daily
Active users
(with
Medium
Engagement)
27

28. Basic Architecture Patterns
PRACTICAL CHOICES
28

29. Baseline
Single MySQL Instance
No Caching
No HA
No Supplemental Technologies
29

30. Simple High Availability
HA with DBaaS (Amazon RDS)
HA with MySQL Replication
HA with Percona XtraDB Cluster
30

31. What do you need to Scale ?
Reads
Writes
Data
Size
31

32. Load Management Tricks
Move the load
In Time (Doing
Batch Processing
at Night)
In Space (Use
Slave for
Reports)
32

33. Queueing
Used by Most high volume architectures
Deal with Load Spikes
Increase Reliability
Scale parts of the system which need to
Many solutions
• RabbitMQ
• Redis
• Gearman
33

34. Scaling Reads
Replication (and Read Write Splitting)
Caching (Memcache, Redis etc)
Percona XtraDB Cluster (Read Anywhere)
34

35. Scaling Writes
• Using Different Servers for
Different Parts of
Application
Functional
Partitioning
• Horizontal Partitioning over
many servers Sharding
35

36. Scaling Data Size
•Depends on the
Application and
Needs
•10TB good rule of
thumb
How Much
We Can
Practically
Store in
Single
MySQL
Instance
these days
36

37. How Is it Possible
“Online” Schema
Change
MySQL 5.6
Pt-online-schema-change
(Percona
Toolkit)
Fast Backup
Percona
Xtrabackup
MySQL Enterprise
Backup
Modern
Filesystem/Volume
Managers
Fast Network
10Gbps affordable
High
Performance
Storage
Flash/SSD
Compression
Innodb
TokuDB
37

38. Summary
MySQL is Mature. Solutions Available for
Many Problems
Make few choices right in the beginning
Hardware and Software advances allow to go
a long way with simple architectures
38

39. 39
Thank You!
Peter Zaitsev
pz@percona.com