This is a copy of the slides for my talk "Database Scalability Patterns", given in March at PGEast 2010.

1. Database Scalability
{Patterns}
/ Robert Treat
Thursday, April 8, 2010

2. robert treat
omniti
postgres
oracle - mysql
mssql - sqlite -nosql
Thursday, April 8, 2010

3. What are “Database Scalability Patterns?”
Thursday, April 8, 2010

4. Part Design Patterns
Part Application Life-Cycle
Thursday, April 8, 2010

5. MyFirstDatabase
Thursday, April 8, 2010

6. Vertical Partitioning
Thursday, April 8, 2010

7. Vertical Scaling
Thursday, April 8, 2010

8. Vertical Scaling
Thursday, April 8, 2010

9. Federated Data Storage
“sharding” Horizontal Partitioning
Read Slaves
Multi-Master
Horizontal Scaling
Thursday, April 8, 2010

10. Read Slaves / Master - Slave
Scale Read Load
Thursday, April 8, 2010

11. Read Slaves / Master - Slave
Scale Read Load
• Typically
• Full Copy of Data
On Each Node
• Asynchronous
Thursday, April 8, 2010

12. Read Slaves / Master - Slave
Scale Read Load
• Typically • Consider
• Full Copy of Data • Partial Copy
On Each Node • Synchronous
• Asynchronous • Don’t use a RDBMS?
Thursday, April 8, 2010

13. Read Slaves / Master - Slave
Scale Read Load
• Typically • Consider
• Full Copy of Data • Partial Copy
On Each Node • Synchronous
• Asynchronous • Don’t use a RDBMS?
Requires Application Changes
Thursday, April 8, 2010

14. Read Slaves / Master - Slave
Scale Read Load
• Typically • Consider
• Full Copy of Data • Partial Copy
On Each Node • Synchronous
• Asynchronous • Don’t use a RDBMS?
Requires Application Changes
“easy”
Thursday, April 8, 2010

15. Scaling Writes
“not easy”
Thursday, April 8, 2010

16. Multi-Master
many different ways to implement this,
few that actually work in production
Thursday, April 8, 2010

17. Multi-Master
many different ways to implement this,
few that actually work in production
write to any node, database syncs data
Thursday, April 8, 2010

18. Multi-Master
many different ways to implement this,
few that actually work in production
write to any node, database syncs data
can reduce cpu, doesn’t reduce i/o
Thursday, April 8, 2010

19. Multi-Master
many different ways to implement this,
few that actually work in production
write to any node, database syncs data
can reduce cpu, doesn’t reduce i/o
failover solution
not a scalability solution
Thursday, April 8, 2010

20. Horizontal Partitioning
• Divide schema by job operations
• Move each piece to own server
• Duplicate some data as needed
Thursday, April 8, 2010

21. Horizontal Partitioning
• Divide schema by job operations
• Move each piece to own server
• Duplicate some data as needed
• You must separate dependencies
in the app code first!
Thursday, April 8, 2010

22. Horizontal Partitioning
• Divide schema by job operations
• Move each piece to own server
• Duplicate some data as needed
• You must separate dependencies
in the app code first!
Each node is a new instance of vertical scaling
Thursday, April 8, 2010

23. Horizontal Scaling
• data split across servers based on algorithm
• data dropped into buckets (multiple?)
• someone must keep track of data, and provide
lookup services
Thursday, April 8, 2010

24. Universal Truths of Scaling Databases
Vertical Scalability is Helpful for Every Pattern
Even in a horizontally scaled, fully distributed
database, the number of nodes needed is
affected by vertical scalability
Thursday, April 8, 2010

25. Universal Truths of Scaling Databases
New Nodes Are Never Free
• Add points of failure
• Add management costs
• Add complexity to architecture
• Add complexity to your app code
Thursday, April 8, 2010

26. MyFirstDB
V
Vertical Partitioning
V
Vertical Scaling
V
Read Slaves
V
Horizontal Partitioning
Thursday, April 8, 2010

27. plan for layered data sources
read / write connections in code
use schemas to separate services
Thursday, April 8, 2010

28. plan for layered data sources
read / write connections in code
use schemas to separate services
use postgres :-)
Thursday, April 8, 2010

29. THANKS!
more:
xzilla.net
@robtreat2
omniti.com
Thursday, April 8, 2010