High Profile Call Girls Nagpur Meera Call 7001035870 Meet With Nagpur Escorts
Accordion - VLDB 2014
1. Accordion: Elastic Scalability
for Database Systems
Supporting Distributed
Transactions
!
Marco Serafini
Qatar Computing Research Institute
!
joint work with:
Essam Mansour, Ashraf Aboulnaga Qatar Computing Research Institute
Kenneth Salem University of Waterloo
Taha Rafiq Amazon.com
Umar Farooq Minhas IBM Research Almaden
2. Leveraging the Cloud
Applications cannot always leverage the cloud
Make partitioned DBMSes scale out and in!
P1
P2
P3
P4
P5
P6
P7
P8
Cloud
layer
DBMS
layer
3. Leveraging the Cloud
Applications cannot always leverage the cloud
Make partitioned DBMSes scale out and in!
P1
P2
P4 P7
P3
P5 P8
P6
Cloud
layer
DBMS
layer
4. Online Solution
Online = Handle workload with unanticipated skews
Partitions suddenly become hot
Overall database load grows/shrinks
Skews change over time
No prior knowledge, no workload trace analysis
5. Accordion
Goal: run a partitioned DBMS on a variable set of servers
DBMS supports ACID distributed transactions
New!
Necessary in many OLTP workloads
Major performance bottleneck
Problems we address: Where & When to move data
10. The dangers of
scaling out
with distributed
transactions
11. Scaling Out:
How Effective is it?
Before scale out: After:
Full DB
1/2 DB
1/2 DB
Does scaling out increase throughput?
12. No Distributed Transactions
35000
30000
25000
20000
15000
10000
5000
0
16 partitions per server
8 partitions per server
8 16 32 64
Server capacity (tps)
Overall number of partitions
YCSB
Max throughput
per server
constant
Overall
throughput
grows linearly
N nodes
2*N nodes
Bars: Smaller DB -
Larger DB -
13. Distributed Transactions
30000
25000
20000
15000
10000
5000
0
16 partitions per server
8 partitions per server
8 16 32 64
Server capacity (tps)
Overall number of partitions
TPC-C
Max throughput
per server
decreases
Overall
throughput
can still
increase N nodes
Bars: Smaller DB -
2*N nodes
Larger DB -
14. Distributed Transactions
= Circular Dependency
Bin Packing Partition Placement
Bin Server
Volume of Bin Maximum throughput
of server
Item Database partition
Volume of Item Transaction rate of
partition
Packing
Constraints Determines
Bin Volume -
not constant!
15. Two Problems
1. Model for server capacity (maximum throughput)
Capacities varies based on placement
Learn model online
2. Partition placement using this model
16. 1: Server Capacity Models
Affinity: Likelihood of co-access among partitions
Aff(P1,P2) = Prob (P2 accessed | P1 accessed)
!
Max throughput capacity per server depends on affinity
Null affinity (YCSB) - Constant capacity
Uniform affinity (TPC-C) - f (# local partitions)
Arbitrary affinity - Must model affinity explicitly
17. 2: Partition Placement
Accordion’s planner uses linear programming
Minimize data
migration
s.t. servers are
not overloaded
Server capacity function
Can be nonlinear
20. Cost Reduction (TPC-C)
40
35
30
25
20
15
10
5
Arbitrary Affinity
up to 9x cost reduction
64 256 1024
Uniform Affinity
30
25
20
15
10
5
0
up to 1.7x cost reduction
Number of servers used
Number of partitions
Accordion
Kairos-SP
Greedy
Static
0
64 256 1024
Number of servers used
Number of partitions
Accordion
Kairos-SP
Greedy
Static
21. Impact on Migration (TPC-C)
40000
35000
sec
per 30000
25000
Transactions 20000
15000
10000
5000
0
0 10 20 30 40 50 Time (min)
Accordion
Kairos-SP
Cold
Shorter reconfiguration time, fewer servers