Make better architectural decisions when it comes to database selection and data modeling.

1. Distributed RDBMS
Challenges, Solutions & Trade-offs
by Ahmed Magdy
Blog: ahmed.a1cv.com

2. DBMS Rank by Popularity
• http://db-engines.com/en/ranking

3. Agenda
• Distributed RDBMS: What & Why
• Fallacies of Distributed Computing
• ACID Challenges
• Two Phase Commit Algorithm (2PC)
• Distributed Relational Challenges
• How to Distribute
• Replication
• Sharding
• Trade-offs
• Latency vs Throughput
• Consistency vs Availability
• Consistency vs Latency
• Some Distributed RDBMS Providers

4. Before we start!
If I had an hour to solve a problem
I'd spend 55 minutes thinking
about the problem and 5 minutes
thinking about solutions.

5. Distributed RDBMS: What & Why
What?
• Relational DBMS installed on multiple machines/VMs sharing the same database
serving common applications.
Why?
• Scalability (more throughput)
• Performance (geographically nearer servers)
• Availability (fail-over)

6. Fallacies of Distributed Computing
• The network is reliable.
• Latency is zero.
• Bandwidth is infinite.
• The network is secure.
• Topology doesn't change.
• There is one administrator.
• Transport cost is zero.
• The network is homogeneous.

7. ACID Challenges

8. Atomicity
• Either all operations occur or nothing
• Preventing partial updates

9. Consistency
• Transactions do not violate data integrity:
• Entity integrity
• Referential integrity
• Domain integrity
• User-defined
• Sequential consistency (Serializability)
• Eventual Consistency

10. Isolation
Isolation level Dirty reads Non-repeatable reads Phantoms
Read Uncommitted may occur may occur may occur
Read Committed - may occur may occur
Repeatable Read - - may occur
Serializable - - -
Read phenomena:
• Dirty reads: read uncommitted writes
• Non-repeatable reads: the data changes between reads
• Phantom reads: 2 identical queries return different number of rows
Isolation Levels: Read Uncommitted, Read Committed,
Repeatable Read, Serializable

11. Durability
• Changes are persisted to disk before reporting
the transaction as committed
Approaches:
• Write representation to disk
• Write operations to transaction log

12. Two Phase Commit Algorithm (2PC)
1. Commit-request Phase (voting phase):
• Coordinator: “Hi Participants, Do you agree to commit this transaction?”
• Participants: “Yes Sir”
2. Commit Phase:
• Coordinator: “Let’s do it guys!”
• Blocks client until all participants
Commit or rollback
• Provides Atomicity & Consistency

13. How to Distribute
• Replication
• Master-slave
• Multi-master
• Partitioning
• Horizontal (Sharding)
• Vertical (like one-to-one relationships)
• Functional (like in Microservice architecture)

14. Replication
Benefits:
• Higher Availability
• Load balancing
• Performance gains by replication to geographically nearer data centers

15. Master-Slave vs Multi-Master Replication
Master-Slave Replication:
• Master for writing, slaves for
reading
• Single point of failure
• A slave can be promoted to be
master, if the master is down
(manually or automatically)
Multi-Master Replication:
• High fault tolerance
• Better load balancing of write and
read operations
• Complex transactional conflict
prevention is required

16. Pessimistic vs Optimistic Replication
Pessimistic Replication:
• Eager / synchronous
• Higher latency (blocking)
• Better Consistency
• Conflicting transactions are
detected before commit so
they can rollback.
Optimistic Replication:
• Lazy / asynchronous
• Lower latency
• Eventual Consistency
• Complex conflict resolution:
• Syntactic
• Semantic

17. Sharding
Strategies
• Lookup (routing table & virtual shards)
• Range (better for range queries)
• Hash (less hotspots for monotonic shard keys)
Best Practices:
• Ensure that shard keys are unique.
• Use stable data for the shard key.
• Keep shards balanced to handle similar volumes of I/O.
• Shard the data to support the most frequently performed queries
• Use parallel tasks if you need to access more than 1 shard
• Minimize operations that affect data in multiple shards
• Shards can be geo-located to reduce latency

18. Latency vs Throughput
Latency = 1 min
Throughput = 1 car / min

19. Latency = 1 min
Throughput = 3 cars / min
Latency vs Throughput [continued]

20. Latency vs Throughput
Latency = 1.5 min
Throughput = 1 car every 1.5 min
= 0.66 cars / min

21. Latency = 1.5 min
Throughput =
3 cars every 2.5 min
= 1.2 cars / min
Latency vs Throughput [continued]

22. • More nodes are added to
improve throughput, but latency
is deteriorated.
Latency vs Throughput [continued]

24. Consistency vs Availability
CAP Theorem
Partitioned

25. Consistency vs Latency
DDBS P+A P+C E+L E+C
Dynamo Yes Yes
Cassandra Yes Yes
Riak Yes Yes
MySQL Yes Yes
MongoDB Yes Yes
PACELC Theorem
PACELC: Partitioned  Availability | Consistency
Else  Latency | Consistency

26. Distributed Relational Challenges
• The Aggregation Challenge
SELECT AVG(salary) FROM employees;
• The Distinctive Values Challenge
SELECT DISTINCT country_id FROM employees;
• The Joins Challenge
SELECT e.first_name, e.last_name, d.name FROM employees AS e INNER JOIN
departments AS d ON e.department_id = d.id;
• The Sub-Queries Challenge
SELECT first_name, last_name FROM employees WHERE department_id IN
(SELECT id FROM departments WHERE rating > 4);
• The “Combination” Challenge
SELECT AVG(salary) FROM employees WHERE department_id IN
(SELECT id FROM departments WHERE rating > 4);

27. Average Salary Query with MapReduce in MongoDB
var mapFunc = function() {
emit(0, this.salary);
};
var reduceFunc = function(key, salaries) {
return Array.sum(salaries) / salaries.length;
};
db.employees.mapReduce(
mapFunc,
reduceFunc,
{
out: {inline: 1}
}
)['results'][0]['value'];

28. Some Distributed RDBMS Providers

29. Questions?