3. 1.1 Who am I, What I Do?
• low-level programming enthusiast
(audio and video DSP routines, tight loop
optimizations)
• embedded systems (graphic EQ with DSP,
fleet management, mobile payment)
• familiar with iOS SDK since 2008
• vox.io (web / mobile sip, xmpp)
• layer.com (messaging)
• obsession with synchronization protocols
16. 2.4.1 Absolute Synchronization (copying)
• Copying (wholesale transfer) is ok
when dealing with small data-sets
(e.g. refreshing weather
forecast, RSVP list ...)
17. 2.4.1 Absolute Synchronization (copying)
• Figuring out differences between previously
fetched data-sets costs CPU and memory
O(n ⋁ m)
18. 2.4.1 Absolute Synchronization (copying)
• Figuring out differences between previously
fetched data-sets costs CPU and memory
O(n ⋁ m)
Dan
Alex
Blake
Emily
George
Caroline
You
19. 2.4.1 Absolute Synchronization (copying)
• Figuring out differences between previously
fetched data-sets costs CPU and memory
O(n ⋁ m)
Dan
Alex
Blake
Emily
George
Caroline
You
Dan
Alex
Emily
George
Caroline
You
≠
20. 2.4.2 Relative Synchronization (changes)
• Getting data up-to-date with changes
instead of full data sets.
(a.k.a. deltas)
21. 2.4 What are Deltas?
Delta encoding is a way to describe differences
between two datasets.
22. 2.5.1 How to Encode Deltas?
insert ― adds new values to dataset
update ― updates existing values in dataset
delete ― deletes existing values from dataset
+
✔
-
• Three primitive operations
26. 2.5.1 How to Encode Deltas?
• Example on how to encode text changes
83: //
84: // Toggles the private ivar `_lightSwitchState` boolean, updates the
85: // background image, plays a sound and transmits the change over network.
86: //
87: func toggleAndSendLightSwitchState() {
88: self.lightSwitchState = !self.lightSwitchState
> 89: self.lightSwitchClient.sendLightSwitchState(self.lightSwitchState)
90: }
27. 2.5.1 How to Encode Deltas?
• Example on how to encode text changes (diff patch)
83: //
84: // Toggles the private ivar `_lightSwitchState` boolean, updates the
85: // background image, plays a sound and transmits the change over network.
86: //
87: func toggleAndSendLightSwitchState() {
88: self.lightSwitchState = !self.lightSwitchState
> 89: self.lightSwitchClient?.sendLightSwitchState(self.lightSwitchState)
90: }
--- 89: self.lightSwitchClient.sendLightSwitchState(self.lightSwitchState)
+++ 89: self.lightSwitchClient?.sendLightSwitchState(self.lightSwitchState)
28. 2.5.1 How to Encode Deltas?
• Example on how to encode text changes (insert operation)
83: //
84: // Toggles the private ivar `_lightSwitchState` boolean, updates the
85: // background image, plays a sound and transmits the change over network.
86: //
87: func toggleAndSendLightSwitchState() {
88: self.lightSwitchState = !self.lightSwitchState
> 89: self.lightSwitchClient?.sendLightSwitchState(self.lightSwitchState)
90: }
{
type: "insert",
offset: 2781,
values: [ "?" ]
}
29. 2.5.1 How to Encode Deltas?
• Example on how to encode custom data model changes
{
guests: [ "Alex", "Blake", "Caroline", "Dan", "Emily", "George" ]
}
30. 2.5.1 How to Encode Deltas?
• Example on how to encode custom data model changes
{
guests: [ "Alex", "Blake", "Caroline", "Dan", "Emily", "George" ]
}
{
type: "delete",
guest: [ "Blake" ]
}
{
guests: [ "Alex", "Caroline", "Dan", "Emily", "George" ]
}
40. 3.2.1 Example (To-do List App Data-model)
• Live synchronized list of
to-do tasks
public struct Todo {
public class List: NSObject {
private let tasks: Array<Task> = []
}
}
41. 3.2.1 Example (To-do List App Data-model)
• Live synchronized list of
to-do tasks
• Task element consists of:
checkbox, label and
color public struct Todo {
public class List: NSObject {
private let tasks: Array<Task> = []
}
}
public struct Todo {
public class Task: NSObject {
public private(set) var identifier: NSUUID
public private(set) var completed: Bool
public private(set) var title: String
public private(set) var label: ColorLabel
public enum ColorLabel: UInt8 {
case None = 0, Red, Orange, Yellow, Green,
Turquoise, Blue, Purple, Pink
}
}
}
42. 3.2.1 Example (To-do List App Data-model)
• Live synchronized list of
to-do tasks
• Task element consists of:
checkbox, label and
color
• Tasks can be added,
edited and removed
public struct Todo {
public class List: NSObject {
private let tasks: Array<Task> = []
public func create(title: String, label: Task.ColorL
public func update(identifier: NSUUID, completed: Bo
public func remove(identifier: NSUUID)
}
}
public struct Todo {
public class Task: NSObject {
public private(set) var identifier: NSUUID
public private(set) var completed: Bool
public private(set) var title: String
public private(set) var label: ColorLabel
public enum ColorLabel: UInt8 {
case None = 0, Red, Orange, Yellow, Green,
Turquoise, Blue, Purple, Pink
}
}
}
43. 3.2.1 Example (To-do List App Data-model)
• Live synchronized list of
to-do tasks
• Task element consists of:
checkbox, label and
color
• Tasks can be added,
edited and removed
44. 3.2.2 Example (To-do List Sync Data-model)
• Todo.List user actions
turn into events (𝚫)
45. 3.2.2 Example (To-do List Sync Data-model)
• Todo.List user actions
turn into events (𝚫)
• Simple concrete objects
describing changes
public struct Sync {
public class Event: NSObject {
public enum Type: UInt8 {
case Insert = 0, Update, Delete
}
public private(set) var type: Type
public private(set) var identifier: NSUUID
public private(set) var completed: Bool?
public private(set) var title: String?
public private(set) var label: Int?
}
}
46. 3.2.2 Example (To-do List Sync Data-model)
• Todo.List user actions
turn into events (𝚫)
• Simple concrete objects
describing changes
• Serializable
public struct Sync {
public class Event: NSObject, Serializable {
public enum Type: UInt8 {
case Insert = 0, Update, Delete
}
public private(set) var type: Type
public private(set) var identifier: NSUUID
public private(set) var completed: Bool?
public private(set) var title: String?
public private(set) var label: Int?
}
}
public protocol Serializable: class {
init(fromDictionary dictionary: Dictionary<String, AnyObject>)
func toDictionary() -> Dictionary<String, AnyObject>
}
47. 3.2.2 Example (To-do List Sync Data-model)
• Todo.List user actions
turn into events (𝚫)
• Simple concrete objects
describing changes
• Serializable
48. 3.2.2 Example (To-do List Sync Data-model)
• Creating new task
{ // serialized event structure
type: 0, // 0 = Insert
identifier: "cb55ceec-b9ae-4bd9-8783-7dbf3e9cb2cd", // client generated id
completed: false, // an incomplete task
title: "Buy Milk", // task description
label: 0 // color tag
}
49. 3.2.2 Example (To-do List Sync Data-model)
• Editing an existing task
{ // event structure
type: 1, // 1 = Update
identifier: "cb55ceec-b9ae-4bd9-8783-7dbf3e9cb2cd", // reference to task
completed: true // new state
}
50. 3.2.3 Example (To-do List Sync and Transport)
• Receive live serialized
Events from the
server. public protocol TransportDelegate: class {
func transport(transport: Transport, didReceiveObject object: Serializable)
func transportDidConnect(transport: Transport)
func transportDidDisconnect(transport: Transport)
}
public struct Sync {
public class Client: NSObject, TransportDelegate {
public private(set) var stream: Stream = Stream()
public private(set) var transport: Transport
public private(set) var todoList: Todo.List
public private(set) var publishedEvents: Array<Event> = []
private func publish(event: Event) -> Bool
public func transport(transport: Transport, didReceiveObject object: Serializab
}
}
51. 3.2.3 Example (To-do List Sync and Transport)
• Receive live serialized
Events from the
server.
• Send serialized
Events to server.
public protocol TransportDelegate: class {
func transport(transport: Transport, didReceiveObject object: Serializable)
func transportDidConnect(transport: Transport)
func transportDidDisconnect(transport: Transport)
}
public struct Sync {
public class Client: NSObject, TransportDelegate {
public private(set) var stream: Stream = Stream()
public private(set) var transport: Transport
public private(set) var todoList: Todo.List
public private(set) var publishedEvents: Array<Event> = []
private func publish(event: Event) -> Bool
public func transport(transport: Transport, didReceiveObject object: Serializab
}
}
52. 3.2.3 Example (To-do List Sync and Transport)
• Receive live serialized
Events from the
server.
• Send serialized
Events to server.
53. 3.3 Let the Streaming Begin
• Data consistent, as long as clients remain connected
54. 3.3 Let the Streaming Begin
• Missing out on events puts the client out-of-sync
55. 3.3 Let the Streaming Begin
• Missing out on events puts the client out-of-sync
{ // event structure
type: 1,
identifier: "cb55ceec-b9ae-4bd9-8783-7dbf3e9cb2cd",
completed: true
}
56. 3.3 Let the Streaming Begin
• Data consistent, as long as clients remain
connected
• Missing out on events puts the client out-of-sync
• Clients can recover from out-of-sync state
• Server's responsibility beside broadcasting should
also be preserving the events
58. 3.4 Persistent Stream
• Think of it as a linear magnetic tape, or as a storage
with a WORM behavior
• Append only
• Immutable events
• Journal of all the events that have happened
59. 3.4 Persistent Stream
• Always copy all the events? (too expensive)
• Integrity check by hashing events? (only detects
mismatches)
How does a client know if it's got all the events?
62. 3.5 Event Discovery
• Sequencing Events on server
public struct Sync {
public class Event: NSObject {
public private(set) var seq: Int?
public private(set) var type: Type
public private(set) var identifier: NSUUID?
public private(set) var completed: Bool?
public private(set) var title: String?
public private(set) var label: Int?
}
}
63. 3.5 Event Discovery
• Sequencing Events on server
• Sequence is a linear function f(x)=x reproducible on client
64. 3.5 Event Discovery
• Sequencing Events on server
• Sequence is a linear function f(x)=x reproducible on client
65. 3.5 Event Discovery
• Client only needs to know the seq value of
the last event f(x<12)=x
66. 3.5 Event Discovery
• Client only needs to know the seq value of
the last event f(x<12)=x
• Figuring out missing events by subtracting the set of seqs
67. 3.5 Event Discovery
// Seq values pulled from all events the client has.
// [ 0, 1, 2, 10, 11, 12 ]
let seqsOfEvents: Set = events.map({ $0.seq })
// Calculated sequence ranging from 0 to 12.
// [ 0, 1, 2, 3 ... 12 ]
let seqsOfAllEvents: Set = [Int](0...12)
// Diffed set of seq values.
// [ 3, 4, 5, 6, 7, 8, 9 ]
let seqsOfMissingEvents: Set = seqOfAllEvents.subtract(seqOfEvents)
• Client only needs to know the seq value of
the last event f(x<12)=x
• Figuring out missing events by subtracting the set of seqs
68. 4. Event and Model Reconciliation
Outbound and inbound reconciliation
82. 4.4 Reducing the Edit Distance
• Events describing the same mutation
83. 4.4 Reducing the Edit Distance
• Causes stream pollution
• Increases the edit distance
84. 4.4 Reducing the Edit Distance
1. Insert Event merges with
Update Events → single Insert Event
2. Update Event merge with
the rest of Update Events → single Update Event
3. Last Update Event defines final state.
4. Delete Event clobbers other Event types
Simple set of rules when queueing:
85. 4.4 Reducing the Edit Distance
public struct Sync {
public class Event: NSObject, Serializable {
var mergedEvents = Array<Event>()
for oldEvent in events.reverse() {
if oldEvent.identifier != self.identifier {
// Event not mergeable, due to the identifier mismatch.
mergedEvents.append(oldEvent)
continue
} else if self.type == Type.Delete {
// Rule #4
self.reset()
self.type = Type.Delete
} else if self.type == Type.Update && (oldEvent.type == Type.Insert || oldEvent.type == Type.Update) {
// Rule #1, #2, #3
self.completed = self.completed ?? oldEvent.completed
self.title = self.title ?? oldEvent.title
self.label = self.label ?? oldEvent.label
}
}
mergedEvents.append(self)
return mergedEvents
}
}
86. 4.5 Conflict Resolution
• Concurrent systems experience conflicts when two
or more nodes (clients) work on the same resource
at the same time.
87. 4.5 Conflict Resolution
• Concurrent systems experience conflicts when two
or more nodes (clients) work on the same resource
at the same time.
• Example: a client deletes a Todo Task before
another client tries to mutate it.
88. 4.5 Conflict Resolution
Possible conflict resolutions:
• Bring the deleted task back (last writer wins)
• Deleted task stays deleted (first writer wins)
• Ask the User what to do? (requires user interaction)
90. 5. Order of Events
Event sequence dictates the order they were written to
stream this puts the Events in total order
• Task objects will be in the exact same order, defined by
the Event.seq
• Task mutations will be applied in the same manner on all
clients
91. 5. Order of Events (total order)
• Queued events must be published in batches
92. 5. Order of Events (total order)
• Queued events must be published in batches
93. 5. Order of Events (total order)
• Queued events must be published in batches
94. 5. Order of Events (total order)
• Queued events must be published in batches
95. 5.1 Total Order (sequential writes)
• Synchronized sequential writes block other clients from
writing
96. 5.1 Total Order (sequential writes)
• Synchronized sequential writes block other clients from
writing - violates our fast concurrent writes requirement
serial
writes
concurrent
writes
99. 5.1 Total Order (offline support)
• Left client loses connection
100. 5.1 Total Order (offline support)
• Offline client adds more To-do tasks to the list
101. 5.1 Total Order (offline support)
• Online client also adds a Todo task to the list
102. 5.1 Total Order (offline support)
• Left client comes back online ― events generated offline
get published and fall at the end (higher seq values)
103. 5.2 Causal Order
Causes must precede their effects - effects come after causes,
and never before
104. 5.2 Causal Order
Causes must precede their effects - effects come after causes,
and never before
cause
effect
105. 5.2 Causal Order
• Generated Event is an effect caused by user taking action /
responding to the UI.
• Events should be reconciled in the same order as they were
generated by clients.
• Events should be applied onto the app model under the same
conditions as it was when author generated the events.
• Total order cannot guarantee Events will be written to stream in
the same order they were generated.
106. 5.2.1 Order Based on Timestamps
Client B's events are
written before Client A's,
even though Client A
generated them first.
109. 5.2.1 Order Based on Timestamps
• No guarantee
time will be the
same on all
devices
• Clock skew
• Manual override
110. 5.2.2 Version Vectors
• Reconstructing Events' order as it was perceived by the author
based on happened-before information.
• Provides causality-tracking basic principle in some optimistic
(lazy) replication algorithms.
• Allows the client to operate independently from the server.
• When all clients eventually publish their events, it brings other online
clients into a consistent state eventual consistency.
111. 5.2.2 Version Vectors
How to encode happened-before information?
public struct Sync {
public class Event: NSObject {
public private(set) var seq: Int?
public private(set) var type: Type
public private(set) var identifier: NSUUID?
public private(set) var completed: Bool?
public private(set) var title: String?
public private(set) var label: Int?
}
}
112. 5.2.2 Version Vectors
1. Information of what's
the last seen event -
event.seq
How to encode happened-before information?
public struct Sync {
public class Event: NSObject {
public private(set) var seq: Int?
public private(set) var precedingSeq: Int
public private(set) var type: Type
public private(set) var identifier: NSUUID?
public private(set) var completed: Bool?
public private(set) var title: String?
public private(set) var label: Int?
}
}
113. 5.2.2 Version Vectors
1. Information of what's
the last seen event -
event.seq
2. Keep unpublished
events in order -
event.clientSeq
How to encode happened-before information?
public struct Sync {
public class Event: NSObject {
public private(set) var seq: Int?
public private(set) var precedingSeq: Int
public private(set) var clientSeq: Int
public private(set) var type: Type
public private(set) var identifier: NSUUID?
public private(set) var completed: Bool?
public private(set) var title: String?
public private(set) var label: Int?
}
}
114. 5.2.2 Version Vectors
1. Information of what's
the last seen event -
event.seq
2. Keep unpublished
events in order -
event.clientSeq
3. Order
How to encode happened-before information?
public struct Sync {
public class Event: NSObject {
/// Event sorting closure
static public let causalOrder = { (e1: Event, e2: Event)
if e1.precedingSeq == e2.precedingSeq {
return e1.clientSeq < e2.clientSeq
}
return e1.precedingSeq < e2.precedingSeq
}
public private(set) var seq: Int?
public private(set) var precedingSeq: Int
public private(set) var clientSeq: Int
public private(set) var type: Type
public private(set) var identifier: NSUUID?
public private(set) var completed: Bool?
public private(set) var title: String?
public private(set) var label: Int?
}
}
121. 5.2.2 Version Vectors
public struct Sync {
public class Event: NSObject, Serializable {
var mergedEvents = Array<Event>()
for oldEvent in events.sort(Event.causalOrder) {
if oldEvent.identifier != self.identifier {
// etc...
public struct Todo {
public class List: NSObject, ModelReconciler {
public func apply(events: Array<Sync.Event>) -> Bool {
for event in events.sort(Sync.Event.causalOrder) {
let success = self.apply(event)
// etc...
Minor adjustment in outbound / inbound reconciliation:
122. 5.2.2 Version Vectors
• Newly published events generated offline are ordered by
their causality.
123. 5.2.2 Version Vectors
• Concurrent writes - no need for batched writes anymore,
due to clientSeq.
124. 5.2.2 Version Vectors
• Concurrent writes - events can be written with undetermined
order; order can be reconstructed on clients
serial
writes
concurrent
writes