Slides for my talk event-sourced architectures with Akka. Discusses Akka Persistence as mechanism to do event-sourcing. Presented at Javaone 2014 and Jfokus 2015.

1. Event-sourced
architectures
with Akka
@Sander_Mak
Luminis Technologies

2. Today's journey
Event-sourcing
Actors
Akka Persistence
Design for ES

3. Event-sourcing
Is all about getting
the facts straight

4. Typical 3 layer architecture
UI/Client
Service layer
Database
fetch ↝ modify ↝ store

5. Typical 3 layer architecture
UI/Client
Service layer
Database
fetch ↝ modify ↝ store
Databases are
shared mutable state

6. Typical entity modelling
Concert
!
artist: String
date: Date
availableTickets: int
price: int
...
TicketOrder
!
noOfTickets: int
userId: String
1 *

7. Typical entity modelling
Concert
!
artist = Aerosmith
availableTickets = 100
price = 10
...
!
TicketOrder
TicketOrder
!
noOfTickets = 3
userId = 1
TicketOrder
!
noOfTickets = 3
userId = 1
noOfTickets = 3
userId = 1

8. Typical entity modelling
Changing the price
Concert
!
artist = Aerosmith
availableTickets = 100
price = 10
...
!
TicketOrder
TicketOrder
!
noOfTickets = 3
userId = 1
TicketOrder
!
noOfTickets = 3
userId = 1
noOfTickets = 3
userId = 1

9. Typical entity modelling
Changing the price
Concert
!
artist = Aerosmith
availableTickets = 100
price = 10
...
!
TicketOrder
TicketOrder
!
noOfTickets = 3
userId = 1
TicketOrder
!
noOfTickets = 3
userId = 1
noOfTickets = 3
userId = 1
✘100

10. Typical entity modelling
Canceling an order
Concert
!
artist = Aerosmith
availableTickets = 100
price = 10
...
!
TicketOrder
TicketOrder
!
noOfTickets = 3
userId = 1
TicketOrder
!
noOfTickets = 3
userId = 1
noOfTickets = 3
userId = 1

11. Typical entity modelling
Canceling an order
Concert
!
artist = Aerosmith
availableTickets = 100
price = 10
...
!
TicketOrder
TicketOrder
!
noOfTickets = 3
userId = 1
!
noOfTickets = 3
userId = 1

12. Update or delete statements in your app?
Congratulations, you are
!
LOSING DATA EVERY DAY

13. Event-sourced modelling
ConcertCreated
!
artist = Aerosmith
availableTickets = 100
price = 10
...
!
TicketsOrdered
TicketsOrdered
!
noOfTickets = 3
userId = 1
TicketsOrdered
!
noOfTickets = 3
!
userId = 1
noOfTickets = 3
userId = 1
time

14. Event-sourced modelling
TicketsOrdered
TicketsOrdered
Changing the price
ConcertCreated
!
artist = Aerosmith
availableTickets = 100
price = 10
...
!
TicketsOrdered
!
noOfTickets = 3
userId = 1
!
noOfTickets = 3
!
userId = 1
noOfTickets = 3
userId = 1
time

15. Event-sourced modelling
TicketsOrdered
TicketsOrdered
Changing the price
ConcertCreated
!
artist = Aerosmith
availableTickets = 100
price = 10
...
!
PriceChanged
!
price = 100
TicketsOrdered
!
noOfTickets = 3
userId = 1
!
noOfTickets = 3
!
userId = 1
noOfTickets = 3
userId = 1
time

16. Event-sourced modelling
ConcertCreated
!
artist = Aerosmith
availableTickets = 100
price = 10
...
!
PriceChanged
!
price = 100
TicketsOrdered
TicketsOrdered
!
noOfTickets = 3
userId = 1
TicketsOrdered
!
noOfTickets = 3
!
userId = 1
noOfTickets = 3
userId = 1
Canceling an order
time

17. Event-sourced modelling
ConcertCreated
!
artist = Aerosmith
availableTickets = 100
price = 10
...
!
PriceChanged
!
price = 100
OrderCancelled
!
userId = 1
TicketsOrdered
TicketsOrdered
!
noOfTickets = 3
userId = 1
TicketsOrdered
!
noOfTickets = 3
!
userId = 1
noOfTickets = 3
userId = 1
Canceling an order
time

18. Event-sourced modelling
‣ Immutable events
‣ Append-only storage (scalable)
‣ Replay events: reconstruct historic state
‣ Events as integration mechanism
‣ Events as audit mechanism

19. Event-sourcing: capture all changes to
application state as a sequence of events
Events: where from?

20. Commands & Events
Do something (active)
It happened.
Deal with it.
(facts)
Can be rejected (validation)
Can be responded to

21. Querying & event-sourcing
How do you query a log?

22. Querying & event-sourcing
How do you query a log?
Command
Query
Responsibility
Segregation

23. CQRS without ES
Command Query
UI/Client
Service layer
Database

24. CQRS without ES
Command Query
UI/Client
Service layer
Database
Command Query
UI/Client
Command
Model
Datastore
Query
Model(s)
DaDtaastatosrtoere Datastore
?

25. Event-sourced CQRS
Command Query
UI/Client
Command
Model
Journal
Query
Model(s)
DaDtaastatosrtoere Datastore
Events

26. Actors
‣ Mature and open source
‣ Scala & Java API
‣ Akka Cluster

27. Actors
"an island of consistency in a sea of concurrency"
Actor
!
!
!
mailbox
state
behavior
async message
send
Process message:
‣ update state
‣ send messages
‣ change behavior
Don't worry about
concurrency

28. Actors
A good fit for event-sourcing?
Actor
!
!
!
mailbox
state
behavior
mailbox is non-durable
(lost messages)
state is transient

29. Actors
Just store all incoming messages?
Actor
!
!
!
mailbox
state
behavior
async message
send
store in journal
Problems with
command-sourcing:
‣ side-effects
‣ poisonous
(failing) messages

30. Persistence
‣ Experimental Akka module
‣ Scala & Java API
‣ Actor state persistence based
on event-sourcing

31. Persistent Actor
PersistentActor
actor-id
!
!
state
!
event
async message
send (command)
‣ Derive events from
commands
‣ Store events
‣ Update state
‣ Perform side-effects
event
journal (actor-id)

32. Persistent Actor
PersistentActor
actor-id
!
!
state
!
event
!
Recover by replaying
events, that update the
state (no side-effects)
!
event
journal (actor-id)

33. Persistent Actor
!
case object Increment // command
case object Incremented // event
!
class CounterActor extends PersistentActor {
def persistenceId = "counter"
!
var state = 0
!
val receiveCommand: Receive = {
case Increment => persist(Incremented) { evt =>
state += 1
println("incremented")
}
}
!
val receiveRecover: Receive = {
case Incremented => state += 1
}
}

34. Persistent Actor
!
case object Increment // command
case object Incremented // event
!
class CounterActor extends PersistentActor {
def persistenceId = "counter"
!
var state = 0
!
val receiveCommand: Receive = {
case Increment => persist(Incremented) { evt =>
state += 1
println("incremented")
}
}
!
val receiveRecover: Receive = {
case Incremented => state += 1
}
}

35. Persistent Actor
!
case object Increment // command
case object Incremented // event
!
class CounterActor extends PersistentActor {
def persistenceId = "counter"
!
var state = 0
!
val receiveCommand: Receive = {
case Increment => persist(Incremented) { evt =>
state += 1
println("incremented")
}
}
!
val receiveRecover: Receive = {
case Incremented => state += 1
}
}
async callback
(but safe to close
over state)

36. Persistent Actor
!
case object Increment // command
case object Incremented // event
!
class CounterActor extends PersistentActor {
def persistenceId = "counter"
!
var state = 0
!
val receiveCommand: Receive = {
case Increment => persist(Incremented) { evt =>
state += 1
println("incremented")
}
}
!
val receiveRecover: Receive = {
case Incremented => state += 1
}
}

37. Persistent Actor
!
case object Increment // command
case object Incremented // event
!
class CounterActor extends PersistentActor {
def persistenceId = "counter"
!
var state = 0
!
val receiveCommand: Receive = {
case Increment => persist(Incremented) { evt =>
state += 1
println("incremented")
}
}
!
val receiveRecover: Receive = {
case Incremented => state += 1
}
}
Isn't recovery
with lots of events
slow?

38. Snapshots
class SnapshottingCounterActor extends PersistentActor {
def persistenceId = "snapshotting-counter"
!
var state = 0
!
val receiveCommand: Receive = {
case Increment => persist(Incremented) { evt =>
state += 1
println("incremented")
}
case "takesnapshot" => saveSnapshot(state)
}
!
val receiveRecover: Receive = {
case Incremented => state += 1
case SnapshotOffer(_, snapshotState: Int) => state = snapshotState
}
}

39. Snapshots
class SnapshottingCounterActor extends PersistentActor {
def persistenceId = "snapshotting-counter"
!
var state = 0
!
val receiveCommand: Receive = {
case Increment => persist(Incremented) { evt =>
state += 1
println("incremented")
}
case "takesnapshot" => saveSnapshot(state)
}
!
val receiveRecover: Receive = {
case Incremented => state += 1
case SnapshotOffer(_, snapshotState: Int) => state = snapshotState
}
}

40. Journal & Snapshot
Cassandra
Cassandra
Kafka Kafka
MongoDB
HBase
DynamoDB
MongoDB
HBase
MapDB
JDBC JDBC
Plugins:

41. Plugins:
Serialization
Default: Java serialization
Pluggable through Akka:
‣ Protobuf
‣ Kryo
‣ Avro
‣ Your own

42. Persistent View
Persistent
Actor
journal
Persistent
View
Persistent
View
Views poll the journal
‣ Eventually consistent
‣ Polling configurable
‣ Actor may be inactive
‣ Views track single
persistence-id
‣ Views can have own
snapshots
snapshot store
other
datastore

43. Persistent View
!
"The Database is a cache
of a subset of the log"
- Pat Helland
Persistent
Actor
journal
Persistent
View
Persistent
View
snapshot store
other
datastore

44. Persistent View
!
case object ComplexQuery
class CounterView extends PersistentView {
override def persistenceId: String = "counter"
override def viewId: String = "counter-view"
var queryState = 0
def receive: Receive = {
case Incremented if isPersistent => {
queryState = someVeryComplicatedCalculation(queryState)
// Or update a document/graph/relational database
}
case ComplexQuery => {
sender() ! queryState;
// Or perform specialized query on datastore
}
}
}

45. Persistent View
!
case object ComplexQuery
class CounterView extends PersistentView {
override def persistenceId: String = "counter"
override def viewId: String = "counter-view"
var queryState = 0
def receive: Receive = {
case Incremented if isPersistent => {
queryState = someVeryComplicatedCalculation(queryState)
// Or update a document/graph/relational database
}
case ComplexQuery => {
sender() ! queryState;
// Or perform specialized query on datastore
}
}
}

46. Persistent View
!
case object ComplexQuery
class CounterView extends PersistentView {
override def persistenceId: String = "counter"
override def viewId: String = "counter-view"
var queryState = 0
def receive: Receive = {
case Incremented if isPersistent => {
queryState = someVeryComplicatedCalculation(queryState)
// Or update a document/graph/relational database
}
case ComplexQuery => {
sender() ! queryState;
// Or perform specialized query on datastore
}
}
}

47. Sell concert tickets
ConcertActor
!
!
price
availableTickets
startTime
salesRecords
!
Commands:
CreateConcert
BuyTickets
ChangePrice
AddCapacity
journal
ConcertHistoryView
!
!
!
!
60
45
30
15
0
100
50
0
$50 $75 $100
code @ bit.ly/akka-es

48. Scaling out: Akka Cluster
Single writer: persistent actor must be singleton, views may be anywhere
Cluster node Cluster node Cluster node
Persistent
Persistent
Actor
id = "1"
Actor
id = "1"
Persistent
Actor
id = "1"
Persistent
Actor
id = "2"
Persistent
Actor
id = "1"
Persistent
Actor
id = "3"
distributed journal

49. Scaling out: Akka Cluster
Single writer: persistent actor must be singleton, views may be anywhere
How are persistent actors distributed over cluster?
Cluster node Cluster node Cluster node
Persistent
Persistent
Actor
id = "1"
Actor
id = "1"
Persistent
Actor
id = "1"
Persistent
Actor
id = "2"
Persistent
Actor
id = "1"
Persistent
Actor
id = "3"
distributed journal

50. Scaling out: Akka Cluster
Sharding: Coordinator assigns ShardRegions to nodes (consistent hashing)
Actors in shard can be activated/passivated, rebalanced
Cluster node Cluster node Cluster node
Persistent
Persistent
Actor
id = "1"
Actor
id = "1"
Persistent
Actor
id = "1"
Persistent
Actor
id = "2"
Persistent
Actor
id = "1"
Persistent
Actor
id = "3"
Shard
Region
Shard
Region
Shard
Region
Coordinator
distributed journal

51. Scaling out: Sharding
val idExtractor: ShardRegion.IdExtractor = {
case cmd: Command => (cmd.concertId, cmd)
}
IdExtractor allows ShardRegion
to route commands to actors

52. Scaling out: Sharding
val idExtractor: ShardRegion.IdExtractor = {
case cmd: Command => (cmd.concertId, cmd)
}
IdExtractor allows ShardRegion
to route commands to actors
val shardResolver: ShardRegion.ShardResolver =
msg => msg match {
case cmd: Command => hash(cmd.concert)
}
ShardResolver assigns new
actors to shards

53. Scaling out: Sharding
val idExtractor: ShardRegion.IdExtractor = {
case cmd: Command => (cmd.concertId, cmd)
}
IdExtractor allows ShardRegion
to route commands to actors
val shardResolver: ShardRegion.ShardResolver =
msg => msg match {
case cmd: Command => hash(cmd.concert)
}
ShardResolver assigns new
actors to shards
ClusterSharding(system).start(
typeName = "Concert",
entryProps = Some(ConcertActor.props()),
idExtractor = ConcertActor.idExtractor,
shardResolver = ConcertActor.shardResolver)
Initialize ClusterSharding
extension

54. Scaling out: Sharding
val idExtractor: ShardRegion.IdExtractor = {
case cmd: Command => (cmd.concertId, cmd)
}
IdExtractor allows ShardRegion
to route commands to actors
val shardResolver: ShardRegion.ShardResolver =
msg => msg match {
case cmd: Command => hash(cmd.concert)
}
ShardResolver assigns new
actors to shards
ClusterSharding(system).start(
typeName = "Concert",
entryProps = Some(ConcertActor.props()),
idExtractor = ConcertActor.idExtractor,
shardResolver = ConcertActor.shardResolver)
Initialize ClusterSharding
extension
val concertRegion: ActorRef = ClusterSharding(system).shardRegion("Concert")
concertRegion ! BuyTickets(concertId = 123, user = "Sander", quantity = 1)

55. Design for event-sourcing

56. DDD+CQRS+ES
DDD: Domain Driven Design
Fully consistent Fully consistent
Aggregate
!
Aggregate
!
Eventual
Consistency
Root entity
enteitnytity entity
Root entity
entity entity

57. DDD+CQRS+ES
DDD: Domain Driven Design
Fully consistent Fully consistent
Aggregate
!
Aggregate
!
Eventual
Consistency
Root entity
enteitnytity entity
Root entity
entity entity
Persistent
Actor
Persistent
Actor
Message
passing

58. DDD+CQRS+ES
DDD: Domain Driven Design
Fully consistent Fully consistent
Aggregate
!
Aggregate
!
Eventual
Consistency
Akka Persistence is not a DDD/CQRS framework
But it comes awfully close
Root entity
enteitnytity entity
Root entity
entity entity
Persistent
Actor
Persistent
Actor
Message
passing

59. Designing aggregates
Focus on events
Structural representation(s) follow ERD

60. Designing aggregates
Focus on events
Structural representation(s) follow ERD
Size matters. Faster replay, less write contention
Don't store derived info (use views)

61. Designing aggregates
Focus on events
Structural representation(s) follow ERD
Size matters. Faster replay, less write contention
Don't store derived info (use views)
With CQRS read-your-writes is not the default
When you need this, model it in a single Aggregate

62. Between aggregates
‣ Send commands to other aggregates by id
!
‣ No distributed transactions
‣ Use business level ack/nacks
‣ SendInvoice -> InvoiceSent/InvoiceCouldNotBeSent
‣ Compensating actions for failure
‣ No guaranteed delivery
‣ Alternative: AtLeastOnceDelivery

63. Between systems
System integration through event-sourcing
topic 1 topic 2 derived
Kafka
Application
Akka
Persistence
Spark
Streaming
External
Application

64. Designing commands
‣ Self-contained
‣ Unit of atomic change
‣ Granularity and intent

65. Designing commands
‣ Self-contained
‣ Unit of atomic change
‣ Granularity and intent
UpdateAddress
street = ...
city = ... vs
ChangeStreet
street = ...
ChangeCity
street = ...

66. Designing commands
‣ Self-contained
‣ Unit of atomic change
‣ Granularity and intent
UpdateAddress
street = ...
city = ... vs
ChangeStreet
street = ...
ChangeCity
street = ...
Move
street = ...
city = ... vs

67. Designing events
CreateConcert

68. Designing events
CreateConcert ConcertCreated
Past tense
Irrefutable

69. Designing events
CreateConcert ConcertCreated
Past tense
Irrefutable
TicketsBought
newCapacity = 99

70. Designing events
CreateConcert ConcertCreated
Past tense
Irrefutable
TicketsBought
newCapacity = 99
TicketsBought
quantity = 1
Delta-based
No derived info

71. Designing events
CreateConcert
ConcertCreated
ConcertCreated
Past tense
Irrefutable
TicketsBought
newCapacity = 99
TicketsBought
quantity = 1
Delta-based
No derived info

72. Designing events
CreateConcert
ConcertCreated ConcertCreated
who/when/..
Add metadata
to all events
ConcertCreated
Past tense
Irrefutable
TicketsBought
newCapacity = 99
TicketsBought
quantity = 1
Delta-based
No derived info

73. Versioning
Actor logic:
v1 and v2
event v2
event v2
event v1
event v1

74. Versioning
Actor logic:
v1 and v2
event v2
event v2
event v1
event v1
Actor logic:
v2
event v2
event v2
event v2
event v1
event v2
event v1

75. Versioning
Actor logic:
v1 and v2
event v2
event v2
event v1
event v1
Actor logic:
v2
event v2
event v2
event v2
event v1
event v2
event v1
Actor logic:
v2
event v2
event v2
snapshot
event v1
event v1

76. Versioning
Actor logic:
v1 and v2
event v2
event v2
event v1
event v1
Actor logic:
v2
event v2
event v2
event v2
event v1
event v2
event v1
Actor logic:
v2
event v2
event v2
snapshot
event v1
event v1
‣ Be backwards
compatible
‣ Avro/Protobuf
‣ Serializer can do
translation
!
‣ Snapshot
versioning:
harder

77. In conclusion
Event-sourcing is...
Powerful
but unfamiliar

78. In conclusion
Event-sourcing is...
Powerful
but unfamiliar
Combines well with
UI/Client
Command
Model
Journal
Query
Model
DaDtaastatosrtoere Datastore
Events
DDD/CQRS

79. In conclusion
Event-sourcing is...
Powerful
but unfamiliar
Combines well with
UI/Client
Command
Model
Journal
Query
Model
DaDtaastatosrtoere Datastore
Events
DDD/CQRS
Not a

80. In conclusion
Akka Actors & Akka Persistence
A good fit for
event-sourcing
PersistentActor
actor-id
!
!
state
!
async message
send
(command)
event
event
journal (actor-id)
Experimental, view
improvements needed

81. Thank you.
!
code @ bit.ly/akka-es
@Sander_Mak
Luminis Technologies