In 2009, I first learned about Event Sourcing and Command Query Responsibility Seggregation (CQRS) at a training Greg Young gave in Utrecht, The Netherlands. I remember being awed by the scalability and architectural simplicity those styles provided. However, I also remembered the technical complexity that comes with it. In 2012, I was in charge of transitioning a CQRS-based system to Event Sourcing. I knew it would be non-trivial, but boy was I in for a surprise. So, over the last four years, I've experienced first-hand how a large group of developers had to deal with the transition. It's a brilliant solution for high-performance or complex business systems, but you need to be aware that this also introduces challenges most people don't tell you about. In this talk, I'd like to share you some of the most powerful benefits of ES, but also show you the flipside of the coin and cover some of the smaller and bigger challenges you'll run into it. Again, I love it and would apply it again without any doubt, but I really want you to understand the trade-offs before you jump on the Event Sourcing train.
7. Events
Transaction 6
Transaction 5
Transaction 4
Transaction 3
Transaction 2
Transaction 1
Temporal
Projector
Read
Store
Time
Projected until
this point
Immutable, thus
auditable and SOX
compliance
Dennis Doomen | @ddoomen | The Continuous Improver
9. Dennis (v4)
Persisted State Changes
Dennis (v3)
Role Granted
PasswordChangedEvent
Dennis (v4)
Dennis (v3)
User Created
Role Granted
Phone Number Added
Password Changed
Dennis (v5)
Role Revoked
Dennis (v6)
Role Granted
Time
Dennis (v7)
Password Changed
Dennis Doomen | @ddoomen | The Continuous Improver
10. Event Store
Projector
Application Application
Network Load Balancer
Event Store
Version 1 Version 2
events
Projection ProjectorProjection
bring off-line
Returns HTTP 503
(Service Unavailable)
Dennis Doomen | @ddoomen | The Continuous Improver
Returns HTTP 503
(Service Unavailable)
13. Dennis Doomen | @ddoomen | The Continuous Improver
Application
Domain
NoSQL / RDBMS
OR/M / DAL
Web UI, HTTP API,
etc
Lucene Index
Document Projector
Web UI, HTTP
API, etc
Web UI, HTTP API, etc
Domain
Commands
Events
Event StoreProjections
Projectors
Uses Event
Sourcing
Uses traditional
CRUD
architecture
Indexing-based
architecture
Subcribe
to
webhooks
Coarse-
grained
HTTP
requests.
Bus
Subscribe
Publish coarse-
grained event
14. Customer #123
Correct Customer
Shipping Address
Command
Customer Shipping
Address Corrected
Event
Event Store
Treat as a
message, not a
type
Order #456
Correct Shipping Address
Command Handler
Application
Customer Shipping Address
Corrected Handler
Order Redirected
Event.
Identity !=
natural key
Identify
partition key
Primitive types
only
Avoid terms like
Create, Update,
Delete, Change.
Avoid property
change events
Don’t use as
inter-domain
contracts
Don’t expose
outside the
domain
Dennis Doomen | @ddoomen | The Continuous Improver
15. Customer #123
Correct Customer
Shipping Address
Command
Customer Shipping
Address Corrected
Event
Event Store
Order #456
Correct Shipping Address
Command Handler
Application
Order Redirected
Event
Dennis Doomen | @ddoomen | The Continuous Improver
16. public class Order
{
private Status status;
public void Cancel()
{
if (status == Status.InProgress)
{
Apply(new OrderCanceledEvent());
}
}
private void When(OrderCanceledEvent e)
{
status = Status.Canceled;
}
}
In single classes
Dennis Doomen | @ddoomen | The Continuous Improver
17. public partial class Order
{
private Status status;
public void Cancel()
{
if (status == Status.InProgress)
{
Apply(new OrderCanceledEvent());
}
}
}
In partial classes
public partial class Order
{
private void When(OrderCanceledEvent e)
{
status = Status.Canceled;
}
}
Dennis Doomen | @ddoomen | The Continuous Improver
18. public class Order
{
private OrderState state;
public void Cancel()
{
if (state.Status == Status.InProgress)
{
state.Apply(new OrderCanceledEvent());
}
}
}
In separate classes
internal class OrderState
{
public Status Status { get; set; }
private void When(OrderCanceledEvent e)
{
Status = Status.Canceled;
}
}
Dennis Doomen | @ddoomen | The Continuous Improver
19. Risk Assessment Level
Changed
Risk Assessment Team
Member Removed
Risk Assessment Team
Member Removed
Risk Assessment
Level Demoted
Versus
Dennis Doomen | @ddoomen | The Continuous Improver
21. Command Service
Some Command
Handler
Customer #123
Event Store
Customer Created
Event
Get<Customer>(“123”)
Customer
Created Event
Converter
Customer Enrolled
Event
May split or
merge events
Can also run as
part of migration
May change the
identity
Unaffected Events
Dennis Doomen | @ddoomen | The Continuous Improver
22.
23. • Column constraints (e.g. data truncation)
• Changes in data invariants (null vs non-
null in event versions)
• Unexpected projection dependencies.
24. • Causing duplicate child records
• Causing large event streams
• Incorrect caching strategy (e.g. on
lookups)
• Identity case-sensitivity
• Incomplete SQL-backed event store
reads.
27. Transaction
Event Event
Transaction
Event Event
Transaction
Event Event
RDBMS
INSERT/UPDATE … WHERE …
Transaction
Event Event
Transaction
Event Event
Transaction
Event Event
RDBMS
INSERT/UPDATE … WHERE …
INSERT/UPDATE … WHERE …
INSERT/UPDATE … WHERE …
Unit of Work
Dennis Doomen | @ddoomen | The Continuous Improver
28. Document #1
Created Event
Event Store
Graph
Projector
Document #1
Closed Event
(other events)
Projector with
active
projections
Archiving
Projector
Start archiving
Document #1
Marked As
Archivable Event
Mark all events
as archivable
Tracks dependencies
between documents
Deletes
projections
related to
Document #1
Can skip all
archivable events
during next
rebuild.
Dennis Doomen | @ddoomen | The Continuous Improver
29. Event Store
Lucene
Projector
Document #1
Marked As
Archivable Event
Allows projectors
to clean up
Lucene Index
Take snapshot
Purge events
Tombstone
$tombstone
stream
Application
Document
Projector
Search
Tracks deleted streams
for future references
Stream Tombstoned
Event
Search
Dennis Doomen | @ddoomen | The Continuous Improver
34. • The Good, The Bad and the Ugly of Event
Sourcing
https://www.continuousimprover.com/search/label/event%20sourcing
• Effective Aggregate Design (Vaughn Vernon)
http://dddcommunity.org/library/vernon_2011/
• Liquid Projections
https://www.liquidprojections.net
• Distributed Event Sourcing (Slides)
https://www.slideshare.net/dennisdoomen/building-occasionally-
connected-applications-using-event-sourcing
• Data schema changes in an event sourced
system (paper)
https://files.movereem.nl/2017saner-eventsourcing.pdf