Event Sourcing, from ancient Mesopotamia to modern software
Patterns are all around us, and in software, the design patterns we choose can give us the power of the ancient kings and queens. Event Sourcing is based on the idea that storing events as they happen is better than storing the current state. Imagine having the power to rewind and replay events, to debug, to generate reporting, to go back to ancient history...
We’ll get up close and personal with Command Query Responsibility Segregation (CQRS), which gives us the bonus of scalability out of the box, we’ll be designing and architecting applications at scale, just like the grand cities of the past, in no time at all!
2. Event Sourcing
Event sourcing workflow
> Audit trails, Sumerian clay tablets and Blockchain
> Advantages over storing state
> Simple form of CQRS, contrasting it to a monolith
> Command side and read side
> How event sourcing fits in
Command Query Responsibility Segregation
> Shopping catalogue example
3. What is Domain Driven Design?
“In order to create good software, you have to
know what that software is all about. You cannot
create a banking software system unless you have
a good understanding of what banking is all about,
one must understand the domain of banking”
– Eric Evans
4. What is Event Sourcing?
Use an append-only store to record the full
series of events that describe actions taken on
data in a domain, rather than storing just the
current state.
5. Event Sourcing Examples
Sumerian clay
tablets
Transactional systems (i.e. Stock)
Accounting ledger
Database transaction log
Source control systems
7. Scenario
You are the team lead of a small team of developers at a
start-up company building an Amazon competitor. Your
product is gaining traction and you expect a lot more users
in the next few months.
• The customers are complaining of slow response times &
crashes during peak times.
• Come up with a plan to make your application more
performant.
• Think about your deployment and scaling strategy.
8. We Have a Monolith
A monolithic application has a single code
base and a single build system which
builds and deploys the entire application
(and dependencies). Modules may be
divided as business features or technical
features.
9. Scenario Continued
Monolithic Application
Big and complex code base
Hard onboarding process for new
developers
Hidden coupling and complexity
Single build pipeline
Release cycle is long
Scaling up is very expensive
Performance issues
Application is offline when
deploying new version
Symptoms A solution path
• CRM
• Online Store
• Supplier
• Stock
• Accounts
• Reports
Identify which parts of the system
are used the most
Logically separate those parts
Deploying the separated parts
independently
Scale the required parts
independently
10. Thought Experiment
You are managing a coffee shop and have 2
employees who work alternate shifts.
• Anna who is a great barista but is slow at
operating the cash register.
• Sam who doesn’t make great coffee but is fast at
operating the register.
• More people are coming to the store and you
want a strategy to reduce wait times for
customers.
11. CQRS to the Rescue
“CQRS stands for Command Query
Responsibility Segregation. At its heart is the
notion that you can use a different model to
update information than the model you use to
read information”
– Martin Fowler
13. Updated Scenario
You have successfully managed to logically separate your
Online-Store module to writes (commands) and reads
(queries).
• It’s become clear that “data models” used for write vs
read are different.
• Investigate whether it is possible to deploy those as
separate applications and have separate data models/
storage mechanisms.
14. CQRS Solution Evolved
What if we build and deploy them
independently?
ShoppingCatalogueReadService
+ Search(criteria): Decimal
+ Browse(category): items[]
ShoppingCatalogueWriteService
+ AddItem(itemInfo): void
+ UpdateItem(item): void
+ RemoveItem(item): void
+ SetDiscount(item): void
We get a solution that has…
Independent scalability
Manageable codebases
But what about…
Shared storage?
Transactions + Locks?
Dependency availability?
15.
16. CQRS Solution Using Event Sourcing
ShoppingCatalogueReadService
+ Search(criteria): Decimal
+ Browse(category): items[]
ShoppingCatalogueWriteService
+ AddItem(itemInfo): void
+ UpdateItem(item): void
+ RemoveItem(item): void
+ SetDiscount(item): void
Domain Events
Event Database
(Append Only)
Read Model Database
Subscribers
listening to events
Queue
Storing an event
publishes it to the
queue
Query
Command
Update Read
Model
17. A Bit More About
Event Sourcing And CQRS
Event Sourcing is naturally CQRS
But… not all CQRS solutions are event sourced
CQRS is not an end goal
Event Sourcing is a natural functional model
State GetNewState(State oldState, Event lastEvent) {
return (oldState + lastEvent);
}
18. Key Benefits of Event Sourcing
• Ability to track all changes done to the domain
Rebuild your state to a point in time
• Less coupling between read and write sides
Reduction of impedance mismatch
Many world views
Independent scalability
• Easier to debug
19. Some Pitfalls When Using Event Sourcing
• Unfamiliar concept to many
• External systems / dependencies
• Eventual consistency
• Event schema changes
20. What Did We Learn?
• Event Sourcing And Command Query
Responsibility Segregation
• What they are?
• When to apply them?
• Key advantages
• Disadvantages
• Basic event sourcing workflow
21. Want More?
• My blog: dasith.me
• ES ‘reference framework’ for .NET: NEventLite
(Search: NEventLite on NuGet or GitHub)
• Domain Driven Design – Eric Evans
• Greg Young on YouTube
• DDD/CQRS Google Group
• ES database: eventstore.org
CQRS and ES are concepts that have roots in DDD.
The complexity at the heart of any software system isn’t in the architecture, user interface or in feature set.
The complexity resides in understanding the domain.
DDD introduces a common set of patterns and principles to model your domain in a way that reduces this complexity.
In its most simple form, event sourcing is nothing but storing all the domain events so your state changes are represented as a series of actions.
Auditability is the most key advantage event sourcing brings
The bank balance example
Blockchain based systems are “naturally event sourced”.
Solves the audit and trust problems in a decentralized way.
Easier to test and debug.
Don’t have to deal with the availability of dependencies.
Anything that changes the state of our system is considered a command while a Query is something that is used to just read state.
What it is : A powerful tool that solves a very specific set of problems.
What it’s not : A silver bullet for every problem domain.
There is no perfect way to solve a problem. It always depends.
CQRS is a stepping stone to an ES system
Command Query Separation (CQRS is based on this) first introduced by Bertrand Meyer in 1988)
Functionally ES state is a left fold of previous state
ES is a application level architecture. If you use ES as a top level architecture like SOA you will end up with an event sourced monolith.