Simon Aubury, Thoughtworks, Principal Data Engineer Building systems around an event-driven architecture is a powerful pattern for creating awesome data intensive applications. Apache Kafka simplifies scalability and provides an event-driven backbone for service architectures. But what can go wrong? Let me share some of my own blunders and lessons learnt in building event driven architecture so you don’t have to repeat my mistakes. https://www.meetup.com/KafkaBayArea/events/273453552/

1. @SimonAubury
Event Driven
Architecture
Mistakes – I’ve made a few …
linkedin.com/in/simonaubury
@SimonAubury
A tale of confessions from
Simon Aubury
https://en.wikipedia.org/wiki/Montparnasse_derailment

2. @SimonAubury
Why am I here?
2
I am Simon Aubury
Principal Data Engineer @ ThoughtWorks

3. @SimonAubury
“
Event-driven architecture (EDA) is a
software architecture paradigm
promoting the production, detection,
consumption of, and reaction to events.
https://en.wikipedia.org/wiki/Event-driven_architecture

4. @SimonAubury
Let’s help a younger
me
4

5. @SimonAubury
Event Driven Architecture - a lot has
happened in 25 years
5

6. @SimonAubury
DDD - existing practices
◉ Problem modelling
○ Contexts - delineate boundary of consistency
◉ Separate our business logic from other
application concerns
◉ Reduce complexity
○ More effective software delivery
◉ Communicate better / A common language
6

7. @SimonAubury
Lesson: it’s hard work
without DDD
DDD gives us the tools to
define our bounded contexts,
which give us our services.
Modelling the domain helps us
identify the events that are
important to the domain.
7

8. @SimonAubury
There is a process to find the events that matter
◉ Use everyone to identify the events that matter
◉ Understand the systems
◉ Start with broad categories
8

9. @SimonAubury
Lesson: the rules are different
within vs across boundaries
Favour asynchrony and eventual
consistency at context boundaries,
embrace the productive coupling of
synchronous within the boundary
9

10. @SimonAubury
Many Meanings of Event Driven
◉ Event Notification
◉ Event-Carried State Transfer
◉ Event-Sourcing
◉ CQRS
10
https://martinfowler.com/articles/201701-event-driven.html

11. @SimonAubury
Lesson: don’t over engineer
Pick an event driven approach – and
be consistent and simple.
Don’t over-engineer an eventing
solution using Event-Sourcing/CQRS
11

12. @SimonAubury
Lesson: know your events
Modelling - discovery & integration …
Use simple language; solicit everyone's
input.
Develop your system inside out, focus
on the domain
12

13. @SimonAubury
Messages are not events
13
https://ibm-cloud-architecture.github.io/refarch-eda/concepts/events-versus-messages/
Messages
Events

14. @SimonAubury
Thinking of events and boundaries
14
https://sarahtaraporewalla.com/architecture/Event-Driven-Architecture-Terminology
Kitchen Waiter Maitre d’ Cashier
Stock
Room
Table 26 orders fish

15. @SimonAubury
Table
26
Too obsessed with microservices
15
Kitchen
Waiter
Maitre d’
Cashier
Stock
Room
Fish Fish for table 26
Fish
Table 26 orders fish

16. @SimonAubury
Understand event boundaries
16
https://sarahtaraporewalla.com/architecture/Event-Driven-Architecture-Terminology
Kitchen Waiter Maitre d’ Cashier
Event Stream
Table 26
Stock
Room
FishFish for table 26Fish for table 26 Fish for table 26

17. @SimonAubury
Choreography vs. orchestration
Which system decides that an action
should be taken?
◉ Orchestration – a manager tells
◉ Choreography (event driven) - a
system takes independent action
17
https://solace.com/blog/microservices-
choreography-vs-orchestration/

18. @SimonAubury
Lesson: event modelling - it really
happened
An event represents a fact, something
happened; it is immutable and
therefore changes how we think
about our domain model (boundary
between services).
18

19. @SimonAubury
Lesson: Beware the passive
aggressive events
An event shouldn’t be used as a
passive-aggressive command.
It’s a “bad smell” if a source system
expects the recipient to carry out an
action yet styles the message as an
event instead.
19

20. @SimonAubury
Now, some Kafka
stuff
20

21. @SimonAubury
Um, so why a Kafka?
Event-first thinking challenges
◉ Observable, trusted; transactional;
scalable
◉ Processing, view projection, windowing
◉ Scale/fan/map out, fan in/collect
chained transformed events
21
https://www.confluent.io/blog/journey-to-event-
driven-part-1-why-event-first-thinking-changes-
everything/

22. @SimonAubury
Why the 💕 Kafka in EDA?
◉ Build apps on top of events - easy to out reporting
and ML later (rather than a bunch of ETL)
◉ Kafka plus functional programming plus
immutability plus polyglot persistence
22

23. @SimonAubury
Event first thinking
◉ Capture facts & behaviour
◉ Represent the real world
◉ Model use cases of how we think
◉ Repeatability & scaling
◉ Common language
23
https://www.confluent.io/blog/journey-to-event-
driven-part-1-why-event-first-thinking-changes-
everything/

24. @SimonAubury
Lesson: Event must have’s
◉ Name – past tense
◉ Correlation ID
◉ Event production time
◉ Originating system
◉ Event creation system (may be different)
◉ A payload of stuff
24

25. @SimonAubury
Plan for schema evolution
Support change - data domains
need to evolve at their own
rate … without breaking
consumers.
TL;DR - Use schema registry
25

26. @SimonAubury
Observability
26
After: add cache for field metadata
5,500 records / sec / table
Before: still slow transform
200 records / sec / table

27. @SimonAubury
Horizontal scaling … scales
horizontally
27
To scale out, you simply start another instance of your
stream processing application, e.g. on another machine. The
instances of your application will become aware of each other
and automatically begin to share the processing work.
https://www.confluent.io/blog/elastic-scaling-in-kafka-streams/

28. @SimonAubury
It went wrong – dead letter queue
Avoid DLQ if you can!
Managing a DLQ is highly dependent
on how crucial is that data, how
difficult it is to source it again and who
owns the source of truth.
28

29. @SimonAubury
Warning: opinions ahead …
◉ A bounded context == a Kafka topic
◉ You don’t need another microservice
framework
◉ You need schemas; mastered by the source
◉ Avoid dead letter queues
◉ Beware of CV-DD
◉ You’ll get things wrong – optimize for change
29

30. @SimonAubury
What did we learn?
30

31. @SimonAubury
Lesson: start .. now
Event Driven Architecture adoption should start now.
Starting the first leads to the next transformational
opportunity. Shift towards EDA is driven by increasing
demands and heightened expectations and system
modernisation
Make your own mistakes …
31

32. @SimonAubury
Any questions ?
Thanks!
32
@SimonAubury
linkedin.com/in/simonaubury
Presentation template by SlidesCarnival