Presented in dotNetZone.gr Meetup #4. Goes through basic DDD concepts, recaps on CQRS, discusses when async messaging should be implemented and shows how with MassTransit.

1. ASYNC MESSAGING in CQRS
PART 1: MASSTRANSIT
DDD GREECE / dotNETZone.gr

2. DDD

3. DDD
TYPICAL DDD SYSTEM LAYERS

4. DDD
DOMAIN MODEL
▸ exists in domain layer
▸ reflects structured knowledge about the domain
▸ ubiquitous language
▸ agreed/shared with domain experts
▸ typically implemented as a SOLID - OO model
▸ “technology” free !

5. DDD
DOMAIN STRUCTURE 1/2
▸ Entities
▸ id
▸ constant throughout its lifecycle
▸ mutable
▸ Value Objects
▸ no id
▸ immutable
▸ diff state = diff object
▸ offloading logic from entities

6. DDD
DOMAIN STRUCTURE 2/2
▸ Domain Services
▸ stateless
▸ behaviour that cannot be modelled with Entities, Value Objects
▸ Domain Events
▸ immutable
▸ timestamped
▸ ref to entities

7. DDD
AGGREGATE
▸ object graph of entities and value objects
▸ single entity is its root
▸ global identity
▸ external entities ref ONLY to the aggregate root
▸ root enforces invariants
▸ consistency boundary

8. CARGO TRACKER

9. CARGO TRACKER
ABOUT
▸ Eric Evans “Big Blue Book”
▸ Published 2003
▸ Imaginary Shipping Company

10. CARGO TRACKER
KEY FUNCTIONALITY 1/2
▸ Book a Cargo
▸ Delivery Spec (Origin, Destination + Arrival DeadLine)
▸ (Re-)Assign an Itinerary
▸ matches Delivery Spec
▸ derives from Legs of 1+ Vessel Voyages

11. CARGO TRACKER
KEY FUNCTIONALITY 2/2
▸ Register a Handling Event
▸ Load, Unload etc
▸ Report the:
▸ Transport Status (In Port, On Board Vessel …)
▸ Routing Status (Not Routed, Routed, …)
▸ Next Expected Handling Event
▸ Delivery History

12. CARGO TRACKER
DOMAIN MODEL - CARGO AGGREGATE 1/3
▸ Cargo (Root)
▸ HandlingEvent
▸ Root’s Methods:
▸ New(DeliverySpec)
▸ AssignItinerary(Itinerary)
▸ ChangeDelivery(DeliverySpec)
▸ RegisterHandlingEvent(HandlingEvent)

13. CARGO TRACKER
DOMAIN MODEL - CARGO AGGREGATE 1/3
▸ Cargo (Root)
▸ HandlingEvent
▸ Root’s Methods:
▸ New(DeliverySpec)
▸ AssignItinerary(Itinerary)
▸ ChangeDelivery(DeliverySpec)
▸ RegisterHandlingEvent(HandlingEvent)

14. CARGO TRACKER
DOMAIN MODEL - CARGO AGGREGATE 1/3
▸ Cargo (Root)
▸ HandlingEvent
▸ Root’s Methods:
▸ New(DeliverySpec)
▸ AssignItinerary(Itinerary)
▸ ChangeDelivery(DeliverySpec)
▸ RegisterHandlingEvent(HandlingEvent)

15. CARGO TRACKER
DOMAIN MODEL - CARGO AGGREGATE 1/3
▸ Cargo (Root)
▸ HandlingEvent
▸ Root’s Methods:
▸ New(DeliverySpec)
▸ AssignItinerary(Itinerary)
▸ ChangeDelivery(DeliverySpec)
▸ RegisterHandlingEvent(HandlingEvent)
WRITE /
COMMANDS

16. CARGO TRACKER
DOMAIN MODEL - CARGO AGGREGATE 2/3
▸ Root’s State:
▸ TransportStatus
▸ RoutingStatus
▸ NextExpected: HandlingEvent
▸ DeliveryHistory
▸ Collection of HandlingEvents

17. CARGO TRACKER
DOMAIN MODEL - CARGO AGGREGATE 2/3
▸ Root’s State:
▸ TransportStatus
▸ RoutingStatus
▸ NextExpected: HandlingEvent
▸ DeliveryHistory
▸ Collection of HandlingEvents
READ / QUERIES

18. CARGO TRACKER
REFACTORING - HANDLING EVENT

19. CARGO TRACKER
REFACTORING - HANDLING EVENT

20. CARGO TRACKER
REFACTORING - DELIVERY HISTORY
▸ “leaves Delivery History with no persistent state”
▸ “no real need to keep it around”
▸ “we could derive Delivery History itself whenever it is
needed to answer some question”

21. CARGO TRACKER
REFACTORING - HANDLING EVENT AGGREGATE
▸ HandlingEvent Aggregate:
▸ no domain logic (anemic)
▸ only repository + event class
▸ repository servicing query needs of other aggregates

22. CQRS RECAP

23. CQRS RECAP
ABOUT
▸ Command-Query Responsibility Segregation
▸ keep separate read (query) and write (command) models
▸ DeliveryHistory: Read/Query
▸ Cargo: Write/Command
▸ practically SRP

24. CQRS RECAP
COMMANDS
▸ BookNewCargo, AssignItinerary etc.
▸ App Layer
▸ CommandHandler
▸ BookNewCargoHandler, AssignItineraryHandler etc.
▸ practically an App Service
▸ SRP

25. CQRS RECAP
EVENTS 1/2
▸ NewCargoBooked, DeliveryChanged etc.
▸ Domain Layer
▸ Emitted by A/R after Command Applying
▸ Dispatched by Command Handler

26. CQRS RECAP
EVENTS 2/2
▸ EventHandler
▸ e.g. NewCargoBookedHandler, DeliveryChangedHandler
▸ App Layer
▸ Typical Handlers:
▸ Aggregate / Bounded Context Integration
▸ wired to another Command Handler
▸ Read Model Update

27. CQRS RECAP
READ MODEL UPDATE 1/3
*
* * * * *
*
* *
* * *
* *
* *
* *
* * *
* * * * *
* * * * * *

28. CQRS RECAP
READ MODEL UPDATE 2/3
1 M
1
M
1
M
1M

29. CQRS RECAP
READ MODEL UPDATE 3/3
*
* * * * *
*
* *
* * *
* *
* *
* *
* * *
* * * * *
* * * * * *

30. CQRS RECAP
MULTIPLE EVENT HANDLERS
*
* * * * *
*
* *
* * *
* *
* *
* *
* * *
* * * * *
* * * * * *
* *
* *
* * *
* * *
* *
*

31. CQRS RECAP
MISCONCEPTIONS 1/2
▸ needs two datastores/databases
▸ must be NoSQL
▸ requires async processing
▸ needs async messaging/queueing framework
▸ consistency only eventual

32. CQRS RECAP
MISCONCEPTIONS 1/2
▸ needs two datastores/databases
▸ must be NoSQL
▸ requires async processing
▸ needs async messaging/queueing framework
▸ consistency only eventual
TRADE OFFS

33. CQRS RECAP
MISCONCEPTIONS 2/2
▸ needs a CQRS framework
▸ works only with event sourcing
▸ applied end-to-end
▸ e.g. CRUD

34. CQRS RECAP
SYNC PROCESSING
▸ single process (e.g. Web App)
▸ decreases availability
▸ if process is UI affects UX
▸ in-process event dispatcher for decoupling
▸ rely on built-in framework facilities
▸ immediate consistency

35. CQRS RECAP
ASYNC PROCESSING
▸ multiple processes (e.g. WebApp and WorkerApp)
▸ higher availability, scaling out
▸ consistency eventual
▸ typically implemented with async messaging
▸ message bus
▸ in-process event dispatcher allows for applying selectively

36. CQRS RECAP
TYPICAL SYSTEM with CQRS + ASYNC PROCESSING
/ *
-BBB
3
B BB #
/ A A *
3 BB # 3 A
A BC A B A C A C B BC C
B 2 3
AB BCB BC C CB
B B A B B CB
C
B BB #
B 3B C 2 3 A B C A 3 3
. 3 - 3
C
2 B BC C#
/ A A * (
3 A A
A 2 BB C
3 AB#
C A 2 BB# C
B B2A A#
/ A A * )
2 ACB CB
C A 3 3
2CB 3 AB BCB
/A C 3

37. ASYNC
MESSAGING

38. ASYNC MESSAGING
MESSAGE BUS 1/2
▸ frameworks: NServiceBus, MassTransit, Rebus etc.
▸ transports: RabbitMQ, Azure Service Bus etc.
▸ at least-once delivery
▸ FIFO
▸ reliable messaging
▸ automatic transport configuration setup
▸ pub/sub, send/receive

39. ASYNC MESSAGING
MESSAGE BUS 1/2
▸ frameworks: NServiceBus, MassTransit, Rebus etc.
▸ transports: RabbitMQ, Azure Service Bus etc.
▸ at least-once delivery
▸ FIFO
▸ reliable messaging
▸ automatic transport configuration setup
▸ pub/sub, send/receive

40. ASYNC MESSAGING
ASYNC & RELIABLE MESSAGING
ALL=CA .KI AK
N ELDAK A AK AM
N L KE AK ( A AE AK
AM
N L KE AK )
1P D= CA -
NANA ( NANA )
1P D= CA . R
)
-/3IK1KKIK
(
A
-/3IK
=
LC A E AKQ EL 142-.41 ( # , 2B
LC EL IM - A Q MDA .KI AK =
1KKIK EL K=ELA E MDA A AK
LC A E AKQ EL 142-.41 ) # , 2B LC EL IM - A IK
= A Q MDA A AE AK MDA LC LM=QL E MDA NANA
LC A E AKQ A MI
A # EL - / ,
A AK EL I A =BMAK
CAMME C MDA -
LC A E AKQ EL
142-.41 # ,
NANAL = A
AKLELMA M = KILL
KI AK KALM=KML
- = EM # 1 - 41
AAEA

41. ASYNC MESSAGING
MESSAGE BUS 2/2
▸ connection management
▸ persistent sagas/process managers
▸ retrying policy on subscriber/receiver errors
▸ alarm service
▸ (de)serialization

42. ASYNC MESSAGING
CQRS & MESSAGING PATTERNS
Command Handler Receiver
Command Issuer Sender
Event Handler Subscriber
Event Emitter Publisher

43. ASYNC MESSAGING
COMMAND FLOW
1. Process A
A. Issue
B. Send
2. Process B
C. Receive
D. Handle - Create Events

44. ASYNC MESSAGING
EVENT FLOW
1. Process B
A. Emit
B. Publish
2. Process C
C. Subscribe
D. Handle

46. IMPLEMENTATION

47. IMPLEMENTATION
WARNING
ASYNC MESSAGING
!=
ASYNC/AWAIT

48. STEP 1:
COMMAND ISSUING/SENDING

49. IMPLEMENTATION
COMMAND ISSUING - (WEB)APP LAYER
▸ IssueAsync implemented with Bus.Send<T>

50. IMPLEMENTATION
COMMAND SENDING / BOOTSTRAPPING MASSTRANSIT
▸ bus is a Single Instance / AppDomain

51. IMPLEMENTATION
MASSTRANSIT / BUS.SEND<T>
▸ Send(er)/Receive(r) Pattern

52. IMPLEMENTATION
MASSTRANSIT / BUS.SEND<T>
▸ Send(er)/Receive(r) Pattern

53. IMPLEMENTATION
COMMAND INTEGRITY 1/2

54. IMPLEMENTATION
COMMAND INTEGRITY 1/2

55. IMPLEMENTATION
COMMAND INTEGRITY 2/2

56. IMPLEMENTATION
COMMAND INTEGRITY 2/2

57. IMPLEMENTATION
COMMAND ISSUING/SENDING SUMMARY
▸ Use Domain Structures in Command
▸ Command Integrity
▸ ICommandIssuer abstraction
▸ MassTransit implementation
▸ Converts Commands to Messages (TbC)
▸ Encapsulate in an App Service
▸ Validation

58. STEP 2:
COMMAND RECEIVING/
HANDLING

59. IMPLEMENTATION
COMMAND RECEIVING / BOOTSTRAPPING MASSTRANSIT

60. IMPLEMENTATION
COMMAND RECEIVING / BOOTSTRAPPING MASSTRANSIT

61. IMPLEMENTATION
COMMAND RECEIVING / BOOTSTRAPPING MASSTRANSIT

62. IMPLEMENTATION
COMMAND RECEIVING / BOOTSTRAPPING MASSTRANSIT

63. IMPLEMENTATION
COMMAND RECEIVING / BOOTSTRAPPING MASSTRANSIT

64. IMPLEMENTATION
MASSTRANSIT / CONSUMER<T>

65. IMPLEMENTATION
MASSTRANSIT / CONSUMER<T>

66. IMPLEMENTATION
MASSTRANSIT RETRING FLOW (SIMPLIFIED)
1. message deserialised
2. identity consumer from message type
A. if consumer identified
‣ run handler in try/catch
‣ on exception check retry policy
‣ either retry on ignore
B. if consumer is NOT identified (TbC)

67. IMPLEMENTATION
COMMAND HANDLING / EVENT DISPATCHING - APP LAYER

68. IMPLEMENTATION
COMMAND HANDLING / EVENT DISPATCHING - APP LAYER

69. IMPLEMENTATION
AGGREGATE EVENTS
▸ A/Rs should inherit from BaseAggregateRoot
▸ exposes Events collection

70. IMPLEMENTATION
AGGREGATE EVENTS
▸ A/Rs should inherit from BaseAggregateRoot
▸ exposes Events collection

71. IMPLEMENTATION
AGGREGATE EVENTS
▸ A/Rs should inherit from BaseAggregateRoot
▸ exposes Events collection

72. IMPLEMENTATION
EVENT DISPATCHER 1/2
▸ wires SYNChronously (in-process) command with event handlers
▸ is the “Emitter”
▸ wired handlers can be
▸ an actual, in-process one (e.g. read model update)
▸ a forwarder (event->message + publish) to the bus
▸ both
▸ even for the same event
▸ infra free implementation

73. IMPLEMENTATION
EVENT DISPATCHER 2/2
*
* * * * *
*
* *
* * *
* *
* *
* *
* * *
* * * * *
* * * * * *
* *
* *
* * *
* * *
* *
*
IN-PROCESS
FORWARD
FORWARD

74. STEP 3:
EVENT EMITTING/PUBLISHING

75. IMPLEMENTATION
EVENT EMITTING/PUBLISHING
▸ as with Commands, Events should use domain structures
▸ IEventEmitter abstraction similar to ICommandIssuer
▸ MassTransit implementation with Bus.Publish<T>
▸ Converts Events to Messages (TbC)

76. IMPLEMENTATION
MASSTRANSIT / BUS.PUBLISH<T>
▸ EndPoint agnostic
▸ Publisher(er)/Subscribe(r) Pattern

77. STEP 4:
EVENT SUBSCRIBING/
HANDLING

78. IMPLEMENTATION
EVENT SUBSCRIBING / BOOTSTRAPPING MASSTRANSIT
▸ bootstrapping identical to command receiving
▸ need a Consumer<T>
▸ translates the message to event
▸ instantiates and invokes event handler

79. IMPLEMENTATION
MASSTRANSIT / CONSUMER<T> (AGAIN)

80. IMPLEMENTATION
MASSTRANSIT / CONSUMER<T> (AGAIN)

81. IMPLEMENTATION
EVENT HANDLING

82. IMPLEMENTATION
EVENT HANDLING

83. IMPLEMENTATION
EVENT HANDLING

84. BEST PRACTICES

85. BEST PRACTICES
MESSAGE DESIGN
▸ messages are POCOS. (period)
▸ do NOT reuse domain structures
▸ SERIOUS danger of message loss
▸ type repetition and mapping are unavoidable
▸ Automapper FTW !
▸ messages might change (aka message versioning)
▸ subscribe to interfaces
▸ ensure correct version or serialiser assembly deployed
▸ JSON .NET

86. BEST PRACTICES
SYSTEM ARCHITECTURE
▸ apply selectively not globally
▸ always forward (i.e. publish) the events
▸ bounded context interface
▸ rely on the retrying facilities
▸ idempotency
▸ fence “expensive” services
▸ perhaps even with sync send/receive (e.g. Routing Service)

87. BEST PRACTICES
SCALING OPTIONS
▸ scale up: can invoke Message Handlers in parallel using a
ThreadPool = Round Robin
▸ bus.SetConcurrencyLimit
▸ scale out: “competing consumers”
▸ same EndPointID
▸ probable concurrency issues
▸ aggregate versioning !

88. BEST PRACTICES
HOUSE-KEEPING
▸ descriptive End Point IDs mostly for subscribers/receivers
▸ web_SERVER1_sender
▸ cmd_rec_worker
▸ evt_sub_booking_reporting
▸ evt_sub_shipping
▸ monitor transport

89. BEST PRACTICES
COMPETING CONSUMERS
1 1
1
1 #
1
#

90. INDICATIONS

91. INDICATIONS
▸ command/event handlers that are:
▸ resource intensive/blocking
▸ involve multiple aggregate instances
▸ e.g. reroute all cargos unloading @ Hong Kong
▸ error prone
▸ involve external service calls
▸ e.g. email multiple recipients
▸ multiple bounded contexts

92. INDICATIONS
▸ separate physical read datastores
▸ high load of commands/events
▸ non-human/UI command issuers

93. CHALLENGES

94. CHALLENGES
EVENTUAL CONSISTENCY
▸ heavily affects UI/UX
▸ contradicts user mentality
▸ need to notify on command handling error
▸ makes UI implementation more complex

95. CHALLENGES
IMPLEMENTATION-RELATED
▸ a LOT of type repetition
▸ more physical processes
▸ changes MIGHT need orchestration
▸ concurrency issues
▸ transport = additional infra

96. RESOURCES

97. RESOURCES
DDD
▸ Domain Driven Design by Eric Evans
http://amzn.to/2uhbOSc
▸ Patterns, Principles and Practices of Domain-Driven Design by
Scott Millett
http://amzn.to/2hsNbLg
▸ Effective Aggregate Design (3 Part Series) by Vaughn Vernon
https://vaughnvernon.co/?p=838
▸ Awesome DDD by Nick Chamberlain
https://github.com/heynickc/awesome-ddd

98. RESOURCES
CQRS
▸ CQRS Journey by Microsoft Patterns & Practises
cqrsjourney.github.io
▸ CQRS by Edument
cqrs.nu
▸ Clarified CQRS by Udi Dahan
udidahan.com/2009/12/09/clarified-cqrs

99. RESOURCES
SAMPLES
▸ Cargo Tracker
▸ github.com/citerus/dddsample-core
(Java) by Citerus AB
▸ github.com/SzymonPobiega/DDDSample.Net
(.ΝΕΤ) by Szymon Pobiega
▸ CQRS, The example - Mark Nijhof
leanpub.com/cqrs

100. RESOURCES
SANDBOX SERVICES
▸ RabbitMQ: CloudAMQP
www.cloudamqp.com
▸ MongoDB: Atlas
www.mongodb.com/cloud
▸ PostgreSQL, SQL Server etc: AWS Free Tier
aws.amazon.com/free/

101. Q&A