Designing and Implementing Information Systems with Event Modeling, Bobby Calderwood, Founder at Evident Systems
https://www.meetup.com/Saint-Louis-Kafka-meetup-group/events/273869005/
2. Hi, I’m Bobby
• Founder of Evident Systems, a Confluent Partner
• Previously:
• Distinguished Engineer, Capital One Tech Fellows team
• Software Engineer, Cognitect’s Datomic team
• Enjoys:
• Functional Programming in Clojure
• Camping with my wife and 3 kids
4. My Client’s Story
• Large organization, partnering with both Confluent and Evident
Systems
• Many cooperating teams re-writing the company’s core applications
• Wanted to expand on the success of their Kafka-based, event-driven
core data platform
• Articulated a vision to build an ecosystem of event-sourced Apps and
Services
5. Benefits of Event-First Application Design
• Event-first thinking is more accurate to the business domain than database-
centric thinking: our brains are designed for narratives!
• Apps and APIs dog-food their own events, ensuring quality and completeness of
what they provide to external teams
• Integration of cross-cutting concerns is first-class, requires no ETL, etc.
• Customer engagement and outreach
• Analytics and ML/AI
• Security & Compliance audit
6. Drawbacks of Event-First Application Design
• Mature and familiar programming languages, data stores, and design tools
promote database-first design
• Event-first is catching up quickly here!
• Kafka ecosystem, functional programming with immutability, polyglot
persistence
• It’s unfamiliar to both product and development teams
• Product specialists don’t know about super-powers they get from event-first
• Lack of a common design language and methodology
7. What My Client Needed
• A common language and design method for event-based information
systems
• Ideally, this design method would:
• Be simple enough to solicit expertise from those with non-developer
skillsets
• Be specific enough to directly facilitate implementation by developers
• Reduce adoption friction by working with the human brain: Narrative
and Visuals
8. A Brief (and possibly biased)
History of Application Design
Methods
9. Big-Design-Up-Front (BDUF)
• Born of waterfall methods
• Includes techniques from UML and the
Rational methodologies
• Too focused on systems-level concerns
(e.g. OOP), not enough on user- and
business-level concerns
• So complex and specific, might as well be
writing code
• Deadly truism: “No plan survives first
contact with reality”
10. No-Design-Up-Front
• aka “Emergent Design”
• Reaction against BDUF in certain Agile
communities
• Heavy reliance on TDD/BDD and refactoring
• Not well suited to the complexities of
distributed systems
• Too focused on the user-level concerns (e.g.
driven by UAT), not enough on the business-
and systems-level concerns
• Deadly truism: “Failing to plan is planning to
fail”
11. Balancing Teleology vs. Emergence
• Goldilocks zone: Lightweight design or Small-Design-Up-Front
• Many such techniques developed by the DDD and Agile communities:
• Event Storming
• Design Thinking
• User Journeys and Wireframing
12. Keys to Lightweight Design
• Must be simple enough for all stakeholders to understand, especially
non-developer stakeholders
• Must be specific and well-specified enough to be actionable by the
implementors
• Both the process of generating the artifact and the artifact itself must
convey knowledge and context among stakeholders
• Artifact must provide value throughout the SDLC, or else won’t be kept
current
14. What is Event Modeling?
• Invented by Adam Dymitruk, founder of Adaptech Group https://
www.adaptechgroup.com/
• With input from Greg Young, coiner of the terms “CQRS” and “Event
Sourcing”
• Intellectual descendent of Event Storming, but simpler and more
solution-focused
• https://www.eventmodeling.org
15.
16. What is Event Modeling?
• 4 “words” in the visual language:
• Wireframe
• Command
• Event
• View (aka Read-Model)
17. What is Event Modeling?
• 4 patterns:
• State Change
• State View
• External State Import
• Internal State Export
18. Events
• Facts or outcomes recorded along the timeline of the business
• Sufficient to describe the whole business process
• Represented by orange notes
• Written as a verb phrase in the past tense
• Can be organized into lanes indicating bounded context/narrative
19.
20. Wireframes
• Visual wireframes or mockups of user interfaces and experience
• Not always graphical, can also be APIs or automated jobs
• Can be organized into swim-lanes, indicating roles/personas
21.
22. Commands & the State Change Pattern
• Encapsulate the intent to change the system: a transactional moment
• Empower users to take action
• Represented by blue notes
• Written as a verb phrase in the imperative tense
• Result in the emission of an Event
23.
24. Read Models & the State View Pattern
• Inform users about the state of the system
• Represented by green notes
• Derived from Events
• Displayed in Wireframes
• Many different Read Models to support various data-access patterns
25.
26. The External State Import Pattern
• Listen to events generated elsewhere:
• Lower-level system events
• External bounded contexts from other apps, services
• Translate those events into the local context
27.
28. The Internal State Export Pattern
• Build up a “todo list” of actions for an external system to perform,
represented by a Read Model
• Some automated process, represented by an automation Wireframe,
invokes Commands on the external system
• Result/reply from external system is recorded as an Event
29.
30. Putting It All Together:
The Workshop
1. Brain Storming
2. The Plot
3. The Story Board
4. Identify Inputs
5. Identify Outputs
6. Apply Conway’s Law
7. Elaborate Scenarios
33. From Design to Implementation
• Transparent Project Management
• Event Model can be sliced into discrete implementation tasks
• No need for a separate user-story creation exercise
• Accelerate Implementation
• Test cases from patterns, sample data generation
• Schema specification, code generation
34. From Implementation to Operations
• Data discovery and integration
• Schema/Stream Registry integration
• Integration among applications and analyses
• Support cooperation within Enterprise
• Design Review, IT operations and incident management
• InfoSec or Compliance audit
36. Assembling the Workshop Group
• Diverse skillsets:
• Software engineers, architects, etc.
• UI/UX designers and user advocates
• Business SMEs and potential users of the system (if internal)
• Diverse levels of org-chart, ideally 3 levels:
• The people doing the work: implementing and using
• Their bosses and skip-levels
37. Analog vs. Digital
• Modeling with sticky notes, sharpies, and butcher paper is great!
• Highly interactive and collaborative
• Tons of serendipitous learning and context transfer
• Building trust within and among teams
• But…
• Artifact is hard to capture for implementation, and is fragile
38. In-Person vs. Remote
• …Also COVID-19 cancelled conference rooms
• Unlike other workshop formats, Event Modeling was explicitly designed
to be remote-friendly
• In-person is great if you can, but remote is also very productive
• Especially with the right tooling!
39. Event Modeling Tooling
• Collaborative whiteboards are okay
• Not purpose-built for Event Modeling, can draw arbitrary picture
• Doesn’t help with Implementation or Operations
• We built our own collaborative Event Modeling platform: oNote
• Free version available now
• Enterprise and paid Solo versions coming soon
40. Conclusion
• Event Modeling is a shared language for designing information systems
• Useful across the SDLC:
• Design — The Workshop
• Implementation — Specification and Project Management
• Operations — Discovery and Integration
• Check out our Event Modeling platform, oNote: https://onote.com