MuleSoft Event Driven Architecture (EDA Patterns in MuleSoft) - VirtualMuleys63

All contents © MuleSoft, LLC
2nd February 2022
18:00 Europe/CET | 17:00 UK/GMT | 22:30 Delhi, India/IST
9:00 San Francisco/PST | 12:00 New York
Event Driven
Architecture with
Mulesoft
David Guest

Virtual Muleys Meetup
(Online Group - English )
Moderator: Angel Alberici
February Edition, 2022
Recording

Virtual Muleys Meetup (Online Group - English)
7
7

Announcements & Updates

● Some of the most popular and useful
self-help resources available to MuleSoft
customers
● MuleSoft Videos Youtube channel:
https://sfdc.co/cmZrjq
6 minutes on Customer facing self-help resources | Friends of
Max overview
11

Product Roadmap December 2021
https://www.mulesoft.com/demo/customers
Watch it if you haven’t!

MuleSoft Partner Kick Oﬀ 2022 - 24th March
MuleSoft’s largest partner event of the year - PKO ’22 - is coming back! PKO ’22 will be
packed with all the information that partners need to thrive with MuleSoft in 2022.
Save the date!

What’s new
● IBM + Mule - Extending Runtime Fabric to run on
Red Hat Openshift (here)
● MuleSoft Transform: Announcement in the next
couple of weeks.
● Aligning Integration Strategy with Business
Objectives - Check it out here.
● Data + APIs - Data economics and composable
data, and shown how APIs can support an
enterprise-wide data strategy. Check out the
articles in order here: one, two, and three.
15
● Roadmap Update - High level view of 14th December:
○ Universal API Management;
○ support for designing & discovering event-driven
APIs;
○ Enhanced API Governance;
○ Mule 4.4;
○ AWS Transit Gateway;
○ Cloudhub 2.0
○ Connectors such as MQTT, Outlook 365, Shopify,
and loads more
○ Accelerators for Retail, Healthcare, SAP, and
Financial Services
○ And a whole lot more!

What’s new: Training
● 3 Day Fundamentals - We are now offering a 3 days version of the Development: Fundamentals
course, which requires learners to have completed an Intro to Anypoint Platform Workshop or the
Getting Started with Anypoint Platform course.
● Earn a certification attempt - Learners will be able to earn their MCD - L1 (Mule 4) certification
attempt by completing the self-paced “Anypoint Platform Development: Fundamentals” course. The
exam will then appear in their learner dashboard for scheduling.
● Flex - The training team are now offering ‘Flex’ courses which take place over a weekend or broken
up over half days.
16

Same address: https://knowledgehub.mulesoft.com/s/
Catalyst Knowledge Hub: We changed its name again!
17

● Azure Git Repository Clone Utility Scripts - Batch Updates - Utility PowerShell scripts, shell scripts, and
git commands and the steps that need to be followed to clone all the repositories from Azure DevOps Git
Repository and do checkouts, rebase, commits, push, etc in batch mode( more than one repository at a
time)
● Mule Development Best Practices - Mule Development - best practices and standards to apply in the
implementation/development phase of the API Development Life Cycle.
● Workday Business Process - Slack Integration - This is a starter utility that might be useful for someone
trying to implement a Slack-ops type of communication standard where Workday is the data source.
Catalyst Knowledge Hub:
18

Meetup - Event Driven
Architecture
David Guest, Customer Success Architect

Agenda
1. Level Set
2. Event Driven (Async) vs Synchronous
3. Event Driven Infrastructure
4. Common Patterns
5. Mulesoft Implementation
6. Tales from the Field

Level Set

History
● Internal messaging systems at application/operating system level
● Flat Files/Formatted Files transfer between systems
● IBM MQSeries came along in 1993
● SOAP / XML appeared around 2000
● Proprietary until Advanced Message Queueing Protocol (AMQP) arrived in 2003
● JSON in 2013
● Modelling APIs
● Modelling Event Driven APIs

Is it really Real-Time?
● Real-time or real time describes various operations in computing or other processes that
must guarantee response times within a specified time (deadline), usually a relatively
short time. A real-time process is generally one that happens in defined time steps of
maximum duration and fast enough to affect the environment in which it occurs, such as
inputs to a computing system.
● The basic premise around real-time APIs is that they are event-driven. According to the
event-driven design pattern, a system should react or respond to events as they happen.

Instant conﬁrmation
Initiate an action
$40M
worth in revenue in
capital markets*
0.1%
Additional revenue
on mobile apps
$.5M
Saved in costs for
logistics
Value of
10 ms
35
What are real-time experiences worth to you?
Source: https://perspectives.mvdirona.com/2009/10/the-cost-of-latency/

Scenario: ATM Card Withheld
Event-Driven Automation
Automate Your
Processes Faster
Support Customers via
Multiple Channels
Achieve Single View of
Customer
Event Store
Bank Systems
ATM Card
withheld event
Notify Branch to
start new ATM
Card process
Trigger SMS to
inform customer
ATM Card State
updated
Case created
1
2
3
4
5

Scenario: Cross-Sell/Up-Sell with Social
Listening
Personalized Experience for Customer
Event Store
Bank Systems
LINE Pay
wedding booking
event
Trigger loan offer
promotion
Check customer
credit rating
Pitch promotion
on LINE
1
2
3
4
Real-Time Marketing Offers
Automated Consumer
Journeys
Expand Your Business
through Cross-Sell/Upsell

All contents © MuleSoft, LLC 38
But, building real-time experiences is hard
Business teams want to... IT teams want to...
..and more
Capture and unlock real-time data
Add or modify services with limited impact
Improve latencies and avoid polling madness
Respond in real-time to customers
Compete by expanding business capabilities
Optimize costs spent on resources
..and more
Real time application

All contents © MuleSoft, LLC 39
Is your digital backbone ready to power real-time?
request
response
98.5% of API polls do not return any new information
Excessive coupling limits ﬂexibility to add new services
Failures cascade through all the coupled services
Response times to the customer are slow
Synchronous communication

Event Driven Infrastructure

Event-Driven Architectures: Core concepts
A simple event-driven system - it used to be easy
JMS
Event broker
Account
Change event
Account
Updated!
Event brokers
Event producers Event consumers

Event grid / hubs / mesh...
Message queues
JMS
Streaming platform
Explosion in # enterprise systems + real-time needs
Event driven is no longer just Pub-Sub
Message brokers
Event producers Event consumers

Event Driven vs Synchronous

Queue Processing vs Synchronous
● Reliability
● Demand Throttling
● Load Balancing
● High Availability
● Multiple Recipients
● Internet of Things
● Integration with modern SaaS applications
● Error Handling
● Avoids Polling

Shift to Event Driven
● Req-reply <= 50% of the equation
● The World is Event Driven
● The Web World operates on HTTP and it isn’t designed for Async interactions
● IoT is Async by nature and is getting bigger & HTTP/Req-Reply doesn’t really cut it
● ‘Fire-and-forget’ processing model
○ In certain situations, a formal response is not needed (Although an acknowledgement
may be required)
○ Very useful when the requirement is only to inform a downstream process (Or processes) that
‘something has happened’

Common Patterns

Message Flow

Event-Driven Architectures: Core concepts
48
Publisher Broker Consumer

Event-Driven Architectures: Multi-subscriber
49
Publisher Broker
Subscriber
Subscriber
Subscriber

Single atomic transaction
Event-Driven Architectures: Atomic
Transactions
50
Publisher Broker
Subscriber
Subscriber
Subscriber

Saga: Choreography-based
51
Order Service
Order
Events
Customer
Events
<<Order Aggregate>>
Customer Service
<<CustomerAggregate>>
Order
Created
Credit
Reserved

Broker
Saga: Orchestration-based
52
Customer
Service
Command
<<Order Aggregate>> <<CustomerAggregate>>
Reserve
Credit
Command
Credit
Reserved
Order Controller
Order Service
Create Order Saga
Customer Service
Create
Order Saga
Reply
Customer
Command Handler
Order
Command
Handler

Sagas in Mule
Request
Response
Request
Response
Event Event
Choreography
Orchestration

Message Content

Thin Event Notiﬁcation
55
Broadcast only the necessary details to notify that an event occurred
Forces subscribers to fetch additional details to complete a task
Use when:
The desire is to prevent subscribers processing stale data. They are forced to fetch the latest data
due to frequent changes or potentially delayed processing
{
“eventType” : “com.example.customer.address”,
“eventId” : “1234567”,
“updatedAt” : “2021-12-25”,
“customerId” : “9876543”
}

Hypermedia-Driven Notification
56
Include hypermedia links in event payloads
Helps consumers find the right API for details
Use when:
You wish to bridge event notifications with API integration
{
“eventId” : “1234567”,
“updatedAt” : “2021-12-25”,
“customerId” : “9876543”,
“_links”: [
{ “rel”:”customer”, “href”:”href”:”/customers/9876543” }

Event-Carried State Transfer
57
Broadcasts all known data at the time of the event
Often contain the entire record, avoiding the need to contact the source system to conduct
further work
Use when:
Subscribers want a snapshot of the data with the event
Sharing data state changes through message streaming (Apache Kafka, Apache Pulsar, etc.) to
support replaying message history.
Using event sourcing /CQRS to capture state changes over time
{
“eventId” : “1234567”,
“updatedAt” : “2021-12-25”,
“customerId” : “9876543”,
“Address” : {

Structured Event Payloads
58
Groups properties as nested structures
Avoids ﬂat structures that require subscribers to ﬁgure out how properties are grouped
Helps drive evolvability as property names are scoped to the parent property (e.g. addressLine1)
Use when:
Event payloads require complex data structures
Event payloads have nested 1-to-many relationships as part of an event payload
{
contactDetails:
firstName
lastName
Email
billingAddress:
addressLine1
addressLine2

Offline and Error Recovery Support
59
Supplement event notification channels with APIs that allow offline consumers to catch up
Also allows consumers to identify and troubleshoot failed deliveries
Use when:
Consumers have limited or no access to internal mechanisms to detect errors
When offline support is necessary to keep consumers in-sync
API: Listing recent dispatches

Separate Internal and External Events
60
Design internal events for coordinating implementation details
Design external events for notiﬁcation and extensibility
Use when:
Event-driven architecture is being used internally, but external events are desired
Prevent leaking implementation details or special knowledge of how your internal systems
operate
{
“eventType”: “authorised”,
“transactionId”: “123456”,
“authorisationServer”: “auth.internal.com”,
“customerId”: “cust1234”,
“transactionAmount”: “1299”,
“transactionCurrency”: “GBP”
…
}
{
“Event”:{
“Type”: “paymentProcessed”,
“OrderId”: “ord1234”,
…
}
}

Mulesoft Implementation

Event grid / hubs / mesh...
Message brokers
JMS
Streaming platform
● What event-driven APIs does
my company have?
● What info to include in the
event?
● What channels? Protocols?
#1: Discovering events and event-driven APIs
Re-use is challenging as custom coding creates black boxes
?

#2: Governing brokers and events consistently
Managing security, access management and validation for APIs and brokers is hard
?
● How are each brokers
managing authentication?
● How do IT apply rules
consistently across brokers,
apps and events?
AMQP
JMS
?
?
● Plaintext
● SASL - Oauth
● mTLS
● LDAP
● IAM users
● IAM Roles
● SecretKey

#3: Monitoring events is …..um “hard”
Even harder to derive insights and take preventive action
Update
account!
SFDC
Updated
Netsuite
error..50
3
!
????????
!
Profile service
SFDC service Netsuite service
● Show me all events related to
customer x?
● How can we look at event
flows, identify malicious
patterns and get alerted?
JMS
Profile
service

Journey to enterprise scale event-driven..
Choose a technology solution that can help you transition seamlessly
DESIGN
with open
standards &
simple
tooling
REUSE
Existing
assets in
design
MANAGE
& SECURE
your APIs
with policies
REGISTER
with event
brokers
`
Now
FY23
Now
FY23

REUSE
Existing
assets in
design
DESIGN
with open
standards
& simple
tooling
REGISTER
with event
brokers
MANAGE
& SECURE
your APIs
with
policies
#1: Discover and reuse your existing assets and best practices
Discover your reusable assets
easily
Explore your Event-driven APIs
quickly
Acquire the right API with limited
support
Now

REUSE
REST
fragments
in design
DESIGN
with open
standards &
simple
tooling
REGISTER
with event
brokers
MANAGE
& SECURE
your APIs
with
policies
#2: Use AsyncAPI to establish an enterprise-grade standard
Well-documented contract on
how to interact with services
Discover, reuse, manage and
govern events in the enterprise
Open-source and protocol agnostic allowing
interoperability
Now

#3: Connect with brokers to stream events across a distributed enterprise
Platform to compose business capabilities
` JMS
Discover Design Manage Secure Monitor
REUSE
REST
fragments
in design
DESIGN
with open
standards
& simple
tooling
REGISTER
with event
brokers
MANAGE
& SECURE
your APIs
with
policies
FY23

#4: Comprehensively monitor your events and request/response in a single view
Centralized &
proactive
monitoring across
your services
Shopping
API
Monitor user and application
behaviour
🕵
Get alerted proactively on
mission critical operations
🚨
Search through events to
identify the root cause of
issues
🔎
REUSE
REST
fragments
in design
DESIGN
with open
standards
& simple
tooling
REGISTER
with event
brokers
MANAGE
& SECURE
your APIs
with
policies
FY23

“AsyncAPI is an open source initiative that seeks to improve the current state of
Event-Driven Architectures (EDA). Our long-term goal is to make working with EDA’s as
easy as it is to work with REST APIs. That goes from documentation to code
generation, from discovery to event management. Most of the processes you apply to
your REST APIs nowadays would be applicable to your event-driven/asynchronous
APIs too.”
AsyncAPI
71
https://www.asyncapi.com/docs/getting-started

Documentation
&
Code Generation
Event Standards Governance
Beneﬁts of Async API
72

● AsyncAPI started as an adaptation of OAS
● The intention is to leverage some re-use between the two
speciﬁcations
● AsyncAPI and OAS are very similar
● Real-world systems may have both RESTful APIs and a
mechanism to handle events
● The ability to share/re-use schemas is valuable
Comparing AsyncAPI to Open API
Speciﬁcation (OAS)
73

Playground - Async Structure
74
https://www.asyncapi.com/docs/getting-started

Internal and External Brokers
75
Mulesoft provides native queuing with Anypoint MQ (https://docs.mulesoft.com/mq/)
Also provides connectors to other queuing technologies
• JMS - https://docs.mulesoft.com/jms-connector/1.8/
• Amazon SQS - https://docs.mulesoft.com/amazon-sqs-connector/5.11/
• Microsoft MQ - https://docs.mulesoft.com/msmq-connector/2.0/
• Apache Kafka - https://docs.mulesoft.com/kafka-connector/4.6/
• Generic - https://docs.mulesoft.com/amqp-connector/1.7/

Internal and External Brokers
76

Sample Use Case

80
Orders
Salesforce
customers
SAP
customers
Customers
Order status
Mobile API
Process
APIs
Experience
APIs
System
APIs
Using elements of event-driven architecture to cancel an
order
Orders
Anypoint MQ
Event Handler
Publish
message
Consume
message Invoke

81
Orders
Salesforce
customers
SAP
customers
Customers
Order status
Mobile API
Process
APIs
Experience
APIs
System
APIs
Using elements of event-driven architecture to update
customer data
Anypoint MQ
Event Handler
Publish
message
Consume
message
Invoke
Customers

82
Orders
Salesforce
customers
SAP
customers
Customers
Order status
Mobile API
Process
APIs
Experience
APIs
System
APIs
Using API-led connectivity to get order details
Anypoint MQ
Event Handler
Customers

83
Web app
API
Orders
Salesforce
customers
SAP
customers
Order history
Customers
Order status
Mobile API
Process
APIs
Experience
APIs
System
APIs
API-led connectivity and event-driven architecture
together
Anypoint MQ Event Handler

84
Web app
API
Orders
Salesforce
customers
SAP
customers
Order history
Customers
Order status
Mobile API
Process
APIs
Experience
APIs
System
APIs
API-led connectivity and event-driven architecture
together
Anypoint MQ
Event Handler
● Abstract event delivery via a
secured RESTful API.
● Avoid Messaging vendor lock in.
● Enable legacy application to
produce events.

Event Pattern ++ API-Led
Experience
APIs
UPS
Shipments
API
Order Command
API/Integration
Order Query API
Orders Mobile API
Accounts
API
Orders
API
Process
APIs
System
APIs
Read
Write
Read and Write operations segregation
Read is sync call whereas Write is async
Reliability, guaranteed delivery
Write operation can be tuned in
accordance to end system capacity
Publish
Subscribe
Publish
Subscribe

CQRS ++ Event Sourcing Pattern
Experience
APIs
UPS
Shipments
API
Order Command
API/Integration
Order Query API
Orders Mobile API
Accounts
API
Orders
API
Account Order
Details API
Process
APIs
System
APIs
Query/Read
Command/Write
Real-time
Replication/Sync
Write all events to event store Replay, rebuild and query
Read and Write operations
segregation
Read is sync whereas Write is async
Reliability, guaranteed delivery
Write operation can be tuned in
accordance to end system capacity
Read operations can be scaled
independently of write as its reading
from different datastore
If all events are stored in write
datastore then they can be replayed,
rebuilt or queried at any point in time
Publish
Subscribe
Publish
Subscribe
“...you should be very cautious about using CQRS. Many information
systems fit well with the notion of an information base that is updated
in the same way that it's read, adding CQRS to such a system can add
significant complexity. I've certainly seen cases where it's made a
significant drag on productivity, adding an unwarranted amount of risk
to the project, even in the hands of a capable team…” - Martin Fowler,
https://martinfowler.com/bliki/CQRS.html

Flow Processing

Flow Processing
88
Acks / Nacks
Error Handling
Dead letter queue reprocessing
Persistent
FIFO

Tales from the Field

eCommerce Order Processing
90
Advantages of Event Driven Architecture
• Decoupling
• Load Balancing
Object Creation
• Timing
• Trigger
• System of Record
Data format of the Object
Object Transfer
• Reference the object
• Pass the object changes
• Pass the object
Publish/Subscribe or Point to Point
Sequencing
Data Consistency/Freshness

Conclusion
● Event Driven Architecture is here already and proving it’s value
● More advanced Mulesoft tools are arriving to help you build this Architecture
● Ensure that your Reference Architecture includes EDA patterns
● Decide what Real Time means to your Organization, instant->quicker than batch
● Don’t be afraid

Thank you!

MuleSoft Event Driven Architecture (EDA Patterns in MuleSoft) - VirtualMuleys63

Recomendados

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente (20)

Similar a MuleSoft Event Driven Architecture (EDA Patterns in MuleSoft) - VirtualMuleys63

Similar a MuleSoft Event Driven Architecture (EDA Patterns in MuleSoft) - VirtualMuleys63 (20)

Más de Angel Alberici

Más de Angel Alberici (10)

Último

Último (20)

MuleSoft Event Driven Architecture (EDA Patterns in MuleSoft) - VirtualMuleys63