1) The document discusses strategies for managing consistency in serverless architectures, including function coordination, choreography, and orchestration approaches. 2) It notes the challenges of distributed transactions and emphasizes the need for eventual consistency and idempotent functions. 3) Common patterns for coordinating functions are examined, including sagas with compensation handling and workflow/orchestration with a coordinator function.

1. @berndruecker
Lost in transaction?
Strategies to manage consistency
in serverless architectures

2. An example for today – the classical one
book
hotel
book
car
book
flight
1. 2. 3.book trip

3. Functions
λ
Book Car
λ
Book
Hotel
λ
Book
Flight

4. Function collaboration
See also e.g. Function Composition in a Serverless World by Timirah James
λ
Book Car
λ
Book
Hotel
λ
Book
Flight
?

5. Merged function
λ
(Trip)
// book har
$.post(carHireUrl,
...
// book hotel
$.post(carHireUrl,
...
Car.js
Hotel.js
Easy to do
No overhead
Big functions
Hard to maintain
No independent scaling

6. Choreography
λ
(Car)
$.post(carHireUrl,
...
invokeHotel()
λ
(Hotel)
$.post(carHireUrl,
...
Chaining
λ
(Car)
$.post(carHireUrl,
...
carBookedEvent()
λ
(Hotel)
$.post(carHireUrl,
...
Event-Chain
No extra framework needed
Hard to understand the flow
Changing the sequence is hard

7. Orchestration: Coordinator function
λ
(Trip)
λ
(Car)
$.post(carHireUrl,
...
λ
(Hotel)
$.post(carHireUrl,
...
// book har
invokeCar()
// book hotel
invokeHotel()
Functions keep independant
Coordinator get‘s long running

8. Orchestration: Workflow
λ
(Trip)
// book har
startTripWorkflow()
λ
(Car)
$.post(carHireUrl,
...
λ
(Hotel)
$.post(carHireUrl,
...
Workflow Engine
Car -> Hotel ->
...
Functions keep independant
No long running function
Extra component involved

9. Warning:
Contains Opinion

10. Berlin, Germany
bernd.ruecker@camunda.com
@berndruecker
Bernd Ruecker
Co-founder and
Technologist of
Camunda

11. Example
λ
(Car)
λ
(Hotel)
Workflow Engine
λ
(Flight)λ
(Trip)

12. Distributed

13. Distributed systems

14. You might need to leverage retrying
λ
(Car)
λ
(Hotel)
Workflow Engine
λ
(Flight)λ
(Trip)

15. You might need to leverage stateful retrying
λ
(Car)
λ
(Hotel)
Workflow Engine
λ
(Flight)λ
(Trip)

16. Photo by pixabay, available under Creative Commons CC0 1.0 license.

17. Requirement: Idempotency of services!
Photo by pixabay, available under Creative Commons CC0 1.0 license.

18. Requirement: Idempotency of services!
Photo by Chr.Späth, available under Public Domain.

19. Make the world a better place.
Make your functions idempotent.

20. @berndruecker
Lost in transaction?
Strategies to manage consistency
in serverless architectures

21. Book
Car
Book
Hotel
All or
nothing
+
try {
tx.begin();
bookCar();
bookHotel();
tx.commit();
} catch (Exception e) {
tx.rollback();
}
Once upon a time:

22. A
C
I
D
Atomicity
Consistency
Isolation
Durability

23. Pat Helland
“
Distributed Systems Guru
Worked at Amazon,
Microsoft & Salesforce

24. Pat Helland
Grown-Ups Don’t Use
Distributed Transactions
“
Distributed Systems Guru
Worked at Amazon,
Microsoft & Salesforce

25. Starbucks does not use two phase commit
https://www.enterpriseintegrationpatterns.com/ramblings/18_starbucks.html
Photo by John Ingle

26. Eric Brewer
Atomicity
Consistency
Isolation
Durability
http://pld.cs.luc.edu/courses/353/spr11/notes/brewer_keynote.pdf

27. That means
Do A
Do B
Temporarily
inconsistent
Eventually
consistent
again
t
Consistent
Local
ACID
Local
ACID
1 function
1 (micro-)service
1 aggregate
1 program
1 resource
Violates „I“
of ACID

28. Youmightknowthisfrom:
Do A
Do B
Temporarily
inconsistent
Eventually
consistent
again
t
Consistent
Photo by Gerhard51, available under Creative Commons CC0 1.0 license.

29. we need to
embrace
eventual consistency

31. function

32. The classical example
Saga
book
hotel
book
car
book
flight
cancel
hotel
cancel
car
1. 2. 3.
5.6.
In case of failure
trigger compensations
book
trip

33. Choreography & orchestration

34. Event-driven choreography
Hotel
Flight
Car
Trip
Trip
booked
Flight
booked
Trip
requested
Hotel
booked
Car
booked
Request
trip

35. Event-driven choreography
Hotel
Flight
Car
Trip
Trip
failed
Trip
requested
Hotel
booked
Car
booked
Request
trip
Flight
failed
Car
canceled
Hotel
canceled
Perform undo
(cancel car booking)
Perform undo
(cancel hotel)

36. The danger is that it's very easy to make
nicely decoupled systems with event
notification, without realizing that you're
losing sight of that larger-scale flow, and
thus set yourself up for trouble in future
years.
https://martinfowler.com/articles/201701-event-driven.html

37. The danger is that it's very easy to make
nicely decoupled systems with event
notification, without realizing that you're
losing sight of that larger-scale flow, and
thus set yourself up for trouble in future
years.
https://martinfowler.com/articles/201701-event-driven.html

38. The danger is that it's very easy to make
nicely decoupled systems with event
notification, without realizing that you're
losing sight of that larger-scale flow, and
thus set yourself up for trouble in future
years.
https://martinfowler.com/articles/201701-event-driven.html

39. Classical example
Saga
book
hotel
book
car
book
flight
cancel
hotel
cancel
car
1. 2. 3.
5.6.
In case of failure
trigger compensations
book
trip

40. If your transaction involves 2 to 4 steps,
choreography might be a very good fit.
However, this approach can rapidly become confusing
if you keep adding extra steps in your transaction
as it is difficult to track which services listen to
which events. Moreover, it also might add a cyclic
dependency between services as they have to
subscribe to one another’s events.
Denis Rosa
Couchbase
https://blog.couchbase.com/saga-pattern-implement-business-transactions-using-microservices-part/

41. Implementing changes in the process
Hotel
Flight
Car
Trip
Trip
failed
Trip
requested
Hotel
booked
Car
booked
Request
trip
Flight
failed
Car
canceled
Hotel
canceled
We have a new basic agreement
with the car rental agency and
can cancel for free within 1 hour
– do that first!

42. Implementing changes in the process
Hotel
Flight
Car
Trip
Trip
failed
Trip
requested
Hotel
booked
Car
booked
Request
trip
Flight
failed
Car
canceled
Hotel
canceled
You have to adjust all services and redeploy at the same time!
We have a new basic agreement
with the car rental agency and
can cancel for free within 1 hour
– do that first!

43. Photo by born1945, available under Creative Commons BY 2.0 license.

44. What we wanted
Photo by Lijian Zhang, available under Creative Commons SA 2.0 License and Pedobear19 / CC BY-SA 4.0

45. Orchestration
Hotel
Flight
Car
Trip
Trip
booked
Request
trip
Book
hotel
Hotel
booked
Car
booked
Flight
booked
Book
car
Book
flight

46. Orchestration
Hotel
Flight
Car
Trip
Trip
booked
Request
trip
Book
hotel
Hotel
booked
Car
booked
Flight
booked
Book
car
Book
flight
We have a new basic agreement
with the car rental agency and
can cancel for free within 1 hour
– do that first!
You have to adjust one service and redeploy only this one!

47. Thoughts on the state machine | workflow engine market

48. For today…
OSS Workflow or
Orchestration Engines
Stack Vendors,
Pure Play BPMS
Low Code Platforms
Homegrown frameworks
to scratch an itch
Integration Frameworks
Cloud Offerings
Data
Pipelines
Camunda, Zeebe, …
AWS Step Functions,
Azure Durable Functions, …

49. Step Functions
https://read.acloud.guru/how-the-saga-pattern-manages-failures-with-aws-lambda-and-
step-functions-bc8f7129f900

50. Describe orchestration with BPMN
Trip
Trip
booked
Request
trip

51. Live hacking
https://github.com/berndruecker/trip-booking-saga-serverless

52. Think of more complicated scenarios… Including stateful retries

53. AWS
Step
Functions
Azure
Durable
Functions
Google
Cloud
Composer
Camunda
BPM
Zeebe
Flow
Language
JSON Source Code
Python,
acyclic!
BPMN BPMN
Visualization
Simple,
proprietary
-
Simple,
proprietary
BPMN BPMN
Performance
& Scale
~ 1k state
transitions / sec*
? ?
Depends on
RDMS
Horizontally
scalable
License and
Distribution
Cloud Cloud
Cloud + OSS
(Apache Airflow)
OSS + EE,
Cloud planned
OSS
Pricing
model
0,025 $ / 1k
state transitions
Billed as
functions
$0.074 / vCPU hour
web,
$0.125 / vCPU hour
db,
$0.273 per GB
storage / month
$0.156 / GB trafiic
no managed
service yet
OSS project
Please remember that I am biased!

54. Graphical models?

55. Clemens Vasters
Architect at Microsoft
http://vasters.com/archive/Sagas.html

56. Clemens Vasters
Architect at Microsoft
http://vasters.com/archive/Sagas.html

57. Clemens Vasters
Architect at Microsoft
http://vasters.com/archive/Sagas.html

58. BPMN
Business Process
Model and Notation
ISO Standard

59. Living documentation for long-running behaviour

60. Visual HTML reports for test cases

61. BizDevOps

62. Thoughts on the state machine | workflow engine market

63. Does it support stateful operations?
Does it support the necessary flow logic?
Does it support BizDevOps?
How to operate?

64. My personal pro-tip for a shortlist ;-)
OSS Workflow or
Orchestration Engines
Stack Vendors,
Pure Play BPMS
Low Code Platforms
Homegrown frameworks
to scratch an itch
Integration Frameworks
Cloud Offerings
Data
Pipelines
Camunda & Zeebe

65. Recap
• Function coordination might include orchestration and
workflow
• Carefully decide about choreography vs. orchestration
• Grown ups don‘t use distributed transactions
but eventual consistency
• Idempotency is super important!
• Know Saga / Compensation as strategy

66. Thank you!

67. mail@berndruecker.io
@berndruecker
https://berndruecker.io
https://medium.com/berndruecker
https://github.com/berndruecker
https://www.infoq.com/articles/events-
workflow-automation
Contact:
Slides:
Blog:
Code:
https://www.infoworld.com/article/3254777/
application-development/
3-common-pitfalls-of-microservices-
integrationand-how-to-avoid-them.html
https://thenewstack.io/5-workflow-automation-
use-cases-you-might-not-have-considered/