The document discusses challenges with microservices integration and distributed systems. It presents strategies like circuit breakers and workflow engines to handle failures and asynchronous communication in a resilient manner. Specifically, it recommends using a workflow engine like Camunda to implement stateful retry, timeouts, compensation to keep failures localized. It also advocates for using external tasks with Camunda to enable polyglot programming, scaling, decoupling and natural transactions in a microservices architecture.

1. Microservices Integration
Challenges and Solutions
Felix Müller
Technical Consultant

2. Who I am?
Felix Müller
@felixlmueller

3. Distributed systems

4. Distributed systems
Asynchronous
communication
Distributed
Transactions
Communication
is complex

5. Communication
is complex

6. Some
Service
Some
Service
Some
Service
Some
Service
Some
Service
Some
Service
Some
Service
Failure will happen.
Accept it!
But keep it local!
Be resilient.

7. Photo by Tookapic, available under Creative Commons CC0 1.0 license.

8. Translation :
„There was an error
while sending your
boarding pass“

9. Check-in
Web-UI
User
Current situation

10. Check-in
Barcode
Generator
Web-UI
Output
Mgmt
Current situation
User

11. Check-in
Barcode
Generator
Web-UI
Output
Mgmt
Current situation
User

12. Circuit
Breaker
Photo by CITYEDV, available under Creative Commons CC0 1.0 license.

13. Check-in
Barcode
Generator
Web-UI
Output
Mgmt
Current situation
Circuit
breaker
User

15. Fail fast
is important

16. Fail fast
is important
but not enough!

17. Check-in
Barcode
Generator
Web-UI
Output
Mgmt
Current situation – the bad part
User

18. Check-in
Barcode
Generator
Web-UI
Output
Mgmt
Current situation – the bad part
User

19. Check-in
Barcode
Generator
Web-UI
Output
Mgmt
Current situation – the bad part
Stateful
Retry
User

20. Translation:
It‘s your problem
now.

21. We are having some technical
difficulties and cannot present you
your boarding pass right away.
But we do actively retry ourselves, so
lean back, relax and we will send it
on time.
…just made this up…

22. Check-in
Barcode
Generator
Web-UI
Output
Mgmt
Possible situation – much better!
User

23. Check-in
Barcode
Generator
Web-UI
Output
Mgmt
Possible situation – much better!
Stateful
Retry
User

24. Check-in
Barcode
Generator
Web-UI
Output
Mgmt
Stateful
Retry
Possible situation – much better!
The failure
never leaves
this scope!
User

25. Persist thing
(Entity, Document, Actor, …)
State machine or
workflow engine
Typical
concerns
DIY = effort,
accidental
complexity
Complex, proprietary,
heavyweight, slow,
don‘t scale,
developer adverse
Scheduling, Versioning,
operating, visibility,
scalability, …
Handling
State

26. Let‘s try this with a simple example
Credit
Card
Payment
REST

27. Live hacking
https://github.com/flowing/flowing-
retail/blob/master/rest/java/payment/src/main/java/io/flowing/retail/payment/port/resthacks/PaymentRestHack
sControllerV3.java

28. Payment
Now you have a state machine!
Credit
Card
REST

29. Distributed systems introduce complexity you have to tackle!
Credit
Card
Payment
REST

30. Distributed
systems

31. It is impossible to
differentiate certain
failure scenarios.
Independant of
communication style!
Service
Provider
Client

32. Distributed systems introduce complexity you have to tackle!
Credit
Card
Payment
REST

33. Distributed systems introduce complexity you have to tackle!
Credit
Card
Payment
REST
Do it reliably

34. Workflows live within
service boundaries
Credit
Card
Payment
REST

35. Check-in
Barcode
Generator
Web-UI
User
Output
Mgmt
Workflows live within
service boundaries

36. Manifold architecture options
https://blog.bernd-ruecker.com/architecture-options-to-run-a-workflow-engine-6c2419902d91

37. Manifold architecture options
https://blog.bernd-ruecker.com/architecture-options-to-run-a-workflow-engine-6c2419902d91

38. Manifold architecture options
https://blog.bernd-ruecker.com/architecture-options-to-run-a-workflow-engine-6c2419902d91

39. generateBoardingPass
HTTP 200 OK
HTTP 202 ACCEPTED
Check-in
A synchronous response is possible in the
happy case, otherwise it is switched to
asynchronous processing.

40. Asynchronous
communication

41. Check-in
Barcode
Generator
Web-UI
User
Output
Mgmt
Asynchronous communication
You need to
monitor
timeouts

42. Check-in
Barcode
Generator
Web-UI
User
Output
Mgmt
Remember…
The failure
never leaves
this scope!

43. Workflow…
Easy to
handle time

44. Workflow…

45. has to implement
Timeout, Retry
has to implement
Idempotence
Client Service Provider

46. Manifold architecture options
The
image
part with
relationsh
ip ID rId4
was not
found in
the file.
https://blog.bernd-ruecker.com/architecture-options-to-run-a-workflow-engine-6c2419902d91

47. Manifold architecture options
https://blog.bernd-ruecker.com/architecture-options-to-run-a-workflow-engine-6c2419902d91
The
image
part with
relationsh
ip ID rId4
was not
found in
the file.

48. Manifold architecture options
https://blog.bernd-ruecker.com/architecture-options-to-run-a-workflow-engine-6c2419902d91
The
image
part with
relationsh
ip ID rId4
was not
found in
the file.

49. Distributed
Transactions
Distributed
Transactions

50. Distributed transactions using compensation *
Compensation
*aka Sagapattern

51. Eventual consistency
Temporarily
inconsistent state
But only
temporarily!

52. has to implement
Timeout, Retry,
Compensation
has to offer
Compensation
has to implement
Idempotence
Client Service Provider

53. has to implement
Timeout, Retry,
Compensation
has to offer
Compensation
has to implement
Idempotency
Client Service ProviderDon‘t forget
about state

54. # Be aware of complexity of distributed systems
# Know strategies and tools to handle it
e.g. Circuit breaker (Hystrix)
Workflow engine for stateful retry, waiting, timeout,
compensation, and visibility (Camunda)

55. Live Coding:
External Tasks

56. Remember this? External Tasks – The Pull Principle

57. Remember this? Let‘s implement it...
Temporarily
inconsistent state
But only
temporarily!

58. Live hacking

59. # External Tasks are the perfect fit for a
decoupled Microservice architecture with Camunda

60. # External Tasks are the perfect fit for a
decoupled Microservice architecture with Camunda
… and help you to solve some other problems along the way:
. Polyglot programming
. Scaling and Clustering
. Decoupling
. Natural transactions for distributed systems

61. Thank you