There are two ways to implement the circuit breaker pattern: white-box à la Hystrix/Resilience4J or black-box à la Istio. Both have pros and cons. Come to this talk to hear more about that! Kubernetes in general, and Istio in particular, have changed a lot the way we look at Ops-related constraints: monitoring, load-balancing, health checks, etc. Before those products became available, there were already available solutions to handle those constraints. Among them are different libraries e.g. Hystrix and Resilience4J in the Java ecosystem. In particular, they both provide an implementation of the Circuit Breaker pattern, which prevents a network or service failure from cascading to other services. But now Istio also provides the same capability. In this talk, we will have a look at how Istio and one among Hystrix/Resilience4J implement the Circuit Breaker pattern, and what pros/cons each of them has. Bonus, you'll be able to choose what library we will focus for the demo. After this talk, you'll be able to decide which one is the best fit in your context.

1. @nicolas_frankel
Istio vs. Hystrix/Resilience4J
Battle of the Circuit Breakers

2. @nicolas_frankel
• Developer Advocate
• Developer until last September
• DevOps and Cloud curious
Me, myself and I

3. @nicolas_frankel
Hazelcast
HAZELCAST IMDG is an operational,
in-memory, distributed computing
platform that manages data using
in-memory storage, and performs
parallel execution for breakthrough
application speed and scale.
HAZELCAST JET is the ultra fast,
application embeddable, 3rd
generation stream processing
engine for low latency batch
and stream processing.

4. @nicolas_frankel
• Some introduction
• The problem
• The circuit-breaker pattern
• Istio implementation
• Hystrix implementation
• Demo
Agenda

5. @nicolas_frankel
• Componentization via Services
• Smart endpoints and dumb pipes
• Decentralized Governance
• Decentralized Data Management
• Infrastructure Automation
• Design for failure
• Evolutionary Design
• Organized around Business Capabilities
• Products not Projects
µservice: a tentative definition
https://martinfowler.com/articles/microservices.html

6. @nicolas_frankel
• Componentization via Services
• Smart endpoints and dumb pipes
• Decentralized Governance
• Decentralized Data Management
• Infrastructure Automation
• Design for failure
• Evolutionary Design
• Organized around Business Capabilities
• Products not Projects
µservice: a tentative definition
https://martinfowler.com/articles/microservices.html

7. @nicolas_frankel
• Microservices are an
organizational solution to an
organizational problem
• They are ill-adapted to most orgs
Word of warning
https://martinfowler.com/bliki/MicroservicePrerequisites.html

8. @nicolas_frankel
"organizations which design systems
... are constrained to produce
designs which are copies of the
communication structures of these
organizations."
Conway’s Law

9. @nicolas_frankel
https://www.nginx.com/blog/adopting-microservices-at-netflix-lessons-for-team-and-process-design/

10. @nicolas_frankel
https://www.nginx.com/blog/adopting-microservices-at-netflix-lessons-for-team-and-process-design/

11. @nicolas_frankel
“I see you have a poorly structured
monolith. Would you like me to
convert it into a poorly structured set
of microservices?”
Rant of the day
https://twitter.com/architectclippy/status/570025079825764352

12. @nicolas_frankel
Webservice, not microservice
Semantics!

13. @nicolas_frankel
• “Anything that can go wrong will go
wrong”
• Apply that to webservices
architecture
Reminder: Murphys’s law

14. @nicolas_frankel

15. @nicolas_frankel
• The network is reliable
• Latency is zero
• Bandwidth is infinite
• The network is secure
• Topology doesn't change
• There is one administrator
• Transport cost is zero
• The network is homogeneous
Reminder: Fallacies of distributed
computing
https://yourlogicalfallacyis.com/

16. @nicolas_frankel
• The network is reliable
• Latency is zero
• Bandwidth is infinite
• The network is secure
• Topology doesn't change
• There is one administrator
• Transport cost is zero
• The network is homogeneous
Reminder: Fallacies of distributed
computing
https://yourlogicalfallacyis.com/

17. @nicolas_frankel
A sample webservice architecture
F B
C1
C2

18. @nicolas_frankel
A sample webservice architecture
F B
C1
C2

19. @nicolas_frankel
A sample webservice architecture
F B
C1
C2

20. @nicolas_frankel
A sample webservice architecture
F B
C1
C2

21. @nicolas_frankel
A sample webservice architecture
F B
C1
C2

22. @nicolas_frankel
“A service client should invoke a remote service via a
proxy that functions in a similar fashion to an electrical
circuit breaker.”
https://microservices.io/patterns/reliability/circuit-breaker.html
Enter the Circuit Breaker pattern

23. @nicolas_frankel
Circuit Breaker state machine

24. @nicolas_frankel
• Number of failed calls
• Elapsed time strategy:
• Fixed
• Doubling
• Something else
• Number of successful calls
Configuration options

25. @nicolas_frankel
What to do in the case of timeout?
The most important configuration
option

26. @nicolas_frankel
1. Recommendation webservice
• “People also bought xyz”
2. Pricing webservice
3. Payment webservice
4. Logging webservice
Use-case: e-commerce webshop

27. @nicolas_frankel
• Fire-and-forget
• Asynchronous calls
Logging

28. @nicolas_frankel
• Synchronous req/response
• Optional
• Fallback options
• Display no recommendations
• Static recommendations set
Recommendation

29. @nicolas_frankel
• Synchronous req/response
• Required
• But better sell at a slightly
outdated price!
• Fallback options
• Accept outdated data from
another source
• In-memory cache
Pricing

30. @nicolas_frankel
• Synchronous req/response
• Required
• Fallback options
• Accept potentially bad payments 🤔
Payment

31. @nicolas_frankel
Available strategies
Strategy Implementations Fits
Black Box ● Proxies
● Service meshes
Fail fast
White Box Libraries
● Hystrix
● Resilience4J
Fallbacks relying
on business logic

32. @nicolas_frankel
“A service mesh is a configurable infrastructure
layer for a microservices application. It makes
communication between service instances flexible,
reliable, and fast. The mesh provides service
discovery, load balancing, encryption,
authentication and authorization, support for the
circuit breaker pattern, and other capabilities.”
https://www.nginx.com/blog/what-is-a-service-mesh/
Service mesh

33. @nicolas_frankel
• Open Source service mesh
• Leverages Kubernetes
• Implements the sidecar pattern
• Uses the Envoy proxy under the
hood
Istio

34. @nicolas_frankel
Sidecar pattern

35. @nicolas_frankel
Istio from a birds-eye view
https://istio.io/docs/concepts/what-is-istio/

36. @nicolas_frankel
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: foo
spec:
host: foo
trafficPolicy:
outlierDetection:
consecutiveErrors: 3
interval: 10s
baseEjectionTime: 1m
maxEjectionPercent: 80
Circuit-breaker configuration in Istio
Number of consecutive errors that open the circuit breaker
Interval between two checks
Duration of opening
Percentage of evicted instances

37. @nicolas_frankel
• No fallback
Cons of Istio

38. @nicolas_frankel
A talk in which you’re the hero!
Go to slide 39 Go to slide 44

39. @nicolas_frankel
“Hystrix is a latency and fault tolerance
library designed to isolate points of access
to remote systems, services and 3rd party
libraries, stop cascading failure and enable
resilience in complex distributed systems
where failure is inevitable.”
Hystrix

40. @nicolas_frankel
• Provided by Netflix
• Currently in maintenance mode ⚠️
• Superseded by Resilience4J
• But not equivalent
Hystrix

41. @nicolas_frankel
• Wraps calls into “commands”
• Run commands asynchronously
from a thread pool
• Measure success/failures
• Circuit-breaker implementation
• Fallback logic
Hystrix features

42. @nicolas_frankel
• A lot of configuration options
• Hard to fine-tune
• No big picture
Cons of Hystrix

43. @nicolas_frankel
• Easy Hystrix integration
• Also:
• Service discovery: Eureka
• Declarative REST client: Feign
• Client-side LB: Ribbon
• etc.
Spring Cloud Netflix

44. @nicolas_frankel
“Resilience4j is a lightweight fault
tolerance library inspired by Netflix
Hystrix, but designed for Java 8 and
functional programming.”
Resilience4J

45. @nicolas_frankel
• Circuit Breaker
• Rate Limiter
• Retry
• Cache
• etc.
Resilience4J’s features

46. @nicolas_frankel
• Each feature is designed as a
function
• Uses Java 8 functional interfaces
• e.g. Supplier
• Based on function composition
• Based on Vavr
• Functional Programming in Java
Resilience4J’s design principles

47. @nicolas_frankel
• Need to be very familiar with
Functional Programming
• No big picture
Cons of Resilience4J

48. @nicolas_frankel

49. @nicolas_frankel
• https://blog.frankel.ch/
• @nicolas_frankel
• https://git.io/JenH9
Thanks