This document discusses creating software architectures that are resilient to stressors. It begins with defining key architectural concepts like what architecture is and what "bend but don't break" means in the context of withstanding stresses. It then explores common software stressors that architectures need to handle, such as failures, unexpected success, or new requirements. The document outlines desired qualities like resilience and innovation. It proposes ways to measure these qualities, including metrics like lead time, error rates, and time to recovery. Finally, it discusses how to design architectures that can adapt to stresses while continuing to function.

1. Architectures that Bend but Don’t
Break
Matt Stine
Global Field CTO / Chief Architect
@mstine
http://www.mattstine.com

2. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Who is this guy?
Matt Stine - Global Field CTO / Chief Architect
at Pivotal
18 year Enterprise IT veteran developer / architect
6 years focused on Cloud Native Platforms and
Architecture
Frequent conference circuit speaker
Host of Software Architecture Radio
http://softwarearchitecturerad.io
I wrote a little cloud book…
https://www.oreilly.com/programming/free/
migrating-cloud-native-application-
architectures.csp
2

3. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
3
What
even is
architecture
?
What
does “bend
but not break”
mean?
What are
software
stressors?
How do
we create
those
qualities?
How do we
measure
those
What
software
qualities do
we want?

4. 4Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
What even is
architecture?

5. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 5
Architecture: decisions that are hard to reverse.
Matt Parker
Software Architecture: the high level shape and
flow of the software that’s independent of the
problem domain, but dependent on the desired
user experience.

6. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 6
Architecture represents the significant
design decisions that shape a system,
where significance is measured by cost of
change.
Simon Brown

7. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 7
Meaningful architecture is a living, vibrant
process of deliberation, design, & decision,
not just documentation.
Grady Booch

8. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 8
The important stuff (whatever that is).
Ralph Johnson

9. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 9
Architecture is the process by which we
make and capture software engineering
decisions that have system-wide impact
and are often expensive to reverse.
Matt’s Definition

10. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
10
What
even is
architecture
?
What
does “bend
but not break”
mean?

11. 11Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
What
does “bend
but not break”
mean?

12. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 12
The oak fought the wind and was broken, the willow
bent when it must and survived.
― Robert Jordan, The Fires of Heaven

13. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 13
Bend but
Don’t Break
Defense (2017)
SOURCE: https://www.patspulpit.com/2018/1/6/16855296/new-england-patriots-defense-took-bend-but-dont-break-to-a-new-level-in-2017
■ Yards per Drive:
32nd (i.e. WORST)
■ Points per Drive:
6th

14. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Break vs. Bend
Break
■ Fragile
Stress → Destruction
■ Robust
Stress → Resistance
14
Bend
■ Resilient
Stress → Adaptation
■ Antifragile
Stress → Innovation

15. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 15
CREDIT: Bilgin Ibryam - https://developers.redhat.com/blog/2016/07/20/from-fragile-to-antifragile-software/

16. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 16
Resilience is the ability of a system to adapt
or keep working when challenges occur.
Resilience
CREDIT: Ines Sombra - https://speakerdeck.com/randommood/full-stack-fest-architectural-patterns-of-resilient-distributed-systems

17. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Defining Resilience
■ Fault-tolerance
■ Evolvability
■ Scalability
■ Failure isolation
■ Complexity Management
17
CREDIT: Ines Sombra - https://speakerdeck.com/randommood/full-stack-fest-architectural-patterns-of-resilient-distributed-systems

18. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 18

19. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Immune Systems
19

20. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Muscular Systems
20

21. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Ant Swarms
21

22. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 22

23. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Software That Improves Itself?
23

24. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Socio-technical Systems
24
!
" #!
" #
!
" #
!
" #

25. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
25
What
even is
architecture
?
What
does “bend
but not break”
mean?
What are
software
stressors?

26. 26Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
What are
software
stressors?

27. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Russ Miles’ Stressors
■ Innovation
■ System Wildly Successful!
■ System Hardly Used
■ System Failures
■ Emergent Need to Reuse
27

28. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
28
What
even is
architecture
?
What
does “bend
but not break”
mean?
What are
software
stressors?
What
software
qualities do
we want?

29. 29Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
What
software
qualities do
we want?

30. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 30
Can we continuously deliver incremental
value to the business through software?
Innovation

31. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 31
Can we deliver systems that are able to
survive and thrive amidst turbulent
conditions in production?
Wild Success / System Failure

32. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
32
What
even is
architecture
?
What
does “bend
but not break”
mean?
What are
software
stressors?
How do we
measure
those
What
software
qualities do
we want?

33. 33Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
How
do we
measure those
qualities?

34. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 34
LEAD TIME: how long does it take you to
go from idea to delivered software?
Innovation

35. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 35
CYCLE TIME: how long does it take you to
make a change and deliver it to production?
Innovation

36. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 36
INVERSE LEAD TIME: how long does it
take for feedback to enter your
development queue?
Innovation

37. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 37
TRASH RATIO: how many of your
experiments do you end up discarding?
Innovation

38. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 38
■
Service Level Objectives
(SLOs)
■
Steady State Metrics
Resilience

39. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 39

40. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
SRE Measures
■ Service Level Indicator:
Quantitative measure of an aspect of level of service provided (e.g.
request latency)
■ Service Level Objective:
Target value/range of values for a service level measured by an SLI
40

41. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Focus on Distributions
41
Not on Averages!
https://landing.google.com/sre/book/chapters/service-level-objectives.html

42. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Four Golden Signals
■ Latency:
Time it takes to service a request.
■ Traffic:
How much demand is placed on the system.
■ Errors:
Rate of requests that fail.
■ Saturation:
How "full" your service is.
42

43. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Steady State
43
Some externally visible business metric with
a low degree of variation.

44. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Netflix: SPS
44
Successful Streaming Starts per Second

45. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Starts Per Second
45
A single indicator that closely approximates our most important activity: viewing.
A single indicator that closely approximates our
most important activity: viewing.

46. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 46
WHAT IS YOUR MOST IMPORTANT ACTIVITY?

47. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 47
WHAT SINGLE INDICATOR COULD YOU USE TO
APPROXIMATE IT?

48. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
The Ultimate Metric
48
TTRTime to Recovery

49. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
49
What
even is
architecture
?
What
does “bend
but not break”
mean?
What are
software
stressors?
How do
we create
those
qualities?
How do we
measure
those
What
software
qualities do
we want?

50. 50Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
How do
we create
those
qualities?

51. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 51
Can we continuously deliver incremental
value to the business through software?
Innovation

52. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 52

53. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 53
This paper discusses modularization as a
mechanism for improving the flexibility and
comprehensibility of a system while allowing
the shortening of its development time. The
effectiveness of a "modularization” is
dependent upon the criteria used in dividing
the system into modules.

54. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 54
The major progress in the area of modular
programming has been the development of
coding techniques and assemblers which (1)
allow one module to be written with little
knowledge of the code used in another
module and, (2) allow modules to be
reassembled and replaced without
reassembly of the whole system.

55. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 55
Let’s build an online store!
Decomposition Techniques

56. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 56
Product
Catalog
Orders Inventory Payment Fulfillment Shipping
Notification
s

57. 57
MVC Controllers / Views
Services
Domain Model
Data
Services
UI

58. Functional Decomposition
Independent
Development
Dependent on shared
data formats and
schema. Must be jointly
defined and agreed
upon across multiple
groups.
Comprehensibility
You need to know
something about how
all of the modules work
to understand the whole
system.
Changes
Struggles with
knowledge spread
across all modules, so
often each module has
to change in response
to a desired functional
change.
58Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/

59. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 59
Let’s Refactor to a
Capability Decomposition
Decomposition Techniques

60. 60

61. Capability Decomposition
Independent
Development
Has abstract interfaces
that encapsulate the
work to be done.
Comprehensibility
You can understand
modules independently.
Changes
Usually isolates a
change to a single
module.
61Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/

62. hexagonal
architecture
DOMAIN
DRIVEN
DESIGN

63. Strategic Design - the View from 35,000
FEET

64. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 64
!
!
!
"
"
"
πελάτης
πελάτης
πελάτης
πελάτης
πελάτης
πελάτης
!
!
!
"
"
"
顧客
顧客
顧客
顧客
顧客
顧客
!
!
!
"
"
"
ग्राहक
ग्राहक
ग्राहक
ग्राहक
ग्राहक
ग्राहक
Bounded
Contexts

65. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
■ Business Invariants
■ Policies
■ Transactions
■ State
■ Persistence
65
Now it’s time to start speaking the
Ubiquitous Language within each Bounded Context.
Tactical Design?
■ Business Invariants
■ Policies
■ Transactions
■ State
■ Persistence

66. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
The Central Concept
66
Aggregate
A cluster of objects
treated as a single unit.
Often modeled as a
state machine.
The atomic unit for any
transactional
behavior.
Only accessed through its
Root Entity.
Responsible for maintaining
any/all business invariants.
Entity
EntityEntity
Value
Object
Value
Object
Value
Object

67. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
67
Application Service
Domain
Model
Adapte
r
Port
Port
Port
Port
Port Port
Adapte
r
Adapte
r
Adapte
r
Adapte
r
Hexagonal
Architecture
■ Domain Model (Aggregates, etc.)
wrapped by an Application Service
■ Ports represent generic
entry/exit points (interface)
■ Adapters represent specific
entry/exit implementations (class)

68. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
68
Adapte
r
Port
Port
Port
Port
Port Port
Adapte
r
Adapte
r
Adapte
r
Adapte
rOrder
(root)
Line Item
Order ID
Order
Repository
Order Placed
Application Service
Hexagonal
Architecture
■ Domain Model (Aggregates, etc.)
wrapped by an Application Service
■ Ports represent generic
entry/exit points (interface)
■ Adapters represent specific
entry/exit implementations (class)

69. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Adapter
Port
Port
Port
Port
Port Port
Adapter
Adapter
Adapter
Adapte
r
Adapte
r
Adapte
r
Adapte
r
Adapte
r
REST
SOAP
AMQP
Adapter
Adapte
r
Kafka
JDBC
69
Application Service
Order
(root)
Line Item
Order ID
Order
Repository
Order Placed
Hexagonal
Architecture
■ Domain Model (Aggregates, etc.)
wrapped by an Application Service
■ Ports represent generic
entry/exit points (interface)
■ Adapters represent specific
entry/exit implementations (class)

70. 70
Software
ModuleAggregate
Entity
EntityEntity
Value
Object
Value
Object
Value
Object
≤
!"
ग्राहकग्राहक
Aggregate
Aggregate≤
Aggregate ≤ SizeOf(Module) ≤ Bounded Context
The Golden Rule

71. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Deployment Topology Spectrum
71
Big
Ball
of
Mud
Monolith Modular Monolith Microservices

72. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 72
■ Individually Deployable
■ Individually Upgradeable
■ Individually Replaceable
■ Individually Scalable
■ Heterogeneous Tech Stacks
Modular Monolith Microservices
■ High Cohesion
■ Low Coupling
■ Business Capability Focus
■ Bounded Contexts / Aggregates
■ Data Encapsulation
■ Substitutable
■ Composable

73. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 73
Can we deliver systems that are able to
survive and thrive amidst turbulent
conditions in production?
Wild Success / System Failure

74. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 74
Leverage Patterns
Resilience

75. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Timeouts
75

76. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Retries
76

77. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Bulkheads
77

78. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Circuit Breakers
78

79. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 79

80. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 80
Embrace Chaos
Antifragility

81. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Socio-technical Systems
81
!
" #!
" #
!
" #
!
" #

82. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 82
https://principlesofchaos.org/

83. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 83

84. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Principles of Chaos Engineering
84
■ Build a Hypothesis around Steady State Behavior
■ Vary Real-world Events
■ Run Experiments in Production
■ Automate Experiments to Run Continuously
■ Minimize Blast Radius

85. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Steady State Behavior
85
A business-facing metric with predictable pattern that fluctuates
significantly when system problems are happening.

86. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Vary Real World Events
86
■ Server Crashes
■ Storage Failures
■ Error Responses
■ Increased Latency between Nodes
■ AZ or Region Failures
■ Thundering Herds

87. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Run Experiments in Production
87
■ Difficult to exactly duplicate the
production environment
■ Impossible to exactly duplicate
production traffic patterns

88. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 88
RUNNING EXPERIMENTS MANUALLY IS
UNSUSTAINABLE
Automate Experiments to Run Continuously

89. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 89
CUSTOMER PAIN WILL HAPPEN.
TAKE STEPS TO ENSURE FALLOUT IS
MINIMIZED AND CONTAINED.
Minimize the Blast Radius

90. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 90

91. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
91
What
even is
architecture
?
What
does “bend
but not break”
mean?
What are
software
stressors?
How do
we create
those
qualities?
How do we
measure
those
What
software
qualities do
we want?

92. > Stay Connected.
Matt Stine
Global Field CTO / Chief Architect
@mstine
http://www.mattstine.com
#springon@s1
See also these sessions:
Nate Schutta - Thinking Architecturally
Matt Parker - Agile Architecture