1.
@cdavisafc
CLOUD-NATIVE
DESIGNING CHANGE-TOLERANT SOFTWARE
Cornelia Davis, Sr. Director of Technology, Pivotal

2.
@cdavisafc
TEXT
To see this presentation delivered live, go to:
http://bit.ly/DesignPatternsReplay
(There is a special (discount) offer embedded within :-))

3.
@cdavisafc
TEXT
ME?
Developer (wasn’t Ops)
Web architectures for >10 years
Cloud-native for 5 years
Cloud Foundry for 5 years
More recently

4.
@cdavisafc
TEXT
A Seattle book store
deploys code, on average,
every second

5.
@cdavisafc
TEXT
A Seattle technology company
hosts applications for thousands
of major corporations

6.
@cdavisafc
TEXT
Major Outage on 20 September 2015
Airbnb, Nest, IMDB and many more experienced downtime
… Including Netflix
Outage lasted 5+ hours
“brief availability blip”
experienced a

7.
MICROSERVICES

8.
It is not that simple

9.
Mental Model

10.
@cdavisafc
▸ Cloud-Native App
(yup, the microservices)
APP APP
APP
APP
APP

11.
@cdavisafc
▸ Cloud-Native App
(yup, the microservices)
APP APP
APP
APP
APP

12.
@cdavisafc
▸ Cloud-Native App
(yup, the microservices)
▸ Cloud-Native Services
APP APP
APP
APP
APP

13.
@cdavisafc
▸ Cloud-Native App
(yup, the microservices)
▸ Cloud-Native Services
(yup, some are apps)
SERVICE
APP
SERVICE
APP
SERVICE
APP
SERVICE
APP
APP

14.
@cdavisafc
▸ Cloud-Native App
(yup, the microservices)
▸ Cloud-Native Services
(yup, some are apps
… and other are other
types of services)
SERVICE
APP
SERVICE
APP
SERVICE
APP
SERVICE
APP
DATA DATA
DATA
APP

15.
@cdavisafc
▸ Cloud-Native App
(yup, the microservices)
▸ Cloud-Native Services
(yup, some are apps
… and other are other
types of services)
▸ Cloud-Native Data
(a distributed data fabric)
SERVICE
APP
SERVICE
APP
SERVICE
APP
SERVICE
APP
DATA DATA
DATA
?
APP

16.
@cdavisafc
CLOUD-NATIVE SOFTWARE
▸ Cloud-Native App
(yup, the microservices)
▸ Cloud-Native Services
(yup, some are apps
… and other are other types
of services)
▸ Cloud-Native Data
(a distributed data fabric)
▸ The Cloud-Native Collective
(implicit and explicit connections)
APP
SERVICE
APP
SERVICE
APP
SERVICE
APP
SERVICE
APP
DATA DATA
DATA
?

17.
@cdavisafc
AGENDA (THE REST OF IT)
▸ Cloud-Native App
▸ Cloud-Native Services
▸ Cloud-Native Data
The Cloud-Native Collective

18.
CLOUD-NATIVE APP

19.
@cdavisafc
CLOUD-NATIVE APP
SCALE HORIZONTALLY
APP APP
APP
APP
APP

20.
@cdavisafc
CLOUD-NATIVE APP
APP APP
APP
APP
APP
APP APP
APP
APP
APP
APP APP
APP
APP
APP
SCALE HORIZONTALLY
APP APP
APP
APP
APP

21.
@cdavisafc
CLOUD-NATIVE APP
APPAPP
APPAPPAPPAPPAPP
APPAPPAPPAPPAPP
APP APP
APP
APP
APP APP
APP
APP
APP APP
APP
APP
APP
SCALE HORIZONTALLY
APP APP
APP
APP
APP

22.
@cdavisafc
CLOUD-NATIVE APP
APPAPP
APPAPPAPPAPPAPP
APPAPPAPPAPPAPP
APP APP
APP
APP
APP APP
APP
APP
APP APP
APP
APP
APP
SCALE HORIZONTALLY
▸ Need an abstraction to treat set of
instances as one logical unit
▸ Load Balancing - DNS? Something
else?
▸ Dynamic!!!
▸ IP Addresses of instances always
changing
APP APP
APP
APP
APP
ROUTER
ROUTER

23.
@cdavisafc
CLOUD-NATIVE APP
STATELESSNESS
ROUTER
APP
(INSTANCE 1)
ValidTokens:
[UserToken]
/login
token=UserToken
token=UserToken

24.
@cdavisafc
CLOUD-NATIVE APP
STATELESSNESS
ROUTER
APP
(INSTANCE 1)
ValidTokens:
[UserToken]
/login
token=UserToken
token=UserToken
APP
(INSTANCE 2)
ValidTokens: []

25.
@cdavisafc
CLOUD-NATIVE APP
STATELESSNESS
ROUTER
APP
(INSTANCE 1)
ValidTokens:
[UserToken]
/login
token=UserToken
token=UserToken
APP
(INSTANCE 2)
Unauthorized ValidTokens: []
token=UserToken

26.
@cdavisafc
CLOUD-NATIVE APP
STATELESSNESS
▸ Do NOT use sticky sessions!
▸ Eventually (and sooner than you
think!) the instance you are sticky to
will disappear
▸ Do make your instances
completely stateless
ROUTER
APP
(INSTANCE 1)
ValidTokens:
[UserToken]
/login
token=UserToken
token=UserToken
APP
(INSTANCE 2)
Unauthorized ValidTokens: []
token=UserToken

27.
@cdavisafc
CLOUD-NATIVE APP
STATELESSNESS
▸ Do NOT use sticky sessions!
▸ Eventually (and sooner than you
think!) the instance you are sticky to
will disappear
▸ Do make your instances
completely stateless
▸ Externalize state
ROUTER
APP
(INSTANCE 1)
ValidTokens:
[UserToken]
token=UserToken
token=UserToken
APP
(INSTANCE 2)
token=UserToken
STATE
STORE

28.
@cdavisafc
CLOUD-NATIVE APP
APP CONFIGURATION
▸ Property Files are the abstraction
used to externalize configuration
▸ They do not hold actual values!
▸ Values coming from the environment
are supplied via environment
variables
▸ Other configuration is controlled just
like source code and injected
RUNTIME CONTEXT
WHERE CONFIGURATION PARAMETERS ARE DEFINED IN ENVIRONMENT VARIABLES
PROPERTY FILE(S)
DEFINE THE CONFIGURATION
PARAMETERS AND DEFAULT VALUES
hostIp=
${CF_INSTANCE_IP}
specialization=
${SPECIALIZATION:
Science}
APP
CODE THAT REFERENCES THE CONFIGURATION PARAMETERS
DEFINED IN THE PROPERTY FILE
public class HelloController {
@Value("${hostIp}")
private String hostIp;
@Value("${specialization}")
private String specialization;
...
}
SYSTEM PROPERTIES
PROVIDED BY THE RUNTIME ENVIRONMENT, VIA
ENVIRONMENT VARIABLES
CF_INSTANCE_IP=10.10.148.29

29.
@cdavisafc
CLOUD-NATIVE APP
APP CONFIGURATION
▸ Other configuration is controlled just
like source code and injected
RUNTIME CONTEXT
WHERE CONFIGURATION PARAMETERS ARE DEFINED IN ENVIRONMENT VARIABLES
PROPERTY FILE(S)
DEFINE THE CONFIGURATION
PARAMETERS AND DEFAULT VALUES
hostIp=
${CF_INSTANCE_IP}
specialization=
${SPECIALIZATION:
Science}
APP
CODE THAT REFERENCES THE CONFIGURATION PARAMETERS
DEFINED IN THE PROPERTY FILE
public class HelloController {
@Value("${hostIp}")
private String hostIp;
@Value("${specialization}")
private String specialization;
...
}
SYSTEM PROPERTIES
PROVIDED BY THE RUNTIME ENVIRONMENT, VIA
ENVIRONMENT VARIABLES
CF_INSTANCE_IP=10.10.148.29
APP PROPERTIES
STORED AND VERSIONED IN A
REPOSITORY SUCH AS GIT
specialization
=Sports
CONFIG SERVER
ROLE IS TO INJECT MANAGED APP
PROPERTIES INTO THE APP

30.
@cdavisafc
CLOUD-NATIVE APP
APPLICATION LIFECYCLE
▸ Application lifecycle events have
rippling effects through the collective
INVOICE APP
ordersvcIP
=10.24.1.35
ORDER SERVICE APP
10.24.1.3510.24.63.116

31.
@cdavisafc
CLOUD-NATIVE APP
APPLICATION LIFECYCLE
▸ Application lifecycle events have
rippling effects through the collective
▸ The application must broadcast
lifecycle change events
▸ (Note: I strongly recommend you
use a framework to help you with
this!)
▸ And an app must be able to absorb
that configuration at the right time
(run time?)
INVOICE APP
ordersvcIP
=10.24.1.35
ORDER SERVICE APP
Some “magic” happens
10.24.63.116
Here I am!
IP Address: 10.24.63.116
Version: 3.4.239
…
I’ll adapt to these changes!
Orders Service new IP:
10.24.63.116

32.
CLOUD-NATIVE
SERVICE

33.
@cdavisafc
CLOUD-NATIVE SERVICE
LET’S TALK ABOUT AUTONOMY
A major benefit of cloud-native architectures is independent:
▸ Application Scaling
▸ Team Scaling
▸ Development Cycles
▸ Experimentation
▸ Resilience

34.
@cdavisafc
CLOUD-NATIVE SERVICE
VERSIONED SERVICES
▸ Use Semantic Versioning
▸ major.minor.patch
▸ Use Blue/Green deploys
▸ Deployments needn’t be replacements
▸ Powerful lever in making
deployments a non-event
SERVICE
APP
SERVICE
APP
APP
SERVICE
APP
APP
… AND PARALLEL DEPLOYS

35.
@cdavisafc
CLOUD-NATIVE SERVICE
VERSIONED SERVICES
▸ Use Semantic Versioning
▸ major.minor.patch
▸ Use Blue/Green deploys
▸ Deployments needn’t be replacements
▸ Powerful lever in making
deployments a non-event
SERVICE
APP
SERVICE
APP
APP
SERVICE
APP
APP
… AND PARALLEL DEPLOYS
SERVICE
APP
1.0.0 2.0.0

36.
@cdavisafc
CLOUD-NATIVE SERVICE
SERVICE DISCOVERY
▸ Let’s dig into the “magic”
(it’s not really magic)
INVOICE APP
orderSvcCoords...
ORDER SERVICE APP
10.24.63.116
Here I am!
IP Address: 10.24.63.116
Version: 3.4.239
…
I’ll adapt to these changes!
Orders Service new IP:
10.24.63.116
Some “magic” happens

37.
@cdavisafc
CLOUD-NATIVE SERVICE
SERVICE DISCOVERY
▸ Dynamic Router maintains routing
table
▸ Messaging pattern to decouple
service from router
▸ Must handle lost broadcast
messages
▸ Request pass through router INVOICE APP
orderSvcCoords
=orderSvc.example.com
ORDER SERVICE APP
10.24.63.116
Here I am!
IP Address: 10.24.63.116
Version: 3.4.239
…
ROUTER
OrderService: 10.24.63.116, …
CustomerService: 10.24.3.1, …

38.
@cdavisafc
CLOUD-NATIVE SERVICE
SERVICE DISCOVERY
▸ Dynamic Router maintains routing
table
▸ Messaging pattern to decouple
service from router
▸ Must handle lost broadcast
messages
▸ Request pass through router INVOICE APP
orderSvcCoords
=10.24.1.13
ORDER SERVICE APP
10.24.63.116
Here I am!
IP Address: 10.24.63.116
Version: 3.4.239
…
SERVICE DISCOVERY SERVER
OrderService: 10.24.63.116, …
CustomerService: 10.24.3.1, …
ORDER SERVICE APP
10.24.1.13
I’ll adapt to these changes!
Orders Service new IP:
10.24.63.116

39.
@cdavisafc
CLOUD-NATIVE SERVICE
SERVICE DISCOVERY
▸ Dynamic Router maintains routing
table
▸ Messaging pattern to decouple
service from router
▸ Must handle lost broadcast
messages
▸ Request pass through router INVOICE APP
orderSvcCoords
=10.24.63.116
ORDER SERVICE APP
10.24.63.116
Here I am!
IP Address: 10.24.63.116
Version: 3.4.239
…
SERVICE DISCOVERY SERVER
OrderService: 10.24.63.116, …
CustomerService: 10.24.3.1, …
ORDER SERVICE APP
10.24.1.13
I’ll adapt to these changes!
Orders Service new IP:
10.24.63.116

40.
@cdavisafc
CLOUD-NATIVE SERVICE
CIRCUIT BREAKERS
▸ You’ve probably seen this picture…
https://martinfowler.com/bliki/CircuitBreaker.html

41.
@cdavisafc
CLOUD-NATIVE SERVICE
CIRCUIT BREAKERS
▸ You’ve probably seen this picture…
▸ … but why do we need circuit breakers
in the first place
https://martinfowler.com/bliki/CircuitBreaker.html

42.
@cdavisafc
CLOUD-NATIVE SERVICE
RETRIES
▸ #1 Fallacy of Distributed Computing:
https://en.wikipedia.org/wiki/Fallacies_of_distributed_computing

43.
@cdavisafc
CLOUD-NATIVE SERVICE
RETRIES
▸ #1 Fallacy of Distributed Computing:
The Network is Reliable
https://en.wikipedia.org/wiki/Fallacies_of_distributed_computing

44.
@cdavisafc
CLOUD-NATIVE SERVICE
RETRIES
▸ #1 Fallacy of Distributed Computing:
The Network is Reliable
It's not.
https://en.wikipedia.org/wiki/Fallacies_of_distributed_computing

45.
@cdavisafc
CLOUD-NATIVE SERVICE
RETRIES
CLIENT
SERVICE
▸ Client must consider failure
▸ Decide on fall-back behavior
▸ Likely including retries
▸ But then we need to handle
downstream consequences of these
(retry) behaviors
Timeouts?
If we don’t hear back,
try again

46.
@cdavisafc
CLOUD-NATIVE SERVICE
CIRCUIT BREAKERS
▸ This is your protection against
DDOSing yourself
https://martinfowler.com/bliki/CircuitBreaker.html

47.
@cdavisafc
CLOUD-NATIVE SERVICE
DISTRIBUTED TRACING

48.
@cdavisafc
CLOUD-NATIVE SERVICE

49.
@cdavisafc
CLOUD-NATIVE SERVICE
DISTRIBUTED TRACING
▸ App must look for and leave tracers
▸ Use a framework to help you do
this!!!
▸ Tools can then read and correlate logs
SERVICE
APP
SERVICE
APP
APP
SERVICE
APP
SERVICE
APP
SERVICE
APP

50.
CLOUD-NATIVE DATA

51.
@cdavisafc
CLOUD-NATIVE DATA
WE ARE LOOSELY COUPLED!

52.
@cdavisafc
CLOUD-NATIVE DATA
OR ARE WE?

53.
@cdavisafc
CLOUD-NATIVE DATA
THIS IS NOT CLOUD-NATIVE DATA

54.
@cdavisafc
CLOUD-NATIVE DATA
DATA APIS (SERVICES)
▸ Microservices do not access data layer directly
▸ Except for those that implement the data API
▸ A surface area to:
▸ Implement access control
▸ Implement throttling
▸ Perform logging
▸ Other policies…

55.
@cdavisafc
CLOUD-NATIVE DATA
ANTI-PATTERN: DATA APIS THAT JUST PROXY

56.
@cdavisafc
CLOUD-NATIVE DATA
VERSIONED DATA APIS (SERVICES)
▸ We are already familiar with versioned
microservices…
V1 V2

57.
@cdavisafc
CLOUD-NATIVE DATA
VERSIONED DATA APIS (SERVICES)
▸ We are already familiar with versioned
microservices…
… and
V1 V2
PARALLEL DEPLOYS

58.
@cdavisafc
CLOUD-NATIVE DATA
EVERY MICROSERVICE NEEDS A CACHE

59.
@cdavisafc
CLOUD-NATIVE DATA
“Caching at Netflix: The Hidden Microservice” https://www.youtube.com/watch?v=Rzdxgx3RC0Q

60.
@cdavisafc
TEXT
We used to:
AUTONOMY!
V2
V1
All clients moved in lockstep
Or another:
V1.0
V1.1 V2.0
Now, one option:
V1.0
V1.1
V2.0
V2.1
Or even:
V1.0 V2.0
(!) Think Unified Log

61.
@cdavisafc
CLOUD-NATIVE DATA
DATABASE PER MICROSERVICES

62.
@cdavisafc
CLOUD-NATIVE DATA
DATABASE PER MICROSERVICES
▸ Supports Polyglot persistence
▸ Independent availability, backup/restore, access
patterns, etc.

63.
@cdavisafc
CLOUD-NATIVE DATA
EVENT LOG AS A SYSTEM OF RECORD
Unified Log
Source of
Record
Teaser for the next webinar…
… It’s not just Request/Response!
Event-driven microservice architectures!

64.
@cdavisafc
CLOUD-NATIVE
▸ Cloud-native Apps
▸ Scale out
▸ Statelessness
▸ Externalize configuration
▸ Implications from changes in the application lifecycle
▸ Services
▸ Versioned services (autonomy)
▸ Service Discovery
▸ Distributed tracing
▸ Retries
▸ Data
▸ Breaking the Data monolith
▸ Data APIs
▸ Caching
▸ Polyglot Persistence
▸ Event sourcing

65.
@cdavisafc
REFERENCES
▸ Scott Mansfield of the Netflix Caching team:
▸ Public video: https://www.youtube.com/watch?
v=Rzdxgx3RC0Q&t=205s
▸ Safari books online video (slightly expanded): https://
www.safaribooksonline.com/library/view/oreilly-
software-architecture/9781491976142/
video289789.html
▸ Slideshare: https://www.slideshare.net/ScottMansfield3/
application-caching-the-hidden-microservice
▸ Netflix on the “brief availability blip”: https://medium.com/
netflix-techblog/chaos-engineering-
upgraded-878d341f15fa

66.
THANK YOU
https://content.pivotal.io/blog/cloud-native-and-the-apparating-app