1.
@cdavisafc
CLOUD-NATIVE
DESIGNING CHANGE-TOLERANT SOFTWARE
Cornelia Davis, Sr. Director of Technology, Pivotal

2.
@cdavisafc
FROM THE 2017 STATE OF DEVOPS REPORT

3.
@cdavisafc
https://www.forbes.com/sites/louiscolumbus/2017/02/18/rightscale-2017-state-of-the-cloud-report-azure-gaining-in-enterprises
FROM THE 2017 STATE OF CLOUD REPORT

4.
@cdavisafc
CLOUD-NATIVE = LOOSELY COUPLED RUNNING IN THE CLOUD
Cloud is where you compute
Cloud-native is how.

5.
@cdavisafc
TEXT
ME?
Developer (wasn’t Ops)
Web architectures for >10 years
Cloud-native for 5 years
Cloud Foundry for 5 years
More recently

6.
@cdavisafc
TEXT
A Seattle technology company
hosts applications for thousands
of major corporations

7.
@cdavisafc
TEXT
Major Outage on 20 September 2015
Airbnb, Nest, IMDB and many more experienced downtime
… Including Netflix
Outage lasted 5+ hours
“brief availability blip”
experienced a

8.
@cdavisafc
CLOUD-NATIVE SERVICE
LET’S TALK ABOUT AUTONOMY
A major benefit of cloud-native architectures is independent:
▸ Application Scaling
▸ Team Scaling
▸ Development Cycles
▸ Experimentation
▸ Resilience

9.
@cdavisafc
CLOUD-NATIVE ARCHITECTURES
▸ Cloud-native Apps
▸ Scale out
▸ Statelessness
▸ Externalize configuration
▸ Implications from changes in the application
lifecycle
▸ Versioned services
▸ Service Discovery
▸ Distributed tracing
▸ Retries
▸ Cloud-native Data
▸ Breaking the Data monolith
▸ Data APIs
▸ Caching
▸ Polyglot Persistence
▸ Event sourcing

10.
CLOUD-NATIVE APP

11.
@cdavisafc
CLOUD-NATIVE APP
SCALE HORIZONTALLY
APP APP
APP
APP
APP

12.
@cdavisafc
CLOUD-NATIVE APP
APP APP
APP
APP
APP
APP APP
APP
APP
APP
APP APP
APP
APP
APP
SCALE HORIZONTALLY
APP APP
APP
APP
APP

13.
@cdavisafc
CLOUD-NATIVE APP
APPAPP
APPAPPAPPAPPAPP
APPAPPAPPAPPAPP
APP APP
APP
APP
APP APP
APP
APP
APP APP
APP
APP
APP
SCALE HORIZONTALLY
APP APP
APP
APP
APP

14.
@cdavisafc
CLOUD-NATIVE APP
APPAPP
APPAPPAPPAPPAPP
APPAPPAPPAPPAPP
APP APP
APP
APP
APP APP
APP
APP
APP APP
APP
APP
APP
SCALE HORIZONTALLY
▸ Need an abstraction to treat set of
instances as one logical unit
▸ Load Balancing - DNS? Something
else?
▸ Dynamic!!!
▸ IP Addresses of instances always
changing
APP APP
APP
APP
APP
ROUTER
ROUTER

15.
@cdavisafc
CLOUD-NATIVE APP
STATELESSNESS
ROUTER
APP
(INSTANCE 1)
ValidTokens:
[UserToken]
/login
token=UserToken
token=UserToken

16.
@cdavisafc
CLOUD-NATIVE APP
STATELESSNESS
ROUTER
APP
(INSTANCE 1)
ValidTokens:
[UserToken]
/login
token=UserToken
token=UserToken
APP
(INSTANCE 2)
ValidTokens: []

17.
@cdavisafc
CLOUD-NATIVE APP
STATELESSNESS
ROUTER
APP
(INSTANCE 1)
ValidTokens:
[UserToken]
/login
token=UserToken
token=UserToken
APP
(INSTANCE 2)
Unauthorized ValidTokens: []
token=UserToken

18.
@cdavisafc
CLOUD-NATIVE APP
STATELESSNESS
▸ Do NOT use sticky sessions!
▸ Eventually (and sooner than you
think!) the instance you are sticky to
will disappear
▸ Do make your instances
completely stateless
ROUTER
APP
(INSTANCE 1)
ValidTokens:
[UserToken]
/login
token=UserToken
token=UserToken
APP
(INSTANCE 2)
Unauthorized ValidTokens: []
token=UserToken

19.
@cdavisafc
CLOUD-NATIVE APP
STATELESSNESS
▸ Do NOT use sticky sessions!
▸ Eventually (and sooner than you
think!) the instance you are sticky to
will disappear
▸ Do make your instances
completely stateless
▸ Externalize state
ROUTER
APP
(INSTANCE 1)
ValidTokens:
[UserToken]
token=UserToken
token=UserToken
APP
(INSTANCE 2)
token=UserToken
STATE
STORE

20.
@cdavisafc
CLOUD-NATIVE APP
APP CONFIGURATION
▸ Property Files are the abstraction
used to externalize configuration
▸ They do not hold actual values!
▸ Values coming from the environment
are supplied via environment
variables
▸ Other configuration is controlled just
like source code and injected
RUNTIME CONTEXT
WHERE CONFIGURATION PARAMETERS ARE DEFINED IN ENVIRONMENT VARIABLES
PROPERTY FILE(S)
DEFINE THE CONFIGURATION
PARAMETERS AND DEFAULT VALUES
hostIp=
${CF_INSTANCE_IP}
specialization=
${SPECIALIZATION:
Science}
APP
CODE THAT REFERENCES THE CONFIGURATION PARAMETERS
DEFINED IN THE PROPERTY FILE
public class HelloController {
@Value("${hostIp}")
private String hostIp;
@Value("${specialization}")
private String specialization;
...
}
SYSTEM PROPERTIES
PROVIDED BY THE RUNTIME ENVIRONMENT, VIA
ENVIRONMENT VARIABLES
CF_INSTANCE_IP=10.10.148.29

21.
@cdavisafc
CLOUD-NATIVE APP
APP CONFIGURATION
▸ Other configuration is controlled just
like source code and injected
RUNTIME CONTEXT
WHERE CONFIGURATION PARAMETERS ARE DEFINED IN ENVIRONMENT VARIABLES
PROPERTY FILE(S)
DEFINE THE CONFIGURATION
PARAMETERS AND DEFAULT VALUES
hostIp=
${CF_INSTANCE_IP}
specialization=
${SPECIALIZATION:
Science}
APP
CODE THAT REFERENCES THE CONFIGURATION PARAMETERS
DEFINED IN THE PROPERTY FILE
public class HelloController {
@Value("${hostIp}")
private String hostIp;
@Value("${specialization}")
private String specialization;
...
}
SYSTEM PROPERTIES
PROVIDED BY THE RUNTIME ENVIRONMENT, VIA
ENVIRONMENT VARIABLES
CF_INSTANCE_IP=10.10.148.29
APP PROPERTIES
STORED AND VERSIONED IN A
REPOSITORY SUCH AS GIT
specialization
=Sports
CONFIG SERVER
ROLE IS TO INJECT MANAGED APP
PROPERTIES INTO THE APP

22.
@cdavisafc
CLOUD-NATIVE APP
APPLICATION LIFECYCLE
▸ Application lifecycle events have
rippling effects through the collective
INVOICE APP
ordersvcIP
=10.24.1.35
ORDER SERVICE APP
10.24.1.3510.24.63.116

23.
@cdavisafc
CLOUD-NATIVE APP
APPLICATION LIFECYCLE
▸ Application lifecycle events have
rippling effects through the collective
▸ The application must broadcast
lifecycle change events
▸ (Note: I strongly recommend you
use a framework to help you with
this!)
▸ And an app must be able to absorb
that configuration at the right time
(run time?)
INVOICE APP
ordersvcIP
=10.24.1.35
ORDER SERVICE APP
Some “magic” happens
10.24.63.116
Here I am!
IP Address: 10.24.63.116
Version: 3.4.239
…
I’ll adapt to these changes!
Orders Service new IP:
10.24.63.116

24.
@cdavisafc
CLOUD-NATIVE SERVICE
VERSIONED SERVICES
▸ Use Semantic Versioning
▸ major.minor.patch
▸ Use Blue/Green deploys
▸ Deployments needn’t be replacements
▸ Powerful lever in making
deployments a non-event
SERVICE
APP
SERVICE
APP
APP
SERVICE
APP
APP
… AND PARALLEL DEPLOYS

25.
@cdavisafc
CLOUD-NATIVE SERVICE
VERSIONED SERVICES
▸ Use Semantic Versioning
▸ major.minor.patch
▸ Use Blue/Green deploys
▸ Deployments needn’t be replacements
▸ Powerful lever in making
deployments a non-event
SERVICE
APP
SERVICE
APP
APP
SERVICE
APP
APP
… AND PARALLEL DEPLOYS
SERVICE
APP
1.0.0 2.0.0

26.
@cdavisafc
CLOUD-NATIVE SERVICE
SERVICE DISCOVERY
▸ Let’s dig into the “magic”
(it’s not really magic)
INVOICE APP
orderSvcCoords...
ORDER SERVICE APP
10.24.63.116
Here I am!
IP Address: 10.24.63.116
Version: 3.4.239
…
I’ll adapt to these changes!
Orders Service new IP:
10.24.63.116
Some “magic” happens

27.
@cdavisafc
CLOUD-NATIVE SERVICE
SERVICE DISCOVERY
▸ Dynamic Router maintains routing
table
▸ Messaging pattern to decouple
service from router
▸ Must handle lost broadcast
messages
▸ Request pass through router INVOICE APP
orderSvcCoords
=orderSvc.example.com
ORDER SERVICE APP
10.24.63.116
Here I am!
IP Address: 10.24.63.116
Version: 3.4.239
…
ROUTER
OrderService: 10.24.63.116, …
CustomerService: 10.24.3.1, …

28.
@cdavisafc
CLOUD-NATIVE SERVICE
SERVICE DISCOVERY
▸ Dynamic Router maintains routing
table
▸ Messaging pattern to decouple
service from router
▸ Must handle lost broadcast
messages
▸ Request pass through router INVOICE APP
orderSvcCoords
=10.24.1.13
ORDER SERVICE APP
10.24.63.116
Here I am!
IP Address: 10.24.63.116
Version: 3.4.239
…
SERVICE DISCOVERY SERVER
OrderService: 10.24.63.116, …
CustomerService: 10.24.3.1, …
ORDER SERVICE APP
10.24.1.13
I’ll adapt to these changes!
Orders Service new IP:
10.24.63.116

29.
@cdavisafc
CLOUD-NATIVE SERVICE
SERVICE DISCOVERY
▸ Dynamic Router maintains routing
table
▸ Messaging pattern to decouple
service from router
▸ Must handle lost broadcast
messages
▸ Request pass through router INVOICE APP
orderSvcCoords
=10.24.63.116
ORDER SERVICE APP
10.24.63.116
Here I am!
IP Address: 10.24.63.116
Version: 3.4.239
…
SERVICE DISCOVERY SERVER
OrderService: 10.24.63.116, …
CustomerService: 10.24.3.1, …
ORDER SERVICE APP
10.24.1.13
I’ll adapt to these changes!
Orders Service new IP:
10.24.63.116

30.
@cdavisafc
CLOUD-NATIVE SERVICE
CIRCUIT BREAKERS
▸ You’ve probably seen this picture…
https://martinfowler.com/bliki/CircuitBreaker.html

31.
@cdavisafc
CLOUD-NATIVE SERVICE
CIRCUIT BREAKERS
▸ You’ve probably seen this picture…
▸ … but why do we need circuit breakers
in the first place
https://martinfowler.com/bliki/CircuitBreaker.html

32.
@cdavisafc
CLOUD-NATIVE SERVICE
RETRIES
▸ #1 Fallacy of Distributed Computing:
https://en.wikipedia.org/wiki/Fallacies_of_distributed_computing

33.
@cdavisafc
CLOUD-NATIVE SERVICE
RETRIES
▸ #1 Fallacy of Distributed Computing:
The Network is Reliable
https://en.wikipedia.org/wiki/Fallacies_of_distributed_computing

34.
@cdavisafc
CLOUD-NATIVE SERVICE
RETRIES
▸ #1 Fallacy of Distributed Computing:
The Network is Reliable
It's not.
https://en.wikipedia.org/wiki/Fallacies_of_distributed_computing

35.
@cdavisafc
CLOUD-NATIVE SERVICE
RETRIES
CLIENT
SERVICE
▸ Client must consider failure
▸ Decide on fall-back behavior
▸ Likely including retries
▸ But then we need to handle
downstream consequences of these
(retry) behaviors
Timeouts?
If we don’t hear back,
try again

36.
@cdavisafc
CLOUD-NATIVE SERVICE
CIRCUIT BREAKERS
▸ This is your protection against
DDOSing yourself
https://martinfowler.com/bliki/CircuitBreaker.html

37.
@cdavisafc
CLOUD-NATIVE SERVICE
DISTRIBUTED TRACING

38.
@cdavisafc
CLOUD-NATIVE SERVICE

39.
@cdavisafc
CLOUD-NATIVE SERVICE
DISTRIBUTED TRACING
▸ App must look for and leave tracers
▸ Use a framework to help you do
this!!!
▸ Tools can then read and correlate logs
SERVICE
APP
SERVICE
APP
APP
SERVICE
APP
SERVICE
APP
SERVICE
APP

40.
CLOUD-NATIVE DATA

41.
@cdavisafc
CLOUD-NATIVE DATA
WE ARE LOOSELY COUPLED!

42.
@cdavisafc
CLOUD-NATIVE DATA
OR ARE WE?

43.
@cdavisafc
CLOUD-NATIVE DATA
THIS IS NOT CLOUD-NATIVE DATA

44.
@cdavisafc
CLOUD-NATIVE DATA
DATA APIS (SERVICES)
▸ Microservices do not access data layer directly
▸ Except for those that implement the data API
▸ A surface area to:
▸ Implement access control
▸ Implement throttling
▸ Perform logging
▸ Other policies…

45.
@cdavisafc
CLOUD-NATIVE DATA
ANTI-PATTERN: DATA APIS THAT JUST PROXY

46.
@cdavisafc
CLOUD-NATIVE DATA
EVERY MICROSERVICE NEEDS A CACHE

47.
@cdavisafc
CLOUD-NATIVE DATA
“Caching at Netflix: The Hidden Microservice” https://www.youtube.com/watch?v=Rzdxgx3RC0Q

48.
@cdavisafc
CLOUD-NATIVE DATA
VERSIONED DATA APIS (SERVICES)
▸ We are already familiar with versioned
microservices…
V1 V2

49.
@cdavisafc
CLOUD-NATIVE DATA
VERSIONED DATA APIS (SERVICES)
▸ We are already familiar with versioned
microservices…
… and
V1 V2
PARALLEL DEPLOYS

50.
@cdavisafc
TEXT
We used to:
AUTONOMY!
V2
V1
All clients moved in lockstep
Or another:
V1.0
V1.1 V2.0
Now, one option:
V1.0
V1.1
V2.0
V2.1
Or even:
V1.0 V2.0
(!) Think Unified Log

51.
@cdavisafc
CLOUD-NATIVE DATA
DATABASE PER MICROSERVICES

52.
@cdavisafc
CLOUD-NATIVE DATA
DATABASE PER MICROSERVICES
▸ Supports Polyglot persistence
▸ Independent availability, backup/restore, access
patterns, etc.

53.
My
Connections
API
Posts
API
New from
Network
API
CLOUD-NATIVE DATA

54.
@cdavisafc
CLOUD-NATIVE DATA
CLIENT SIDE JOIN My
Connections
API
Posts
API
New from
Network
API

55.
@cdavisafc
EVENT-DRIVEN SYSTEMS
My
Connections
API
Posts
API
New from
Network
API
CLOUD-NATIVE DATA

56.
@cdavisafc
CLOUD-NATIVE DATA
INDEPENDENT DATABASES - SHARED ENTITIES
▸ We’ve started to break up the data monolith
▸ BUT our data integration “strategy” is rather
brittle and bespoke
▸ How are changes to data in one bounded
context reflected in the other?
Sales
Support
?

57.
My
Connections
API
Posts
API
New from
Network
API

58.
My
Connections
API
Posts
API
New from
Network
API

59.
My
Connections
API
Posts
API
New from
Network
API
Unified Log

60.
@cdavisafc
CLOUD-NATIVE DATA
SOURCE OF TRUTH
My
Connections
API
Posts
API
New from
Network
API
Unified Log

61.
@cdavisafc
CLOUD-NATIVE ARCHITECTURES
▸ Cloud-native Apps
▸ Scale out
▸ Statelessness
▸ Externalize configuration
▸ Implications from changes in the application
lifecycle
▸ Versioned services
▸ Service Discovery
▸ Distributed tracing
▸ Retries
▸ Cloud-native Data
▸ Breaking the Data monolith
▸ Data APIs
▸ Caching
▸ Polyglot Persistence
▸ Event sourcing

62.
@cdavisafc
REFERENCES
▸ Scott Mansfield of the Netflix Caching team:
▸ Public video: https://www.youtube.com/watch?
v=Rzdxgx3RC0Q&t=205s
▸ Safari books online video (slightly expanded): https://
www.safaribooksonline.com/library/view/oreilly-
software-architecture/9781491976142/
video289789.html
▸ Slideshare: https://www.slideshare.net/ScottMansfield3/
application-caching-the-hidden-microservice
▸ Netflix on the “brief availability blip”: https://medium.com/
netflix-techblog/chaos-engineering-
upgraded-878d341f15fa

63.
THANK YOU
https://content.pivotal.io/blog/cloud-native-and-the-apparating-app