Communication Amongst Microservices: Kubernetes, Istio, and Spring Cloud by Angela Chin

1. © Copyright 2017 Pivotal Software, Inc. All rights Reserved. Version 1.0
Communication Amongst
Microservices:
Kubernetes, Istio, and Spring Cloud
Angela Chin, Senior Software Engineer, Pivotal
Twitter: @AngelaSChin

7. I’m Angela and I love
● (Software) Networking
● Hot yoga
● Hiking
● Desserts
● Food

8. Thanks for joining me.
Let’s get started!

9. Confidential
What is Kubernetes?

10. Confidential
k8s!

11. Confidential
What is k8s?

12. Confidential
Shorthand for Kubernetes!

13. Confidential

14. Confidential
Kubernetes
● Container orchestrator built around Docker

15. Confidential
Kubernetes
● Container orchestrator built around Docker
● “Have a datacenter on their laptop”

16. Confidential
Kubernetes
● Container orchestrator built around Docker
● “Have a datacenter on their laptop”
● Dev env == production env

17. Confidential
Kubernetes
● Container orchestrator built around Docker
● “Have a datacenter on their laptop”
● Dev env == production env
● Multi-cloud

18. Confidential
Kubernetes
● Container orchestrator built around Docker
● “Have a datacenter on their laptop”
● Dev env == production env
● Multi-cloud
● Community convergence & 3rd party integrations

19. Confidential
Kubernetes
● Container orchestrator built around Docker
● “Have a datacenter on their laptop”
● Dev env == production env
● Multi-cloud
● Community convergence & 3rd party integrations
● Flexibility

20. Confidential
Kubernetes
● Runs applications
● Reduces infrastructure code
● Reduces toil

21. Confidential

22. Confidential
Pivotal Container Service (PKS)

23. Confidential
PKS
● Managed Kubernetes for multiple clusters

24. Confidential
PKS
● Managed Kubernetes for multiple clusters

25. Confidential
PKS
● Managed Kubernetes for multiple clusters

26. Confidential
PKS
● Managed Kubernetes for multiple clusters
● Focused on Day 2 Operations

27. Confidential
PKS
● Managed Kubernetes for multiple clusters
● Focused on Day 2 Operations
● Uses BOSH

28. Confidential
BOSH
● Day 2 Operations
● Large-scale cloud software
● Security fixes within 48 hours
● Security aware
● Monitors and resurrects VMs
● Reproducible deployments

29. Embedded OS
(Windows & Linux)
NSX-T
CPI (15 methods)
v1
v2
v3
...
CVEs
Product Updates
Java | .NET | NodeJS
Pivotal Application
Service (PAS)
Application Code & Frameworks
Buildpacks | Spring Boot | Spring Cloud |
Steeltoe
Elastic | Packaged Software | Spark
Pivotal Container
Service (PKS)
>cf push >kubectl run
YOU build the containerWE build the container
vSphere
Azure &
Azure StackGoogle CloudAWSOpenstack
Pivotal
Network
“3Rs”
Github
Concourse
Concourse
Pivotal Services
Marketplace
Pivotal and
Partner Products
Continuous
delivery
Public Cloud
Services
Customer
Managed
Services
Repair
— CVEs
Repave Rotate
— Credhub
Pivotal Cloud Foundry

30. Confidential
I haz a Kubernetes.
What does that mean?

31. Confidential

32. Confidential
Master

33. Confidential
Master Worker

34. Confidential
Master Worker
Kubernetes Cluster

35. Confidential
Master Worker
Kubernetes Cluster
Master
Master
Worker
Worker

36. Confidential
Master Worker
Kubernetes Cluster
Master
Master
Worker
Worker
kube-apiserver

37. The Kubernetes API is the single
source of truth.

38. Confidential
Master
Master
Master
Master
Master
Worker
kube-apiserver

39. Confidential
Master
Master
Master
Master
Master
Worker
kube-apiserver
kubectl
apply -f myApp.yml

40. Confidential
Master
Master
Master
Master
Master
Worker
kube-apiserver etcd
kubectl
apply -f myApp.yml

41. Confidential
Master
Master
Master
Master
Master
Worker
kube-apiserver etcd
kubectl
apply -f myApp.yml
scheduler

42. Confidential
Master
Master
Master
Master
Master
Worker
kube-apiserver etcd
kubectl
apply -f myApp.yml
scheduler
kube-controller-
manager

43. Confidential
Master
Master
Master
Master
Master
Worker
kube-apiserver etcd
kubectl
apply -f myApp.yml
scheduler
kube-controller-
manager
kubelet

44. Confidential
Master
Master
Master
Master
Master
Worker
kube-apiserver etcd
kubectl
apply -f myApp.yml
scheduler
kube-controller-
manager
kubelet

45. Confidential
Master
Master
Master
Master
Master
Worker
kube-apiserver etcd
kubectl
apply -f myApp.yml
scheduler
kube-controller-
manager
kubelet
myApp

46. Confidential

47. Confidential
So… k8s is cool

48. Confidential
But why do I care again?

49. Confidential
What am I deploying??

50. Confidential

51. Confidential
The Microservices Use Case

52. So you’re building some
microservices...

53. Monolith

54. Microservices

56. Microservices
● Smaller, simpler and easier to test
● Develop, deploy and run independently
● Organization can scale to many teams

57. Tradeoffs with Microservices
Pros Cons
Strong Modular Boundaries Distributed Systems
Independent Deployment Eventual Consistency
Diversity of Technology Operational Complexity

58. Fallacies of Distributed systems
● The network is reliable.
● Latency is zero
● Bandwidth is infinite
● The network is secure.
● Topology doesn't change.
● There is one administrator.
● Transport cost is zero.
● The network is homogeneous.

60. backendfrontend

61. Things your microservices ought to do...
• Client-side retries
• Load balancing
• Mutual TLS
• Configurable timeouts
• Collecting metrics

62. Confidential
how do?

63. Confidential
Kubernetes
● Container orchestrator built around Docker
● “Have a datacenter on their laptop”
● Dev env == production env
● Multi-cloud
● Community convergence & 3rd party integrations
● Flexibility

64. Confidential
Kubernetes
● Container orchestrator built around Docker
● “Have a datacenter on their laptop”
● Dev env == production env
● Multi-cloud
● Community convergence & 3rd party integrations
● Flexibility

65. Confidential
Istio

66. Project to connect, secure, manage and observe microservices.
● Open source
● Platform agnostic
● Polyglot services
What is Istio?

67. Project to connect, secure, manage and observe microservices.
● Utilizes sidecar proxies
● Service mesh
What is Istio?

69. Things your microservices ought to do...
• Client-side retries
• Load balancing
• Mutual TLS
• Configurable timeouts
• Collecting metrics

70. Retries
Proxy
Frontend
Proxy
Backend 3
Proxy
Backend 2
Proxy
Backend 1

71. Retries
Proxy
Frontend
Proxy
Backend 3
Proxy
Backend 2
Proxy
Backend 1
Istio
subscribe!

72. Retries
Proxy
Frontend
Proxy
Backend 3
Proxy
Backend 2
Proxy
Backend 1
Istio
backends!

73. Retries
Proxy
Frontend
Proxy
Backend 3
Proxy
Backend 2
Proxy
Backend 1
Istio

74. Retries
Proxy
Frontend
Proxy
Backend 3
Proxy
Backend 2
Proxy
Backend 1
Istio

75. Retries
Proxy
Frontend
Proxy
Backend 3
Proxy
Backend 2
Proxy
Backend 1
Istio

76. Retries
Proxy
Frontend
Proxy
Backend 3
Proxy
Backend 2
Proxy
Backend 1
Istio

77. Istio Terminology
• Service: unit of (destination) application.
• Service Versions: variants of (destination) application binary.
• Service Registry: keeps track of pods/VMs of a service
• Virtual Service: rule defining what service to send a request to
• Destination Rule: rule defining what to do after destination
service is identified

78. apiVersion: config.istio.io/v1alpha3
kind: VirtualService
metadata:
name: bananagram
spec:
hosts:
- banana
http:
- route:
- destination:
host: banana
Example Virtual Service Config

79. apiVersion: config.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: bananagram
spec:
hosts:
- banana
trafficPolicy:
connectionPool:
tcp:
maxConnections: 1
Destination Rule

80. Confidential
Istio & Kubernetes: Better Together

82. $ helm template install/kubernetes/helm/istio --name istio --
namespace istio-system > $HOME/istio.yaml
$ kubectl create namespace istio-system
$ kubectl apply -f $HOME/istio.yaml

83. Confidential
Master
Master
Master
Master
Master
Worker
kube-apiserver
scheduler
kube-controller-
manager
kubelet
istio-pilot
istio-mixer
… etc

85. Confidential

86. But wait...

87. What about Spring Cloud Services?

88. Spring Cloud
Mission to eliminate boilerplate associated with distributed systems problems for
Spring Boot applications.
Started by using established libraries and servers (nothing was baked into platforms)
● Netflix OSS (Eureka, Ribbon, Hystrix)
● Hashicorp Consul
● Apache Zookeeper

89. Level of abstraction
Lorem ipsum dolor sit amet,
consectetuer adipiscing elit. Aenean
commodo ligula eget dolor. Aenean
massa. Cum sociis natoque penatibus
et magnis dis parturient montes,
nascetur ridiculus mus. Donec quam
felis, ultricies nec, pellentesque
● Spring and Istio address the
problems of distributed
systems at different layers of
abstraction
● “Istio helps decouple
operations of a cluster from the
application developer” - Eric
Brewer, Google (VP,
Infrastructure)
Application
Platform Or Container
Orchestrator
IaaS

90. Istio or Spring Cloud (or both?)
● If the capability is provided by the platform (Istio), use it. Except for …
○ Fallbacks
○ Tracing Propagation
○ Security
● Istio or a service mesh architecture works better for polyglot environments
● But it also depends on your risk propensity
○ Istio GA’d in July 2018
○ Spring Cloud has been around for 4 years
● Performance is also a factor
For more info, watch: A Tale of Two Frameworks

91. Confidential
To Recap

92. We Covered
● Kubernetes
● Pivotal Container Service (PKS)
● Microservices
● Istio
● Spring Cloud Services

93. Its been a journey gif

98. Confidential
Thank You
@AngelaSChin

102. Confidential
Example: Retries

103. Retries: Istio
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: foo
spec:
hosts:
- foo
http:
- route:
- destination:
host: foo
subset: v1
retries:
attempts: 3

104. Retries: Spring
@Service
class Service {
@Retryable(RemoteAccessException.class)
void service() {
// ... do something
}
@Recover
void recover(RemoteAccessException e)
{
// ... panic
}
}
● Annotation or Template
● Stateless or Stateful
● Flexible Retry Policies
● Backoff Policies (e.g.
exponential)

105. Confidential
In process vs out of process
architecture

106. Frontend Backend
A Library!
In-process architecture

110. Multiple languages, Multiple libraries
Each with…
• Different features
• Different configuration
• Different quirks
Polyglot shouldn't be painful!

111. Outline
• What your microservices need
• How sidecars help
• Envoy & Istio
• Kubernetes Integration
• Cloud Foundry Integration

112. Out-of-process architecture
Your App Some service
Separate
process!

113. Out-of-process architecture
Your App
- retries
- load balancing
- mutual TLS
- timeouts
- metric collection
- etc.
Your
business
logic
here

114. Your App
(acting as
a service)
Client app
Out-of-process architecture

115. Confidential