Developing and managing hundreds (or maybe thousands) of microservices at scale is a challenge for both development and operations teams. We have seen over the last years the appearance of new frameworks dedicated to deliver ‘Cloud Native’ applications by providing a set of (out of box) building blocks. Most of these frameworks integrate microservices concerns at the code level. Recently, we have seen the emerging of a new pattern known as sidecar or proxy promoting to push all these common concerns outside of the business code and provides them on the edge by integrate a new layer to the underlying platform called Service Mesh. Istio is one of the leading Service Mesh implementing sidecar pattern. We will go during the presentation throw the core concepts behind Istio, the capabilities that provides to manage, secure and observe microservices and how it gives a new breath for both developers and operations. The presentation will be guided by a sequence of demo exposing Istio capabilities.

1. Managing microservices with
Istio Service Mesh
Rafik Harabi, INNOVSQUARE
CNCF Tunisia Meetup 12/2018

2. Who I am?
● Solution Architect at INNOVSQUARE focusing on Digital
Transformation and Go2Cloud.
● Deloitte UK (London) Associate
● #Kubernetes, #Istio and #SpringPlatform are my daily
routines.
@rafik8_
https://fr.linkedin.com/in/rafikharabi

3. Moving to microservices network challenges
Network Reliability
Fault tolerance and resiliency
Monitoring and Observability

4. Challenges deep-dive
Network Reliability
Service have to handle
the network facts:
● Network latency /
bandwidth
● Transport cost
● Topology and
administration
Fault Tolerance
Service have to be able
to handle outright failure
and timeouts:
● Avoid cascading
failure
● Retries
● Circuit breaking
Monitoring
We have to:
● monitor the
delivered
microservices and
their interactions
● Trace requests and
identify potential
hotspots

5. The evolution of microservices frameworks: from
NetFlix OSS to Istio

6. 2011
NetFlix OSS
first microservices patterns
and libraries open-sourced
2013
Spring Cloud
Enterprise microservice framework
for Java
2014
Docker
Containerization
2015
Kubernetes
Workload orchestration
2018
Istio
Service mesh

7. Microservices challenges
Challenge 1 Challenge 2 Challenge 3
- N to N communications.
- Distributed software interconnection and troubleshooting is hard.
- Containers should stay thin and platform agnostic.
- Upgrade of polyglot microservices is hard at scale.

8. Microservices building blocks
Challenge 1 Challenge 3Configuration Service
Service Registry / Discovery
Circuit Breaker / Retry
Rate Limiting
API Gateway
Load Balancing / Intelligent Routing
Authentication & Authorization
Monitoring
Distributed tracingEvent Driven Messaging (Async)
Log AggregationAudit

9. Microservices building blocks
Challenge 3
Business Value
Configuration Service
Service Registry / DiscoveryCircuit Breaker / Retry
Rate Limiting
Event Driven Messaging (Async)
Audit
Load Balancing / Intelligent Routing
API Gateway
Authentication & Authorization Monitoring
Distributed tracing Log Aggregation

10. Code oriented frameworks
Challenge 3
Service A Service B
Business logic Business logic
Circuit Breaker
Rate limiting
Tracing
Metrics
Circuit Breaker
Rate limiting
Tracing
Metrics

11. Code oriented pattern
Challenge 1
Challenge 3
Configuration Service
Service Registry / Discovery
Circuit Breaker/Retry Rate Limiting
API Gateway
Load Balancing / Intelligent Routing
Authentication & Authorization
Monitoring
Distributed tracingEvent Driven Messaging (Async)
Log Aggregation
Audit
Business Service
Foundation
Monitoring and ObservabilityCommunication
Business Values

12. Code oriented solutions limits
- Language oriented.
- Error prone (implementation).
- Hard to upgrade each microservice when system grow.
- Add technical challenges and duties to development teams.
- Different team in the same organization may have different implementation.
- Each team should maintien his implementation.

13. Desired state
- Keep microservice concerns separate from the business logic.
- The network should be transparent to applications.
- Developers should focus on delivering business capabilities and not
implementing microservices common concerns.
- Microservices interconnection should be language agnostic.
- Easy to upgrade solution.

14. Service Mesh
Definition
A service mesh is a dedicated
infrastructure layer for handling
service-to-service communication.
It’s responsible for the reliable
delivery of requests through the
complex topology of services that
comprise a modern, cloud native
application.
buoyant.io

15. Service Mesh
The design
Each service will have its own proxy
service and all these proxy services
together form the “Service Mesh”.
All the requests to and from each
service will go through the mesh
proxies.
Proxies are also known as sidecars.

16. Sidecar pattern
Service A
Business logic
Circuit Breaker
Rate limiting
Tracing
Metrics
Proxy
Service B
Business logic
Circuit Breaker
Rate limiting
Tracing
Metrics
Proxy
Injected
Network concerns
become transparent
Service to service communication

17. History of Istio
- Envoy proxy (Istio data plane) created by Lyft and open-sourced in 2016.
- IBM and Google launch the project in May 2017.
- First major version released in July 2018.
- Version 1.1 under development.

18. Solution
Istio Promises
● Focus on business logic and
spent less time with common
concerns.
● No change in the service code.
● Central configuration
management.
● Easy to upgrade
● Security

19. Istio do:
- Service discovery
- Load Balancing & Intelligent
Routing
- Resiliency: Circuit Breaker &
Retry
- Rate Limiting
- Authentication and
Authorization
- Service to Service mTLS
- Policy enforcement
- Observability
- Monitoring metrics
- Distributed tracing
- User authentication and
authorization: we still need an
IAM.
- Event Driven Asynchronous
communication
- Service Orchestration
Istio do not:

20. Sidecar pattern
Challenge 1
Challenge 3
Configuration Service
Service Registry / Discovery
Circuit Breaker/Retry
Rate Limiting
API Gateway
Authentication & Authorization
Monitoring Distributed tracing
Event Driven Messaging (Async)
Log Aggregation Audit
Business Service
Foundation
Monitoring and ObservabilityCommunication
Business Values
Load Balancing / Intelligent Routing
Business Service
Business Service

21. Architecture

22. Challenge 1 Challenge 2 Challenge 3

23. Istio building blocks 1/2
Component Description
Pilot Responsible for service discovery and for configuring the Envoy
sidecar proxies
Citadel Automated key and certificate management
Mixer Istio-Policy: policy enforcement
Istio-Telemetry: gather telemetry data
Galley Configuration ingestion for istio components
Ingress Gateway manage inbound connection to the service mesh
Egress Gateway manage outbound connection from the service mesh
Sidecar injector Inside sidecar for enabled namespaces

24. Istio building blocks 1/2
Component Description
Prometheus Metrics collection
Grafana Monitoring dashboard
Jaeger Distributed tracing
Kiali Observability dashboard
Service Graph Service dependencies

25. Challenge 1 Challenge 2 Challenge 3

26. Demo Application - BookInfo
Challenge 1 Challenge 2 Challenge 3
Product Page Application
Details Service
Review Service
Rating Service
http://bookinfo.europe-west1-b.innovlabs.io

27. Demo Application - BookInfo

28. Application Deployment
istio-init istio-proxy Service container
1 2 3
kubectl describe po/pod-id -n bookinfo-app

29. Envoy proxy initialization - istio-init
kubectl logs po/pod-id -c istio-init -n bookinfo-app

30. Dashboard
Challenge 1 Challenge 2 Challenge 3
● Monitoring: http://grafana.europe-west1-b.innovlabs.io/
● Observability: http://kiali.europe-west1-b.innovlabs.io/console/overview
● Metrics: http://prometheus.europe-west1-b.innovlabs.io/graph
● Tracing: http://tracing.europe-west1-b.innovlabs.io/search
● Service graph: http://servicegraph.europe-west1-b.innovlabs.io/dotviz

31. Service Discovery
Challenge 1 Challenge 2 Challenge 3
Kubernetes provide service discovery, why do I need an extra one ?
Istio supports:
- HTTP L7 filter
- HTTP L7 routing (based on http headers and cookies)
- First class HTTP/2
- gRPC support
- Fine-grained traffic splitting

32. Ingress Gateway
Challenge 1 Challenge 2 Challenge 3
Define access to services from the outside the service mesh:
Challenge 1 Challenge 3
Product Page
Proxy
Details
Proxy
Ingress Gateway
1
2
3 4
5
6 7
8

33. Ingress Gateway
Challenge 1 Challenge 2 Challenge 3
Gateway: defines a load balancer operating at the edge of the mesh receiving
incoming (Ingress) or outgoing (Egress) HTTP/TCP connections.
VirtualService: defines a set of traffic routing rules to apply when a host is
addressed.
DestinationRule: defines policies that apply to traffic intended for a service after
routing has occurred.

34. Ingress Gateway
Challenge 1 Challenge 2 Challenge 3

35. Ingress Gateway
Challenge 1 Challenge 2 Challenge 3

36. Ingress Gateway
Challenge 1 Challenge 2 Challenge 3
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: bookinfo-gateway
spec:
selector:
istio: ingressgateway
servers:
- port:
number: 80
name: http
protocol: HTTP
hosts:
- "bookinfo.europe-west1-b.innovlabs.io"
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: bookinfo
spec:
hosts:
- "*"
gateways:
- bookinfo-gateway
http:
- match:
- uri:
exact: /productpage
route:
- destination:
host: productpage
port:
number: 9080

37. Traffic Management / Splitting
Challenge 1 Challenge 3
EDS: Endpoint Discovery Service
CDS: Cluster Discovery Service
RDS: Route Discovery Service
LDS: Listener Discovery Service
istioctl proxy-status
istioctl proxy-config clusters -n namespace pod-id
Product Page
Proxy
Details
Proxy
Pilot
xDS Sync xDS Sync

38. Traffic Management / Splitting
● Achieve affinity by using Cookie, Header, or Source IP.
● Each service can have any number of versions (a.k.a. subsets).
● Pilot translates high-level rules into low-level configurations and distributes
this config to Envoy instances.
● Pilot uses three types of configuration resources to manage traffic within its
service mesh: Virtual Services, Destination Rules, and Service Entries.

39. Traffic Management / Splitting
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: reviews
spec:
host: reviews
subsets:
- name: v1
labels:
version: v1
- name: v2
labels:
version: v2
- name: v3
labels:
version: v3
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: reviews
spec:
hosts:
- reviews
http:
- route:
- destination:
host: reviews
subset: v1
weight: 80
- destination:
host: reviews
subset: v2
weight: 20

40. A/B testing - Canary release
100%
0%

41. A/B testing - Canary release
100%
0%
80%
20%

42. A/B testing - Canary release
100%
0%
80%
20%
50%
50%

43. A/B testing - Canary release
100%
0%
80%
20%
50% 50%
100%
0%

44. Fault Injection apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: details-service-503
spec:
hosts:
- details
http:
- route:
- destination:
host: details
fault:
abort:
percent: 20
httpStatus: 503
Inject faults to test the resiliency of
your application.

45. Circuit Breaker
Circuit breaking allows to write
applications that limit the impact of
failures, latency spikes, and other
network effects.
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: details
spec:
host: details
trafficPolicy:
connectionPool:
tcp:
maxConnections: 1
http:
http1MaxPendingRequests: 1
maxRequestsPerConnection: 1
outlierDetection:
consecutiveErrors: 1
interval: 1s
baseEjectionTime: 3m
maxEjectionPercent: 100

46. Retry strategy
Enable automatic retry when a
temporary network issue occurs.
Timeout could be also specified.
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: details
spec:
hosts:
- details
http:
- route:
- destination:
host: details
retries:
attempts: 3
perTryTimeout: 2s

47. Egress Gateway

48. Debugging
kubectl logs po/POD-ID -c istio-init -n namespace
istioctl proxy-status
istioctl proxy-config clusters -n namespace POD-ID
istioctl proxy-config listeners -n namespace POD-ID

49. Istio Service Mesh - next steps
- Mesh Extension: enable managing external services.
- Istio CNI: v1.1
https://deploy-preview-2902--preliminary-istio.netlify.com/docs/setup/kuberne
tes/istio-cni-install/
- Service Mesh interoperability over cloud providers.

50. Deep dive with istio 1/2
- IBM workshop: https://istio101.gitbook.io/lab/
- Redhat interactive learn: https://learn.openshift.com/servicemesh
- Redhat developer demos:
https://github.com/redhat-developer-demos/istio-tutorial/
- Istio by example, Ray Tsang:
https://github.com/saturnism/istio-by-example-java

51. Deep dive with istio 2/2
Istio in Action, Christian Posta

52. “Istio can (and should) be adopted incrementally.”
Christian Posta, Istio in Action

53. Thank you!
Any questions ?

54. References
- Microservice 4.0 Journey - From Spring NetFlix OSS to Istio Service Mesh and
Serverless, Daniel Oh
- https://blog.buoyant.io/2017/04/25/whats-a-service-mesh-and-why-do-i-need-
one/
- https://istio.io/blog/2017/0.1-using-network-policy/