In this WebHack talk I shared my experience about microservices, Docker, Kubernetes and Kong, an API gateway by Mashape. Since they are based on a real working system, this slides is majorly for how to build the whole thing up, not about detailed internal implementation. Although I included some details and reference in order to make it more comprehensive.
2. Speaker
@GregWeng
First languages: Haskell, ECMAScript
Second languages: Python, Ruby, C/C++
Domain:Functional Programming,
Compiler,
Dynamic Language Engine,
Distributed System (as an user),
Web FrontEnd
Natural Language: Chinese, English, Japanese 勉強中
Slides:
11. Microservices
Frontend app
User
Backing Recipe Service
Ingredient
Recipe Rating
Logging
Balancer
Logging
Logging
Logging
Health
Monitor
And the system get complicated soon...
Deployment
CI/CD
CI/CD
CI/CD
21. Container
Develop & deploy a Dockerized service
Nodes
Write code
Write Docker file
Build Docker image
Deploy image to nodes
22. Container
Develop & deploy a Dockerized service
Nodes
Write code
Write Docker file
Build Docker image
Deploy image to nodes
Install code & dependencies
into the image
Instructions to build the image
(many details here, actually)
For your service
23. Container
Update a Dockerized service
Nodes
Patch the code or
Dockerfile
Re-build Docker image
Re-deploy the image
(much more details here)
24. Container
Why Docker/Container?
Lightweight than VM (runtime overhead & image size)
but still keep isolation among services*
Hardware
Host OS
File System
Network
Process
Kernel
Process
Network
Process
File System
Guest OS
Process
* Isolation enough for most of cases, but can be leaky sometimes
25. Container
Why Docker/Container?
Build your own images based on others' work
Rating Service Image
Python3 Image
Debian Image
based on
based on
## In the Dockerfile ##
FROM python:3.6.3-jessie
COPY ./app /opt/app
WORKDIR /opt/app
...
31. Kubernetes
How to automatically deploy 1 image to 1000 nodes?
......
While make sure some deployment failures
won't disturb the whole procedure
32. Kubernetes
And how to upgrade to new version of image not to
interrupt the service?
......
Plus the capability to auto-rollback when
new version actually has regression
33. Kubernetes
Annoying service config and secrets...
Database: table,
username/password
Service port, host
Logging level, ...
Database: table,
username/password
Service port, host
Logging level, ...
Database: table,
username/password
Service port, host
Logging level, ...
Service A Service B Service C
Where to put, keep, apply and update?
34. Kubernetes
Any map for exposing service to public, or connect
containers as internal services for each other?
Public
36. Kubernetes
Different types of resources in Kubernetes
Deployment
Job
Service
Ingress
...
Define these files and command Kubernetes
to create/delete/update on Nodes
37. Kubernetes
Tiny Kubernetes-based microservice in brief
Containers
Pod
Containers
Pod
Containers
Pod
Containers
Pod
replica = 3
label: app=rating
label: app=recipe
Rating
Service
selector:
app=rating
Service
Recipe
Service
selector:
app=recipe
Service
/rating:
Rating
Service
/recipe:
Recipe
Service
Ingress
Backing
Recipe
Ingress
Public
38. Kubernetes
Create a Deployment in Kubernetes
Containers
PodName
Label
Annotation
Port
restartPolicy
imagePullPolicy
readinessProbe
livenessProbe
env
image
command
...
kubectl create -f rating-deployment.yml
rating-deployment.yml
Kubernetes will start the
deployment and pull the image
Build & push Docker image to registry
39. Kubernetes
Update a Deployment in Kubernetes
Containers
PodName
Label
Annotation
Port
restartPolicy
imagePullPolicy
readinessProbe
livenessProbe
env
image'
command
...
kubectl replace -f rating-deployment.yml
rating-deployment.yml
Kubernetes will start to roll out
the new version
*can be done w/o file updated: `kubectl set image deployment/rating rating=rating:new-version`
40. Kubernetes
Create a service in Kubernetes
kubectl create -f rating-service.yml
Kubernetes will create the service and
set rules to connect selected pods
Name
Label
Annotation
Type
Port
Selector
How to expose the service (and balancing)
What is the service port & Pods port
Which Pods should be the backend
Containers
Pod
Rating
Service
selector:
app=rating
Service
label: app=rating
*actually Service can be very powerful & complicated. Please refer the official k8s document
41. Kubernetes
Load Balancing in Kubernetes*
kube-proxy kube-proxy kube-proxy
Service
rating
Pods
Service
recipe
Pods
Node A Node B Node C
Load Balancer
(Provider or External)
*can be very detailed (ex: userspace proxy or iptable based) and have different ways to set up; reference: slides from KubeConf EU 2016
42. Kubernetes
How Kubernetes detect unhealthy Pod?
Containers
Pod
readinessProbe
livenessProbe
rating-deployment.yml
httpGet:
path: /
port: 8001
initialDelaySeconds: 30
periodSeconds: 180
Kubernetes detected it not alive
after the container get ready
respect restartPolicy to
restart the container
*For multi-container Pod, the overall Pod status depends on the restartPolicy and container state
43. SQL connection
Kubernetes
Why multi-container Pod?
GCloud SQL
Proxy
Pod: api-gateway
Gateway
Kong
Google
Cloud SQL
Proxied connection
External service
GCloud SQL
secrets
Mounted secrets
Raw username/
password
45. Kubernetes
Regression test for microservice (the most low-tech way)
test-job.yml
app
test-db
Job Pod
HTTP
client
script
Test Container Service Containers
trigger job
collect result & kill the pod
create job
resource
46. Kubernetes
And test it locally before push to production
Minikube on local console
(local kubernetes)
Kubernets platform for
Production
https://github.com/kubernetes/minikube
47. Frontend app
User
So now we have the infrastructure to manage our service
Ingredient
Recipe Rating
Logging
Balancer
Logging
Logging
Logging
Health
Monitor
Deployment
CI/CD
CI/CD
CI/CD
Kubernetes
51. We want to manage all request & feature at once
Kong
Add and manage rate limiting, oauth2, logging, IP whitelist... easily
RESTfulRESTfulRESTful RESTful
52. But to manage them on each service is just
stockpiling the complexity from management
Kong
55. Kong
Kong is...
Kong core.lua
OpenResty
lua modules
Nginx
Plugin
lua modules
Admin API
API CRUD
Plugin CRUD
API endpoints
RESTfulRESTfulRESTful
"Proxy"
Interface
Consumer CRUD
User
Postgres/
Cassandra
Operator
56. Kong
Kong is...
Admin API
API endpoints
RESTfulRESTfulRESTful
"Proxy"
Interface
User
Postgres/
Cassandra
OperatorResource CRUD
plugin states
Rate
Limiting
OAuth
2.0
Logging
plugins
CORSACL
Bot
Detection
JWT Datadog
AWS
Lambda
(and more)
+plugins
57. Kong
Kong is...
Admin API
API endpoints
RESTfulRESTfulRESTful
"Proxy"
Interface
User
Postgres/
Cassandra
OperatorResource CRUD
Kong
Kong
Kong
Kong
Kong
Kong
Kong
cluster states
cluster
58. Kong is...
External Load Balancer
ex: from Kubernetes
Kong
Kong
Kong
Frontend app
User
Internal Upstream
Load Balancer
(DNS record based)
service.upstream.domain
192.168.0.1
192.168.0.2
192.168.0.3
...
59. How plugin works
diagram from: https://github.com/openresty/lua-nginx-module
Kong
handler.lua
nginx-kong.conf
access_by_lua_block
rewrite_by_lua_block
header_filter_by_lua
...
plugin
:rewrite
:access
:header_filter
...
iterate each plugin
(defined priority)
62. Kong
Kong
Kong
Kong
Kong
Kong
Kong
Kong & database migration
Postgres/
Cassandra
Kong
migration up
Kong
Kong
Kong
Kong
Kong
Kong
Postgres/
Cassandra
Kong
migration up
Kong
migration up
Kong
migration up
Kong
migration up
Kong
migration up
Kong
migration up
Kong
migration up
63. Kong
Combine all the things together
Recipe
RESTful
Ingredient Rating
RESTfulRESTful
Frontend app
User
Search
RESTful
API gateway
Deployment
Balancer
Health
Monitor
Logging
CI/CD
Postgres
Cassandra
Operator
Admin API
Endpoints +plugins
64. Advantages
1. Nginx is fast, therefore Kong is fast
2. Built-in plugins are good
3. Plugin system make customisation easy and clear
4. OpenResty is a great module collection
5. Docker ready, and also for other cloud formats, like AMI
6. Admin interface is also RESTful: it is possible to set up in pure cURL
7. Lua is easy to learn, and it is also friendly to bind C code
8. All the benefits we want for manage multiple RESTful microservices
9. Mashape provides enterprise support
Kong
65. Issues
1. Migration racing issue (Job + Deployment in k8s)
2. Plugins are not dynamic loaded
3. No resource limiting for individual plugin
4. Kong is not multi-tenant
5. No official web dashboard
6. Multiple configuration for Nginx and Kong
7. Restriction from the underlying Nginx & OpenResty
8. Lua is good, but not so good (also, hiring problem)
9. Cannot only upgrade Nginx for security reason means to re-build
whole the whole stack (maybe possible in embedded mode)
Kong
70. Container
For development & deployment (unit)
Deployment
CI/CD
CI/CD
CI/CD
Ingredient
Recipe Rating
Logging
Balancer
Logging
Logging
Logging
Health
Monitor
IngredientRecipe Rating Deployment
Container (Docker)
71. Container
Develop & deploy a Dockerized service
Nodes
Write code
Write Docker file
Build Docker image
Deploy image to nodes
Install code & dependencies
into the image
Instructions to build the image
(many details here, actually)
For your service
72. Container
Why Docker/Container?
Lightweight than VM (runtime overhead & image size)
but still keep isolation among services*
Hardware
Host OS
File System
Network
Process
Kernel
Process
Network
Process
File System
Guest OS
Process
* Isolation enough for most of cases, but can be leaky sometimes
75. Kubernetes
Different types of resources in Kubernetes
Deployment
Job
Service
Ingress
...
Define these files and command Kubernetes
to create/delete/update on Nodes
76. Kubernetes
Tiny Kubernetes-based microservice in brief
Containers
Pod
Containers
Pod
Containers
Pod
Containers
Pod
replica = 3
label: app=rating
label: app=recipe
Rating
Service
selector:
app=rating
Service
Recipe
Service
selector:
app=recipe
Service
/rating:
Rating
Service
/recipe:
Recipe
Service
Ingress
Backing
Recipe
Ingress
Public
77. Kubernetes
Create a Deployment in Kubernetes
Containers
PodName
Label
Annotation
Port
restartPolicy
imagePullPolicy
readinessProbe
livenessProbe
env
image
command
...
kubectl create -f rating-deployment.yml
rating-deployment.yml
Kubernetes will start the
deployment and pull the image
Build & push Docker image to registry
78. Frontend app
User
So now we have the infrastructure to manage our service
Ingredient
Recipe Rating
Logging
Balancer
Logging
Logging
Logging
Health
Monitor
Deployment
CI/CD
CI/CD
CI/CD
Kubernetes
81. Kong
Kong is...
Kong core.lua
OpenResty
lua modules
Nginx
Plugin
lua modules
Admin API
API CRUD
Plugin CRUD
API endpoints
RESTfulRESTfulRESTful
"Proxy"
Interface
Consumer CRUD
User
Postgres/
Cassandra
Operator
82. Kong
Kong is...
Admin API
API endpoints
RESTfulRESTfulRESTful
"Proxy"
Interface
User
Postgres/
Cassandra
OperatorResource CRUD
plugin states
Rate
Limiting
OAuth
2.0
Logging
plugins
CORSACL
Bot
Detection
JWT Datadog
AWS
Lambda
(and more)
+plugins
83. Kong
Kong is...
Admin API
API endpoints
RESTfulRESTfulRESTful
"Proxy"
Interface
User
Postgres/
Cassandra
OperatorResource CRUD
Kong
Kong
Kong
Kong
Kong
Kong
Kong
cluster states
cluster
84. Kong
Combine all the things together
Recipe
RESTful
Ingredient Rating
RESTfulRESTful
Frontend app
User
Search
RESTful
API gateway
Deployment
Balancer
Health
Monitor
Logging
CI/CD
Postgres
Cassandra
Operator
Admin API
Endpoints +plugins
85. Advantages
1. Nginx is fast, therefore Kong is fast
2. Built-in plugins are good
3. Plugin system make customisation easy and clear
4. OpenResty is a great module collection
5. Docker ready, and also for other cloud formats, like AMI
6. Admin interface is also RESTful: it is possible to set up in pure cURL
7. Lua is easy to learn, and it is also friendly to bind C code
8. All the benefits we want for manage multiple RESTful microservices
9. Mashape provides enterprise support
Kong
86. Issues
1. Migration racing issue (Job + Deployment in k8s)
2. Plugins are not dynamic loaded
3. No resource limiting for individual plugin
4. Kong is not multi-tenant
5. No official web dashboard
6. Multiple configuration for Nginx and Kong
7. Restriction from the underlying Nginx & OpenResty
8. Lua is good, but not so good (also, hiring problem)
9. Cannot only upgrade Nginx for security reason means to re-build
whole the whole stack (maybe possible in embedded mode)
Kong
87. Advantages
1. Nginx is fast, therefore Kong is fast
2. Built-in plugins are good
3. Plugin system make customisation easy and clear
4. OpenResty is a great module collection
5. Docker ready, and also for other cloud formats, like AMI
6. Admin interface is also RESTful: it is possible to set up in pure cURL
7. Lua is easy to learn, and it is also friendly to bind C code
8. All the benefits we want for manage multiple RESTful microservices
9. Mashape provides enterprise support
Kong