Talk about how and why we decided to migrate from a monolithic applocation to microservices at seedtag and how we solved the complexities that we found

1. @Laura_Morillo@Laura_Morillo
Development And
Continuous Deployment
with Microservices

2. @Laura_Morillo

3. @Laura_Morillo
seedtag?

4. @Laura_Morillo
1.
Past seedtag app

5. @Laura_Morillo
Monolithic
Urls
Publishers
Users Events
Campaigns Analytics
Admin

6. @Laura_Morillo
Monolithic: Development
Urls
Publishers
Users Events
Campaigns Analytics
Admin

7. @Laura_Morillo
Monolithic: CI

8. @Laura_Morillo
Monolithic: NO CD

9. @Laura_Morillo
Monolithic: Deployment

10. @Laura_Morillo
Monolithic: Production

11. @Laura_Morillo
Monolithic: Scalability
Scalability server

12. @Laura_Morillo
Monolithic: Scalability
Scalability server

13. @Laura_Morillo
2.
Current seedtag
structure

14. @Laura_Morillo
Microservices
TMS
Events
Campaigns
Analytics
Sherlock Watson
Users
Event
Aggregation
Url
processing
Reporting
Reporting

15. @Laura_Morillo
Advantages

16. @Laura_Morillo
Microservices: Advantages(I)
Bounded code

17. @Laura_Morillo
Microservices: Advantages(II)
Better resources
exploitation

18. @Laura_Morillo
Microservices: Advantages(III)
Technology flexibility

19. @Laura_Morillo
TMS
Events
Campaigns
Analytics
Sherlock Watson
Users
Event
Aggregation
Url
processing
Reporting
Reporting
Urls
Publishers
Users Events
Campaigns Analytics
Admin
VS

20. @Laura_Morillo
Drawbacks

21. @Laura_Morillo
Microservices: Drawbacks(I)
Complexity

22. @Laura_Morillo
Microservices: Drawbacks(II)
How to divide?

23. @Laura_Morillo
Microservices: Drawbacks(III)
Communication &
Service Discovery

24. @Laura_Morillo
Microservices: Production

25. @Laura_Morillo
Microservices: Node.js Dockerfile
FROM eu.gcr.io/project-id/node8-base:v26
USER node
WORKDIR /code
RUN npm set progress=false
ADD package.json /code
RUN yarn
ADD ./src /code/src
EXPOSE 3000
CMD ["yarn", "run", "start"]

26. @Laura_Morillo
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: my-service
spec:
replicas: 4
template:
metadata:
labels:
run: my-service
spec:
containers:
- name: seedtag-my-service
image: eu.gcr.io/project-id/my-service:latest
*kubernetes: deployment.yaml

27. @Laura_Morillo
*kubernetes: service.yaml
apiVersion: v1
kind: Service
metadata:
name: geolocation-service
labels:
run: geolocation-service
spec:
ports:
- port: 3000
targetPort: 3000
selector:
run: geolocation-service
type: ClusterIP

28. @Laura_Morillo
*kubernetes
Deployment
Instance 1 v1
Instance 2 v1
Instance 4 v1
Service
Instance 3 v1

29. @Laura_Morillo
apiVersion: extensions/v1beta1
spec:
template:
spec:
containers:
- name: seedtag-geolocation-service
image: eu.gcr.io/project-id/geolocation-service:latest
...
resources:
requests:
cpu: "1"
memory: "400Mi"
limits:
cpu: "1.5"
memory: "1Gi"
*kubernetes: deployment.yaml
resources

30. @Laura_Morillo
apiVersion: extensions/v1beta1
spec:
template:
spec:
containers:
- name: seedtag-geolocation-service
image: eu.gcr.io/project-id/geolocation-service:latest
...
readinessProbe:
httpGet:
path: /status
port: 3000
initialDelaySeconds: 5
periodSeconds: 1
timeoutSeconds: 5
*kubernetes: deployment.yaml
Readiness probe

31. @Laura_Morillo
*kubernetes: status.js
Readiness probe
const notOk = res => res.status(502).send('');
module.exports.get = (req, res) => {
const int = setTimeout(() => {
notOk(res);
responseSent = true;
}, 500);
mongoose.connection.db.admin().ping((err, result) => {
if (responseSent) return null; // Pong is no longer useful
clearTimeout(int);
if (err) return notOk(res);
return res.send(result);
});
};

32. @Laura_Morillo
Circuit Breaker

33. @Laura_Morillo
*kubernetes: scaler.yaml
apiVersion: autoscaling/v1
kind: HorizontalPodAutoscaler
Metadata:
name: seedtag-geolocation-service
namespace: default
Spec:
scaleTargetRef:
apiVersion: extensions/v1beta1
kind: Deployment
name: seedtag-geolocation-service
minReplicas: 2
maxReplicas: 10
targetCPUUtilizationPercentage: 70

34. @Laura_Morillo
Min
*kubernetes
Deployment
Instance 1 v1
Instance 2 v1
Instance 10 v1
Service
.
.
.
Max

35. @Laura_Morillo
Production

36. @Laura_Morillo
Microservices: Development

37. @Laura_Morillo
Microservices: docker-compose
Dev Environment
SVC1 SVC2
docker-compose

38. @Laura_Morillo
Microservices: docker-compose
Dev Environment
SVC1 SVC2
docker-compose

39. @Laura_Morillo
Microservices: docker-compose (global)
service2:
build: ./service2
volumes:
- ./service2:/code
expose:
- "3000"
ports:
- "3000:3000"
command: npm run dev
environment:
NODE_ENV: 'development'
depends_on:
- service1
service1:
build: ./service1
volumes:
- ./service1:/code
expose:
- "3000"
ports:
- "3000:3000"
command: npm run dev
environment:
NODE_ENV: 'development'

40. @Laura_Morillo
Microservices: docker-compose
(individual)
service2:
build: .
volumes:
- .:/code
expose:
- "3000"
ports:
- "3000:3000"
command: npm run dev
environment:
NODE_ENV: 'development'
depends_on:
- service1

41. @Laura_Morillo
Microservices: docker-compose
dependencies
service1:
image: ajnasz/api-mock
volumes:
- ./mock/service1/api.md:/usr/src/app/api.md

42. @Laura_Morillo
Microservices: Mock server
/public/api/token
Retrieve messages
## GET
+ Response 200 (application/json)
{"accessToken":"dummy-access-token"}
# /public/userinfo
Retrieve messages
## GET
+ Response 200 (application/json)
{"id": "1", "username":"seedtag","permissions":["admin"]}

43. @Laura_Morillo
Development

44. @Laura_Morillo
Microservices: dev-environment

45. @Laura_Morillo
Microservices: seedtag UI

46. @Laura_Morillo
Microservices: seedtag UI
✘ Sync repositories
✘ Build services
✘ Visualize development status
✘ Start/Stop kafka consumer/producers
✘ Load fixtures
✘ Dump/Restore database

47. @Laura_Morillo
Development

48. @Laura_Morillo
Microservices: continuous deployment

49. @Laura_Morillo
Microservices: CD (I)
docker-compose up -d --build
(using service independant docker-compose)

50. @Laura_Morillo
Microservices: CD (II)
docker-compose exec -T service yarn run ci
(Check linter, run unit/integration/acceptance tests)

51. @Laura_Morillo
Microservices: CD (III)
docker build -t ${IMAGE}:v${BUILD} .
docker tag -f ${IMAGE}:v${BUILD} ${IMAGE}:latest

52. @Laura_Morillo
Microservices: CD (IV)
gcloud docker -- push ${IMAGE}:v${BUILD}
gcloud docker -- push ${IMAGE}:latest

53. @Laura_Morillo
Microservices: CD (V)
kubectl set image
deployment/service=${IMAGE}:v${BUILD}

54. @Laura_Morillo
*kubernetes
Deployment
Instance 1 v1
Instance 2 v1
Instance n v1
.
.
.
Service

55. @Laura_Morillo
*kubernetes
Deployment
Instance 1 v2
Instance 2 v1
Instance n v1
.
.
.
Service

56. @Laura_Morillo
*kubernetes
Deployment
Instance 1 v2
Instance 2 v2
Instance n v2
.
.
.
Service

57. @Laura_Morillo
Continuous Deployment

58. @Laura_Morillo
Advantages:
❏ Modularized resulting code
❏ Better resources exploitation
❏ Technology flexibility
Summary(I)

59. @Laura_Morillo
Advantages:
❏ Modularized resulting code
❏ Better resources exploitation
❏ Technology flexibility
Summary(I)

60. @Laura_Morillo
Advantages:
❏ Modularized resulting code
❏ Better resources exploitation with kubernetes
❏ Technology flexibility
Summary(I)

61. @Laura_Morillo
Advantages:
❏ Modularized resulting code
❏ Better resources exploitation with kubernetes
❏ Technology flexibility (node.js, scala/spark,
python)
Summary(I)

62. @Laura_Morillo
Summary(II)
Problems:
❏ Communication problems
❏ Service discovery
❏ Complexity

63. @Laura_Morillo
Summary(II)
Problems:
❏ Communication problems solved with circuit
breaker and kafka
❏ Service discovery
❏ Complexity

64. @Laura_Morillo
Summary(II)
Problems:
❏ Communication problems solved with circuit
breaker and kafka
❏ Service discovery solved with kubernetes and
docker compose
❏ Complexity

65. @Laura_Morillo
Summary(II)
Problems:
❏ Communication problems solved with circuit
breaker and kafka
❏ Service discovery solved with kubernetes and
docker compose
❏ Complexity reduced with kubernetes, docker
compose and seedtag ui

66. @Laura_Morillo
Development
Continuous Deployment
Production

67. @Laura_Morillo
thanks!
Any questions?
We are hiring!
@laura_morillo
laura.morillovelarde@gmail.com
Presentation template by SlidesCarnival