Микросервисы: огонь, вода и девопс

1. 18+
Микросервисы
огонь, вода и DevOps

2. @tolkv
@lavcraft

3. @aatarasoff
@aatarasoff

4. DISCLAIMER
No warranty guarantee
4

5. Agenda
1. Theory
2. Practice
3. Q&A
5

6. 6
● делается с первого раза
● не меняется годами
● не зависит от людей
● не зависит от процессов

7. 7
У всех нас конечно же так

8. 8
У всех нас конечно же так

9. 9
Точка зрения: архитектор

10. 10
Работа идеального архитектора

11. 11
Работа идеального архитектора

12. 12
Работа идеального архитектора

13. 13
Точка зрения: разработчик

14. Немного деталей не помешает
14

15. Немного деталей не помешает
15

16. Архитектура
16

17. Архитектура
Что это?
17

18. Что же такое архитектура?
18

19. Что же такое архитектура?
19

20. Что же такое архитектура?
20
https://www.youtube.com/watch?v=_Kex5hwGE-w

21. 21

22. 22
Закон Конвея

23. Big Ball of Mud
23

24. Путь к микросервисам
● Нагрузки
● Масштабирование
24

25. Путь к микросервисам
● Нагрузки
● Масштабирование
● Управление
● Разделение
25

26. 12 April 1996
Первое упоминание SOA
https://www.gartner.com/doc/302868/service-oriented-architectures-
26

27. Принципы SOA
1. Standardized service contract
2. Loose coupling
3. Encapsulation
4. Reusability
5. Autonomy
6. Statelessness
7. Discoverability
27

28. Принципы SOA
1. Standardized service contract
2. Loose coupling
3. Encapsulation
4. Reusability
5. Autonomy
6. Statelessness
7. Discoverability
28

29. 5 January 2009
SOA is Dead
http://apsblog.burtongroup.com/2009/01/soa-is-dead-long-live-services.html
29

30. Classic SOA
● centralized orchestration
● complicated (hi, SOAP)
● smart pipes (hi, ESB)
30

31. SOA != SOAP (WS-*)
31

32. 32

33. 33

34. A=F(Rq, FRq, NFRq,..........???............)
34

35. Microservices = SOA 2.0
35

36. A=F(Rq, FRq, NFRq,..........???............)
DevOps
OSS Domination
36

37. 37
Про тренды

38. 38
Про тренды

39. In short, the microservice architectural style is an approach to developing
a single application as a suite of small services, each running in its own
process and communicating with lightweight mechanisms, often an HTTP
resource API. These services are built around business capabilities and
independently deployable by fully automated deployment machinery.
There is a bare minimum of centralized management of these services,
which may be written in different programming languages and use
different data storage technologies.
-- James Lewis and Martin Fowler
Что такое микросервисы?
39

40. In short, the microservice architectural style is an approach to developing
a single application as a suite of small services, each running in its own
process and communicating with lightweight mechanisms, often an HTTP
resource API. These services are built around business capabilities and
independently deployable by fully automated deployment machinery.
There is a bare minimum of centralized management of these services,
which may be written in different programming languages and use
different data storage technologies.
-- James Lewis and Martin Fowler
Что такое микросервисы?
40

41. Размер имеет значение?
● Method/Function = Microservice?
● 10-300 LOC = Microservice?
● 1 week = Microservice?
● 1 developer = Microservice?
41

42. Размер не имеет значения*
● Single Responsibility
● One capability
● Bounded context
“In your organization, you should be thinking not in terms
of data that is shared, but about the capabilities those
contexts provide the rest of the domain.”
– Sam Newman, Building Microservices
*до разумных пределов конечно
42

43. Domain-Driven Design
43

44. In short, the microservice architectural style is an approach to developing
a single application as a suite of small services, each running in its own
process and communicating with lightweight mechanisms, often an HTTP
resource API. These services are built around business capabilities and
independently deployable by fully automated deployment machinery.
There is a bare minimum of centralized management of these services,
which may be written in different programming languages and use
different data storage technologies.
-- James Lewis and Martin Fowler
Что такое микросервисы?
44

45. Process segregation
45

46. In short, the microservice architectural style is an approach to developing
a single application as a suite of small services, each running in its own
process and communicating with lightweight mechanisms, often an HTTP
resource API. These services are built around business capabilities and
independently deployable by fully automated deployment machinery.
There is a bare minimum of centralized management of these services,
which may be written in different programming languages and use
different data storage technologies.
-- James Lewis and Martin Fowler
Что такое микросервисы?
46

47. Независимый деплой
build test
47

48. In short, the microservice architectural style is an approach to developing
a single application as a suite of small services, each running in its own
process and communicating with lightweight mechanisms, often an HTTP
resource API. These services are built around business capabilities and
independently deployable by fully automated deployment machinery.
There is a bare minimum of centralized management of these services,
which may be written in different programming languages and use
different data storage technologies.
-- James Lewis and Martin Fowler
Что такое микросервисы?
48

49. Language/technology segregation
49

50. Data segregation
50

51. Team segregation
51

52. Problem → So Many Teams
52

53. 53

54. 54
Java
Python
JS

55. 55
● Spring Boot/Spring Cloud
● Ratpack
● Dropwizard
● Vert.x
● Restlet
● Spark
● KumuluzEE
?

56. 56
● Spring Boot/Spring Cloud
● Ratpack
● Dropwizard
● Vert.x
● Restlet
● Spark
● KumuluzEE
Выбирайте то,
что больше
нравится
/
в чем есть
экспертиза

57. 57

58. 58

59. Принцип LSD
- L языков программирования
- S среднее число фреймворков на язык
- D типов источников данных
complexity = L * S * D
59

60. Немного LSD для вас
- три языка программирования
- два в среднем фреймворка на язык
- семь типов источников данных
- legacy WS, mongo db
- OLTP, OLAP
- elasticsearch, neo4j
- Мишкина база %)
complexity = 3 * 2 * 7 = 42 (!)
60

61. Чем нельзя пожертвовать?
min (L * S * D) → ?
61

62. min (L * S * D) → max (D)
62

63. Problem → Many Teams Again
63

64. 64

65. 65
t0

66. DRY off
66

67. @Getter // generate getters
@Setter // generate setters
@Aspect // we are an aspect
@ToString // generate toString()
@EnableWs // SOAP is so enterprisy, we definitely need it
@Endpoint // Seriously, just read above
@EnableWebMvc // we want MVC
@EnableCaching // and we want to cache stuff
@Configuration // this class can configure itself
@RestController // we want some REST
@XmlRootElement // this component is marshallable
@EnableWebSocket // we want web socket, it's so new-generation
@RedisHash("cat") // this class is an entity saved in redis
@EnableScheduling // we want scheduled tasks
@EnableWebSecurity // and some built-in security
@NoArgsConstructor // generate no args constructor
@ContextConfiguration // we want context configuration for unit testing
@SpringBootApplication // this is a Sprint Boot application
@Accessors(chain = true) // getters/setters are chained (ala jQuery)
@EnableAspectJAutoProxy // we want AspectJ auto proxy
@EnableAutoConfiguration // and auto configuration
@EnableRedisRepositories // since it is an entity we want to enable spring data
repositories for redis
@EnableWebSocketMessageBroker // we want a broker for web socket messages 67
Fluent annotations

68. @Getter // generate getters
@Setter // generate setters
@Aspect // we are an aspect
@ToString // generate toString()
@EnableWs // SOAP is so enterprisy, we definitely need it
@Endpoint // Seriously, just read above
@EnableWebMvc // we want MVC
@EnableCaching // and we want to cache stuff
@Configuration // this class can configure itself
@RestController // we want some REST
@XmlRootElement // this component is marshallable
@EnableWebSocket // we want web socket, it's so new-generation
@RedisHash("cat") // this class is an entity saved in redis
@EnableScheduling // we want scheduled tasks
@EnableWebSecurity // and some built-in security
@NoArgsConstructor // generate no args constructor
@ContextConfiguration // we want context configuration for unit testing
@SpringBootApplication // this is a Sprint Boot application
@Accessors(chain = true) // getters/setters are chained (ala jQuery)
@EnableAspectJAutoProxy // we want AspectJ auto proxy
@EnableAutoConfiguration // and auto configuration
@EnableRedisRepositories // since it is an entity we want to enable spring data
repositories for redis
@EnableWebSocketMessageBroker // we want a broker for web socket messages 68
Fluent annotations

69. @ShutUpAndTakeMyMoney
69
Fluent annotations

70. Payment Service[jar,doc] Insurance Service [jar,doc]
One Point of View
[UberDoc.zip]
Rent Service[jar,doc] Other Service [jar,doc]
Агрегация документации
70

71. Analyst, PM Developer Tester Docs
Word, PDF...
71

72. Analyst, PM Developer Tester Docs
Word, PDF... Code + Tests
72

73. Analyst, PM Developer Tester Docs
Word, PDF... Code + Tests Test cases
73

74. Analyst, PM Developer Tester Docs :(
Word, PDF... Code + Tests Test cases
74

75. Analyst, PM Developer Tester Docs :)
Markdown/Asciidoctor
Docs :)
75

76. = Hippo Rent Service
This is documentation for Open API of our hippo renting service
== Methods
=== Rent
==== Request specification
===== Headers
//тут опишем http-заголовки
===== Example
//а здесь будут примеры вызова
==== Response specification
===== Response fields
//здесь описание полей ответа
===== Example
//ну и пример того, что вообще ожидать в ответе
Вот такой документ
76

77. ./gradlew ... asciidoc publishDocs
77
Компиляция документа

78. document(
"rent-hippo",
preprocessResponse(prettyPrint()),
requestHeaders(
headerWithName("User-ID").description("User unique identifier")),
responseFields(
fieldWithPath("hippoRemain").description("Hippo remain count"),
fieldWithPath("parrot_fee").description("Fee in virtual parrots"),
fieldWithPath("ins").description("Insurance number. Yeah, we sell
it"),
fieldWithPath("hash").description("Blockchain block hash"))
)
Имя сниппета
78

79. document(
"rent-hippo",
preprocessResponse(prettyPrint()),
requestHeaders(
headerWithName("User-ID").description("User unique identifier")),
responseFields(
fieldWithPath("hippoRemain").description("Hippo remain count"),
fieldWithPath("parrot_fee").description("Fee in virtual parrots"),
fieldWithPath("ins").description("Insurance number. Yeah, we sell
it"),
fieldWithPath("hash").description("Blockchain block hash"))
)
Тестируем заголовки
79

80. document(
"rent-hippo",
preprocessResponse(prettyPrint()),
requestHeaders(
headerWithName("User-ID").description("User unique identifier")),
responseFields(
fieldWithPath("hippoRemain").description("Hippo remain count"),
fieldWithPath("parrot_fee").description("Fee in virtual parrots"),
fieldWithPath("ins").description("Insurance number. Yeah, we sell it"),
fieldWithPath("hash").description("Blockchain block hash"))
)
Тестируем поля ответа
80

81. generated-snippets
rent-hippo
curl-request.adoc
http-request.adoc
http-response.adoc
httpie-request.adoc
request-headers.adoc
response-fields.adoc
Получаем сниппеты
81

82. = Hippo Rent Service
This is documentation for Open API of our hippo renting service
== Methods
=== Rent
==== Request specification
===== Headers
include::{snippets}/rent-hippo/request-headers.adoc[]
===== Example
include::{snippets}/rent-hippo/http-request.adoc[]
==== Response specification
===== Response fields
include::{snippets}/rent-hippo/response-fields.adoc[]
===== Example
include::{snippets}/rent-hippo/http-response.adoc[]
Вставляем их в документ
82

83. = Hippo Rent Service
This is documentation for Open API of our hippo renting
service
== Methods
=== Rent
==== Request specification
===== Headers
include::{snippets}/rent-hippo/request-headers.adoc[]
===== Example
include::{snippets}/rent-hippo/http-request.adoc[]
==== Response specification
===== Response fields
include::{snippets}/rent-hippo/response-fields.adoc[]
===== Example
include::{snippets}/rent-hippo/http-response.adoc[]
83

84. 84

85. Payment Service[logs] Insurance Service[logs]
One Log
Rent Service[logs] Other Service[logs]
Агрегация логов
85

86. 86
RentService
No TraceId
No SpanId
TraceId = X
SpanId = A

87. 87
RentService
PaymentService
No TraceId
No SpanId
TraceId = X
SpanId = A
TraceId = X
SpanId = B
TraceId = X
SpanId = C

88. 88
RentService
PaymentService
BlockchainService
No TraceId
No SpanId
TraceId = X
SpanId = A
TraceId = X
SpanId = B
TraceId = X
SpanId = C
TraceId = X
SpanId = D
TraceId = X
SpanId = D
TraceId = X
SpanId = F

89. 89
RentService
PaymentService
SecurityServiceBlockchainService
No TraceId
No SpanId
TraceId = X
SpanId = A
TraceId = X
SpanId = B
TraceId = X
SpanId = C
TraceId = X
SpanId = D
TraceId = X
SpanId = D
TraceId = X
SpanId = E
TraceId = X
SpanId = E
TraceId = X
SpanId = F
TraceId = X
SpanId = G

90. 90
RentService
PaymentService
SecurityServiceBlockchainService
No TraceId
No SpanId
TraceId = X
SpanId = A
TraceId = X
SpanId = B
TraceId = X
SpanId = B
TraceId = X
SpanId = C
TraceId = X
SpanId = C
TraceId = X
SpanId = D
TraceId = X
SpanId = D
TraceId = X
SpanId = E
TraceId = X
SpanId = E
TraceId = X
SpanId = F
TraceId = X
SpanId = G

91. 91
RentService
PaymentService
SecurityServiceBlockchainService
TraceId = X
SpanId = A
No TraceId
No SpanId
TraceId = X
SpanId = A
TraceId = X
SpanId = A
TraceId = X
SpanId = B
TraceId = X
SpanId = B
TraceId = X
SpanId = C
TraceId = X
SpanId = C
TraceId = X
SpanId = D
TraceId = X
SpanId = D
TraceId = X
SpanId = E
TraceId = X
SpanId = E
TraceId = X
SpanId = F
TraceId = X
SpanId = G

92. 92

93. Парадокс централизации
Чтобы эффективно разрабатывать распределённые
приложения, нам нужны очень хорошие
централизованные библиотеки и инструменты
Например: логирование, health-checking, метрики,
обработка типовых ошибок, автодокументирование
93

94. Парадокс централизации
Чтобы эффективно разрабатывать распределённые
приложения, нам нужны очень хорошие
централизованные библиотеки и инструменты
Но: не выносите бизнес-логику или доменные объекты!
Не размывайте бизнес-контекст вашего API
94

95. Изоляция данных
95

96. 96
Одно приложение - одна БД
Всё просто

97. N сервисов → 1 БД
Изоляция на уровне таблиц или схем
97

98. N сервисов → 1 БД
Изоляция через хранимые
процедуры 98

99. 99
Transport Layer
API API API API API

100. 100
Transport Layer
API API API API API

101. N сервисов → 1 БД
Распределённый монолит
101

102. N сервисов → N БД
102

103. N сервисов → N БД
Распределённые транзакции –
это больно 103

104. 104
Transport Layer
API API API API API
Очень
много
кода

105. N сервисов → N БД
Ваше legacy тянет вас на дно
105

106. 106

107. M сервисов → 1 БД
L сервисов → L БД
M + L = N, M << L
107

108. 108
Transport Layer
APIAPI API API API

109. M сервисов → 1 БД
L сервисов → L БД
M + L = N, M << L
109

110. Доставка
110

111. war/ear
111
Зависимость от сервера приложений

112. executable fatJar
112
Зависимость от системных
библиотек

113. 113
not war < make jar

114. executable fatJar/npm-пакет
114
Разный менеджмент для разных
форматов дистрибуции

115. docker/rkt/packer
115

116. 116
not jar < make docker

117. scp fat.jar root@prod101:/home/app/
117

118. scp fat.jar root@prod101:/home/app/
Что делать с консистентостью?
Как быть с доступностью?
118

119. ./gradlew deployToArtifactory
ansible -i [stage,dev,test] -t deploy
119

120. ./gradlew deployToArtifactory
ansible -i [stage,dev,test] -t deploy
“Прибитая молотком” конфигурация
120

121. 121
Как быстро масштабироваться?

122. Нужна оркестрация
122

123. 123

124. mesos / kubernetes / ∞
124

125. Как использовать ресурсы
125

126. 126
t
Bare
Metal

127. 127
t
Bare
Metal
Virtual
Machines

128. 128
t
Bare
Metal
Virtual
Machines
Lightweight
Containers

129. 129
t
Bare
Metal
Virtual
Machines
Lightweight
Containers

130. 130

131. Использование группы машин как
одной
WEB
WASTED
CACHE
WASTED WASTED
HADOOP
131

132. Представьте, что кластер это
просто одна большая машина
WEB
WASTED
CACHE
WASTED WASTED
HADOOP
WASTED
WEB
CACHE
HADOOP FREE FREE
132

133. yoman
Yo
start.spring.io
Ss
python
Py
lazybones
Lz
java
Ja
spring boot
Sb
gradle
Gr
asciidoctor
Ad
thrift
Th
docker
Dr
mesos
Ms
marathon
Ma
chronos
Ch
aurora
Au
Artifactory
Ar
Kubernetes
Kb
eureka
Eu
consul
Cl
etcd
Ed
zookeeper
Zk
hystrix
Hx
sleuth
Sl
zipkin
Zn
groovy
Gy
133
Data Isolation
Di
Centralization
paradox
Cp
LSD principle
Ls
DDD
Dd
Smart Docs
Sd
Dynamic Sharing
Ds
Smart Libs
Sl
SOA
So
ansible
An
Conway’s Law
Co

134. yoman
Yo
start.spring.io
Ss
python
Py
lazybones
Lz
java
Ja
spring boot
Sb
gradle
Gr
asciidoctor
Ad
thrift
Th
docker
Dr
mesos
Ms
marathon
Ma
chronos
Ch
aurora
Au
Artifactory
Ar
Kubernetes
Kb
eureka
Eu
consul
Cl
etcd
Ed
zookeeper
Zk
hystrix
Hx
sleuth
Sl
zipkin
Zn
groovy
Gy
134
Data Isolation
Di
Centralization
paradox
Cp
LSD principle
Ls
DDD
Dd
Smart Docs
Sd
Dynamic Sharing
Ds
Smart Libs
Sl
SOA
So
ansible
An
Conway’s Law
Co
?

135. A=F(Rq, FRq, NFRq,..........???............)
DevOps
OSS Domination
135

136. A=F(Rq, FRq, NFRq,.....человеки…..)
136

138. «Колодцы»

140. Преодолевать барьеры всегда
дорого

141. Manager
DBA
BA
UX
Developer QA
Operations
Software Engineer

142. Выводы
142
● Микросервисы - это просто

143. Выводы
143
● Микросервисы - это просто
● Микросервисы - это сложно

144. Выводы
144
● Микросервисы - это просто
● Микросервисы - это сложно
● А DevOps это не человек

145. Выводы
145
● Микросервисы - это просто
● Микросервисы - это сложно
● А DevOps это не человек
● Больше свободы

146. Выводы
146
● Микросервисы - это просто
● Микросервисы - это сложно
● А DevOps это не человек
● Больше свободы и больше ответственности

147. Спасибо! Готовы ответить на ваши вопросы
@tolkv
@aatarasoff
147
@lavcraft
@aatarasoff