Se ha denunciado esta presentación.
Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.

Distributed systems - Fun, Fast & Useful

142 visualizaciones

Publicado el

Have you ever build your own distributed system? Do you think it is very complicated task? All those CAP theorem, transport protocols and consensus algorithms, right? But I can say definitely - NO. It is not so hard and complicated, even more, it can be easy and fun if you are using good tools and frameworks. My example will be build on Atomix and I will show you how easy and funny it can be.

Publicado en: Ingeniería
  • Sé el primero en comentar

  • Sé el primero en recomendar esto

Distributed systems - Fun, Fast & Useful

  1. 1. Distributed Systems Fast, Fun & Useful
  2. 2. Me !2
  3. 3. Me !2 andriyrav@gmail.com JandriyR @AndriyRymar
  4. 4. Me !2 andriyrav@gmail.com JandriyR @AndriyRymar Found / Co-found: &
  5. 5. Me !2 andriyrav@gmail.com JandriyR @AndriyRymar Morning@Lohika Found / Co-found: Program committee: & &
  6. 6. Me !2 andriyrav@gmail.com JandriyR @AndriyRymar Morning@Lohika Feb 23, Lviv Found / Co-found: Program committee: Organize: & &
  7. 7. Content !3
  8. 8. Content !3 What?
  9. 9. Content !3 What? Why?
  10. 10. Content !3 What? Why? How?
  11. 11. FUN https://mbtskoudsalg.com/images/alien-clipart-1.png
  12. 12. FUN FAST https://mbtskoudsalg.com/images/alien-clipart-1.pnghttps://image.flaticon.com/icons/png/512/263/263385.png
  13. 13. FUN FAST USEFUL https://mbtskoudsalg.com/images/alien-clipart-1.pnghttps://ubisafe.org/images/astronaut-transparent-3.pnghttps://image.flaticon.com/icons/png/512/263/263385.png
  14. 14. What?
  15. 15. What? !6
  16. 16. What? !6 Features
  17. 17. What? !6 Features Goals
  18. 18. What? !6 Features Goals Challenges
  19. 19. Features !7
  20. 20. Features !7
  21. 21. Features !7
  22. 22. Features !7
  23. 23. Features !7
  24. 24. Goals !8
  25. 25. Goals !8 Transparency
  26. 26. Goals !8 Transparency
  27. 27. Goals !9
  28. 28. Goals !9 Openness
  29. 29. Goals !9 Openness
  30. 30. Goals !9 Openness
  31. 31. Goals !10
  32. 32. Goals !10 Reliability
  33. 33. Goals !10 Reliability
  34. 34. Goals !10 Reliability
  35. 35. Goals !10 Reliability
  36. 36. Goals !10 Reliability
  37. 37. Goals !11
  38. 38. Goals !11 Performance
  39. 39. Goals !11 Performance
  40. 40. Goals !11 Performance
  41. 41. Goals !11 Performance
  42. 42. Goals !12
  43. 43. Goals !12 Scalability
  44. 44. Goals !12 Scalability
  45. 45. Goals !12 Scalability
  46. 46. Goals !12 Scalability
  47. 47. Goals !13
  48. 48. Goals !13 Transparency
  49. 49. Goals !13 Transparency Openness
  50. 50. Goals !13 Transparency Openness Reliability
  51. 51. Goals !13 Transparency Openness Reliability Performance
  52. 52. Goals !13 Transparency Openness Reliability Performance Scalability
  53. 53. Challenges !14
  54. 54. Challenges !14 Security http://i.imgur.com/VdzZAqx.png
  55. 55. !15
  56. 56. Challenges !16 Security
  57. 57. Challenges !16 Security
  58. 58. Challenges !16 Security
  59. 59. Challenges !17
  60. 60. Challenges !17 Fault tolerance
  61. 61. Challenges !17 Fault tolerance
  62. 62. Challenges !17 Fault tolerance
  63. 63. Challenges !18
  64. 64. Challenges !18 Coordination
  65. 65. Challenges !18 Coordination
  66. 66. Challenges !18 Coordination
  67. 67. Challenges !19
  68. 68. Challenges !19 Security
  69. 69. Challenges !19 Security Fault tolerance
  70. 70. Challenges !19 Security Fault tolerance Coordination
  71. 71. What? !20
  72. 72. What? !20 Features
  73. 73. What? !20 Features Goals
  74. 74. What? !20 Features Goals Challenges
  75. 75. Why?
  76. 76. Why? !22
  77. 77. Why? !22 Data Processing
  78. 78. Why? !22 Data Processing Task Execution
  79. 79. !23 Data Processing
  80. 80. !23 Data Processing MR Map Reduce
  81. 81. !23 Data Processing DAG Distributed Acyclic Graph
  82. 82. !24 Distributed Acyclic Graph
  83. 83. !24 Distributed Acyclic Graph Source
  84. 84. !24 Distributed Acyclic Graph Source Tokenize
  85. 85. !24 Distributed Acyclic Graph Source Tokenize Accumulate
  86. 86. !24 Distributed Acyclic Graph Source Tokenize Accumulate Combine
  87. 87. !24 Distributed Acyclic Graph Source Tokenize Accumulate Combine Result
  88. 88. !24 Distributed Acyclic Graph Source Tokenize Accumulate Combine Result Word count
  89. 89. !24 Distributed Acyclic Graph Source Tokenize Accumulate Combine Text lines Result Word count
  90. 90. !24 Distributed Acyclic Graph Source Tokenize Accumulate Combine Text lines Extracted words Result Word count
  91. 91. !24 Distributed Acyclic Graph Source Tokenize Accumulate Combine Text lines Extracted words Word 2 count pairs Result Word count
  92. 92. !24 Distributed Acyclic Graph Source Tokenize Accumulate Combine Text lines Extracted words Word 2 count pairs Result Word count Unique pairs
  93. 93. !25 Task execution
  94. 94. !25 Task execution
  95. 95. !25 Task execution Job coordinator
  96. 96. !25 Task execution Job coordinator
  97. 97. !26 Task execution
  98. 98. !26 Task execution Self organized
  99. 99. !26 Task execution Self organized
  100. 100. !26 Task execution Self organized
  101. 101. Why? !27
  102. 102. Why? !27 Scalability
  103. 103. Why? !27 Scalability Reliability http://files.channel9.msdn.com/thumbnail/6ea2b003-389b-4e21-abbe-af00822ae852.png
  104. 104. Why? !27 Scalability Reliability Speed http://files.channel9.msdn.com/thumbnail/6ea2b003-389b-4e21-abbe-af00822ae852.png
  105. 105. How?
  106. 106. !29
  107. 107. How? !30
  108. 108. How? !30 Cluster
  109. 109. How? !30 Cluster Communication
  110. 110. How? !30 Cluster Communication Coordination
  111. 111. How? !30 Cluster Communication Coordination Failure detection
  112. 112. How? !30 Cluster Communication Coordination Failure detection Consensus
  113. 113. Cluster !31
  114. 114. Cluster !31
  115. 115. Cluster !31
  116. 116. Cluster !31
  117. 117. Cluster !31
  118. 118. Cluster !32 ?
  119. 119. Cluster !33
  120. 120. Cluster !33
  121. 121. Cluster !33
  122. 122. Cluster !33
  123. 123. Cluster !33
  124. 124. Communication !34
  125. 125. Communication !34 Data format
  126. 126. Communication !34 Data format
  127. 127. Communication !35
  128. 128. Communication !35 Routing
  129. 129. Communication !35 Routing
  130. 130. Communication !35 Routing
  131. 131. Communication !36
  132. 132. Communication !36 Acknowledgements
  133. 133. Communication !36 Acknowledgements
  134. 134. Communication !36 Acknowledgements
  135. 135. Communication !36 Acknowledgements
  136. 136. Communication !37
  137. 137. Communication !37 Timeouts & retries
  138. 138. Communication !37 Timeouts & retries
  139. 139. Communication !37 Timeouts & retries
  140. 140. Communication !37 Timeouts & retries
  141. 141. Communication !37 Timeouts & retries
  142. 142. Communication !38
  143. 143. Communication !38 Direction
  144. 144. Communication !38 Direction
  145. 145. Communication !38 Direction
  146. 146. Communication !38 Direction
  147. 147. Communication !38 Direction ?
  148. 148. Communication !39
  149. 149. Communication !39 Sequence & Flow
  150. 150. Communication !39 Sequence & Flow
  151. 151. Communication !39 Sequence & Flow
  152. 152. Communication !40
  153. 153. Communication !40 Epidemic algorithms
  154. 154. Communication !40 Epidemic algorithms Gossip
  155. 155. Gossip !41
  156. 156. Gossip !41
  157. 157. Coordination !42
  158. 158. Coordination !42 Data distribution
  159. 159. Coordination !42 Data distribution Neighborhood selection
  160. 160. Data distribution !43
  161. 161. Data distribution !43
  162. 162. Data distribution !43 - smthng + smthng
  163. 163. Data distribution !43 Long - smthng + smthng
  164. 164. Data distribution !44 Long - smthng + smthng
  165. 165. Neighborhood selection !45
  166. 166. Neighborhood selection !45 Snitch funcitons
  167. 167. Neighborhood selection !45 Snitch funcitons https://vignette.wikia.nocookie.net/harrypotter/images/b/b9/Golden_Snitch..png/revision/latest?cb=20161120075526
  168. 168. Neighborhood selection !46 Snitch funcitons
  169. 169. Neighborhood selection !47 Snitch funcitons
  170. 170. Neighborhood selection !47 Snitch funcitons
  171. 171. Neighborhood selection !47 Snitch funcitons
  172. 172. Neighborhood selection !47 1000M ly Snitch funcitons
  173. 173. Neighborhood selection !47 1000M ly Snitch funcitons
  174. 174. Neighborhood selection !47 1000M ly 450M ly Snitch funcitons
  175. 175. Neighborhood selection !47 1000M ly 450M ly Snitch funcitons
  176. 176. Failure detection !48
  177. 177. Failure detection !48 Exact value
  178. 178. Failure detection !48 Exact value Continuous value
  179. 179. Exact failure value !49
  180. 180. Exact failure value !49
  181. 181. Exact failure value !49
  182. 182. Exact failure value !49
  183. 183. Continuous failure value !50
  184. 184. Continuous failure value !50
  185. 185. Continuous failure value !50
  186. 186. Continuous failure value !50
  187. 187. Continuous failure value !50
  188. 188. Continuous failure value !50
  189. 189. Continuous failure value !50
  190. 190. Continuous failure value !50
  191. 191. Continuous failure value !50
  192. 192. Consensus !51
  193. 193. Consensus !51 https://images.vexels.com/media/users/3/152300/isolated/preview/ae0fc81df564354bb6b22d3b4fca7e85-astronauta-segurando-bandeira-dos-desenhos-animados-by-vexels.png
  194. 194. Consensus !52
  195. 195. Consensus !52 Raft
  196. 196. Consensus !52 Raft Primary-backup
  197. 197. Raft !53
  198. 198. Raft !53
  199. 199. Raft !53
  200. 200. Raft !53
  201. 201. Raft !53
  202. 202. !54 Raft
  203. 203. !54 Raft
  204. 204. !54 Leder Raft
  205. 205. !54 Leder Follower Raft
  206. 206. !54 Leder Follower Candidate Raft
  207. 207. !54 Leder Follower Candidate sync/req Raft
  208. 208. !54 Leder Follower Candidate sync/req Raft
  209. 209. !54 Leder Follower Candidate sync/req init election Raft
  210. 210. !54 Leder Follower Candidate sync/req init election start election Raft
  211. 211. !54 Leder Follower Candidate sync/req init election start election change to leader Raft
  212. 212. !54 Leder Follower Candidate sync/req init election start election change to leader Raft change to follower
  213. 213. Primary-backup !55
  214. 214. Primary-backup !55
  215. 215. Primary-backup !55
  216. 216. Primary-backup !55
  217. 217. Primary-backup !55
  218. 218. Primary-backup !56
  219. 219. Primary-backup !56 Client
  220. 220. Primary-backup !56 Client Primary
  221. 221. Primary-backup !56 Client Primary Backup
  222. 222. Primary-backup !56 Client Primary Backup synk synk
  223. 223. Primary-backup !56 Client Primary Backup synk synk req
  224. 224. Primary-backup !56 Client Primary Backup synk synk req upd
  225. 225. Primary-backup !56 Client Primary Backup synk synk req resp upd
  226. 226. Primary-backup !56 Client Primary Backup synk synk synk req resp upd
  227. 227. Primary-backup !56 Client Primary Backup synk synk synk req resp upd
  228. 228. Primary-backup !56 Client Primary Backup synk synk synk req resp upd I’m Primary
  229. 229. How? !57
  230. 230. How? !57 Cluster
  231. 231. How? !57 Cluster Communication
  232. 232. How? !57 Cluster Communication Coordination
  233. 233. How? !57 Cluster Communication Coordination Failure detection
  234. 234. How? !57 Cluster Communication Coordination Failure detection Consensus
  235. 235. DEMO
  236. 236. Atomix !59
  237. 237. Atomix !60
  238. 238. Atomix !61https://vignette.wikia.nocookie.net/starwars/images/6/62/X-70B-Phantom-TOR.png/revision/latest?cb=20161011022806
  239. 239. Demo plan !62
  240. 240. Galaxy race !63
  241. 241. Galaxy race !63
  242. 242. !64 Galaxy race
  243. 243. !64 Galaxy race
  244. 244. !64 Galaxy race
  245. 245. !65 Galaxy race
  246. 246. !66 Galaxy race
  247. 247. !66 Galaxy race
  248. 248. !66 Galaxy race
  249. 249. !66 Galaxy race Boolean
  250. 250. !66 Galaxy race Boolean Start race Finish race
  251. 251. !66 Galaxy race Boolean Map Start race Finish race
  252. 252. !66 Galaxy race Boolean Map Start race Finish race Racer name Racer sate
  253. 253. !66 Galaxy race Boolean Map Start race Finish race Racer name Racer sate 0…..n
  254. 254. !66 Galaxy race Boolean Map Start race Finish race Racer name Racer sate 0…..n UI
  255. 255. !67
  256. 256. Hope you like it !68
  257. 257. Summary !69
  258. 258. Summary !69 What?
  259. 259. Summary !69 What? Why?
  260. 260. Summary !69 What? Why? How?
  261. 261. Demo repo !70 https://github.com/flylight/atomix-space-racing
  262. 262. Thanks !71 Nick Mkrtchyan https://codepen.io/WebSonick/
  263. 263. Resources !72 • https://images.vexels.com/media/users/3/150026/isolated/preview/580cca5efa65c7c69ba5cad9e954698d-ilustra--o-de-foguete-espacial- by-vexels.png • http://www.transparentpng.com/download/space/space-rocket-silver-icon-png-30.png • https://pixler.com.au/wp-content/uploads/2017/05/rocket.png • https://dumielauxepices.net/sites/default/files/spaceship-clipart-772365-6555786.png • https://apprecs.com/android/com.nanoki.warspace/war-space-free-strategy-mmo • https://mbtskoudsalg.com/explore/spaceship-png-images/ • https://media.indiedb.com/images/games/1/51/50676/sun.png • http://planetlazer.net/beta/wp-content/uploads/2014/09/fireplanet.png • https://img00.deviantart.net/16cf/i/2012/137/9/5/space_station_3_by_mysticmorning-d504aab.png • https://ubisafe.org/images/astronaut-transparent-3.png • https://vignette.wikia.nocookie.net/starwars/images/c/c4/XWAPLT2-StationStub.png/revision/latest?cb=20160904144954 • https://i.pinimg.com/originals/a1/40/a5/a140a5c1206d3cd334cf1781d0d41d0b.png • https://cdn150.picsart.com/upscale-252635731007212.png?r1024x1024 • https://cdn.pixabay.com/photo/2017/07/29/04/54/cartoon-2550646_1280.png • https://wallscover.com/images/showtime-wallpaper-5.jpg • http://mightynice.com.au/wp-content/uploads/2015/07/Spaceman2.png
  264. 264. !73 Thank You! https://i.gifer.com/Bf.gif
  265. 265. !73 Thank You! https://i.gifer.com/Bf.gif
  266. 266. Questions? !74 andriyrav@gmail.com JandriyR @AndriyRymar Become a speakerContacts Morning@Lohika JavaDay Lviv

×