Life Beyond the Illusion of Present
Video: https://www.parleys.com/tutorial/life-beyond-illusion-present Summary: The idea of the present is an illusion. Everything we see, hear and feel is just an echo from the past. But this illusion has influenced us and the way we view the world in so many ways; from Newton’s physics with a linearly progressing timeline accruing absolute knowledge along the way to the von Neumann machine with its total ordering of instructions updating mutable state with full control of the “present”. But unfortunately this is not how the world works. There is no present, all we have is facts derived from the merging of multiple pasts. The truth is closer to Einstein’s physics where everything is relative to one’s perspective. As developers we need to wake up and break free from the perceived reality of living in a single globally consistent present. The advent of multicore and cloud computing architectures meant that most applications today are distributed systems—multiple cores separated by the memory bus or multiple nodes separated by the network—which puts a harsh end to this illusion. Facts travel at the speed of light (at best), which makes the distinction between past and perceived present even more apparent in a distributed system where latency is higher and where facts (messages) can get lost. The only way to design truly scalable and performant systems that can construct a sufficiently consistent view of history—and thereby our local “present”—is by treating time as a first class construct in our programming model and to model the present as facts derived from the merging of multiple concurrent pasts. In this talk we will explore what all this means to the design of our systems, how we need to view and model consistency, consensus, communication, history and behaviour, and look at some practical tools and techniques to bring it all together.
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Life Beyond the Illusion of Present
Similar to Life Beyond the Illusion of Present (11)
More from Jonas Bonér
More from Jonas Bonér (19)
Recently uploaded
Recently uploaded (20)
Life Beyond the Illusion of Present
- 1. Life BEYOND the ILLUSION of PRESENT Jonas Bonér CTO Typesafe @jboner
- 2. "Time is what prevents everything from happening at once." - John Archibald Wheeler
- 4. THIS SIMPLIFIED MODEL IS VERY APPEALING TO US
- 6. BACK THEN LIFE WAS GOOD
- 7. THEN, ALONG CAME CONCURRENCY MADE LIFE MISERABLE
- 10. WELL, ALONG CAME DISTRIBUTION MADE LIFE MISERABLE, again...
- 11. But don't be surprised UNFORTUNATELY, THIS IS NOT HOW THE WORLD WORKS
- 12. "The future is a function of the past." - A. P. Robertson
- 13. "The (local) present is a merge function of multiple concurrent pasts." — Me
- 14. val newLocalPresent = observedPasts. foldLeft(oldLocalPresent) { _ merge _ }
- 15. WE NEED TO EXPLICITLY MODEL THE LOCAL PRESENT AS FACTS DERIVED FROM THE MERGING OF MULTIPLE CONCURRENT PASTS
- 16. INFORMATION IS ALWAYS FROM THE PAST
- 17. THE TRUTH IS CLOSER TO EINSTEIN'S PHYSICS
- 19. THE COST OF MAINTAINING THIS ILLUSION IS INCREASED CONTENTION & COHERENCY
- 20. AS LATENCY GETS HIGHER, THE ILLUSION CRACKS EVEN MORE
- 22. THE NETWORK IS RELIABLE...NAT
- 23. "If a tree falls in a forest and no one is around to hear it, does it make a sound?" — Charles Riborg Mann
- 24. Information CAN (and will) GET LOST
- 25. HOW DO WE DEAL WITH INFORMATION LOSS IN REAL LIFE?
- 26. WE USE A SIMPLE PROTOCOL OF Confirm, Wait & Repeat
- 27. We fill in THE BLANKS
- 28. ...AND IF WE ARE WRONG, WE TAKE COMPENSATING ACTIONAPOLOGY-ORIENTED PROGRAMMING - PAT HELLAND (IN MEMORIES, GUESSES, AND APOLOGIES)
- 29. The bottom line: WE CAN'T FORCE THE WORLD INTO A SINGLE GLOBALLY CONSISTENT PRESENT
- 30. Should we just GIVE UP?
- 31. I BELIEVE THAT THERE IS A PATH FORWARD
- 32. WE NEED TO TREAT TIME AS A FIRST CLASS CONSTRUCT
- 33. WHAT IS TIME, really?
- 34. TIME IS THE SUCCESSION OF CAUSALLY RELATED EVENTS
- 36. Think in FACTS
- 37. What is a FACT?
- 38. "A Fact is something that truly exists or happens: something that has actual existence, a true piece of information." — Merriam Webster
- 40. So, do variables HAVE A PURPOSE IN LIFE?
- 41. "The assignment statement is the von Neumann bottleneck of programming languages and keeps us thinking in word-at-a-time terms in much the same way the computer's bottleneck does." — John Backus (Turing Award lecture 1977)
- 42. MUTABLE STATE NEEDS TO BE CONTAINED
- 43. Ok, but how should we MANAGE FACTS?
- 48. "When bookkeeping was done with clay tablets or paper and ink, accountants developed some clear rules about good accounting practices. One never alters the books; if an error is made, it is annotated and a new compensating entry is made in the books. The books are thus a complete history of the transactions of the business. Update-in-place strikes many systems designers as a cardinal sin: it violates traditional accounting practices that have been observed for hundreds of years." — Jim Gray (1981)
- 49. CRUD
- 50. "Database is a cache of a subset of the log." — Pat Helland (2007)
- 51. Store facts in an EVENT LOG
- 53. Can we REWRITE THE PAST?
- 54. Allows us to shift our focus from DATA AT REST, to DATA IN MOTION
- 56. CONSTRUCTING A SUFFICIENTLY CONSISTENT LOCAL PRESENT MEANS EMPLOYING CONSISTENCY MECHANISMS
- 58. WE NEED TO DECOMPOSE THE SYSTEM USING CONSISTENCY BOUNDARIES
- 59. INSIDE DATA: OUR CURRENT PRESENT OUTSIDE DATA: BLAST FROM THE PAST BETWEEN SERVICES: HOPE FOR THE FUTURE — PAT HELLAND (DATA ON THE INSIDE VS DATA ON THE OUTSIDE)
- 60. MicroSERVICE
- 61. AGGREGATE Root
- 62. WITHIN THE CONSISTENCY BOUNDARY
- 63. EVENT Sourcing
- 65. Decoupling in TIME / SPACE
- 67. Here, we are living in the LOOMING SHADOW OF IMPOSSIBILITY THEOREMS
- 69. PROTOCOLS CLIMB THE LADDER OF KNOWLEDGE Cɸ: Common Knowledge (infinite number of i) Eiɸ: (Everyone knows * i) ɸ E3ɸ: (Everyone knows * 3) ɸ E2ɸ: Everyone knows Everyone knows ɸ E1ɸ: Everyone knows ɸ Sɸ: Someone knows ɸ
- 70. COMMON KNOWLEDGE IS NOT ATTAINABLE VIA PROTOCOL- JOSEPH HALPERN
- 72. Dissecting CAP
- 73. "The first principle of successful scalability is to batter the consistency mechanisms down to a minimum." – James Hamilton
- 74. EVENTUAL CONSISTENCY What does it really mean?
- 75. Tracking TIME is tracking CAUSALITY
- 76. RELYING ON TIMESTAMPS IS A BAD IDEA
- 77. Instead, rely on LOGICAL TIME
- 78. Lamport CLOCKS GLOBAL CAUSAL ORDERING BETWEEN
- 79. Vector CLOCKSPARTIAL CAUSAL ORDERING BETWEEN EVENTS
- 81. What CONSISTENCY DO YOU REALLY NEED AND when?
- 83. CONFLICT-FREE REPLICATED DATA TYPES
- 85. WE ARE JUST GETTING STARTED
- 86. WE HAVE A LONG ROAD AHEAD OF US...
- 89. Life BEYOND the ILLUSION of PRESENT Jonas Bonér CTO Typesafe @jboner