In this talk, I'll explain how the current web applications are no more simple DB projections but more complex and distributed applications that require technologies and architectures different from the past. We start with a simple & classic application to move to a more complex and feature-rich application that need a more advanced architecture.
Global Identity Enrolment and Verification Pro Solution - Cizo Technology Ser...
From a web application to a distributed system
1. From a web application to a
distributed system
How engineering is hard but could be harder without the BEAM
2. GPad
Born to be a developer with an interest in distributed system.
I develop using many languages like C++, C#, js and ruby. I had fallen in love with
functional programming, especially with elixir, erlang.
● Twitter: https://twitter.com/gpad619
● Github: https://github.com/gpad/
● Medium: https://medium.com/@gpad
CTO & founder of coders51
4. Web development history ...
A long a time ago in Galaxy Far Away there was a DB, with some scripts and a
web-server in front of these.
Every request was a bunch of queries on DB and nothing more …
Everything was fine and consistent …
Then …
5. Today
You have a bunch of services that need to talk to one another.
You need to maintain some state for a time span longer than a http request.
You have more DB (no transaction).
You need to scale.
7. The BEAM
Who came from classic web development, especially with OOP, was trained to:
1. Don’t keep the state in memory
2. Create object/struct during http request
3. Destroy everything at the end
It works!
9. But ...
I need to maintain state for more than an http transaction (long operation).
I need to talk/listen to other systems and “react” to these event.
I don’t have a single DB so I don’t have transaction.
I need to keep the connection open and send some data to my “client”.
11. Question
How can we create a web application that takes
advantage of the characteristics of the BEAM using
the resources of a single node and eventually scale
to a multi-node application?
12. Example
Simple e-commerce app where you can:
1. Create users
2. Get products by a third party system
3. Buy some products
4. Pay
14. Embrace the memory state
We can maintain the state of our app in memory.
In the BEAM world every process is isolated and can keep the state.
Use the DB as the backup of you state.
We can store on it and read from it but don’t use the DB as your state.
15. Embrace the memory state - Customer
Every customer can be mapped to a process that maintains the state of the customer in
memory and saves it on DB.
If a customer crashes, it doesn’t affect the other customers.
The operation of a single customer is serialized through a single process.
Show me to code ...
25. Embrace the KV pattern
We could build our app using the KV pattern.
We could find this “entities” and use the process as boundaries between them.
Every entity should be something that can be executed as a unit.
Every entity should talk to other entities.
The crash of a single entity should affect only a single entities.
26. Embrace the KV pattern
Use other processes (activities) to interact and keep track of the interaction with other
entities.
Every process can run independently from the others, it can crash and recover to
conclude the interaction with the other entities/systems.
Show me the code ...
52. Embrace the world ...
With this KV pattern you can “scale” your application horizontally.
Every entity can be “transparently located” inside the cluster.
You can see your application as composed by two different parts: an “application level”
composed by entities and activities and another level “system level” that take care of
distribution.
You should have a “platform” that take care of spinning, move and locate your entities
inside your cluster.
Do you remember something ?!?
53. Embrace the world - BEAM
The BEAM has “natively” a lot of mechanism to take care of a lot of problems that can
raise from a distributed system.
You can contact a process inside your cluster in a transparently way.
You have a registry (local or remote) that can help you to find an entity (process).
It’s a good solution as a starting point ...
54. Embrace the world - Library
There are a lot of libraries that can help you in this wide and dangerous world.
riak_core - Now fully managed by community
erleans - Inspired by orleans
lasp - A entire suite of libraries
Show me the code ...
61. Recap
Embrace the memory state - Keep the state of your app in memory and use the DB as
storage. DB is not your App!!!
Embrace the KV pattern - Identify the entities of your app and try to build it as
isolated as possible.
Rule the world - You are running on the BEAM!!!
62. References
● Thinking like an Erlanger
● Selling Food With Elixir
● To spawn, or not to spawn?
● Life Beyond Distributed Transactions
● Create a riak_core application in elixir
● Erleans
● Lasp
● Repo of the example