2. Awesome architectures
in Magento 2.3
Alessandro Ronchi
Bitbull
@aleron75
This presentation has been realized with kind permission of Riccardo Tempesta (The
Rick); the original one was presented by him at Meet Magento Germany 2019 and the
slides you are seeing here are just a refactoring of his original work. Most of the
content is the same, especially the jokes :)
3. @MageTitansES @aleron75
And, in turn, most of the work presented here is built standing on the shoulders of
heroes like the Magento architect you see here.
- Igor Miniailo
- Eugene Shakhsuvarov
- Anton Kril
If you are wondering where all this hero culture comes from…
5. - COO at Bitbull
- Mage Titans Italy organizer
- Magento Master 2017, 2019
- Top 50 Contributor 2017, 2018
- Magento Community Maintainer
Alessandro Ronchi
@MageTitansES @aleron75
Never stop exploring!
Just a couple of words about me.
6. Agenda
1. The Magento legacy
2. Software architectures evolution
3. Evolution of Magento 2
4. Some applied concepts
7. Confused about
Magento development?
We’ve heard that someone feels a bit stressed by developing on Magento 2 and this
probably comes from a bit of confusion.
Nevertheless I’ve talked with Mark, a 23yo guy, who told me that developing on
Magento 2 is not stressful at all.
8. “Being a Magento 2 developer
is not stressful at all”
~ Mark, 23 years old
@MageTitansES @aleron75
You can see the quote from Mark…
And yes, you spotted out the joke because, actually, Mark is only 20yo :)
9. CONFUSION
from Latin: “cum” + “fusus”
So where this stress come from?
To understand, let’s see where this confusion comes from.
10. M2 !== M1
It comes from the fact that M2 is not a revolution but an evolution of M1: M1 is our
legacy, merged into M2.
Despite this, we shouldn’t develop on M2 the same way we did on M1, here is where
the confusion likely comes from.
11. Software architecture
evolution: 1990’s to 2010’s
To better understand our legacy and the direction M2 is getting, let me briefly run
through the evolution of software architectures of last decades.
And indulge me to do it like any legit Italian should do it.
13. Software architecture: 1990s
@MageTitansES @aleron75
- no standards
- fast development
- no maintenance
At that time we had very few standards for web applications, the market required fast
development and the concept of software maintenance was pretty new.
As a result, code suffered from high coupling, we couldn’t change one piece of code
without affecting another and, infact, we didn’t do it.
Rather, we preferred to copy logic here and there because applying changes could
have brought undesired side effects (known as regressions).
Without tests, we couldn’t perform refactoring and w/o refactoring we couldn’t clean
the kitchen, we couldn’t increase the quality and maintainability of our code.
15. Software architecture: 2000s
@MageTitansES @aleron75
- bigger complexity
- testability
- scalability (as a whole)
In the 2000s the market required more complex, testable and scalable web
applications.
We realized that we could obtain this by isolating some responsibilities into layers.
So n-layered applications came out.
Typical 3-layered: presentation, business logic, data handling.
This architecture is not bad per se, this is how, for example, M1 was built.
But there are some limitations:
- not possible to scalable separately read/write
- app deployed as a whole (monolithic application)
- layers tend to reflect the data model, UIs are typically CRUD oriented
16. Software architecture: 2010s
@MageTitansES @aleron75
Finally we realized that we could split a monolith into smaller pieces, each with single
specific responsibility, with something good inside exposed through public interfaces
(APIs).
It was time to serve ravioli!
17. Software architecture: 2010s
@MageTitansES @aleron75
- composition
- independent deployability
- independent scalability
This kind of applications allows us to compose functionality, deploy and scale them
independently.
This is what we called Service Oriented Architectures and, more trendily,
microservices.
23. @MageTitansES @aleron75
...that can brings us to situations like the one illustrated.
This comes as a legacy from Magento 1 and it became evident thanks to DI.
Why having all these dependencies (which btw is not good) was not evident before?
24. final class Mage
Because of the infamous “God” class of M1
It allowed a class to easily depend on other classes at the cost of “hiding” this
dependency (while adding more and more responsibilities).
DI made all these dependencies evident in M2 and revealed the architectural
problems of M1: classes had too much responsibilities.
26. MODULES !== MODULARITY
Not quite: having modules doesn’t mean having a real modular architecture.
Real modularity is the very first step towards Service Isolation.
28. @MageTitansES @aleron75
Bounded Contexts
InventoryCatalog Checkout Customer ...
Bounded Contexts is a concept that comes from Domain Driven Design; in brief:
DDD is an approach to software development that helps us taking decisions about
how to design our application speaking the ubiquitous language of the Domain
Experts (our customer) and trying to avoid ambiguity and focusing on design
simplification.
A Bounded Context is a unit in which all the elements that we design have an
ubiquitous meaning and possibly a single responsibility, allowing us to reach
design simplification.
29. Another Inventory
@MageTitansES @aleron75
Bounded Contexts: replaceability
Inventory
Catalog Checkout Customer ...
Isolating elements (concepts) in bounded context easily allows to obtain
replaceability, one of the key goals of Service Isolation.
30. @MageTitansES @aleron75
Bounded Contexts: example
Catalog
- sku
- name
- description
...
Let’s see an example: the representation of a product in different contexts.
31. @MageTitansES @aleron75
Bounded Contexts: example
Catalog
- sku
- name
- description
...
Inventory
- sku
- physical qty
- in stock
...
Here we see that a product in different contexts can have different attributes.
32. @MageTitansES @aleron75
Bounded Contexts: example
Catalog
- sku
- name
- description
...
Inventory
- sku
- physical qty
- in stock
...
Checkout
- sku
- salable qty
- unit price
...
Different responsibilities, different behaviors, no need to generalize in a unique
concept that would be:
- charged with too many responsibilities
- prone to ambiguity: what does a “qty” property represent? Physical
availability? Salable quantity?
33. @MageTitansES @aleron75
Bounded Contexts: example
Catalog
Product
Inventory
Source Item
Checkout
Quote Item
In different contexts, the same concept maps to different entities.
Different entities can be simpler, more specialized, less ambiguous.
40. @MageTitansES @aleron75
Contexts Interaction in M1: coupled
Inventory Sales
MUTUAL DEPENDENCY
This is how context interaction works in M1 (and still works in the majority of M2
codebase): we have mutual dependency between different contexts.
That’s not what we want.
41. @MageTitansES @aleron75
Contexts Interaction in M2: decoupled
Inventory
Sales
Inventory Sales
Uses plugins to
add/change logic
This, instead, is how ideally we can achieve decoupled context interaction in M2:
through the usage of a specific context which makes the connection b/w two isolated
contexts that don’t share anything with each other.
This third element implements specialized business logic and can use plugins to
interact with the other contexts.
43. @MageTitansES @aleron75
Observers: why not
Extension point
Observer
Event
Event
Extension point
- soft dependencies
- rely only on existing extension points
Observers don’t allow us to define hard evident dependencies meaning that, if the
event that is observed is not dispatched, the observer is not triggered.
If something is not working we don’t have evidence of that.
Furthermore, we can only rely on extension points that have been previously defined:
no event triggered, no extension point.
44. @MageTitansES @aleron75
Preferences: why not always
- some classes still do too much
- can be used on single responsibility classes
- all or none
MANY RESPONSIBILITIES
TOO MANY POTENTIAL SIDE EFFECTS
SINGLE RESPONSIBILITY
(WHEN AVAILABLE)
Preferences allow us to replace a core class with our own implementation. That can
work sometimes but the truth is that a lot of classes in M2 still have a lot of
responsibilities; replacing a class means taking the ownership of all its business logic.
Maybe we need to change only a few behaviors.
Through preferences, there can be only one class replacing another at the same time,
no space for others.
45. @MageTitansES @aleron75
Plugins: the best option
- hard & explicit dependencies
- no extension point needed (more or less)
- work at method level
Plugins are the best option, because they don’t have the drawbacks seen:
- dependency is hard and explicit because we declare the class which we are
pluginizing by passing it as the $subject parameter
- we don’t need the core team to have foreseen the possible extension point
because we can pluginize every public method (read more here)
- they work at method level and multiple plugins can coexist on the same class
and even on the same method
46. MODULARITY
API FIRST
Last, but not least, real modularity can be achieved by designing a system that rely
on abstractions, that is, interfaces or, in other words, APIs.
50. @MageTitansES @aleron75
API first: replaceability
MyModuleApi MyModule
AnotherModule
- modules only depend on abstractions
Again, depending on abstractions allows us to obtain replaceability
53. @MageTitansES @aleron75
API first: modules splitting
http://bit.ly/mage2si
This is a picture made among others by Anton Kril, Director of Architecture at
Magento, that summarizes the possible scenarios before and after applying this
approach.
You can find this and other diagrams in the Service Isolation repository linked before:
http://bit.ly/mage2si
54. @MageTitansES @aleron75
Modules splitting: so many modules!
MSI is 40+ modules, so far...
A consequence of this approach is the growing number of modules required to
implement a functionality.
60. FULL MODULAR SYSTEM CLEAR CONTEXT BOUNDARIES
@MageTitansES @aleron75
Service isolation
This approach allows us to have:
- full modularity
- clear context boundaries
61. FULL MODULAR SYSTEM CLEAR CONTEXT BOUNDARIES EASY TO FIND RESPONSIBILITIES
@MageTitansES @aleron75
Service isolation
This approach allows us to have:
- full modularity
- clear context boundaries
- classes that are easier to find because they have fewer responsibilities,
located in proper namespaces
62. FULL MODULAR SYSTEM CLEAR CONTEXT BOUNDARIES EASY TO FIND RESPONSIBILITIES
REPLACEABILITY
@MageTitansES @aleron75
Service isolation
This approach allows us to have:
- full modularity
- clear context boundaries
- classes that are easier to find because they have few responsibilities located
in proper namespaces
- we get replaceability, making it easier to override entire parts of core module
63. FULL MODULAR SYSTEM CLEAR CONTEXT BOUNDARIES EASY TO FIND RESPONSIBILITIES
REPLACEABILITY INDEPENDENT SCALABILITY
@MageTitansES @aleron75
Service isolation
This approach allows us to have:
- full modularity
- clear context boundaries
- classes that are easier to find because they have few responsibilities located
in proper namespaces
- we get replaceability, making it easier to override entire parts of core module
- and finally, we can scale pieces of application independently thanks to DB
isolation
65. @MageTitansES @aleron75
DB isolation: foreign keys?
FK
Source Item
product_id product_id
Product
In Magento 1 (and still in the majority of Magento 2 as well), we can’t split the DB not
to break referential integrity.
66. FK
Source Item
product_id product_id
Product
FK: THE DATABASE MUST BE THE SAME
@MageTitansES @aleron75
DB isolation: foreign keys?
Because of this “internal” link, different parts of the application must share the same
DB.
67. Source Item
sku sku
Product
@MageTitansES @aleron75
DB isolation: foreign keys?
use business keys: no DB dependency
So what can we do to avoid problems with foreign keys?
Get rid of them! Using business keys, instead.
68. Source Item
sku sku
Product
use business keys: no DB dependency
@MageTitansES @aleron75
DB isolation: foreign keys?
Pay attention, this comes at a cost: we lose something, of course, because we are
entering in the field of distributed systems where immediate data consistency is
not always guaranteed.
70. @MageTitansES @aleron75
The CAP theorem
Reference:
https://www.dataversity.net/acid-vs-base-the-shifting-ph-of-database-transaction-proc
essing/
There are three essential system requirements necessary for the successful
design, implementation and deployment of distributed applications.
They are Consistency, Availability and Partition Tolerance – or CAP:
- Consistency: all nodes see the same data at the same time
- Availability: node failures don’t prevent the system to continue working
- Partition Tolerance: network failures don’t prevent the system to
continue working
CA: typical of ACID systems (Atomicity, Consistency, Isolation, Durability: M1) - not
good for scaling
CP: typical of BASE systems (Basically Available, Soft state, Eventual consistency:
M2)
71. in distributed systems
we can guarantee
only two!
@MageTitansES @aleron75
The CAP theorem
The CAP Theorem states that in distributed systems we can guarantee only two of the
three requirements.
Magento 1 is an ACID system where we have Consistency and Availability
guaranteed.
72. in large scalable
distributed systems
we give up on
immediate consistency
@MageTitansES @aleron75
The CAP theorem
The Service Isolated architecture towards which Magento 2 is evolving requires it to
abandon Immediate Consistency shifting from an ACID to a BASE system.
73. in large scalable
distributed systems
we rely on
eventual consistency
@MageTitansES @aleron75
The CAP theorem
Eventual Consistency means that the system will eventually become consistent
once it stops receiving input.
The data will propagate to everywhere it should sooner or later, but the system will
continue to receive input and is not checking the consistency of every transaction
before it moves onto the next one.
Examples of eventual consistency we experience everyday as users:
- Dropbox
- Git
- Google Docs
74. We read data from hereWe write data here
Separate contexts can have different representation of an information.
Data projection doesn’t happen necessarily in real-time and can take some time.
DATA PROJECTION
@MageTitansES @aleron75
Eventual consistency
A diagram illustrating how data becomes consistent b/w different systems.
76. Is this the real life?
Is this just fantasy?
Let me anticipate a question that can come from the audience.
All the ideas expressed here are the result of how Magento Inventory (formerly MSI)
is developed and chances are that the future architecture of Magento 2 (or should we
call it Magento 3?) is going towards this.
As an old Chinese saying tells us: "A journey of a thousand miles begins with a single
step"
77. @MageTitansES @aleron75
Reference: MSI Magento Inventory
http://bit.ly/mageinventory
That’s why I heavily recommend to start looking at the Magento Inventory Project.
78. Good reads
http://bit.ly/dsbook1 http://bit.ly/dsbook2 http://bit.ly/dsbook3
FREE!
Here are some recommended books about distributed systems:
- Designing Data-Intensive Applications http://bit.ly/dsbook1
- Building Microservices http://bit.ly/dsbook2
- Distributed Systems (introductory, easy, free, must read!) http://bit.ly/dsbook3
79. What I learned
1. M2: CONTINUOUS IMPROVEMENT
2. SERVICE ISOLATION
3. DISTRIBUTED SYSTEM (M3?)
4. LEARN BY CONTRIBUTING
To recap the lessons learned:
- Magento is under continuous improvement
- At date, Service Isolation is the number one goal of architectural evolution
- In the future, Magento could become a distributed platform
- All the information provided in this presentation (and many, many more) were
learned by contributing to the Magento core
80. “there’s no better way to learn!”
Paraphrasing a famous flying company, contributing is the better way to learn (in my
opinion).
81. @MageTitansES @aleron75
Get involved in the evolution!
It’s up to you: hate or help!
https://magentocommeng.slack.com/messages#appdesign
Jump in, if you want.
82. I thank you all!
Alessandro Ronchi
Bitbull
@aleron75
“You brought me fame and fortune and everything that goes with it.
I thank you all” ~ Freddy Mercury