Businesses are inherently complex with many interrelated parts. This complexity can be managed by applying principles of minimalism and containment at different levels of software architecture. At the lowest levels, pure functions are used to contain complex business logic. At higher levels, services isolate complexity by having single responsibilities and well-defined external contracts. The overall system complexity is managed through infrastructure as code, standard service templates, and evolving the system into platforms and products over time. By separating concerns and containing complexity at each level, complex systems can be understood and evolved.

1. Complex
Made
Bearable
20/11/2019
Lucas Cavalcanti
@lucascs

2. http://www.infoq.com/presentations/Simple-Made-Easy

3. Immutable data structures
Functions
Standard library
Few syntax elements
Smart Defaults: Language
SIMPLE MADE EASY

5. 5
MINIMAL
STY LE
Simple is
now Easy.
What
about
Complex?

6. Businesses are usually
inherently complex
SIMPLE MADE EASY

9. 4,0
8,0
12,0
16,0
Set-14 Mar-15 Set-15 Mar-16 Set-16 Mar-17 Set-17 Mar-18 Set-18 Mar-19 Sep-19
GROWING RAPIDLY IN A COMPLEX DOMAIN
Customers
15M
Engineers
400
Kafka Messages/day
1B
Microservices
350
Deploys/day
10s
Shards
15
Teams
50
HTTP Requests/day
1B

10. MINIMAL
STY LE
Abstraction
levels:
Software is
a fractal
https://ccsearch.creativecommons.org/photos/baba59b2-31f7-469c-a2c2-0ca7843495f1

11. MINIMAL
STY LE
We can
have
enough
complexity
at each
level
https://ccsearch.creativecommons.org/photos/baba59b2-31f7-469c-a2c2-0ca7843495f1

12. MINIMAL
STY LE
As long as
we are
dealing with
one
complexity
at time
https://ccsearch.creativecommons.org/photos/baba59b2-31f7-469c-a2c2-0ca7843495f1

13. Function level: Business
logic may be complex

14. - Clojure spec
- invariants
- generative tests
- example based tests
- contained complexity
Pure functions
(s/defn revolving-corrections :- [s.amortization/Correction]
[type :- (s/enum :revolving :late),
amortizations :- [models.amortization/Amortization]]
(->> amortizations
(group-by :amortization/post-date)
(map (partial revolving-correction type))
(sort-by :post-date)))

15. Controllers
Fetches data from db, other services
Runs business logic
Alters database, produces events
Idempotent! (whenever possible)
(s/defn new-payment!
[{:keys [account-id post-date]
:as payment} :- schemata.payment/Payment
as-of :- types.time/LocalDateTime
http :- IHttpClient
producer :- IProducer]
(let [balance-sheet (http/balance-sheet! account-id post-date http)
amortization (l.payment/amortization payment balance-sheet)
revolved (l.payment/revolved balance-sheet amortization)]
(producer/debt-payment! amortization as-of producer)
(when revolved
(producer/new-revolving! revolved as-of producer))))

16. Do not couple
logic with side
effects,
compose them

17. Service level: Integration
may be complex

18. - Infrastructure to run functions
- Single responsibility
- Takes care of few entities
Services

19. Ports
Implements external communication
Components
Libraries shared between services
Standard protocols
HTTP, Kafka, S3, Datomic
(defprotocol Producer
(produce! [component produce-map]
"Produce an asynchronous message,
with details defined in the produce-map”))
(defprotocol HttpClient
"Protocol for making HTTP requests (outbound)"
(req! [component req-map] [component defaults req-map]
"Make a request, optionally overriding the defaults”))

20. - Schemas for endpoints
- Adapting internal to external
schemas
- Handlers are just functions that
receive maps
- Wires ports to controllers
- Has access to all components, so
dependencies are explicit
Adapters
(defn balances
[{account-id :uuid
{:keys [datomic]} :components}]
{:status 200
:body (-> (c.account/balances account-id datomic)
(adapters.balances/wire-document))})
(def routes
#{["/api/accounts/:id/balances"
:get
(conj base-interceptors
(doc/desc "Gets account balances")
(auth/allow? #{"admin"})
(adapt/externalize! {200 out.balances/Document})
balances)]
...})

21. Do not couple
controller with
entry points

22. Do not couple
entrypoints with
exchange
technology or
frameworks

23. Diplomat Architecture
Ports and Adapters for Functional Programming

24. Ports
- Complex communication
hidden in a simple protocol

25. Adapters
- Complex wiring done via
functions that receive
maps and return maps

26. Controllers
- Complex wiring of logic
and side effects done via
composing function calls

27. Pure functions
- Complex business logic
done via values and use
of standard libraries

28. Services
- Keep deployable unit small
enough to be bearable

29. System level: Infrastructure
is complex

30. Service template with the diplomat structure
Default set of components and libraries
Clojure, Datomic, Kafka, S3
Metadata on endpoints
Smart defaults: Service
SERVICE LEVEL

31. Infrastructure as
code
- Leverages homogeneous
services
- EDN files
- Abstracts complex
provisioning logic
- Most engineers don’t have to
know how it works, but all
benefit from it

32. Contracts +
Tolerant readers
- All endpoints declare their schemas
- Sachem validates if consumers and
producers are compatible
- Consumer declares only what’s needed

33. Lego pieces
- Each service looks like the same
- Can be seen as a function on HTTP
- Can be seen as a channel +
transducer on Kafka

34. Product level: Evolving is
complex

35. Business
Domains
- A cohesive set of services
- Conway’s Law
- Takes care of a part of the
business
- Interacts with other domains

36. Do not couple
domains,
compose them
- Very hard to do when you
don’t fully understand the
domain

37. Products and
platforms
- At the beginning product and
platform are the same
- Once you understand the
domain enough, turn it into a
platform
- Products are configurations
on top of platforms
- Platforms have simple and
well documented APIs

38. Software is
a fractal,
each level
is complex
If you don’t couple complexities at
different levels
If you manage to contain the
complexity
You can use it at a higher level and
understand it in a lower level.

39. Thanks!
Lucas Cavalcanti
@lucascs
https://nu.careers