Malli is a new Clojure/Script library for elegant data-driven data modelling. It is born from the real world needs of dynamic, distributed and multi-tenant systems: Schemas to describe the data should be first-class data. They should drive the runtime value transformations, web forms, rules and processes. Configuration should be explicit – no magic, macros or mutability. We should be able to infer, create and modify the models at runtime, persist and load them back from database and share over the wire, for both Clojure and ClojureScript. Like JSON Schema, but for EDN & Clojure/Script. In this talk, I’ll walk through the core concepts, features and the lessons learned on creating Malli. I’ll also demonstrate some cool new things we can do with literal data schemas and compare it to prior art, including clojure.spec, plumatic schema and JSON Schema.

2. Inside
Data-driven
Schemas
Tommi Reiman
29.2.2019
ClojureD
malli

5. Chapter 1:
Expectations

6. • Data-oriented - represent entities and their relationships as data.
• Data-driven - create DSLs in data structures and write interpreters for
them
Clojure <3 Data
https:!"clojureverse.org/t/clojure-a-pragmatic-data-driven-language/4565/5Eric Normand

8. It’s just data!
;; restaurant
{:name "Lie Mi"
:tags #{:street :vietnamese}
:stars 4.3
:address {:street "Hämeenkatu 14"
:city “Tampere"}}

9. It’s just data!
{:type :restaurant
:name "Lie Mi"
:tags #{:street :vietnamese}
:stars 4.3
:address {:street "Hämeenkatu 14"
:city “Tampere"}}

10. It’s just data!
{:type :customers.1234/restaurant
:name "Lie Mi"
:tags #{:street :vietnamese}
:stars 4.3
:address {:street "Hämeenkatu 14"
:city “Tampere"}}

11. It’s just data!
{:mydomain/type :mydomain.types/restaurant
:mydomain.restaurant/name "Lie Mi"
:mydomain.restaurant/tags #$:street :vietnamese}
:mydomain.restaurant/stars 4.3
:mydomain.restaurant/address {:mydomain.address/street "Hämeenkatu 14"
:mydomain.address/city "Tampere"}}

12. It’s just data!
{:type :restaurant
:name "Lie Mi"
:tags #{:street :vietnamese}
:stars 4.3
:address {:street "Hämeenkatu 14”
:city "Tampere"}}

13. It’s just data!
(-> {:type :restaurant
:name "Lie Mi"
:tags #{:street :vietnamese}
:stars 4.3
:address {:street "Hämeenkatu 14"
:city "Tampere"}})
(select-keys [:name :tags :address]))
(assoc-in [:address :zip] 33100))
;{:name "Lie Mi"
; :tags #{:vietnamese :street}
; :address {:street "Hämeenkatu 14"
; :city "Tampere"
; :zip 33100}}

14. https:!"devrant.com/rants/2192467/on-category-theory

15. Types, Schemas and Specs
Malcolm 16:45
Awesome!
Awesome!
Awesome!

16. One tool for everything?

17. Expectations
We are building dynamic multi-tenant systems where data-
models should be data too: they should drive the runtime value
transformations, forms and processes.We should be able to edit
the models at runtime, persist them and load them back from
database and over the wire, for both Clojure and ClojureScript.
Think of JSON Schema, but for EDN and Clojure/Script.

18. If you have expectations (of others) that aren't
being met, those expectations are your own
responsibility.You are responsible for your own
needs. If you want things, make them.
- Rich Hickey
https:!"gist.github.com/richhickey/1563cddea1002958f96e7ba9519972d9

19. Chapter 2:
DIY

20. The Idea
• Schemas as data (hiccup, reagent, internal.*, reitit)
• Take the best parts of Plumatic Schema, clojure.spec & JSON Schema
• Programming with Schemas, first class meta-data
• Focus on Developer Experience
• Small core, extendable
• Create a Prototype, experiment, have fun

21. Malli
/ˈmɑlːi/, [ˈmɑlːi]

22. Syntax (Schema AST)
[:vector {:min 0, :max 10} int?]
type %& [type ?properties & children]
{:type :vector
:properties {:min 0, :max 10}
:children [int?]}

23. Schemas
int?
[:= 43]
[:tuple double? double?]
[:and int? [:> 18]]
[:map
[:x int?]
[:y int?]]
AST '( (resolve) '( IntoSchema opts '( Schema

24. Validation
(require '[malli.core :as m])
(m/validate int? 1)
; => true
(m/validate int? "kikka")
; => false
(m/validate [:maybe keyword?] nil)
; => true
(m/validate [:vector int?] [1 2 3])
; => true

25. Complex Schema
(def CompositeMap
[:and
[:map {:closed true}
[:x pos-int?]
[:y pos-int?]]
[:fn {:error/message "x )* y"}
(fn [{:keys [x y]}] (> x y))]])
(m/validate CompositeMap {:x 3, :y 2})
; '( true
(m/validate CompositeMap {:x 1, :y 2})
; '( false

26. Complex Schema
(def CompositeMap
[:and
[:map {:closed true}
[:x pos-int?]
[:y pos-int?]]
[:fn {:error/message "x )* y"}
'(fn [{:keys [x y]}] (> x y))]])
(m/validate CompositeMap {:x 3, :y 2})
; '( true
(m/validate CompositeMap {:x 1, :y 2})
; '( falseMichiel Borkent

27. Forms
(def CompositeMap
[:and
[:map {:closed true}
[:x pos-int?]
[:y pos-int?]]
[:fn {:error/message "x )* y"}
'(fn [{:keys [x y]}] (> x y))]])
(m/form CompositeMap)
;[:and
; [:map {:closed true}
; [:x pos-int?]
; [:y pos-int?]]
; [:fn {:error/message "x )* y"}
; (fn [{:keys [x y]}] (> x y))]]

28. • Works on all of JVM, ClojureScript & GraalVM
Durable Schemas
Michiel Borkent
(require '[malli.edn :as edn])
(+, CompositeMap
(edn/write-string)
(edn/read-string)
(m/validate {:x 3, :y 2}))
; '( true

29. • Inspired by clojure.spec
Explaining Data
(m/explain CompositeMap {:x -1, :y -2, :EXTRA true})
;{:schema …,
; :value {:x -1, :y -2, :EXTRA true},
; :errors [{:path [1 2 1], :in [:x], :schema pos-int?, :value -1}
; {:path [1 3 1], :in [:y], :schema pos-int?, :value -2}
; {:path [1], :in [:EXTRA], :schema …, :type :malli.core/extra-key}]}
AlexRich

30. Errors for Humans
(require '[malli.error :as me])
(+, CompositeMap
(m/explain {:x -3, :y -2, :EXTRA true})
(me/humanize))
;{:x ["should be positive int"]
; :y ["should be positive int"]
; :EXTRA ["disallowed key"]
; :malli/error ["x )* y"]}
Alexander Kiel

31. GeneratingValues
(require '[malli.generator :as mg])
(mg/generate keyword?)
; '( :?
(mg/generate [:enum "a" "b" "c"] {:seed 42})
; '( "a"
(mg/generate
[:re #"^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9-.]+.[a-zA-Z]{2,63}$"]
{:seed 42, :size 10})
; '( "CaR@MavCk70OHiX.yZ"

32. GeneratingValues
(mg/generate
[:and {:gen/elements ["kikka" "kukka" "kakka"]} string?])
; '( "kikka"
(mg/generate
[:and {:gen/fmap '(partial str "kikka_")} string?]
{:seed 10, :size 10})
/0 '( "kikka_WT3K0yax2"
(require '[clojure.test.check.generators :as gen])
(mg/generate
[:sequential {:gen/gen (gen/list gen/neg-int)} int?]
{:size 42, :seed 42})
; '( (-37 -13 -13 -24 -20 -11 -34 -40 -22 0 -10)
Gary Frederics

33. Inferring Schemas
(def lie-mi
{:type :restaurant
:name "Lie Mi"
:tags #$:street :vietnamese}
:stars 4.3
:address {:street "Hämeenkatu 14"
:city "Tampere"}})
(def wanha-tappi
{:type :restaurant
:name "Wanha Tappi"
:tags #$:burgers :beer}
:address {:street "Jokipohjantie 18"
:city "Tampere"}})
Stathis Sideris
(require '[malli.provider :as mp])
(def Restaurant
(mp/provide [lie-mi wanha-tappi]))
Restaurant
;[:map
; [:type keyword?]
; [:name string?]
; [:tags [:set keyword?]]
; [:stars {:optional true} double?]
; [:address
; [:map
; [:street string?]
; [:city string?]]]]

34. Programming with Schemas
(require '[malli.util :as mu])
(+, [:map
[:a int?]
[:b string?]
[:c [:map
[:d int?]
[:e keyword?]]]]
(mu/closed-schema)
(mu/select-keys [:a :c])
(mu/update-in [:c] mu/dissoc :e))
;[:map {:closed true}
; [:a int?]
; [:c [:map {:closed true}
; [:d int?]]]]
Metosin

35. Walking over Schemas
• Generic schema walking using theVisitor-pattern (CLJ-2251)
• To JSON Schema, Swagger2, Map-syntax etc.
(require '[malli.json-schema :as json-schema])
(json-schema/transform
[:map {:title "a map"}
[:x int?]
[:y {:optional true} inst?]])
;{:type "object"
; :title "a map"
; :properties {:x {:type "integer", :format "int64"}
; :y {:type "string", :format "date-time"}
; :required [:x]}

36. Walking over Schemas
(defn closed-schema
"Closes recursively all :map schemas by adding `{:closed true}`
property, unless schema explicitely open with `{:closed false}`"
([schema]
(closed-schema schema nil))
([schema options]
(m/accept
schema
(m/schema-visitor
(fn [schema]
(if (-open-map? schema options)
(update-properties schema c/assoc :closed true)
schema))))))

38. • Open Schemas to embrace growth (and typing errors)
• Closed Schemas and Spell Checking to the rescue?
Spell Checking
(def nante-eck
{:type :restaurant
:name "Nante-Eck"
:tags #$:traditional :vegan}
:starz 4.2
:address {:streez "Unter den Linden 35"
:city "Berlin"}})
Restaurant
;[:map
; [:type keyword?]
; [:name string?]
; [:tags [:set keyword?]]
; [:stars {:optional true} double?]
; [:address
; [:map
; [:street string?]
; [:city string?]]]]

39. Spell Checking
(+, Restaurant
(m/explain nante-eck)
(me/humanize))
; {:address {:street ["missing required key"]}}
(+, Restaurant
(mu/closed-schema)
(m/explain nante-eck)
(me/humanize))
;{:address {:street ["missing required key"]
; :streez ["disallowed key"]}
; :starz ["disallowed key"]}
(+, Restaurant
(mu/closed-schema)
(m/explain nante-eck)
(me/with-spell-checking)
(me/humanize))
;{:address {:streez ["should be spelled :street"]}
; :starz ["should be spelled :stars"]}
Bruce Hauman

40. • Fully re-written engine, built with interceptors
• Encode and decode chains with enter & leave phases
• Json, string, strip-extra-keys & default-value-transformer
• Schemas can define transformation via properties
• Fast
ValueTransformers
(m/decode int? "42" (mt/string-transformer))
; '( 42
(m/encode int? 42 (mt/string-transformer))
; '( ”42”

41. (require '[malli.transform :as mt])
(def json-transformer
(mt/transformer
(mt/strip-extra-keys-transformer)
(mt/json-transformer)
(mt/default-value-transformer)))
(def json+,Restaurant
(m/decoder Restaurant json-transformer))
(json+,Restaurant
{:type "restaurant"
:DARK "ORKO"
:name "Mustafas Gemüse Kebab"
:tags ["kebab" "yoghurt"]
:address {:street "Mehringdamm 32"
:EXTRA "KEY"
:city "Berlin"}})
;{:type :restaurant,
; :name "Mustafas Gemüse Kebab",
; :tags #$:kebab :yoghurt},
; :address {:street "Mehringdamm 32"
; :city "Berlin"}}

42. (require '[criterium.core :as cc])
/0 1.8µs
(cc/quick-bench
(+, json
(update :type keyword)
(update :tags (comp set (partial map keyword)))
(select-keys [:type :name :tags :address])
(update :address select-keys [:street :city])))
/0 1.9µs
(cc/quick-bench (json+,Restaurant json))
(def json+,Order
(m/decoder
[:map
[:id int?]
[:tags string?]
[:address
[:map
[:street string?]
[:zip string?]
[:country string?]]]]
(mt/json-transformer)))
json+,Order
; '( #object[clojure.core$identity]

43. (require '[criterium.core :as cc])
/0 1.8µs
(cc/quick-bench
(+, json
(update :type keyword)
(update :tags (comp set (partial map keyword)))
(select-keys [:type :name :tags :address])
(update :address select-keys [:street :city])))
/0 1.9µs
(cc/quick-bench (json+,Restaurant json))
(def json+,Order
(m/decoder
[:map
[:id int?]
[:tags [:vector string?]]
[:address
[:map
[:street string?]
[:zip string?]
[:country string?]]]]
(mt/json-transformer)))
json+,Order
; '( #object[clojure.core$identity]

44. (require '[criterium.core :as cc])
/0 1.8µs
(cc/quick-bench
(+, json
(update :type keyword)
(update :tags (comp set (partial map keyword)))
(select-keys [:type :name :tags :address])
(update :address select-keys [:street :city])))
/0 1.9µs
(cc/quick-bench (json+,Restaurant json))
(def json+,Order
(m/decoder
[:map
[:id int?]
[:tags [:vector string?]]
[:address
[:map
[:street string?]
[:zip string?]
[:country string?]]]]
(mt/json-transformer)))
json+,Order
; '( #object[clojure.core$identity]

45. Chapter 3:
The Future

46. • reitit-malli coercion module, since 0.4.0
• Orders of magnitude faster coercion than with reitit-spec
• Small example application: https://github.com/metosin/reitit/tree/
master/examples/ring-malli-swagger
• A new module & configuration system?
reitit + malli

47. • … and custom Schemas to inline external stuff, maybe like:
Schema providers (F#)
(m/validate
[:json-schema
{:type "object"
:properties {:x {:type "integer", :format "int64"}
:y {:type "integer", :format "int64"}}
:required [:x]}]
{:x 1, :y 2})
; => true

48. • Have generated forms in four decades, just one more time ;)
• Good companion for FSMs and rule engines, rapid prototyping
• https://www.youtube.com/watch?v=IekPZpfbdaI
• https://github.com/domino-clj + malli
Generating UIs
Dmitri Carmen Nikola
Schema '( UI %& Schema UI-Schema '( UI

49. • Inspired by expound, backed by metosin/virhe
Errors for Developers
Ben Brinkerhoff

50. • malli-powered code checking: https://github.com/teknql/aave
• Add support schematized defn and fn
• Emit clj-kondo type annotations
• Schemas for Schemas AST
• Test utilities
MoreTooling
Ryan Schukler
Michiel Borkent

52. • Pre-alpha, still experimenting, but core is 90% code complete
• Was public announcement by the community long time ago
• Many active contributors and some users too
• Big thanks to all people and libraries for source of innovation
• Goal to get official release out in a month or two
• Learned a lot, has been fun to develop
Current Status

53. • Understand and manage your expectations
• Clojure is awesome tool for building data-driven systems
• Malli is a fresh new schema library for Clojure/Script, try it out
• If you’re interested in type and schema systems, consider contributing
• https://github.com/metosin/malli
• #malli on slack
Takeaways

54. Questions?
Tommi Reiman
tommi@metosin.fi
@ikitommi
https://www.github.com/metosin/malli
malli.io

55. Multi-Schemas
(def MultiSchema
[:multi {:dispatch :type}
[:sized [:map
[:type [12 :sized]]
[:size int?]]]
[:human [:map
[:type [12 :human]]
[:name string?]]]])
(m/validate MultiSchema {:type :sized, :size 10})
; '( true
(m/validate MultiSchema {:type :human, :name “Tiina"})
; '( true

56. Multi-Schemas
(def MultiSchema
[:multi {:dispatch 'first}
[:sized [:tuple [12 :sized] [:map [:size int?]]]]
[:human [:tuple [12 :human] [:map [:name string?]]]]])
(m/validate MultiSchema [:sized {:size 10}])
; '( true
(m/validate MultiSchema [:human {:name "Tiina"}])
; '( true