1. Functional
Programming in Clojure
Juan Manuel Gimeno Illa
jmgimeno@diei.udl.cat
Master in Open Source Engineering
2. Clojure is a Functional
Language
• In Functional Programming, functions are the
fundamental building-blocks for programs
• In imperative programming, instructions are
these blocks
• A Functional Language is one that promotes
and eases Functional Programming
• Sometimes FP is called a style
3. Imperative program
• The task is accomplished by
• executing sequences of instructions
• which modify program state
• until a desired result is achieved
• Variables represent memory addresses
4. Imperative program
Program Flow
Program State
Write Variable
Read & Modify Variable
Variable
Read Variable &
Control Flow Variable
Modify Variable
Variable
Control Flow
Write Variable
5. Functional program
• The task is accomplished by function
composition: passing the result of one function as
parameter to the next
• the entire program can be viewed as a single
function defined in terms of smaller ones
• program execution is evaluation of expressions
• the nesting structure determines program flow
• Variables represent values (as in maths)
6. Functional program
s
ment
Argu Function
e nts value
m urn
A rgu Function Ret
Arguments
e
Input v alu
et urn Function
R Return value
Function Arg
um ents
Output
Retu
rn v Function
alue
Recursive call
7. Pure functions
• A pure function is one that
• depends upon nothing but its parameters
• and does nothing but return a value
Arguments
Performs calculations
and may call itself or
other pure functions
Return Value
8. Impure functions
• An impure function either
• reads from anywhere except its
parameters
• or changes anything in the program state
Arguments Performs calculations Reads external state
and may call any
function (pure or
impure)
Return Value Writes external state
(side effect)
9. FP and purity
• FP is largely concerned with the careful
management of
• state
• side effects
• Both are necessary but are regarded as
necessary evil
• so use them as little as possible
11. Function definition
• Functions are the main building blocks in
functional programming
• So, there are various means to define
functions
• defn macro (for top level functions)
• anonymous functions
• high-order functions
12. defn macro
symbol to reference the function
(defn name optional documentation string
doc-string?
optional metadata map (we will
attr-map?
ignore this aspect of clojure)
[params*]
body) parameter vector
function body
13. defn macro
(defn greeting
"Returns a greeting of the form 'Hello, username.'"
[username]
(str "Hello, " username))
=> #‘user/greeting
(greeting "world")
=> "Hello, world"
(doc greeting)
-------------------------
user/greeting
([username])
Returns a greeting of the form 'Hello, username.'
=> nil
14. Multiple parameter lists
and bodies
(defn greeting
"Returns a greeting of the form 'Hello, username.'
Default username is 'world'."
([] (greeting "world"))
([username] (str "Hello, " username)))
=> #‘user/greeting
(greeting)
=> "Hello, world"
15. Variable arity
sequence of arguments after the
second (nil if none)
(defn date [person-1 person-2 & chaperones]
(println person-1 "and" person-2
"went out with" (count chaperones)
"chaperones."))
=> #’user/date
(date "Romeo" "Juliet" "Friar Lawrence" "Nurse")
Romeo and Juliet went out with 2 chaperones.
=> nil
16. Anonymous Functions
• There are three reasons to create an anomynous
function:
• The function is so brief and self-explanatory that
giving it a name makes the code harder to read.
• The function is used only from inside another
function and needs a local name, not a top-level
binding.
• The function is created (dynamically at
runtime)inside another function for the purpose of
closing over some data (i.e. a closure).
17. Brief AFs
• (defn indexable-word? [word]
(> (count word) 2))
(filter indexable-word?
(split #”W+” “A fine day it is”))
=> (“fine” “day”)
• (filter (fn [word] (> (count word) 2))
(split #”W+” “A fine day it is”))
• (filter #(> (count %) 2)
(split #”W+” “A fine day it is”))
18. Local functions
• (defn indexable-words [text]
(let [indexable-word? (fn [w]
(> (count w) 2))]
(filter indexable-word?
(split #”W+” text))))
• This combination says two things:
• indexable-word? is interesting enough to have a
name (maybe as documentation)
• but it is relevant only inside indexable-words
19. Closures
• (defn make-greeter
[greeting-prefix]
(fn [username]
(str greeting-prefix ", " username)))
(def hello-greeting (make-greeter “Hello”))
(hello-greeting “world”)
=> “Hello, world”
• hello-greeting remembers the value of greeting-prefix
• Formally, the greeting functions are closures over the
value of greeting-prefix
20. Currying
• Currying refers to the process of transforming
a function into a function with less arguments
wrappimg it into a closure.
• Manipulating functions like this allows for the
creation of customized functions w/o having to
write explicit definitions for them.
• Currying is so important in functional
programmming that, for instance, in Haskell all
functions are curried.
21. Currying with partial
• In Clojure any function can be curried using
the partial function.
• For instance:
(def times-pi (partial * 3.14159))
• is equivalent to:
(defn times-pi [x]
(* 3.14159 x))
22. Composing
• In one sense every function is a compositions,
since all functions must use other functions in
their definitions.
• However, the comp function creates news
functions by combining existing ones w/o
specifying an actual function body.
• the new function will call all the functions,
from right to left, passing the result as the
argument for the next one
23. Composing with comp
• For instance:
(def concat-and-reverse
(comp (partial apply str) reverse str))
• Is equivalent to:
(defn concat-and-reverse [& strs]
(apply str (reverse (apply str strs))))
• (concat-and-reverse "hello" "clojuredocs")
=> "scoderujolcolleh"
25. FP is naturally recursive
• Consider this definition of the factorial
(a.k.a. “the hello world of fp”)
int factorial(int n) { • Its execution model
int fact = 1; can only be defined as
int i = 1; changing values of
while ( i <= n) { variables
fact *= i;
++i; • Can’t be seen as the
evaluation of an
}
return fact; expression
}
26. FP is naturally recursive
• Compare with this version:
(defn fact [n]
(if (zero? n)
1
(* n (fact (dec n)))))
• In this cas execution can be viewed only as
the evaluation of an expression
28. Blowing the stack
• Recursive functions have the problem that
use stack space
• So, if the number of calls to return to is
big, the stack overflows.
• For instance, calling (fact 10000) produces
java.lang.StackOverflowError
29. Tail call
• A tail call is a function call producing a
return value which is then returned by the
calling function.
• the call site is then said to be in tail
position.
• Tail calls are important because they can be
implemented without consuming space in
the call stack (tail call optimization or TCO)
30. Tail recursion
• A function is tail recursive if all its recursive
calls are made in tail position.
• e.g. the fact function presented before is
not tail-recursive because the result given
by (fact (dec n)) is further multiplied by n
• Often it is easy to construct a tail recursive
version by means of an accumulator.
31. Tail recursive factorial
(defn fact2
([n] (fact2 n 1))
([n f] (if (zero? n)
f
(fact2 (dec n) (* f n)))))
• But (fact2 10000) blows the stack too !!!
• The problem is that the JVM does not make
TCO and Clojure uses the JVM call mechanism.
32. The recur special form
• To have TCO in Clojure, it is necessary to
indicate it explicitly by means of the recur
special form.
(defn fact2
([n] (fact2 n 1))
([n f] (if (zero? n)
f
(recur (dec n) (* f n)))))
33. The loop special form
• Simplifies the definition of TR functions by
defining an anonymous fn, making the initial
call and allowing recur to it.
(defn fact3 [n]
(loop [n n
f 1]
(if (zero? n)
f
(recur (dec n) (* f n)))))
34. Trampolining
• recur does not solve mutual recursive tail-
recursion.
(declare my-odd? my-even?)
(defn my-odd? [n]
(if (= n 0) false #(my-even? (dec n))))
(defn my-even? [n]
(if (= n 0) true #(my-odd? (dec n))))
(trampoline my-even? 1000000)
35. No recursion using
reduce
• Some recursions are avoidable using the high order function
reduce
• (reduce f v c)applies f to v and the first element of c
to get v’, then f to v’ and the second element of c, etc.
• if we don’t pass v, the first element of c is used
• unless c is empty, and we return (f)
• When the list is exhausted the last computed value is
returned.
• Using reduce we substitute explicit recursion (and
accumulation) by function application.
37. Continuation passing
style
• Instead of returning values, a function
written in CPS takes an explicit
continuation" argument
• This function represents what must be
done after the result of the function is
computed
• This style of programming allows for some
high level control abstractions
• In CPS every call is a tail call
38. CPS example
(defn dec-cp [x k] (k (dec x)))
(defn *-cps [x y k] (k (* x y)))
(defn factorial-cps
([n]
(factorial-cps n identity))
([n k]
(if (zero? n)
(k 1)
(dec-cps n (fn [nn]
(factorial-cps nn
(fn [ff]
(*-cps n ff k))))))))
39. A tail recursive version
• The continuation is a function that reifies
the process of the execution stack
(defn factorial-tail-rec
([n]
(factorial-tail-rec n identity))
([n k]
(if (zero? n)
(k 1)
(recur (dec n) (fn [v] (k (* n v)))))))
41. Immutability
• One cornerstone principle in Clojure is
immutability
• so data structures in clojure are immutable
• (except references and transients)
• Immutability at the language level:
• allows pure functions on data structures
• simplifies concurrent programming
42. Comparing to Java
• Creating immutable classes in Java is possible:
• both the class and all its fields should be labeled as final
• this reference should not escape during construction
• any internal mutable objects should
• originate within the class
• never scape
• Some constraints are not enforced by the language
• Immutability by convention !!!!
43. Why immutability?
• Invariants
• are defined only in the construction mechanism
• never violated afterwards
• Reasoning
• about its possible states and transitions is simplified
• Equality has meaning
• equality in the presence of mutability has no meaning
• equality in the face of mutability and concurrency is utter
lunacy
44. Why immutability?
• Sharing is cheap
• if an object will never change, sharing it only needs
providing a reference
• the object can be interned
• Fosters concurrent programming
• immutable objects are always thread safe
• no need to synchronize nor lock
• (clojure isolates mutation to its reference types which
have a well defined concurrency semantics)
45. Naïve implementation
• A naïve implementation of immutable data consist
on copying lots and lots of data
• F.i., adding an element to a list w/o mutating the list
consist in copying it, and adding the element to the
copy
• the original list is preserved
• the new list contains both the new element and
those from the old list
• This is unacceptable both in time and space
46. Structural sharing
• As lists are immutable, one can share the
common elements in both lists.
• (as one can see baselist as a historical
version of both lst1 and lst2, this kind
of structures are called persistent)
(def baselist (list :barnabas :adam))
(def lst1 (cons :willie baselist))
(def lst2 (cons :phoenix baselist))
(identical? (rest lst1) (rest lst2))
=> true
48. A persistent tree
• Clojure’s vectors and maps are also
persistent and implemented by a hash-tree
(an explanation of PersistentVector
implementation can be found in this post)
• Let’s build a persistent binary search tree
implementation to “understand” how
structural sharing can work in a tree
49. A persistent tree
• Each node in the tree will have three fields: a
value, the left branch and the right branch.
• The empy tree will be simply nil.
• We will define a function xconj that adds a
value in the tree
(defn xconj [t v]
(cond
(nil? t) {:val v :L nil :R nil})
....
50. A persistent tree
• Now we have to handle the case of adding to a non-
empty tree
• As is an ordered tree we must test if the insertion goes
on the right or on the left
• Suppose (< v (:val t))
• if this were a mutable tree we must have substituted :L
with the tree resulting from the insertion
• instead, we should build a new node, copying parts of
the old node that don’t need to change
58. Abstracting structures
• At a low level programs work with structures such as
strings, lists, vectors and trees
• At a higher level, these same data structure
abstractions come again and again
• XML data is a tree
• DB query results can be viewed as lists or vectors
• Directory hierarchies are trees
• Files are often viewed as one big string or as a
vector of lines
59. Sequence abstraction
• In clojure all these data structures can be accesses
through a single abstraction: the sequence (or seq).
• A seq is a logical list (not tied to implementation)
• Collections that can be viewed as seqs are called
seq-able
• The sequence library is a set of functions that can
work with any seq-able
• They are the functional and immutable equivalents
to iterators
60. Seq-able collections
• All Clojure collections
• All Java collections
• Java arrays and strings
• Regular expression matches
• Directory structures
• I/O streams
• XML trees
• .....
61. Sequence
• A sequence has three core capabilities:
• you can get the first item in a sequence:
(first aseq)
first returns nil if its argument is empty or nil
• you can get everything after the first element:
(rest aseq)
rest returns an empty seq (not nil) if there are no more
items
• you can construct a new sequence by adding an element
to the front:
(cons elem aseq)
62. Sequences
• These three capabilities are declared in the Java
interface clojure.lang.ISeq
• The seq function will return a seq on any seq-able
collection:
(seq coll)
seq will return nil if coll is empty of nil
• The next function will return the seq of items after
the first:
(next aseq)
and is equivalent to (seq (rest aseq))
64. Beware of the REPL
• When you apply rest or cons to a vector, the
result is a seq not a vector
• In the REPL seqs print just like lists
• (class (rest [1 2 3]))
=> clojure.lang.PersistentVector$ChunkedSeq
76. List comprehensions
• A list comprehension creates a list setting the properties
the result list must satisfy
• A list comprehension will consist of the following
• input list(s)
• placeholder names for elements in the input lists
• predicates on the elements
• an output form that generates the elements from the
elements in the input that satisfy the predicates
• (for [binding-form coll-expr filter-expr? ...] expr)
79. Seqs are conceptual
• All sequences share the same conceptual
model: a singly linked list.
first rest
first rest
first rest
Item Item Item (empty)
80. Introducing lazyness
• The rest of a sequence doesn’t need to
exist, provided it can be created when
necessary.
first rest
Instructions to generate the next component
sequence
Item
81. Benefits of lazy seqs
• You can postpone expensive computations that
may not in fact be needed.
• You can work with huge data sets that do not
fit into memory
• even infinite ones !!
• You can delay I/O until it is absolutely needed
• (there exist functional languages such as
Haskell in which all evaluation is lazy)
82. Caution with infinite
sequences
• Some care is required with infinite sequences not
to attempt to realize an infinite number of values.
• Trying to print an infinite lazy sequence in the
REPL is an usual mistake
• use take (or similar)
• set! a non-nil value to *print-length*, for
instance:
(set! *print-length* 10)
83. “Seeing” the lazyness
• To “see” lazyness in action we must create a
non-pure function (why?)
(def squares (map (fn [n] (print *) (* n n))
(iterate inc 1)))
(first squares)
=> *1
Only the necessary
(second squares) part is realized
=> *4
(first squares) Realized elements
=> 1 are cached
(take 4 squares)
=> (1 *4 *9 16)
84. Constructing LS
directly
• To built a LS manually, use the built-in
lazy-seq macro.
(defn lazy-counter [base increment]
(lazy-seq
(cons base
(lazy-counter (+ base increment) increment))))
(take 10 (lazy-counter 0 2)
=> (0 2 4 6 8 10 12 14 16 18)
(nth (lazy-counter 2 3) 1000000)
=> 3000002
85. Constructing LS using
generator functions
• It’s often easier to use a sequence
generator function than lazy-seq
• For instance, iterate generates an infinite
sequence of items by callimng a fn and
passing the previous value as argument.
(defn lazy-counter-iterate [base increment]
(iterate (fn [n] (+ n increment)) base))
86. Constructing LS using
generator functions
• It’s often easier to use a sequence
generator function than lazy-seq
• For instance, iterate generates an infinite
sequence of items by callimng a fn and
passing the previous value as argument.
(defn lazy-counter-iterate [base increment]
(iterate (partial + increment) base))
87. Circular programs
• A circular program creates a data structure
whose computation depends upon itself or
refers to itself.
• Circular programs provide a very
appropriate formalism to model multiple
traversal algorithms as elegant and concise
single traversal solutions.
89. Forcing Sequences
• We have seen using take to prevent the REPL to
evaluate the entire sequence.
• Sometimes we have the opposite problem: we want
to force the complete evaluation of a sequence
• for instance because the code generating the
sequence has side-effects
• For instance:
(def x (for [i (range 1 3)] (do (println i) i)))
=> #‘user/x
90. Forcing Sequences
• doall forces Clojure to walk the elements of a sequence
and returns the elements as a result:
(doall x)
|1
|2
=> (1 2)
• dorun walks the elements w/o keeping past elements in
memory (and returns nil)
• As Clojure discourages side-effects you should use
them rarelly (clojure.core calls each one once in 4kloc).
91. LS and memory
management
• Using LS it is possible to use large, even infinite, sequences in a
memory efficient way.
• But it is also possible to inadvertently consume large amounts of
memory (even resulting in OutOfMemoryError).
• Use the following guidelines to reason about how LS consume
memory:
• LS not realized consume no appreciable memory
• Once realized it consumes memory for all values that contains
• provided there is a reference to the realized sequence
• until the reference is discarded and the sequence garbage
collected
92. LS and memory
management
• ~60 Mb of heap
(def integers (iterate inc 0))
(nth integers 1000000)
• Almost nothing
(nth (iterate inc 0) 1000000)
• nth itself does not maintain any references
(it retrieves rest from each entry, and drops
any references to the sequence itself)
93. ;; Lazy quick-sort (The Joy of Clojure, Listing 6.3)
(defn- cons-when [v coll]
(if (empty? v) coll (cons v coll)))
(defn- sort-parts [work]
"Lazy, tail-recursive, incremental quicksort. Works against
and creates partitions based on the pivot, defined as work"
(lazy-seq
(loop [[part & parts :as work] work]
(when work
(if (coll? part)
(let [[pivot & xs] part
smaller? #(< % pivot)]
(recur (cons-when
(filter smaller? xs)
(cons pivot
(cons-when
(remove smaller? xs)
parts)))))
(cons part (sort-parts parts)))))))
(defn qsort [xs]
(sort-parts (list xs)))
95. Destructuring
• Destructuring allows us to bind locals based on
an expected form for a composite data structure.
• You can use destructuring bindings in:
• function parameters
• let forms
• binding forms
• (and other macros based on them)
96. Destructuring with a
vector
• This is the simplest form of destructuring and allows to
pick apart a sequential thing
• you give each item a name
• you can also use an ampersand (&) to indicate the
remaining values, which are bound to a (possibly lazy)
seq
• :as can be used to bind a local to the entire collection
• if you are not interested on an element its idiomatic
to use the name _
97. Destructuring with a
vector
(let [range-vec (vec (range 10))
[a _ c & more :as all] range-vec]
(println "a c are:" a c)
(println "more is:" more)
(println "all is:" all))
a c are: 0 2
more is: (3 4 5 6 7 8 9)
all is: [0 1 2 3 4 5 6 7 8 9]
=> nil
98. Destructuring with a
map
• As we have seen, usually we represent
records by maps using keys as keywords
• We can use destructuring to access the
different fields in those maps
(defn name-to-string
[{f-name :f-name
m-name :m-name
l-name :l-name}]
(str l-name “, “ f-name “ “ m-name))
99. Destructuring with a
map
• Using similar names for the locals and the
keywords is so usual that there exists
special syntax for it.
• There also exist :strs and :syms for
strings and symbols.
(defn name-to-string
[{:keys [f-name m-name l-name]}]
(str l-name “, “ f-name “ “ m-name))
100. Destructuring with a
map
• If the destructuring map looks up a key not
in the source map
• normally bound to nil
• different defaults with :or
(defn name-to-string
[{:keys [title f-name m-name l-name]
:or {title “Mr.”}}]
(str title “ “ l-name “, “ f-name “ “ m-name))
101. Named arguments and
default values
• Since 1.2 functions can have named
optional arguments.
(defn slope
[& {:keys [p1 p2] :or {p1 [0 0] p2 [1 1]}}]
(float (/ (- (p2 1) (p1 1))
(- (p2 0) (p1 0)))))
(slope :p1 [4 15] :p2 [3 21])
=> -6.0
(slope :p2 [2 1])
=> 0.5
103. Six Rules of Clojure FP
1. Avoid direct recursion
4. Be careful not to realize
(the JVM cannot optimize
more of a lazy sequence
recursive calls)
than needed.
2. User recur when you are
5. Know the sequence
producing scalar values
library.
or small, fixed sequences
(Clojure will optimize 6. Subdivide even simple
those calls). problems into smaller
pieces and you will often
3. When producing large or
find solutions in the
variable-sized sequences,
sequence library.
always be lazy.
104. Bibliography
• L.VanderHart and S. Sierra, Practical Clojure,
Apress, 2010.
• S.Halloway, Programming Clojure, Pragmatic
Bookshelf, 2009.
• M. Fogus and C. Houser, The Joy of Clojure,
Manning, (to be publised).
• R. Hickey, Persistent Data Structures and
Managed References (video and slides).