1. Clojure STM
@marblejenka
2011 5 24

2. • Clojure STM
• STM STM
•
Clojure/Java java.util.concurrent.*
2
2011 5 24

3. Agenda
• Clojure
• STM
• Clojure STM
• Clojure STM API
• Clojure STM
• Clojure STM
3
2011 5 24

4. Clojure
• Clojure
–
–
•
–
4
2011 5 24

5. Clojure
•
• Clojure
• ref STM:Software Transactional Memory
– ref
atom/agent/var
5
2011 5 24

6. Agenda
• Clojure
• STM
• Clojure STM
• Clojure STM API
• Clojure STM
• Clojure STM
6
2011 5 24

7. STM
• Software Transactional Memory
• ACID ACI
– Atomic
•
– Consistency
•
– Isolation
•
– Durabilty
• STM
D
• DB
• MVCC
7
2011 5 24

8. STM
reference
F
v=42, t=0
The Joy of Clojure Figure 10.1 8
2011 5 24

9. Tx A STM
reference
F
v=42, t=0
The Joy of Clojure Figure 10.1 8
2011 5 24

10. Tx A STM
reference
A
F
v=42, t=0
The Joy of Clojure Figure 10.1 8
2011 5 24

11. Tx A STM
reference
A
F
v=42, t=0
read
The Joy of Clojure Figure 10.1 8
2011 5 24

12. Tx A STM
reference
A
F
v=42, t=0
read
The Joy of Clojure Figure 10.1 8
2011 5 24

13. Tx A STM
reference
A
F
v=42, t=0
read
F
v=42, t=0
The Joy of Clojure Figure 10.1 8
2011 5 24

14. Tx A STM
reference
A
F
v=42, t=0
read
F
v=42, t=0 calc
The Joy of Clojure Figure 10.1 8
2011 5 24

15. Tx A STM
reference
A
F
v=42, t=0
read
F
v=42, t=0 calc
local write
The Joy of Clojure Figure 10.1 8
2011 5 24

16. Tx A STM
reference
A
F
v=42, t=0
read
F
v=42, t=0 calc
local write
F
v=9, t=0
The Joy of Clojure Figure 10.1 8
2011 5 24

17. Tx A STM
reference
A
F
v=42, t=0
read Tx B
F
v=42, t=0 calc
local write
F
v=9, t=0
The Joy of Clojure Figure 10.1 8
2011 5 24

18. Tx A STM
reference
A
F
v=42, t=0
read Tx B
F
v=42, t=0 calc B
local write
F
v=9, t=0
The Joy of Clojure Figure 10.1 8
2011 5 24

19. Tx A STM
reference
A
F
v=42, t=0
read Tx B
F
v=42, t=0 calc B
local write read
F
v=9, t=0
The Joy of Clojure Figure 10.1 8
2011 5 24

20. Tx A STM
reference
A
F
v=42, t=0
read Tx B
F
v=42, t=0 calc B
local write read
F
v=9, t=0
The Joy of Clojure Figure 10.1 8
2011 5 24

21. Tx A STM
reference
A
F
v=42, t=0
read Tx B
F
v=42, t=0 calc B
local write read
F
v=42, t=0
F
v=9, t=0
The Joy of Clojure Figure 10.1 8
2011 5 24

22. Tx A STM
reference
A
F
v=42, t=0
read Tx B
F
v=42, t=0 calc B
local write read
F
v=42, t=0
F calc
v=9, t=0
The Joy of Clojure Figure 10.1 8
2011 5 24

23. Tx A STM
reference
A
F
v=42, t=0
read Tx B
F
v=42, t=0 calc B
local write read
F
v=42, t=0
F calc
v=9, t=0
local write
The Joy of Clojure Figure 10.1 8
2011 5 24

24. Tx A STM
reference
A
F
v=42, t=0
read Tx B
F
v=42, t=0 calc B
local write read
F
v=42, t=0
F calc
v=9, t=0
local write
F
v=27, t=0
The Joy of Clojure Figure 10.1 8
2011 5 24

25. Tx A STM
reference
A
F
v=42, t=0
read Tx B
F
v=42, t=0 calc B
local write read
F
v=42, t=0
F calc
v=9, t=0 conflict?
local write
F
v=27, t=0
The Joy of Clojure Figure 10.1 8
2011 5 24

26. Tx A STM
reference
A
F
v=42, t=0
read Tx B
F
v=42, t=0 calc B
local write read
F
v=42, t=0
F calc
v=9, t=0 conflict?
local write
F
v=27, t=0
The Joy of Clojure Figure 10.1 8
2011 5 24

27. Tx A STM
reference
A
F
v=42, t=0
read Tx B
F
v=42, t=0 calc B
local write read
F
v=42, t=0
F calc
v=9, t=0 conflict?
NO local write
F
v=27, t=0
The Joy of Clojure Figure 10.1 8
2011 5 24

28. Tx A STM
reference
A
F
v=42, t=0
read Tx B
F
v=42, t=0 calc B
local write read
F
v=42, t=0
F calc
v=9, t=0 conflict?
NO local write
F
commit v=27, t=0
The Joy of Clojure Figure 10.1 8
2011 5 24

29. Tx A STM
reference
A
F
v=42, t=0
read Tx B
F
v=42, t=0 calc B
local write read
F
v=42, t=0
F calc
v=9, t=0 conflict?
NO local write
F
commit v=27, t=0
F
v=9, t=1
The Joy of Clojure Figure 10.1 8
2011 5 24

30. Tx A STM
reference
A
F
v=42, t=0
read Tx B
F
v=42, t=0 calc B
local write read
F
v=42, t=0
F calc
v=9, t=0 conflict?
NO local write
F
commit v=27, t=0
F
v=9, t=1 conflict?
The Joy of Clojure Figure 10.1 8
2011 5 24

31. Tx A STM
reference
A
F
v=42, t=0
read Tx B
F
v=42, t=0 calc B
local write read
F
v=42, t=0
F calc
v=9, t=0 conflict?
NO local write
F
commit v=27, t=0
F
v=9, t=1 conflict?
The Joy of Clojure Figure 10.1 8
2011 5 24

32. Tx A STM
reference
A
F
v=42, t=0
read Tx B
F
v=42, t=0 calc B
local write read
F
v=42, t=0
F calc
v=9, t=0 conflict?
NO local write
F
commit v=27, t=0
F
v=9, t=1 conflict?
YES
The Joy of Clojure Figure 10.1 8
2011 5 24

33. Tx A STM
reference
A
F
v=42, t=0
read Tx B
F
v=42, t=0 calc B
local write read
F
v=42, t=0
F calc
v=9, t=0 conflict?
NO local write
F
commit v=27, t=0
F
v=9, t=1 conflict?
YES
retry!
The Joy of Clojure Figure 10.1 8
2011 5 24

34. STM
• STM
–
•
– Transactional
• atmic
• STM
–
–
I/O
–
9
2011 5 24

35. Agenda
• Clojure
• STM
• Clojure STM
• Clojure STM API
• Clojure STM
• Clojure STM
10
2011 5 24

36. Clojure STM
•
– ref
– Clojure immutable
– ref
• ref all-or-noting
– dosync
– ref
all-of-nothing
– 11
2011 5 24

37. Clojure STM
• a,b
• def
– a,b
– a b
– a b
– a,b
• def dosync
– a,b
– a,b
12
2011 5 24

38. Clojure STM
(def a 0)
(def b 1)
(inc a) ; a +1
(dec b) ; b -1
(println a b) ; -> 1 0
• a,b
• Clojure immutable println
a,b
13
2011 5 24

39. Clojure STM
(def a (ref 0))
(def b (ref 1))
(dosync
(alter a inc)
(alter b dec))
(println @a @b)
• a,b all-or-nothing
• ref println
1,0
14
2011 5 24

40. Clojure STM
(def a (ref 0))
(def b (ref 1))
(dosync
(alter a inc)
(alter b dec))
(println @a @b)
• a,b all-or-nothing
• ref println
1,0
14
2011 5 24

41. Agenda
• Clojure
• STM
• Clojure STM
• Clojure STM API
• Clojure STM
• Clojure STM
15
2011 5 24

42. Clojure STM API
• API(Clojure STM
)
• Java
• URL
– https://github.com/clojure/clojure/blob/master/src/clj/clojure/
core.clj#L1831
16
2011 5 24

43. API
• ref
– ref
• deref
–
• ref-set
– ref
• alter
– ref
• commute
– alter
• ensure
– in-transaction-value
17
2011 5 24

44. ref
(defn ref
{:added "1.0"
:static true
}
([x] (new clojure.lang.Ref x))
([x & options]
(let [r ^clojure.lang.Ref (setup-reference (ref x) options)
opts (apply hash-map options)]
(when (:max-history opts)
(.setMaxHistory r (:max-history opts)))
(when (:min-history opts)
(.setMinHistory r (:min-history opts)))
r)))
•
• ref dosync
• 18
2011 5 24

45. deref
(defn deref
"Also reader macro: @ref/@agent/@var/@atom/@delay/@future/@promise. Within a transaction,
returns the in-transaction-value of ref, else returns the
most-recently-committed value of ref. When applied to a var, agent
or atom, returns its current state. When applied to a delay, forces
it if not already forced. When applied to a future, will block if
computation not complete. When applied to a promise, will block
until a value is delivered. The variant taking a timeout can be
used for blocking references (futures and promises), and will return
timeout-val if the timeout (in milliseconds) is reached before a
value is available. See also - realized?."
{:added "1.0"
:static true}
([^clojure.lang.IDeref ref] (.deref ref))
([^clojure.lang.IBlockingDeref ref timeout-ms timeout-val] (.deref ref timeout-ms timeout-val)))
• ref
– dosync dosync in-
transaction-value
• future
19
2011 5 24

46. ref-set
(defn ref-set
"Must be called in a transaction. Sets the value of ref.
Returns val."
{:added "1.0"
:static true}
[^clojure.lang.Ref ref val]
(. ref (set val)))
• ref
20
2011 5 24

47. alter
(defn alter
"Must be called in a transaction. Sets the in-transaction-value of
ref to:
(apply fun in-transaction-value-of-ref args)
and returns the in-transaction-value of ref."
{:added "1.0"
:static true}
[^clojure.lang.Ref ref fun & args]
(. ref (alter fun args)))
• ref
•
21
2011 5 24

48. commute
(defn commute
"Must be called in a transaction. Sets the in-transaction-value of
ref to:
(apply fun in-transaction-value-of-ref args)
and returns the in-transaction-value of ref.
At the commit point of the transaction, sets the value of ref to be:
(apply fun most-recently-committed-value-of-ref args)
Thus fun should be commutative, or, failing that, you must accept
last-one-in-wins behavior. commute allows for more concurrency than
ref-set."
{:added "1.0"
:static true}
[^clojure.lang.Ref ref fun & args]
(. ref (commute fun args)))
• ref
• alter
22
2011 5 24

49. ensure
(defn ensure
"Must be called in a transaction. Protects the ref from modification
by other transactions. Returns the in-transaction-value of
ref. Allows for more concurrency than (ref-set ref @ref)"
{:added "1.0"
:static true}
[^clojure.lang.Ref ref]
(. ref (touch))
(. ref (deref)))
• in-
transaction-value
• ref ref
ref
23
2011 5 24

50. dosync
(defmacro dosync
"Runs the exprs (in an implicit do) in a transaction that encompasses
exprs and any nested calls. Starts a transaction if none is already
running on this thread. Any uncaught exception will abort the
transaction and flow out of dosync. The exprs may be run more than
once, but any effects on Refs will be atomic."
{:added "1.0"}
[& exprs]
`(sync nil ~@exprs))
•
24
2011 5 24

51. sync
(defmacro sync
"transaction-flags => TBD, pass nil for now
Runs the exprs (in an implicit do) in a transaction that encompasses
exprs and any nested calls. Starts a transaction if none is already
running on this thread. Any uncaught exception will abort the
transaction and flow out of sync. The exprs may be run more than
once, but any effects on Refs will be atomic."
{:added "1.0"}
[flags-ignored-for-now & body]
`(. clojure.lang.LockingTransaction
(runInTransaction (fn [] ~@body))))
•
• “flags-ignored-for-now”
25
2011 5 24

52. io!
(defmacro io!
"If an io! block occurs in a transaction, throws an
IllegalStateException, else runs body in an implicit do. If the
first expression in body is a literal string, will use that as the
exception message."
{:added "1.0"}
[& body]
(let [message (when (string? (first body)) (first body))
body (if message (next body) body)]
`(if (clojure.lang.LockingTransaction/isRunning)
(throw (new IllegalStateException ~(or message "I/O in transaction")))
(do ~@body))))
• dosync io
• STM I/O
• I/O sync
26
2011 5 24

53. set/get-validator
(defn set-validator!
"Sets the validator-fn for a var/ref/agent/atom. validator-fn must be nil or a
side-effect-free fn of one argument, which will be passed the intended
new state on any state change. If the new state is unacceptable, the
validator-fn should return false or throw an exception. If the current state
(root
value if var) is not acceptable to the new validator, an exception
will be thrown and the validator will not be changed."
{:added "1.0"
:static true}
[^clojure.lang.IRef iref validator-fn] (. iref (setValidator validator-fn)))
(defn get-validator
"Gets the validator-fn for a var/ref/agent/atom."
{:added "1.0"
:static true}
[^clojure.lang.IRef iref] (. iref (getValidator)))
•
•
27
2011 5 24

54. alter/reset-meta
(defn alter-meta!
"Atomically sets the metadata for a namespace/var/ref/agent/atom to be:
(apply f its-current-meta args)
f must be free of side-effects"
{:added "1.0"
:static true}
[^clojure.lang.IReference iref f & args] (.alterMeta iref f args))
(defn reset-meta!
"Atomically resets the metadata for a namespace/var/ref/agent/atom"
{:added "1.0"
:static true}
[^clojure.lang.IReference iref metadata-map] (.resetMeta iref metadata-map))
•
•
28
2011 5 24

55. ref-history
(defn ref-history-count
"Returns the history count of a ref" • ref
{:added "1.1"
:static true}
[^clojure.lang.Ref ref]
(.getHistoryCount ref))
•
(defn ref-min-history
"Gets the min-history of a ref, or sets it and returns the ref"
{:added "1.1"
:static true}
([^clojure.lang.Ref ref]
(.getMinHistory ref))
([^clojure.lang.Ref ref n]
(.setMinHistory ref n)))
(defn ref-max-history
"Gets the max-history of a ref, or sets it and returns the ref"
{:added "1.1"
:static true}
([^clojure.lang.Ref ref]
(.getMaxHistory ref))
([^clojure.lang.Ref ref n]
(.setMaxHistory ref n)))
29
2011 5 24

56. Agenda
• Clojure
• STM
• Clojure STM
• Clojure STM API
• Clojure STM
• Clojure STM
30
2011 5 24

57. Clojure STM
• STM
• STM
31
2011 5 24

58. •
32
2011 5 24

59. 33
2011 5 24

60. Ref
• STM
–
TVal
• Ref TVal
34
2011 5 24

61. Ref.Tval
• ref
•
35
2011 5 24

62. LockingTransaction
•
•
– 10000000nano second
kill
• vals vals Ref
• sets/commutes/ensures
– sets ref comutes commute
– alter ref-set ref-set
– ensures commute
36
2011 5 24

63. LockingTransaction.Info
• ref
ref
transaction ref
•
• running/commiting/retry/killed/
commited
•
37
2011 5 24

64. LockingTransaction.RetryEx
•
•
38
2011 5 24

65. LockingTransaction.AbortException
•
39
2011 5 24

66. • ref-set
40
2011 5 24

67. (def current-track (ref "initial state of current track"))
(dosync (ref-set current-track "changed"))
• ref
• dosync ref-set
41
2011 5 24

68. ref
42
2011 5 24

69. transaction run
43
2011 5 24

70. ref-set
44
2011 5 24

71. ref
45
2011 5 24

72. Agenda
• Clojure
• STM
• Clojure STM
• Clojure STM API
• Clojure STM
• Clojure STM
46
2011 5 24

73. Clojure STM Transaction
• Multiversion Concurrency Control
–
• Snapshot Isolation
–
– Oracle/PostgereSQL RDBMS
SERIALIZABLE
47
2011 5 24

74. • Clojure
– conj
–
•
– Map Idel Hash Tree
• STM
– Snapshot Isolation
ref
48
2011 5 24

75. STM I/O
• No I/O in STM transactions?
– http://groups.google.com/group/clojure/
browse_thread/thread/
d967171ed4b0f20c/9f55aead761f3e99?
lnk=gst&q=STM#9f55aead761f3e99
• Agent
49
2011 5 24

76. • Extending Clojure's STM with external
transactions
– http://groups.google.com/group/clojure/
browse_thread/thread/
aa22a709501a64ac
• RDBMS JTA
–
– 50
2011 5 24

77. • Long-running STM updates will never
complete... is there a solution?
– http://groups.google.com/group/clojure/
browse_thread/thread/
7edd12e22eb120ea/207888a0d12ade38?
lnk=gst&q=STM#207888a0d12ade38
• ensure
–
51
2011 5 24

78. • LL tiger clojure
–
– http://www.slideshare.net/makingx/ll-tiger-clojure
• The Art of Multiprocessor Programming
– 18 Transactional Memory STM
Java Java
– http://amzn.to/kJCkyT
52
2011 5 24