Part 3/3 of our Devoxx university session!

1. The WGJD - Modern Java
Concurrency
Ben Evans and Martijn Verburg
(@kittylyst @karianna)
http://www.teamsparq.net
Slide Design by http://www.kerrykenneally.com

2. Are you a fluffy animal lover..?
Leave now.
2

3. Why Modern Java Concurrency is important
• The WGJD wants to utilise modern hardware
• The WGJD wants to write concurrent code
– Safely
– Without fear
– With an understanding of performance implications
• The WGJD wants to take advantage of:
– JVM support for parallelised operations
– An API that avoids synchronized
• Modern Java Concurrency lets you do all of this
3

4. About this section
• We only have ~60mins hour to talk today
• Huge amount to talk about
• This subject fills 2 (or even 4) days worth of training
• We will give you some highlights today
• For die-hard low latency fiends
– Locks are actually bad OK!
– Come talk to us afterwards to find out more
4

5. Modern Java concurrency
• Not a new subject
– Underwent a revolution with Java 5
• More refinements since
– Still more coming in 7
• java.util.concurrent (j.u.c) really fast in 6
– Better yet in 7
• However, still under-appreciated by a surprising number
• Too much Java 4-style concurrency code still in PROD
– Why...?
5

6. Java concurrency - Why not upgrade?
• Too much legacy code?
– People scared to refactor?
• People don’t know j.u.c is easier than “classic” Java concurrency?
• People don’t know j.u.c is faster than classic?
• People don’t know that you can mix-and-match (with a bit of care)?
• Still not being taught at Universities?
• Not enough people reading Doug Lea or Brian Goetz?
6

7. Modern Java concurrency
• Perhaps........
– Previous treatments haven’t involved enough otters.
• We will rectify this.
• If you suffer from lutraphobia, you may want to leave now...
– We did warn you about fluffy animals
– Ones that BITE
7

8. Otterly Amazing Tails of Modern Java
Concurrency
• Srsly.
• Otters!
• See those
teeth?
8

9. Why Otters?
• Otters are a very good metaphor
for concurrency
• Not just because they look a bit
like threads (i.e. long and thin)
• Collaborative, Competitive & Sneaky
• Can hare off in opposite directions
• Capable of wreaking havoc
if not contained
9

10. Otter Management (aka the 4 forces)
• Safety
– Does each object stay self-consistent?
– No matter what other operations are happening?
• Liveness
– Does the program eventually progress?
– Are any failures to progress temporary or permanent?
• Performance
– How well does the system take advantage of processing cores?
• Reusability
– How easy is it to reuse the system in other applications?
10

11. Some History
• Until recently, most computers had only one processing core
• Multithreading was simulated on a single core
– Not true concurrency
• Serial approach to algorithms often sufficed
• Interleaved multithreading can mask errors
– Or be more forgiving than true concurrency
• Why and how (and when) did things change?
11

12. Moore’s Law
• “The number of transistors on an economic-to-produce chip roughly
doubles every 2 years”
• Originally stated in 1965
– Expected to hold for the 10 years to 1975
– Still going strong
• Named for Gordon Moore (Intel founder)
– About transistor counts
– Not clock speed
– Or overall performance
12

13. Transistor Counts
13

14. Moore’s Law - Problems
• Not about overall performance
• Memory latency exponent gap
– Need to keep the processing pipeline full
– Add memory caches of faster SRAM “close” to the CPU
• (L1, L2 etc)
• Code restricted by L1 cache misses rather than CPU speed
– After JIT compilation
14

15. Spending the transistor budget
• More and more complex contortions...
• ILP, CMT, Branch prediction, etc, etc
15

16. Multi-core
• If we can’t increase clock speed / overall performance
– Have to go multi-core
– Concurrency and performance are tied together
• Real concurrency
– Separate threads executing on different cores at the same moment
• The JVM runtime controls scheduling
– Java thread scheduling does NOT behave like OS scheduling
• Concurrency becomes the performance improver
16

17. Classic Java Concurrency
• Provides exclusion
• Need locking to make
mutation concurrency-safe
• Locking gets complicated
• Can become fragile
• Why synchronized?
17

18. I wrap sychronized....
around entire classes
Safe as Fort Knox
18

19. Three approaches to Concurrent Type Safety
• Fully-synchronized Objects
– Synchronize all methods on all classes
• Immutability
– Useful, but may have high copy-cost
– Requires programmer discipline
• Be Very, Very Careful
– Difficult
– Fragile
– With Java - Often the only game in town
19

20. The JMM
• Mathematical
description of memory
• Most impenetrable
part of the Java
language spec
• JMM makes Primary concepts:
minimum guarantees • synchronizes-with
• happens-before
• Real JVMs (and CPUs) • release-before-acquire
may do more • as-if-serial
20

21. Synchronizes-with
• Threads have their own
description of an object’s state
• This must be flushed to
main memory and other threads
• synchronized means that this
local view has been synchronized-with
the other threads
• Defines touch-points where threads must perform synching
21

22. java.util.concurrent
• Thanks, Doug!
• j.u.c has building blocks for
concurrent code
– ReentrantLock
– Condition
– ConcurrentHashMap
– CopyOnWriteArrayList
– Other Concurrent Data Structures
22

23. Locks in j.u.c
• Lock is an interface
• ReentrantLock is the usual implementation
23

24. I always wanted
to be an actor
24

25. Conditions in j.u.c
25

26. Conditions in j.u.c
26

27. ConcurrentHashMap (CHM)
• HashMap has:
– Hash function
– Even distribution of buckets
• Concurrent form
– Can lock independently
– Seems lock-free to users
– Has atomic operations
27

28. CopyOnWriteArrayList (COWAL)
• Makes separate copies of
underlying structure
• Iterator will never throw
ConcurrentModificationException
28

29. CountDownLatch
• A group consensus construct
• countDown() decrements
the count
• await() blocks until
count == 0
– i.e. consensus
• Constructor takes an int (the count)
• Quite a few use cases
– e.g. Multithreaded testing
29

30. Example - CountDownLatch
30

31. Handoff Queue (in-mem)
• Efficient way to hand off work
between threadpools
• BlockingQueue a good pick
• Has blocking ops with timeouts
– e.g. for backoff / retry
• Two basic implementations
– ArrayList and LinkedList backed
• Java 7 introduces the shiny new TransferQueue
31

32. Just use JMS!
32

33. Example - LinkedBlockingQueue
• Imagine multiple producers and consumers
33

34. Executors
• j.u.c execution constructs
– Callable, Future,
FutureTask
• In addition to the venerable
– Thread and Runnable
• Stop using TimerTask!
• Executors class provides factory methods for making threadpools
– ScheduledThreadPoolExecutor is one standard choice
34

35. Animals were harmed in the
making of this presentation
Crab got executed
35

36. Example - ThreadPoolManager
36

37. Example - QueueReaderTask
37

38. Fork/Join
• Java 7 introduces F/J
– similar to MapReduce
– useful for a certain class of problems
– F/J executions are not really threads
• In our example, we
subclass RecursiveAction
• Need to provide a compute() method
– And a way of merging results
• F/J provides an invokeAll() to hand off more tasks
38

39. Fork/Join
• Typical divide and conquer style problem
– invokeall() performs the threadpool, worker & queue magic
39

40. Concurrent Java Code
• Mutable state (objects) protected by locks
• Concurrent data structures
– CHM, COWAL
• Be careful of performance
– especially COWAL
• Synchronous multithreading - explicit synchronization
• Executor-based threadpools
• Asynch multithreading communicates using queue-like handoffs
40

41. Stepping Back
• Concurrency is key to the future of
performant code
• Mutable state is hard
• Need both synch & asynch state sharing
• Locks can be hard to use correctly
• JMM is a low-level, flexible model
– Need higher-level concurrency model
– Thread is still too low-level
41

42. Imagine a world...
• The runtime & environment helped out the programmer more:
– Runtime-managed concurrency
– Collections were thread-safe by default
– Objects were immutable by default
– State was well encapsulated and not shared by default
• Thread wasn’t the default choice for unit of concurrent execution
• Copy-on-write was the basis for mutation of collections /
synchronous multithreading
• Hand-off queues were the basis for asynchronous multithreading
42

43. What can we do with Java?
• We’re stuck with a lot of heritage in Java
– But the JVM and JMM are very sound
• You don’t have to abandon Java
– Mechanical sympathy and clean code get you far
– The JIT compiler just gets better and better
• If we wanted to dream of a new language
– It should be on the JVM
– It should build on what we’ve learned in 15 years of Java
43

44. New Frontiers in Concurrency
• There are several options now on the JVM
– New possibilities built-in to the language syntax
– Synch and asynch models
• Scala offers an Actors model
– And the powerful Akka framework
• Clojure is immutable by default
– Has agents (like actors) & shared-nothing by default
– Also has a Software Transactional Memory (STM) model
• Groovy has GPARs
44

45. Acknowledgments
• All otter images Creative Commons or Fair Use
• Matt Raible
• Ola Bini
• Photos owned by Flickr Users
– moff, spark, sodaro, lonecellotheory, tomsowerby
– farnsworth, prince, marcus_jb1973, mliu92, Ed Zitron,
– NaturalLight & monkeywing
• Dancing Otter by the amazing Nicola Slater @ folksy
45

46. Where is our beer!?
46

47. Thanks for listening! (@kittylyst, @karianna)
• http://www.teamsparq.net
• http://www.java7developer.com 47