Microsoft 365 Copilot: How to boost your productivity with AI – Part one: Ado...
Modern Java Concurrency Highlights
1. The WGJD - Modern Java
Concurrency
Ben Evans and Martijn Verburg
(@kittylyst @karianna)
http://www.teamsparq.net
Slide Design by http://www.kerrykenneally.com
3. Why Modern Java Concurrency is important
• The WGJD wants to utilise modern hardware
• The WGJD wants to write concurrent code
– Safely
– Without fear
– With an understanding of performance implications
• The WGJD wants to take advantage of:
– JVM support for parallelised operations
– An API that avoids synchronized
• Modern Java Concurrency lets you do all of this
3
4. About this section
• We only have ~60mins hour to talk today
• Huge amount to talk about
• This subject fills 2 (or even 4) days worth of training
• We will give you some highlights today
• For die-hard low latency fiends
– Locks are actually bad OK!
– Come talk to us afterwards to find out more
4
5. Modern Java concurrency
• Not a new subject
– Underwent a revolution with Java 5
• More refinements since
– Still more coming in 7
• java.util.concurrent (j.u.c) really fast in 6
– Better yet in 7
• However, still under-appreciated by a surprising number
• Too much Java 4-style concurrency code still in PROD
– Why...?
5
6. Java concurrency - Why not upgrade?
• Too much legacy code?
– People scared to refactor?
• People don’t know j.u.c is easier than “classic” Java concurrency?
• People don’t know j.u.c is faster than classic?
• People don’t know that you can mix-and-match (with a bit of care)?
• Still not being taught at Universities?
• Not enough people reading Doug Lea or Brian Goetz?
6
7. Modern Java concurrency
• Perhaps........
– Previous treatments haven’t involved enough otters.
• We will rectify this.
• If you suffer from lutraphobia, you may want to leave now...
– We did warn you about fluffy animals
– Ones that BITE
7
8. Otterly Amazing Tails of Modern Java
Concurrency
• Srsly.
• Otters!
• See those
teeth?
8
9. Why Otters?
• Otters are a very good metaphor
for concurrency
• Not just because they look a bit
like threads (i.e. long and thin)
• Collaborative, Competitive & Sneaky
• Can hare off in opposite directions
• Capable of wreaking havoc
if not contained
9
10. Otter Management (aka the 4 forces)
• Safety
– Does each object stay self-consistent?
– No matter what other operations are happening?
• Liveness
– Does the program eventually progress?
– Are any failures to progress temporary or permanent?
• Performance
– How well does the system take advantage of processing cores?
• Reusability
– How easy is it to reuse the system in other applications?
10
11. Some History
• Until recently, most computers had only one processing core
• Multithreading was simulated on a single core
– Not true concurrency
• Serial approach to algorithms often sufficed
• Interleaved multithreading can mask errors
– Or be more forgiving than true concurrency
• Why and how (and when) did things change?
11
12. Moore’s Law
• “The number of transistors on an economic-to-produce chip roughly
doubles every 2 years”
• Originally stated in 1965
– Expected to hold for the 10 years to 1975
– Still going strong
• Named for Gordon Moore (Intel founder)
– About transistor counts
– Not clock speed
– Or overall performance
12
14. Moore’s Law - Problems
• Not about overall performance
• Memory latency exponent gap
– Need to keep the processing pipeline full
– Add memory caches of faster SRAM “close” to the CPU
• (L1, L2 etc)
• Code restricted by L1 cache misses rather than CPU speed
– After JIT compilation
14
15. Spending the transistor budget
• More and more complex contortions...
• ILP, CMT, Branch prediction, etc, etc
15
16. Multi-core
• If we can’t increase clock speed / overall performance
– Have to go multi-core
– Concurrency and performance are tied together
• Real concurrency
– Separate threads executing on different cores at the same moment
• The JVM runtime controls scheduling
– Java thread scheduling does NOT behave like OS scheduling
• Concurrency becomes the performance improver
16
17. Classic Java Concurrency
• Provides exclusion
• Need locking to make
mutation concurrency-safe
• Locking gets complicated
• Can become fragile
• Why synchronized?
17
19. Three approaches to Concurrent Type Safety
• Fully-synchronized Objects
– Synchronize all methods on all classes
• Immutability
– Useful, but may have high copy-cost
– Requires programmer discipline
• Be Very, Very Careful
– Difficult
– Fragile
– With Java - Often the only game in town
19
20. The JMM
• Mathematical
description of memory
• Most impenetrable
part of the Java
language spec
• JMM makes Primary concepts:
minimum guarantees • synchronizes-with
• happens-before
• Real JVMs (and CPUs) • release-before-acquire
may do more • as-if-serial
20
21. Synchronizes-with
• Threads have their own
description of an object’s state
• This must be flushed to
main memory and other threads
• synchronized means that this
local view has been synchronized-with
the other threads
• Defines touch-points where threads must perform synching
21
22. java.util.concurrent
• Thanks, Doug!
• j.u.c has building blocks for
concurrent code
– ReentrantLock
– Condition
– ConcurrentHashMap
– CopyOnWriteArrayList
– Other Concurrent Data Structures
22
23. Locks in j.u.c
• Lock is an interface
• ReentrantLock is the usual implementation
23
27. ConcurrentHashMap (CHM)
• HashMap has:
– Hash function
– Even distribution of buckets
• Concurrent form
– Can lock independently
– Seems lock-free to users
– Has atomic operations
27
29. CountDownLatch
• A group consensus construct
• countDown() decrements
the count
• await() blocks until
count == 0
– i.e. consensus
• Constructor takes an int (the count)
• Quite a few use cases
– e.g. Multithreaded testing
29
31. Handoff Queue (in-mem)
• Efficient way to hand off work
between threadpools
• BlockingQueue a good pick
• Has blocking ops with timeouts
– e.g. for backoff / retry
• Two basic implementations
– ArrayList and LinkedList backed
• Java 7 introduces the shiny new TransferQueue
31
34. Executors
• j.u.c execution constructs
– Callable, Future,
FutureTask
• In addition to the venerable
– Thread and Runnable
• Stop using TimerTask!
• Executors class provides factory methods for making threadpools
– ScheduledThreadPoolExecutor is one standard choice
34
38. Fork/Join
• Java 7 introduces F/J
– similar to MapReduce
– useful for a certain class of problems
– F/J executions are not really threads
• In our example, we
subclass RecursiveAction
• Need to provide a compute() method
– And a way of merging results
• F/J provides an invokeAll() to hand off more tasks
38
39. Fork/Join
• Typical divide and conquer style problem
– invokeall() performs the threadpool, worker & queue magic
39
40. Concurrent Java Code
• Mutable state (objects) protected by locks
• Concurrent data structures
– CHM, COWAL
• Be careful of performance
– especially COWAL
• Synchronous multithreading - explicit synchronization
• Executor-based threadpools
• Asynch multithreading communicates using queue-like handoffs
40
41. Stepping Back
• Concurrency is key to the future of
performant code
• Mutable state is hard
• Need both synch & asynch state sharing
• Locks can be hard to use correctly
• JMM is a low-level, flexible model
– Need higher-level concurrency model
– Thread is still too low-level
41
42. Imagine a world...
• The runtime & environment helped out the programmer more:
– Runtime-managed concurrency
– Collections were thread-safe by default
– Objects were immutable by default
– State was well encapsulated and not shared by default
• Thread wasn’t the default choice for unit of concurrent execution
• Copy-on-write was the basis for mutation of collections /
synchronous multithreading
• Hand-off queues were the basis for asynchronous multithreading
42
43. What can we do with Java?
• We’re stuck with a lot of heritage in Java
– But the JVM and JMM are very sound
• You don’t have to abandon Java
– Mechanical sympathy and clean code get you far
– The JIT compiler just gets better and better
• If we wanted to dream of a new language
– It should be on the JVM
– It should build on what we’ve learned in 15 years of Java
43
44. New Frontiers in Concurrency
• There are several options now on the JVM
– New possibilities built-in to the language syntax
– Synch and asynch models
• Scala offers an Actors model
– And the powerful Akka framework
• Clojure is immutable by default
– Has agents (like actors) & shared-nothing by default
– Also has a Software Transactional Memory (STM) model
• Groovy has GPARs
44
45. Acknowledgments
• All otter images Creative Commons or Fair Use
• Matt Raible
• Ola Bini
• Photos owned by Flickr Users
– moff, spark, sodaro, lonecellotheory, tomsowerby
– farnsworth, prince, marcus_jb1973, mliu92, Ed Zitron,
– NaturalLight & monkeywing
• Dancing Otter by the amazing Nicola Slater @ folksy
45
* Hands up for Java 6, 5, 4....\n* Hands up for j.u.c\n
Hands up if you’re daunted by the refactoring that would be required\n\n
\n
\n
They are Apex Predators - think of them in that same way\n
The practice of managing your threads (or Otters!) is governed by four forces (after Doug Lea)\n
\n
\n
It’s a little-known fact that Otters are scared of log-linear graphs\n
Very successful within own frame of reference, but caveats\nReference Mechanical sympathy again here\n
* So you can contort more and more, but you’re chasing diminishing returns\n* Ultimately, that exponent gap between clock speed and memory will do for you\n
* Raise your hand if you use the process monitor on Ubuntu or another Linux. OK, have you seen how Java processes behave under that?\n
* Explain that we’re going to show replacements for using synchronized\n* Raise your hand if you know why we use the keyword “synchronized” to denote a critical section in Java\n
\n
How performant do we think FS objects are?\nImmutability is good, but watch the copy-cost\n
happens-before defines a “partial order” (if you’re a mathematician)\nJMM is even worse than the generics constraints part of the spec!\n\n
* Hands up if you know what a NUMA architecture is?\n* In some ways, this is actually easier to explain on a NUMA arch...\n
\n
Lock can in some cases directly replace synchronized, but is more flexible\nMONITORENTER & MONITOREXIT\nWe use reentrant lock else recursive code deadlocks real quick\n
\n
Condition takes the place of wait() / notify() (Object monitors)\nTalk to the cases - 1 putter, 1 taker, many putters, few takers, few putters, many takers - think about this stuff at the design stage\n
\n
Basically it’s a drop-in replacement for regular HashMap\n“What’s the worst thing that can happen if you’re iterating over the keys of a regular HashMap and someone alters it underneath you?”\n
Not quite a drop-in replacement - Performance needs to be thought about\nMV: Need to fix code sample so we have iterators\n
\n
\n
BlockingQueue offers several different ways to interact with it (see the Javadoc)\n\n
\n
offer is similar to add but doesn’t throw exceptions\nProducers adding trade orders for example\nTheatresports\n
Final building block for modern concurrent applications with Java\n
\n
Let’s step through a quasi-realistic example\nThat cancel() code begs for some of that lambda treatment huh!\n
\n
As well as RecursiveAction there’s the more general ForkJoinTask\n\n
Multithreaded Quicksort - shows a speedup from O(nlog n) to O(n) - not quite linear, but not bad\n
So this is pretty much a statement of the state-of-the-art in Java concurrency\n
* Thread is the assembly language of concurrency\n* We need to move to a more managed model\n