Overview of Java Concurrency Utilities from Goetz and Bloch books

1. JavaJava
ConcurrencyConcurrency
●
Carol McDonaldCarol McDonald
–AvailityAvaility

2. . 4
Why Concurrency:
• Benefits of threads:
> Multitasking, Exploit multiple cores or CPUs
> Multi-threading good for blocking for I/O or other blocking ops
• Java < 1.5 concurrency primitives:
> wait(), notify(), sleep(), interrupt(),
synchronized
> Easy to use incorrectly
> Incorrect use can produce poor performance

3. . 5
Risks of Threads
• Safety hazards
> Syncronization, contention
• Liveness hazards
> Race conditions, deadlock
• Performance hazards
> Context switching overhead

4. . 6
Concurrency Utilities Goals
• set of concurrency building blocks
> library for concurrency like Collections for data structures
• Enhance scalability, performance, and thread safety
• Java Memory Model for Multi tasking

5. . 7
Do you see the error?
01 class PingPong {
02 public static synchronized void main(String[] a) {
03 Thread t = new Thread() {
04 public void run() {
05 pong();
06 }
07 };
08
09 t.run();
10 System.out.print("Ping");
11 }
12
13 static synchronized void pong() {
14 System.out.print("Pong");
15 }
16 }

6. . 9
What Does It Print?
(a) PingPong
(b) PongPing
(c) It varies
Not a multithreaded program!

7. . 10
Example How to start a thread
public class HelloRunnable implements Runnable {
public void run() {
System.out.println("Hello from a thread!");
}
public static void main(String args[]) {
(new Thread(new HelloRunnable())).start();
}
}

8. . 11
Another Look
01 class PingPong {
02 public static synchronized void main(String[] a) {
03 Thread t = new Thread() {
04 public void run() {
05 pong();
06 }
07 };
08
09 t.run(); // Common typo!
10 System.out.print("Ping");
11 }
12
13 static synchronized void pong() {
14 System.out.print("Pong");
15 }
16 }

9. . 12
How Do You Fix It?
01 class PingPong {
02 public static synchronized void main(String[] a) {
03 Thread t = new Thread() {
04 public void run() {
05 pong();
06 }
07 };
08
09 t.start();
10 System.out.print("Ping");
11 }
12
13 static synchronized void pong() {
14 System.out.print("Pong");
15 }
16 }

10. . 14
Concurrency Utilities:
• Task Scheduling Framework: Executor
interface replaces direct use of Thread
• Callable and Future
• Synchronisers
> Semaphore, CyclicBarrier, CountDownLatch
• Concurrent collections
• Lock
• Atomic, Visible

11. . 15
Concurrency Utilities:
• Task Scheduling Framework: Executor
• Separate task submission from
execution policy
• No more direct Thread invocation
> Use myExecutor.execute(aRunnable);
> Not new Thread(aRunnable).start();
public interface Executor {
void execute (Runnable command);
}

12. . 16
Executor
an object whose job it is to run Runnables:
public interface Executor {
void execute (Runnable command);
}
public interface ExecutorService extends Executor {
..
}
public class Executors {
static ExecutorService newFixedThreadPool(int poolSize);
...
}
Executor pool = Executors.newFixedThreadPool(5);
pool.execute (runnable);
manages the lifecycle of the execution service
static factory methods for Executor implementations

13. . 17
Creating Executors
• Factory methods in the Executors class
public class Executors {
static ExecutorService
newSingleThreadedExecutor();
static ExecutorService
newFixedThreadPool(int poolSize);
static ExecutorService
newCachedThreadPool();
static ScheduledExecutorService
newScheduledThreadPool();
// Other methods not listed
}

14. . 18
How not to manage tasks
class UnreliableWebServer {
public static void main(String[] args) {
ServerSocket socket = new ServerSocket(80);
while (true) {
final Socket connection = socket.accept();
Runnable r = new Runnable() {
public void run() {
handleRequest(connection);
}
};
// Don't do this!
new Thread(r).start();
}
}
}
Could create too many threads !

15. . 19
Thread Pool Example
class WebService {
public static void main(String[] args) {
Executor pool = Executors.newFixedThreadPool(5);
ServerSocket socket = new ServerSocket(999);
for (;;) {
final Socket connection = socket.accept();
Runnable task = new Runnable() {
public void run() {
new Handler().process(connection);
}
}
pool.execute (task);
}
}
}
class Handler { void process(Socket s); }
myExecutor.execute(aRunnable)

16. . 20
ExecutorService for Lifecycle Support
• ExecutorService supports graceful and
immediate shutdown
public interface ExecutorService extends
Executor {
void shutdown();
List<Runnable> shutdownNow();
boolean isShutdown();
boolean isTerminated();
boolean awaitTermination(long timeout,
TimeUnit unit);
// additional methods not listed
}

17. . 21
Callables and Futures
•Callable interface provides way to get a result
from a thread
●Implement call() method rather than run()
•Callable is submitted to ExecutorService
●Call submit() not execute()
●submit() returns a Future object
•Retrieve result using get() method of Future
object
●If result is ready it is returned
●If not ready calling thread will block

18. . 22
Callable Example
class CallableExample implements
Callable<String> {
public String call() {
String result = null;
/* Do some work and create a result */
return result;
}
}

19. . 23
Future Example
ExecutorService es =
Executors.newSingleThreadedExecutor();
Future<String> f =
es.submit (new CallableExample());
/* Do some work in parallel */
try {
String callableResult = f.get();
} catch (InterruptedException ie) {
/* Handle */
} catch (ExecutionException ee) {
/* Handle */
}

20. . 24
ScheduledExecutorService
• Deferred and recurring tasks
> Schedule execution of Callable or Runnable to run
once after a fixed delay
> schedule()
> Schedule a Runnable to run periodically at a fixed rate
> scheduleAtFixedRate()
> Schedule a Runnable to run periodically with a fixed delay
between executions
> scheduleWithFixedDelay()
• Submission returns a ScheduledFuture
> Can be used to cancel task

21. . 26
Recommendation
• Hold Locks for as short a time as possible
• Do not perform CPU intensive or I/O ops inside a
synchronized method or while locking

22. . 27
Synchronize Critical Section
• E.g., shared resource is an customer account. Certain methods called
by multiple threads.
• Hold monitor lock for as short a time as possible.
synchronized double getBalance() {
Account acct = verify(name, password);
return acct.balance;
} Lock held for long time
double getBalance() {
synchronized (this) {
Account acct = verify(name, password);
return acct.balance;
}
}
Current object is locked
Equivalent to above
double getBalance() {
Account acct = verify(name, password);
synchronized (acct) { return acct.balance};
}
Better
Only acct object is locked – for shorter tim

23. . 28
Locks
• Java provides basic locking via synchronized
• Good for many situations, but some issues
> Single monitor per object
> Not possible to interrupt thread waiting for lock
> Not possible to time-out when waiting for a lock
> Block structured approach
> Aquiring multiple locks is complex
> Advanced techniques not possible
• New Lock interface addresses these issues

24. . 29
Lock Interface
• No automatic unlocking
Interface Lock {
void lock();
void lockInterruptibly() throws IE;
boolean tryLock();
boolean tryLock(long t, TimeUnit u) throws IE;
//returns true if lock is aquired
void unlock();
Condition newCondition() throws
UnsupportedOperationException;
}
IE = InterruptedException

25. . 31
Lock Example
Lock lock = new RentrantLock();
public void accessProtectedResource()
throws IllegalMonitorStateException {
lock.lock();
try {
// Access lock protected resource
} finally {
// Ensure lock is always released
lock.unlock();
}
}

26. . 32
ReadWriteLock Interface
• Has two locks controlling read and write access
> Multiple threads can aquire the read lock if no threads have a
write lock
> Only one thread can aquire the write lock
> Methods to access locks
rwl.readLock().lock();
rwl.writeLock().lock();
> Better performance for read-mostly data access

27. . 33
ReadWriteLock Example
ReentrantReadWriteLock rwl = new ReentrantReadWriteLock();
Lock r = rwl.readLock();
Lock w = rwl.writeLock();
ArrayList<String> data = new ArrayList<String>();
public String getData(int pos) {
r.lock();
try { return data.get(pos); }
finally { r.unlock(); }
}
public void addData(int pos, String value) {
w.lock();
try { data.add(pos, value); }
finally { w.unlock(); }
}

28. . 34
Synchronizers
• Co-ordinate access and control -can eliminate most
uses of wait or notify
• Semaphore
> Manages a fixed sized pool of resources
• CountDownLatch
> One or more threads wait for a set of threads to complete an
action
• CyclicBarrier
> Set of threads wait until they all reach a specified point
• Exchanger
> Two threads reach a fixed point and exchange data

29. . 35
Semaphore Example
private Semaphore available;
private Resource[] resources;
private boolean[] used;
public Resource(int poolSize) {
available = new Semaphore(poolSize);
/* Initialise resource pool */
}
public Resource getResource() {
try { available.aquire() } catch (IE) {}
}
public void returnResource(Resource r) {
/* Return resource to pool */
available.release();
}

30. . 36
Recommendation
• Prefer Concurrency utilities to wait and notify

31. . 37
BlockingQueue Interface
• thread safe Producer Consumer Pattern
Interface BlockingQueue<E> {
void put(E o) throws IE;
boolean offer(E o) throws IE;
boolean offer(E o, long t, TimeUnit u) throws IE;
E take() throws IE;
E poll() throws IE;
E poll(long t, TimeUnit u) throws IE;
int drainTo(Collection<? super E> c);
int drainTo(Collection<? super E> c, int max);
// Other methods not listed
}

32. . 39
Consumer Blocking Queue Example:
Logger thread
private BlockingQueue<String> msgQueue;
public Logger(BlockingQueue<String> mq) {
msgQueue = mq;
}
public void run() {
try {
while (true) {
String message = msgQueue.take();
/* Log message */
}
} catch (InterruptedException ie) { }
}

33. . 40
Producer Blocking Queue Example:
using the Logger thread
private ArrayBlockingQueue messageQueue =
new ArrayBlockingQueue<String>(10);
Logger logger = new Logger(messageQueue);
// runnables :
public void run() {
String someMessage;
try {
while (true) {
/* Do some processing */
/* Blocks if no space available */
messageQueue.put(someMessage);
}
} catch (InterruptedException ie) { }
}

34. . 41
Concurrent Collections
• ConcurrentMap (interface)
> Extends Map interface with atomic operations
• ConcurrentHashMap
> Fully concurrent retrieval
> Tunable concurrency for updates
> Constructor takes number of expected concurrent threads
• ConcurrentLinkedQueue
> Unbounded, thread safe queue, FIFO
• CopyOnWriteArrayList
> Optimised for frequent iteration, infrequent modifications

35. . 42
Java SE 6:
Concurrency Features
•Deque interface for double ended queues
>ArrayDeque, LinkedBlockingDeque,
ConcurrentLinkedDeque
•Concurrent skiplists
>ConcurrentSkipListMap,
ConcurrentSkipListSet
•AbstractQueuedLongSynchronizer
>Version of AbstractQueuedSynchronizer that uses a
long for internal state information

36. Concurrency:Concurrency:
Atomic VariablesAtomic Variables

37. . 44
Java Memory Model
• key ideas:
• all threads share the main
memory
• each thread uses a local
working memory
• refreshing local memory
to/from main memory must
comply to JMM rules

38. . 45
Java Memory Model
•
• Volatile
writing to a volatile forces a flush to
shared memory
• a read of volatile variable always
returns the most recent write by any
thread

39. . 46
Atomics
• java.util.concurrent.atomic
> classes that support lock-free thread-safe programming
on single variables
> use efficient machine level atomic processor instructions
> AtomicBoolean, AtomicInteger, AtomicLong,
AtomicLongArray,AtomicReference, AtomicReferenceArray
AtomicInteger balance = new AtomicInteger(0);
public int deposit(integer amount) {
return balance.addAndGet(amount);
}

40. . 47
Prefer immutable objects/data
•An immutable object is one whose
> State cannot be changed after construction
> All fields are private final , no setters
•Immutable objects are automatically thread-safe!
•Simpler
•Safer
> Can be freely shared
•More scalable
> No synchronization required

41. . 48
Multithreaded Lazy Initialization is tricky
public class Singleton {
private static Singleton instance;
public static Singleton getInstance() {
if (instance == null) {
instance = new Singleton();
}
return instance;
}
private Singleton() {}
}
Shared
mutable
Non deterministic type bug can result

42. . 49
Eager Initialization
public class Singleton {
private static final Singleton instance=
new Singleton();
public static Singleton getInstance() {
return instance;
}
private Singleton() {}
}
immutable

43. . 50
Servlet Counter
public class HelloWithHits extends HttpServlet {
int hits; // Shared mutable variable (danger)
protected void doGet(HttpServletRequest request,
HttpServletResponse response)
throws IOException {
++hits; // Risks atomicity + visibility failure
...
}
}

44. . 51
Servlet Counter
public class HelloWithHits extends HttpServlet {
final AtomicInteger hits = new AtomicInteger();
protected void doGet(HttpServletRequest request,
HttpServletResponse response)
throws IOException {
hits.incrementAndGet();
...
}
}

45. . 52
Audit Web
• Find the shared variables
> Search all servlet classes for instance fields
> Search for all instances of getAttribute() on HttpSession or
ServletContext
> Search JSPs for session- or application-scoped beans
• For each, determine if it is mutable
> If so, can you replace it with an immutable object?
• Identify variables that will change at the same time
> Ensure that read/multiple write accesses are atomic
> One way is to encapsulate in an immutable holder
http://www.ibm.com/developerworks/library/j-jtp09238.html

46. . 53
Reduce locking
• Do As little work as possible inside
synchronized blocks
• guard different state with different locks
> reduces likelihood of lock contention
> (but disciplined order or deadlock)

47. . 54
Reduce locking
• reduce locking
> replace mutable objects with immutable ones
> replace shared objects with thread-local ones
> java.util.concurrent.atomic, and
> Concurrent collections in java.util.concurrent

48. . 55
VisualVM Monitoring threads

49. . 56
VisualVM

50. . 57
VisualVM

51. . 58
Trends
trend towards concurrent, asynchronous
computing

52. . 59
Actors
• message passing concurrency
• Share Nothing
• Communicate through messages
• Asynchronous non-blocking

53. 60
EBay: Async Everywhere
Pattern: Event Queue
> Primary application writes data and queues event
> Consumers subscribe to event

54. 62
Ebay: Partition Everything
Pattern: Functional Segmentation
> Segment processing into pools, services, and stages
> Segment data by entity and usage pattern

55. 63
LinkedIn Best Practices
Storage and architecture
> Keep the system stateless

eBay, Google, etc.
> Partition data and services

Facebook, eBay
> Cache data
> Replicate your data

56. . 64
MySQL replication
• Writes replicated to In memory memcache for fast reads
master
slave

57. Resources

58. . 68
More Info/References
•Books
> Java Concurrency in Practice (Goetz, et al)
> See http://www.jcip.net
> Concurrent Programming in Java (Lea)
> Effective Java (Bloch)
• Articles
> http://www.ibm.com/developerworks/java/tutorials/j-
concur/section4.html
> http://blogs.sun.com/carolmcdonald/entry/some_concurr
ency_tips

59. . 69
References
• http://www.ibm.com/developerworks/library/j-jtp09238.html
• http://www.angelikalanger.com/
• Best Practices for Large-Scale Web Sites -- Lessons from eBay
http://java.sun.com/javaone/2009/articles/gen_ebay.jsp
• Lessons from Linked In
> http://developers.sun.com/learning/javaoneonline/j1sessn.jsp?sessn=TS
• http://www.ibm.com/developerworks/library/j-jtp03304/

60. . 70
For More Information
• Memory management white paper
> http://java.sun.com/j2se/reference/whitepapers/
• Destructors, Finalizers, and Synchronization
> http://portal.acm.org/citation.cfm?id=604153
• Finalization, Threads, and the Java Technology
Memory Model
> http://developers.sun.com/learning/javaoneonline/2005/corep
latform/TS-3281.html
• Memory-retention due to finalization article
> http://www.devx.com/Java/Article/30192

61. . 71
For More Information
• FindBugs
> http://findbugs.sourceforge.net
• Heap analysis tools
> Monitoring and Management in 6.0
> http://java.sun.com/developer/technicalArticles/J2SE/monitoring/
> Troubleshooting guide
> http://java.sun.com/javase/6/webnotes/trouble/
> VisualVM
> http://download-
llnw.oracle.com/javase/6/docs/technotes/guides/visualvm/threads.html
> JConsole
> http://java.sun.com/developer/technicalArticles/J2SE/jconsole.html

62. Thank You!Thank You!
●
Carol McDonaldCarol McDonald
–