Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Operating Systems - Advanced Synchronization
1. Operating Systems
CMPSCI 377
Advanced Synchronization
Emery Berger
University of Massachusetts Amherst
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science
2. Why Synchronization?
Synchronization serves two purposes:
Ensure safety for shared updates
Avoid race conditions
Coordinate actions of threads
Parallel computation
Event notification
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 2
3. Synch. Operations
Safety:
Locks
Coordination:
Semaphores
Condition variables
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 3
4. Safety
Multiple threads/processes – access shared
resource simultaneously
Safe only if:
All accesses have no effect on resource,
e.g., reading a variable, or
All accesses idempotent
E.g., a = abs(x), a = highbit(a)
Only one access at a time:
mutual exclusion
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 4
5. Safety: Example
“The too much milk problem”
Model of need to synchronize activities
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 5
6. Why You Need Locks
thread A thread B
if (no milk && no note) if (no milk && no note)
leave note leave note
buy milk buy milk
remove note remove note
Does this work? milk
too much
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 6
7. Mutual Exclusion
Prevent more than one thread from
accessing critical section
Serializes access to section
Lock, update, unlock:
lock (&l);
update data; /* critical section */
unlock (&l);
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 7
8. Too Much Milk: Locks
thread A thread B
lock(&l) lock(&l)
if (no milk) if (no milk)
buy milk buy milk
unlock(&l) unlock(&l)
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 8
9. Exercise!
Simple multi-threaded program
N = number of iterations
Spawn that many threads to compute
expensiveComputation(i)
Add value (safely!) to total
Use:
pthread_mutex_create, _lock, _unlock
pthread_mutex_t myLock;
pthread_create, pthread_join
pthread_t threads[100];
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 9
11. Solution
#include <pthread.h>
int main (int argc, char * argv[]) {
int total = 0;
int n = atoi(argv[1]);
pthread_mutex_t lock;
// mutex init
void * wrapper (void * x) {
pthread_mutex_init (&lock);
int v = *((int *) x); // allocate threads
delete ((int *) x); pthread_t * threads = new pthread_t[n];
// spawn threads
int res = expComp (v);
for (int i = 0; i<n; i++) {
pthread_mutex_lock (&lock);
// heap allocate args
total += res;
int * newI = new int;
pthread_mutex_unlock (&lock); *newI = i;
return NULL; pthread_create (&threads[i], wrapper,
(void *) newI);
}
}
// join
for (int i = 0; i<n; i++) {
pthread_join (threads[i], NULL);
}
// done
printf (“total = %dn”, total);
return 0;
}
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 11
12. Atomic Operations
But: locks are also variables, updated
concurrently by multiple threads
Lock the lock?
Answer: use hardware-level atomic
operations
Test-and-set
Compare-and-swap
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 12
13. Test&Set Semantics
int testAndset (int& v) {
int old = v;
v = 1;
return old;
}
pseudo-code: red = atomic
What’s the effect of
testAndset(value)
when:
value = 0?
(“unlocked”)
value = 1?
(“locked”)
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 13
14. Lock Variants
Blocking Locks
Spin locks
Hybrids
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 14
15. Blocking Locks
Suspend thread immediately
Lets scheduler execute another thread
Minimizes time spent waiting
But: always causes context switch
void blockinglock (Lock& l) {
while (testAndSet(l.v) == 1) {
sched_yield();
}
}
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 15
16. Spin Locks
Instead of blocking, loop until lock released
void spinlock (Lock& l) {
while (testAndSet(l.v) == 1) {
;
}
}
void spinlock2 (Lock& l) {
while (testAndSet(l.v) == 1) {
while (l.v == 1)
;
}
}
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 16
17. Other Variants
Spin-then-yield:
Spin for some time, then yield
Fixed spin time
Exponential backoff
Queuing locks, etc.:
Ensure fairness and scalability
Major research issue in 90’s
Not used (much) in real systems
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 17
18. “Safety”
Locks can enforce mutual exclusion,
but notorious source of errors
Failure to unlock
Double locking
Deadlock
Priority inversion
not an “error” per se
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 18
19. Failure to Unlock
pthread_mutex_t l;
void square (void) {
pthread_mutex_lock (&l);
// acquires lock
// do stuff
if (x == 0) {
return;
} else {
x = x * x;
}
pthread_mutex_unlock (&l);
}
What happens when we call square()
twice when x == 0?
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 19
20. Scoped Locks with RAI
Scoped Locks:
acquired on entry, released on exit
C++: Resource Acquisition is Initialization
class Guard {
public:
Guard (pthread_mutex_t& l)
: _lock (l)
{ pthread_mutex_lock (&_lock);}
~Guard (void) {
pthread_mutex_unlock (&_lock);
}
private:
pthread_mutex_t& _lock;
};
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 20
21. Scoped Locks: Usage
Prevents failure to unlock
pthread_mutex_t l;
void square (void) {
Guard lockIt (&l);
// acquires lock
// do stuff
if (x == 0) {
return; // releases lock
} else {
x = x * x;
}
// releases lock
}
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 21
22. Double-Locking
Another common mistake
pthread_mutex_lock (&l);
// do stuff
// now unlock (or not...)
pthread_mutex_lock (&l);
Now what?
Can find with static checkers –
numerous instances in Linux kernel
Better: avoid problem
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 22
23. Recursive Locks
Solution: recursive locks
If unlocked:
threadID = pthread_self()
count = 1
Same thread locks increment count
Otherwise, block
Unlock decrement count
Really unlock when count == 0
Default in Java, optional in POSIX
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 23
24. Increasing Concurrency
One object, shared among threads
W R R W R
Each thread is either a reader or a writer
Readers – only read data, never modify
Writers – read & modify data
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 24
25. Single Lock Solution
thread A thread B thread C
lock(&l) lock(&l) lock(&l)
Read data Modify data Read data
unlock(&l) unlock(&l) unlock(&l)
thread D thread E thread F
lock(&l) lock(&l) lock(&l)
Read data Read data Modify data
unlock(&l) unlock(&l) unlock(&l)
Drawbacks of this solution?
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 25
26. Optimization
Single lock: safe, but limits
concurrency
Only one thread at a time, but…
Safe to have simultaneous readers
Must guarantee mutual exclusion for
writers
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 26
27. Readers/Writers
thread A thread B thread C
rlock(&rw) wlock(&rw) rlock(&rw)
Read data Modify data Read data
unlock(&rw) unlock(&rw) unlock(&rw)
thread D thread E thread F
rlock(&rw) rlock(&rw) wlock(&rw)
Read data Read data Modify data
unlock(&rw) unlock(&rw) unlock(&rw)
Maximizes concurrency
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 27
28. R/W Locks – Issues
When readers and writers both queued up,
who gets lock?
Favor readers
Improves concurrency
Can starve writers
Favor writers
Alternate
Avoids starvation
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 28
29. Synch. Operations
Safety:
Locks
Coordination:
Semaphores
Condition variables
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 29
30. Semaphores
What’s a “semaphore” anyway?
A visual signaling
apparatus with
flags, lights, or
mechanically
moving arms, as
one used on a
railroad.
Regulates traffic at critical section
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 30
31. Semaphores in CS
Computer science: Dijkstra (1965)
A non-negative
integer counter with
atomic increment &
decrement.
Blocks rather than
going negative.
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 31
32. Semaphore Operations
P(sem), a.k.a. wait = V(sem), a.k.a. signal =
decrement counter increment counter
If sem = 0, block until Wake 1 waiting process
greater than zero V = “verhogen”
P = “prolagen” (proberen (“increase”)
te verlagen, “try to
decrease”)
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 32
33. Semaphore Example
More flexible than locks
By initializing semaphore to 0,
threads can wait for an event to occur
thread A thread B
// wait for thread B // do stuff, then
// wake up A
sem.wait();
sem.signal();
// do stuff …
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 33
34. Counting Semaphores
Controlling resources:
E.g., allow threads to use at most 5 files
simultaneously
Initialize to 5
thread A thread B
sem.wait(); sem.wait();
// use a file // use a file
sem.signal(); sem.signal();
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 34
35. Synch Problem: Queue
Suppose we have a thread-safe queue
insert(item), remove()
Options for remove when queue empty:
Return special error value (e.g., NULL)
Throw an exception
Wait for something to appear in the queue
Wait = sleep()
But sleep when holding lock…
Goes to sleep
Never wakes up!
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 35
36. Condition Variables
Wait for 1 event, atomically grab lock
wait(Lock& l)
If queue is empty, wait
Atomically releases lock, goes to sleep
Reacquires lock when awakened
notify()
Insert item in queue
Wakes up one waiting thread, if any
notifyAll()
Wakes up all waiting threads
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 36
37. Deadlock
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 37
38. Deadlocks
Deadlock = condition where two
threads/processes wait on each other
thread A thread B
printer->wait(); disk->wait();
disk->wait(); printer->wait();
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 38
39. Deadlocks, Example II
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 39
40. Deadlocks - Terminology
Deadlock
When several threads compete for finite number
of resources simultaneously
Deadlock prevention algorithms
Check resource requests & availability
Deadlock detection (rarely used today)
Finds instances of deadlock when threads stop
making progress
Tries to recover
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 40
41. Rules for Deadlock
All of below must hold:
Mutual exclusion:
1.
Resource used by one thread at a time
Hold and wait
2.
One thread holds resource while waiting for
another; other thread holds that resource
No preemption
3.
Thread can only release resource voluntarily
No other thread or OS can force thread to release
resource
Circular wait
4.
Set of threads {t1, …, tn}: ti waits on ti+1,
tn waits on t1
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 41
42. Circular Waiting
If no way to free resources (preemption),
deadlock
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 42
43. Deadlock Detection
Define graph with vertices:
Resources = {r1, …, rm}
Threads = {t1, …, tn}
Request edge from thread to resource
(rj → ti)
Assignment edge from resource to thread
(rj → ti)
OS has allocated resource to thread
Result:
No cycles no deadlock
Cycle may deadlock
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 43
44. Example
Deadlock or not?
Request edge from
thread to resource ti -> rj
Thread: requested
resource but not
acquired it
Assignment edge from
resource to thread rj -> ti
OS has allocated
resource to thread
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 44
45. Multiple Resources
What if there are multiple
interchangeable instances of a resource?
Cycle deadlock might exist
If any instance held by thread outside
cycle, progress possible when thread
releases resource
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 45
46. Deadlock Detection
Deadlock or not?
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 46
47. Exercise: Resource Allocation Graph
Draw a graph for the following event:
Request edge from thread
to resource ti -> rj
Thread: requested
resource but not
acquired it
Assignment edge from
resource to thread rj -> ti
OS has allocated
resource to thread
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 47
48. Resource Allocation Graph
Draw a graph for the following event:
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 48
49. Detecting Deadlock
Scan resource allocation graph
for cycles & break them!
Different ways to break cycles:
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 49
50. Detecting Deadlock
Scan resource allocation graph
for cycles & break them!
Different ways to break cycles:
Kill all threads in cycle
Kill threads one at a time
Force to give up resources
Preempt resources one at a time
Roll back thread state to before acquiring resource
Common in database transactions
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 50
51. Deadlock Prevention
Instead of detection, ensure at least one
of necessary conditions doesn’t hold
Mutual exclusion
Hold and wait
No preemption
Circular wait
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 51
52. Deadlock Prevention
Mutual exclusion
Make resources shareable (but not all
resources can be shared)
Hold and wait
Guarantee that thread cannot hold one
resource when it requests another
Make threads request all resources they
need at once and release all before
requesting new set
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 52
53. Deadlock Prevention, continued
No preemption
If thread requests resource that cannot be
immediately allocated to it
OS preempts (releases) all resources thread
currently holds
When all resources available:
OS restarts thread
Problem: not all resources can be
preempted
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 53
54. Deadlock Prevention, continued
Circular wait
Impose ordering (numbering) on resources
and request them in order
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 54
55. Avoiding Deadlock
Cycle in locking graph = deadlock
Standard solution:
canonical order for locks
Acquire in increasing order
E.g., lock_1, lock_2, lock_3
Release in decreasing order
Ensures deadlock-freedom,
but not always easy to do
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 55
56. The End
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 56