1. aicas Technology
Multicore for Real-Time
and Safety-Critical Software:
Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas
Dr. James J. Hunt, CEO aicas
MultiCoreExpo 2011, 5th May 2011
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
1
2. Agenda
Race conditions Reaching peak
Synchronization performance
CPU affinities
Atomic operations
Multicore scheduling
Memory model
Lock free algorithms
Values out of thin air Compare and Swap
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
2
3. Typical Problems on Multicore
typical code sequence (C/C++ or Java)
int counter;
void increment()
{
counter++;
}
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
3
4. Typical Problems on Multicore
typical code sequence (C/C++ or Java)
int counter;
void increment()
{
counter++;
} r1 = counter;
r2 = r1 + 1;
counter = r2;
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
4
5. Typical Problems on Multicore
typical code sequence (C/C++ or Java)
int counter;
void increment() Thread 1 Thread 2
{
counter++;
} r1 = counter;
r2 = r1 + 1;
counter = r2;
r1 = counter;
r2 = r1 + 1;
counter = r2;
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
5
6. Typical Problems on Multicore
typical code sequence (C/C++ or Java)
int counter;
void increment() Thread 1 Thread 2
{
counter++;
} r1 = counter;
r2 = r1 + 1;
counter = r2;
r1 = counter;
r2 = r1 + 1;
counter = r2;
An increment can get lost!
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
6
7. Typical Problems on Multicore
typical code sequence (C/C++ or Java)
int counter;
void increment()
{
counter++;
}
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
7
8. Typical Problems on Multicore
typical code sequence (C/C++ or Java)
int counter;
void increment()
{
counter++;
}
code lacks synchronization
but on a single core, it practically always works!
on a multicore, chances for failure explode!
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
8
10. Atomic Operations
What is the result of
int a, b; /* 32 bit, initially 0 */
Thread 1 Thread 2
b = a; a = 1;
?
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
10
11. Atomic Operations
What is the result of
int a, b; /* 32 bit, initially 0 */
Thread 1 Thread 2
b = a; a = 1;
?
b == 0
b == 1
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
11
12. Atomic Operations
What is the result of
long a, b; /* 64 bit, initially 0 */
Thread 1 Thread 2
b = a; a = 1;
?
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
12
13. Atomic Operations
What is the result of
long a, b; /* 64 bit, initially 0 */
Thread 1 Thread 2
b = a; a = 1;
?
b == 0
b == 1
b == 4294967296
b == 4294967295
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
13
14. Cache Structure
CPUs may have local caches for performance
Main Memory
L2 Cache L2 Cache
L1 Cache L1 Cache L1CPU0
Cache L1CPU1
Cache
CPU0 CPU1 CPU2
CPU0 CPU1
CPU3
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
14
15. Cache Structure
Main Memory
L2 Cache L2 Cache
L1 Cache L1 Cache L1 Cache L1 Cache
CPU0 CPU1 CPU2 CPU3
Modifications do not become visible immediately
Modifications may be re-ordered
Reads may refer to outdated (cached) data
Reads may be re-ordered
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
15
16. Typical Problems on Multicore
polling update
long counter;
[..]
do
{
doSomething();
}
while (counter < MAX);
counter is incremented by parallel thread
on a Multicore, changes to counter may not
become visible!
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
16
17. Typical Problems on Multicore
polling update
long counter;
[..]
do
{
doSomething();
}
while (counter < MAX);
counter is incremented by parallel thread
on a Multicore, changes to counter may not
become visible!
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
17
20. Understanding the Memory Model
Memory model specifies what optimizations are
permitted by the compiler or underlying
hardware
C/C++ programs have undefined semantics in
case of race conditions
Java defines a strict memory model
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
20
21. Java's Memory Model
ordering operations are
synchronized block
accessing a volatile variable
The presence of an ordering operation
determines the visible state in shared memory
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
21
22. Java's Memory Model: Defined Order
all reads are completed before
entering synchronized block, or
reading a volatile variable
read fence
all writes are completed before
exiting a synchronized block, or
writing a volatile variable
write fence
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
22
23. Java's Memory Model: Data Races
data races are not forbidden in Java
you can use shared memory variables
your code has to tolerate optimizations
examples
collecting debugging / profiling information
useful if occasional errors due to data races are
tolerable
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
23
24. Memory Model Example
Shared memory communication
Ptr p;
boolean p_valid;
Thread 1
p = new Ptr();
p_valid = true;
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
24
25. Memory Model Example
Shared memory communication
Ptr p;
boolean p_valid;
Thread 1 Thread 2
p = new Ptr();
p_valid = true;
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
25
26. Memory Model Example
Shared memory communication
Ptr p;
boolean p_valid;
Thread 1 Thread 2
p = new Ptr(); if (p_valid)
p_valid = true; p.call();
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
26
27. Memory Model Example
Shared memory communication
Thread 1 Thread 2
p = new Ptr(); if (p_valid)
p_valid = true; p.call();
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
27
28. Memory Model Example
Shared memory communication
Thread 1 Thread 2
p = new Ptr(); if (p_valid)
p_valid = true; p.call();
What may happen:
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
28
29. Memory Model Example
Shared memory communication
Thread 1 Thread 2
p = new Ptr(); if (p_valid)
p_valid = true; p.call();
What may happen:
t1 = new Ptr();
t2 = true;
p_valid = t2;
p = t1;
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
29
30. Memory Model Example
Shared memory communication
Thread 1 Thread 2
p = new Ptr(); if (p_valid)
p_valid = true; p.call();
What may happen:
t1 = new Ptr();
t2 = true;
p_valid = t2;
p = t1;
Writes reordered!
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
30
31. Memory Model Example
Shared memory communication
Thread 1 Thread 2
p = new Ptr(); if (p_valid)
p_valid = true; p.call();
What may happen:
t1 = new Ptr();
t2 = true;
p_valid = t2;
p = t1;
Writes reordered!
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
31
32. Memory Model Example
Shared memory communication
Thread 1 Thread 2
p = new Ptr(); if (p_valid)
p_valid = true; p.call();
What may happen:
t1 = new Ptr(); t3 = p;
t2 = true; if (p_valid)
p_valid = t2; t3.call();
p = t1;
Writes reordered! Reads reordered!
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
32
33. Memory Model Example
Shared memory communication
Thread 1 Thread 2
p = new Ptr(); if (p_valid)
p_valid = true; p.call();
What may happen:
t1 = new Ptr(); t3 = p;
t2 = true; if (p_valid)
p_valid = t2; t3.call();
p = t1;
Writes reordered! Reads reordered!
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
33
34. Example Use of Java's Memory Model
Shared memory communication
volatile Ptr p;
volatile boolean p_valid;
Thread 1 Thread 2
p = new Ptr(); if (p_valid)
p_valid = true; p.call();
in Java
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
34
35. Example Use of Java's Memory Model
Shared memory communication
volatile Ptr p;
volatile boolean p_valid;
Thread 1 Thread 2
p = new Ptr(); if (p_valid)
p_valid = true; p.call();
in Java
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
35
36. Example Use of Java's Memory Model
Shared memory communication
volatile Ptr p;
volatile boolean p_valid;
Thread 1 Thread 2
p = new Ptr(); if (p_valid)
p_valid = true; p.call();
in Java
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
36
37. Example Use of C's Memory Model
Shared memory communication
volatile Obj *p;
volatile boolean p_valid;
Thread 1 Thread 2
p = malloc(..); if (p_valid)
p_valid = TRUE; p>f = ..;();
in C?
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
37
38. Example Use of C's Memory Model
Shared memory communication
volatile Obj *p;
volatile boolean p_valid;
Thread 1 Thread 2
p = malloc(..); if (p_valid)
p_valid = TRUE; p>f = ..;();
in C?
CPU may reorder memory accesses!
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
38
39. Example Use of C's Memory Model
Shared memory communication
volatile Obj *p;
volatile boolean p_valid;
Thread 1 Thread 2
p = malloc(..); if (p_valid)
p_valid = TRUE; p>f = ..;();
in C?
CPU may reorder memory accesses!
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
39
40. Example Use of C's Memory Model
Shared memory communication
volatile Obj *p;
volatile boolean p_valid;
Thread 1 Thread 2
p = malloc(..); if (p_valid)
p_valid = TRUE; p>f = ..;();
in C?
CPU may reorder memory accesses!
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
40
41. Example Use of C's Memory Model
Shared memory communication
volatile Obj *p;
volatile boolean p_valid;
Thread 1 Thread 2
p = malloc(..); if (p_valid)
p_valid = TRUE; p>f = ..;
How to fix it?
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
41
42. Example Use of C's Memory Model
Shared memory communication
volatile Obj *p;
volatile boolean p_valid;
Thread 1 Thread 2
p = malloc(..); if (p_valid)
asm volatile( p>f = ..;
"sfence":::"memory");
p_valid = TRUE;
How to fix it? Add memory fences!
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
42
43. Example Use of C's Memory Model
Shared memory communication
volatile Obj *p;
volatile boolean p_valid;
Thread 1 Thread 2
p = malloc(..); if (p_valid)
asm volatile( {
"sfence":::"memory"); asm volatile(
p_valid = TRUE; "lfence":::"memory");
p>f = ..;
}
How to fix it? Add memory fences!
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
43
44. Example Use of C's Memory Model
Shared memory communication
volatile Obj *p;
volatile boolean p_valid;
Thread 1 Thread 2
p = malloc(..); if (p_valid)
asm volatile( {
"sfence":::"memory"); asm volatile(
p_valid = TRUE; "lfence":::"memory");
p>f = ..;
}
How to fix it? Add memory fences!
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
44
45. Out of Thin Air
imagine this code
int x = 0, n = 0;
Thread 1 Thread 2
for (i=0; i<n; i++) x = 42;
x += f(i); print(x);
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
45
46. Out of Thin Air
imagine this code
int x = 0, n = 0;
Thread 1 Thread 2
for (i=0; i<n; i++) x = 42;
x += f(i); print(x);
can only print 42 in Java
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
46
47. Out of Thin Air: Introduction of Writes
loop optimization in C/C++
int x = 0, n = 0;
Thread 1 Thread 2
tmp = x;
for (i=0; i<n; i++) x = 42;
tmp += f(i);
x = tmp;
print(x);
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
47
48. Out of Thin Air: Introduction of Writes
loop optimization in C/C++
int x = 0, n = 0;
Thread 1 Thread 2
tmp = x;
for (i=0; i<n; i++) x = 42;
tmp += f(i);
x = tmp;
print(x);
can print 0 in C/C++
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
48
49. Out of Thin Air
imagine this code
int x = 0, y = 0;
Thread 1 Thread 2
r1 = x; r2 = y;
y = r1; x = r2;
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
49
50. Out of Thin Air
imagine this code
int x = 0, y = 0;
Thread 1 Thread 2
r1 = x; r2 = y;
y = r1; x = r2;
Expected result
x == 0; y == 0;
Only possible result in Java
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
50
51. Out of Thin Air: Optimization in C/C++
imagine this code
int x = 0, y = 0;
Thread 1 Thread 2
y = 42; r2 = y;
r1 = x; x = r2;
if (r1 != 42)
y = r1;
Possible results in upcoming C++ MM
x == 42; y == 42;
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
51
52. Performance on Multicore: Example
Single core application, 3 threads
All threads synchronize frequently on same lock
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
52
53. Performance on Multicore: Example
Single core application, 3 threads
All threads synchronize frequently on same lock
while (true)
{
synchronized (lock)
{
counter++;
}
doSomething();
}
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
53
54. Performance on Multicore: Example
Single core application, 3 threads
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
54
55. Performance on Multicore: Example
Single core application, 3 threads
On a multicore
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
55
56. Performance on a Multicore
Frequent synchronization can kill the performance
Typical non-RTOS will use heuristics to improve
average performance
spin-lock for a short time
blocking for longer periods
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
56
57. Performance on a Multicore
These heuristics introduce priority-inversion and
generally destroy predictability
A typical semaphore implementation
does not take thread priority into account
does not limit worst-case-execution-time
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
57
58. CPU Affinities
OSes provide APIs to lock threads certain CPUs
This limits the decision space of the RTOS
a global scheduler for n CPUs would always run the n
highest priority threads
with affinities, this may not be possible, e.g., if the two
highest priority threads are locked to the same CPU
CPU affinities can introduce priority inversion!
So what are CPU affinities good for?
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
58
59. CPU Affinities: No More interrupts
Locking interrupts to a dedicated CPU
protects all other CPUs from interrupts, and
invalidated cashes
WCETA is simplified considerably
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
59
60. CPU Affinities: No More interrupts
Locking interrupts to a dedicated CPU
protects all other CPUs from interrupts, and
invalidated cashes
WCETA is simplified considerably Interrupt
CPU0 non-RT Task
CPU1 A A A A A A A A A A A RT Task
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
60
61. CPU Affinities: Separating Threads
Locking thread A to one CPU and threads B, C, ...
to other CPUs may increase A's performance.
A will not be preempted by B, C, ..
A will not see its caches invalidated by B, C, ...
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
61
62. CPU Affinities: Separating Threads
Locking thread A to one CPU and threads B, C, ...
to other CPUs may increase A's performance.
A will not be preempted by B, C, ..
A will not see its caches invalidated by B, C, ...
CPU0 B B B B B non-RT Task
CPU1 A A A A A A A A A A A RT Task
CPU2 B C B C B C
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
62
63. CPU Affinities: Grouping Threads
Locking threads A and B to the same CPU
will increase shared memory communication between
A and B
will avoid performance degradation on locking
will enable simple scheduling analysis (RMA)
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
63
64. CPU Affinities: Grouping Threads
Locking threads A and B to the same CPU
will increase shared memory communication between
A and B
will avoid performance degradation on locking
will enable simple schedule feasibility analysis (RMA)
CPU0 C D C DC D E G D RT Task A
CPU1 A B A BA B A RT Task B
CPU2 E E G E E C E Other Task
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
64
65. CPU Affinities: Difficult Decisions
what if A and B both computation intensive and
both access the same shared memory?
How can we use idle time?
...
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
65
66. Multicore Scheduling for Realtime
A pure priority based scheduler is not sufficient:
Imagine three tasks A, B, C on 2 CPUS
A
B
C
Release Deadline
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
66
67. Multicore Scheduling for Realtime
Priorities will cause deadline miss
CPU0 A pri=10 C pri=9
CPU1 B pri=10
Release Deadline
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
67
68. Multicore Scheduling for Realtime
Priorities will cause deadline miss
CPU0
CPU1
A pri=10
B pri=10
C pri=9
Release Deadline
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
68
69. Multicore Scheduling for Realtime
CPU affinities do not help
CPU0 A pri=10, {CPU0} C pri=9
CPU1 B pri=10, {CPU1}
Release Deadline
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
69
70. Multicore Scheduling for Realtime
Round robin could help
CPU0 A C B A C B
CPU1 B A C B A C
Release Deadline
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
70
71. Multicore Scheduling for Realtime
Round robin could help
CPU0 A C B A C B
CPU1 B A C B A C
Release Deadline
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
71
72. Performance on Multicore
Synchronization is expensive
Can synchronization be avoided?
Can lock free algorithms be used?
Use compare and swap (CAS) instructions instead
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
72
73. Lock Free Algorithms
Typical code sequence
do
{
x = counter;
result = CAS(counter,x,x+1);
}
while (result != x);
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
73
74. Compare-and-Swap Issues
Typical code sequence
do
{
x = counter;
result = CAS(counter,x,x+1);
}
while (result != x);
What is the WCET? ∞?
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
74
75. Compare-and-Swap Issues
On dual core:
Typical code sequence
do
{
x = counter;
result = CAS(counter,x,x+1);
}
frequency
while (result != x);
What is the WCET? ∞?
# iterations
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
75
76. Lock Free Library Code
Using libraries helps
AtomicInteger counter = new AtomicInteger();
void increment()
{
(void)counter.incrementAndGet();
}
Code is easier and safer
Hand made lock free algorithms are not for
normal application development
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
76
77. Compare-and-Swap Solutions
One way state changes, without retry
Bounded number of retries
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
77
78. One Way State Changes Using CAS
Example code
for (i=0; i<N; i++)
{
new Thread()
{
public void run()
{
CAS(state,INIT,STARTING);
[..]
}
}.start();
}
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
78
79. Bounding Retries for CAS
introduce long enough code sections in between
2 compare-and-swap loops
then, if a retry is required, one other CPU was
successful
after n-1 conflicts, one can be sure that all other
CPUs are outside the CAS loop
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
79
80. Bounding Retries for CAS: Example
AtomicInteger counter = new AtomicInteger();
static final int GRANULARITY = 64;
[..]
new Thread(){
int local_counter;
public void incCounter() {
local_counter++;
if (local_counter >= GRANULARITY) {
local_counter = 0;
counter.addAndSet(GRANULARITY);
}
}
}.start();
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
80
81. Measurements
Number of CAS tries is bounded:
1E+11
1E+10
1E+9
1E+8
1
1E+7 2
3
1E+6
4
1E+5 5
6
1E+4
7
1E+3 8
1E+2
9
1E+1
1E+0
1E-1
alloc free sweep available Memory
On 8-CPU x86 system (Linux)
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
81
82. CAS Lists
List modifaction using CAS, single linked list
A B C
head next next next
data data data
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
82
83. CAS Lists
Add
A B C
head next next next
data data data
X
next
data
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
83
84. CAS Lists
Add
A B C
head next next next
data data data
X
next
data
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
84
85. CAS Lists
Add: CAS(head,A,X)
A B C
head next next next
data data data
X
next
data
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
85
86. CAS Lists
Add: CAS(head,A,X)
A B C
head next next next
data data data
X
next
data
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
86
87. CAS Lists
Add
A B C
head next next next
data data data
X
next
data
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
87
88. CAS Lists
Remove
A B C
head next next next
data data data
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
88
89. CAS Lists
Remove: CAS(head,A,B)
A B C
head next next next
data data data
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
89
90. CAS Lists
Remove: CAS(head,A,B)
A B C
head next next next
data data data
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
90
91. CAS Lists
Remove
B C
head next next
data data
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
91
92. CAS Lists, ABA Problem
Now, consider concurrent modifications:
Thread 1 head
A
next
B
next
C
next
Thread 2
data data data
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
92
93. CAS Lists, ABA Problem
Thread 1 head
A
next
B
next
C
next
Thread 2
remove: data data data
CAS(head,A,B)
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
93
94. CAS Lists, ABA Problem
Thread 1 head
A
next
B
next
C
next
Thread 2
remove: data data data
pre
e
mp
ted
CAS(head,A,B)
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
94
95. CAS Lists, ABA Problem
Thread 1 head
A
next
B
next
C
next
Thread 2
remove: data data data
remove:
CAS(head,A,B)
pre
e
mp
ted
CAS(head,A,B)
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
95
96. CAS Lists, ABA Problem
Thread 1 head
A
next
B
next
C
next
Thread 2
remove: data data data
remove:
CAS(head,A,B)
B C
pre
head next next
data data
e
mp
ted
CAS(head,A,B)
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
96
97. CAS Lists, ABA Problem
Thread 1 head
A
next
B
next
C
next
Thread 2
remove: data data data
remove:
CAS(head,A,B)
B C
pre
head next next
data data
remove:
e
CAS(head,B,C)
mp
ted
CAS(head,A,B)
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
97
98. CAS Lists, ABA Problem
Thread 1 head
A
next
B
next
C
next
Thread 2
remove: data data data
remove:
CAS(head,A,B)
B C
pre
head next next
data data
remove:
e
CAS(head,B,C)
mp
C
head next
t
data
ed
CAS(head,A,B)
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
98
99. CAS Lists, ABA Problem
Thread 1 head
A
next
B
next
C
next
Thread 2
remove: data data data
remove:
CAS(head,A,B)
B C
pre
head next next
data data
remove:
e
CAS(head,B,C)
mp
C
head next
t
data
ed
add A:
CAS(head,C,A)
CAS(head,A,B)
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
99
100. CAS Lists, ABA Problem
Thread 1 head
A
next
B
next
C
next
Thread 2
remove: data data data
remove:
CAS(head,A,B)
B C
pre
head next next
data data
remove:
e
CAS(head,A,B)
mp
C
head next
t
data
ed
A C add A:
head next next CAS(head,C,A)
data data
CAS(head,A,B)
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
100
101. CAS Lists, ABA Problem
Thread 1 head
A
next
B
next
C
next
Thread 2
remove: data data data
remove:
CAS(head,A,B)
B C
pre
head next next
data data
remove:
e
CAS(head,A,B)
mp
C
head next
t
data
ed
A C add A:
head next next CAS(head,C,A)
data data
CAS(head,A,B) B A C
head next next next
data data data
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
101
102. CAS Lists, ABA Problem
Thread 1 head
A
next
B
next
C
next
Thread 2
remove: data data data
remove:
CAS(head,A,B)
B C
pre
head next next
data data
remove:
e
CAS(head,A,B)
mp
C
head next
t
data
ed
A C add A:
head next next CAS(head,C,A)
data data
CAS(head,A,B) B A C
head next next next
data data data
B was re-introduced!
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
102
103. ABA Problem: Solutions
Non solution: In C: free() after remove
Reason: newly allocated block may be same
Solution: In Java: only add new references
Reason: GC ensures old values no longer visible
Solution: Sync all threads before reuse
Example: Phase 1 moves elements from List1 to
List2, Phase 2 moves elements back
Solution: Use 64-/128-bit CAS and mod. counter
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
103
104. ABA Solved via Modification Counter
Thread 1 head: 42
A
next
B
next
C
next
Thread 2
remove: data data data
pre
e
mp
ted
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
104
105. ABA Solved via Modification Counter
Thread 1 head: 42
A
next
B
next
C
next
Thread 2
remove: data data data
remove:
CAS(head,A:42,
B:43)
pre
e
mp
ted
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
105
106. ABA Solved via Modification Counter
Thread 1 head: 42
A
next
B
next
C
next
Thread 2
remove: data data data
remove:
CAS(head,A:42,
B C
B:43)
head: 43 next next
data data
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
106
107. ABA Solved via Modification Counter
Thread 1 head: 42
A
next
B
next
C
next
Thread 2
remove: data data data
remove:
CAS(head,A:42,
B C
B:43)
head: 43 next next
data data
remove:
C CAS(head,B:43,
head: 44 next C:44)
data
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
107
108. ABA Solved via Modification Counter
Thread 1 head: 42
A
next
B
next
C
next
Thread 2
remove: data data data
remove:
CAS(head,A:42,
B C
B:43)
head: 43 next next
data data
remove:
C CAS(head,B:43,
head: 44 next C:44)
data
A C add A:
head: 45 next next CAS(head,C:44,
data data A:45)
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
108
109. ABA Solved via Modification Counter
Thread 1 head: 42
A
next
B
next
C
next
Thread 2
remove: data data data
remove:
CAS(head,A:42,
B C
B:43)
head: 43 next next
data data
remove:
C CAS(head,B:43,
head: 44 next C:44)
data
A C add A:
head: 45 next next CAS(head,C:44,
CAS(head,A:42, data data A:45)
B:43)
retry!
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
109
110. ABA Solved via Modification Counter
Thread 1 head: 42
A
next
B
next
C
next
Thread 2
remove: data data data
remove:
CAS(head,A:42,
B C
B:43)
head: 43 next next
data data
remove:
C CAS(head,B:43,
head: 44 next C:44)
data
A C add A:
head: 45 next next CAS(head,C:44,
CAS(head,A:42, data data A:45)
B:43) C
retry! head: 46 next
CAS(head,A:45, data
C:46)
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
110
111. Conclusion
Code that runs well on single CPU may fail on
multicore
Clear semantics of concurrent code is required
for functional correctness
Cost of locking may be prohibitive
CPU affinities may help, but it is difficult to make
the application more efficient
Lock free code is very hard to get right
A reliable memory model and good concurrent
libraries are basis to avoid pitfalls.
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
111
112. Thank you.
Please come see us at booth 2306
Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
112