This is a 1-day parallel programming workshop, I gave more than a year ago.

1. Migration to Multi-Core Zvi Avraham, CTO [email_address]

3. Multi-Core vs. Many-Core Multi-Core: ≤8 cores/threads Many-Core: >8 cores/threads

12. Parallel Programming Models

15. Flynn’s Taxonomy Single Instruction Multiple Instruction Single Data SISD MISD Multiple Data SIMD MIMD

16. Flynn’s Taxonomy (2)

21. Active Object Design Pattern

22. Rational’s Capsule ROOM – Real-Time Object Oriented Modeling

30. Scatter / Gather

31. Fork / Join Barries – “joins” Parallel Regions Master Thread

34. Map / Reduce

38. CPU Affinity Changing Process and Thread Affinity

44. CPU Affinity – Task Manager

45. CPU Affinity – Task Manager

50. Parallel Programming APIs

53. Capabilities Comparison Intel TBB OpenMP Threads Task level parallelism + + - Data decomposition support + + - Complex parallel patterns (non-loops) + - - Generic parallel patterns + - - Scalable nested parallelism support + - - Built-in load balancing + + - Affinity support - + + Static scheduling - + - Concurrent data structures + - - Scalable memory allocator + - - I/O dominated tasks - - + User-level synch. primitives + + - Compiler support is not required + - + Cross OS support + + -

55. Win32 Threads API It’s assumed, that reader/listener is familiar with basic multithreading and corresponding Win32 API

57. Win32 API Hierarchy for Concurrency Windows OS Job Job Process Primary Thread Thread Process Fiber Fiber

62. Win32 Threads API Example: CreateThread

63. Win32 Threads API: Critical Section

64. CCriticalSection C++ Class

65. CCriticalSection & CLock example

68. Our running Example: The PI program Numerical Integration  4.0 (1+x 2 ) dx =  0 1  F(x i )  x   i = 0 N Mathematically, we know that: We can approximate the integral as a sum of rectangles: Where each rectangle has width  x and height F(x i ) at the middle of interval i. F(x) = 4.0/(1+x 2 ) 4.0 2.0 1.0 X 0.0

70. PI Program: an example static long num_steps = 100000; double step; void main () { int i; double x, pi, sum = 0.0; step = 1.0/(double) num_steps; for (i=1;i<= num_steps; i++){ x = (i-0.5)*step; sum = sum + 4.0/(1.0+x*x); } pi = step * sum; }

71. OpenMP PI Program : Parallel for with a reduction #include <omp.h> static long num_steps = 100000; double step; #define NUM_THREADS 2 void main () { int i; double x, pi, sum = 0.0; step = 1.0/(double) num_steps; omp_set_num_threads(NUM_THREADS); #pragma omp parallel for reduction(+:sum) private(x) for (i=1;i<= num_steps; i++){ x = (i-0.5)*step; sum = sum + 4.0/(1.0+x*x); } pi = step * sum; } OpenMP adds 2 to 4 lines of code

72. OpenMP

77. Intel® TBB Thread Building Blocks C++ Library

78. Calculate PI using TBB

79. Calculate PI in MPI

83. Any Questions?

84. Thank you!