Victor Haydin presented on high-performance computing (HPC). He began by defining HPC as using supercomputers and computer clusters to solve advanced computation problems, such as those involving teraflops-level performance. He discussed why HPC is needed for applications in areas like finances, healthcare, fluid dynamics, and genetics. Finally, he outlined some approaches to implementing HPC, such as using commodity hardware, GPU-based systems, distributed and load-balanced architectures, and middleware to handle errors and allow the same code to run on CPUs and GPUs.

1. Victor Haydin
Head of R&D, ELEKS

2. Agenda
1. What is HPC?
2. Why does somebody need it?
3. How to do it?

3. What?

4. Definition
Wikipedia: “High-performance computing (HPC)
uses supercomputers and computer clusters to solve
advanced computation problems.
Today, computer systems approaching the teraflops-
region are counted as HPC-computers.”

5. Definition
advanced
computation
problems

6. Modeling and Simulation

7. Low-latency processing

8. Big Data

9. A.I.

10. Supercomputers
Computer clusters
Teraflops performance

11. HPC systems comparison
100000000
10000000
1000000
100000
10000
HPC
1000
100
10
1
CPU (Intel Ivy Bridge) 100xCPU GPU (NVIDIA Kepler) 100xGPU IBM Sequoia

12. Why?

13. Finances

14. Healthcare

15. Fluid- and Aerodynamics

16. Genetics

17. Computer Vision and Image Processing

18. How?

19. Disclaimer

20. Commodity Hardware

21. Specialized
VS.

22. GPU-based

23. Example 1:
Financial Risk Analysis
Using Monte-Carlo method
On GPGPU

28. Distribute

29. Run

30. Define

31. Store

32. Feed

33. Present

34. Survive

35. High-level architecture

36. Middleware

37. Worker

38. Example 2:
Image Search platform
Using local feature detection
On GPGPU

52. High-level architecture

53. Middleware

54. Load Balancing

57. Unicast
140
• Computation time – 1 second 120
• Sending time – 120 seconds! 100
80
Unicast
• More workers – slower speed
60
40
20
0
9 workers 18 workers

58. Multicast
140
• Computation time – 1 second 120
• Sending time –25 seconds 100
80
Unicast
Multicast
• Almost 5 times faster
60
40
20
0
1 2

59. Middleware

60. Worker

62. ERROR: CUDA ERROR CODE 30 (“UNKNOWN ERROR”)

63. Run same code on CPU and GPU

64. Kernel
CUDA_KERNEL foo(…)
{
CUDA_DEFINE_PARAMS;
// your code here
}
CUDA_CALL(threads, blocks, foo(…))

65. Generated code
// GPU mode // CPU mode
__global__ void foo (…) void foo(…)
{
{
// same code here
// your code here }
}
// LOOP OVER threads and blocks
foo<<<threads, blocks>>>(…) {
foo(…)
}

66. Pros & Cons
• Same code for CPU and • Shared memory
GPU • __syncthreads()
• Debugging
• Range checking
• No CUDA ERROR 30

68. @victor_haydin
linkedin.com/in/victorhaydin
victor.haydin@gmail.com

69. Got a question?
Ask!