This document discusses various technologies for accelerating computation, including managed vs unmanaged code, CPU optimization techniques, GPGPUs like CUDA, Intel Xeon Phi coprocessors, and FPGAs. It provides an overview of these acceleration options and considerations for each, such as portability, safety, and programmability. The talk will focus on accelerating CPU-bound problems and optimization mechanisms like instruction-level parallelism rather than I/O-bound problems.

1. dmitrinesteruk@gmail.com

2. • Quant
• Programmer (C++, .NET, MATLAB)
• Microsoft MVP Visual C# (since 2009)
• Pluralsight course author
(MATLAB, CUDA, D, Boost,…)
• Technical Evangelist @ JetBrains

3. • An overview of available technologies for
computation
• A look at managed vs. unmanaged code
• How to leverage capabilities of x86 architecture
• What COTS and specialized acceleration h/w exists
and how to use it

4. • Native code
• Managed code

5. • More portable. But С++ is also portable provided you do
not use platform-specific things.
• In theory gets optimized for various platforms. In
practice, this isn’t great.
• Does not permit low-level interaction with the processor.
• Additional safety («managed») – array bound checks,
type conversion checks, etc.

6. • Not always portable (e.g. .NET is only partially
portable, excluding UI, WCF, …)
• Typically supports garbage collection.
• Has ways of interacting with native code (JNI,
P/Invoke, C++/CLI).

7. • Developer vs. software productivity?
• Managed languages simpler to use

8. • This talk focuses on CPU bound problems
• Some problems bottleneck on I/O
• SSD made things a lot better
• Optimization mechanisms

9. • Don’t expect CPU clock speed to pick up
• PC/server architecture does not scale
• The only way to accelerate computation is to provide
more entities to compute on.

10. • Instruction-level
• Thread-level
• Machine-level

11. • Via inline assembly
• Via ‘intrinsics’
• Compiler vectorization
• Use magical compilers (e.g. Intel SPMD)

12. • SIMD things

13. • Processing data in an array
• OpenMP
• Intel Threading Building Blocks/
Parallel Patterns Library (MS)

14. • GPGPU
• Expansion boards
• Custom chips

15. • Hardware Platforms – NVIDIA, ATI
• Software platforms for computation – CUDA,
OpenCL, C++ AMP

16. • Typically 2, effectiveness drop-off after that
• PCI bus congestion, but depends on usage patterns

17. • CUDA is the principal commercially successful GPGPU platform
• CUDA is supported by many software manufacturers
(Photoshop, MATLAB, etc.)
• In many domains (e.g. video transcoding), the situation with GPU
leveraging is dire
• In terms of performance, it is thought that CUDA has better
floating-point, AMD better integral math

18. • CUDA is actually a managed technology
• CUDA is not device-independent
• CUDA C is the primary development language

19. • A GPU has several streaming multiprocessors (SM)
• Each SM has lots of processors (SP)
• We can launch a large number of threads in parallel
• Very large number of SPs ensures that even at lower
clock speeds, GPU wins out over CPU

20. • A look at CUDA development

21. • GPU does not support ordinary x86.
• Running several tasks on a GPU is difficult
• Branch divergence – branching code (a simple if)
turns computation from parallel to sequential.

22. • How do you plug in a few CPUs into a
motherboard? You cannot. The architecture doesn’t
scale. (And never will.)
• An alternative is to put a coprocessor on the PCI bus

23. • Commercial coprocessor
implementation from Intel
• PCI board with 60x cores
• Supports x86!!!!!!!!!111111
• Supports different technologies
• Runs its own micro Linux (not a driver)
• Can be used in either independent or offload mode
• Requires special development tools (Intel C++ compiler)

24. • Intel makes a lot of tools for С++ developers
• To work with Xeon Phi, you need

25. • Offload mode
• Native execution mode
• Symmetric execution

26. • Programming the Xeon Phi

27. • 60 processors
• 4 hardware threads per core
• 8Gb memory
• 512-bit SIMD

28. • Same as in ordinary PCs, i.e.,
• OpenMP, MPI
• pthreads
• Other models coming soon

29. • FPGA – Field Programmable
Gate Array
• Design your own CPU
processing mechanic
• Middle ground between
hard-wired ASIC and very
flexible general-purpose CPU
• Uses special hardware description
languages (HDL) – VHDL, Verilog. There are others (SystemC,
OpenCL) and higher-level solutions (e.g., MATLAB, Embeddr).

30. • Intrinsically parallel
• Low-power
• Better scalability
• Not a COTS solution

31. • FPGA lets us offload some tasks from the CPU
• FPGA is a lot less flexible. Not so good for math.
• FPGA is a low-level construct.
• FPGAs are relatively expensive to operate.

32. • FPGAs do not directly compete with ordinary CPUs
• Gain an advantage due to a highly asynchronous
nature
• The goal is to pre-program an FPGA to solve a
single problem very quickly
• E.g., protocol parsing in hardware (so called ‘feed
handler’)

33. • JetBrains is working on the C++ IDE
• And C++ support in ReSharper
• Questions?