Accelerate Your C/C++ Applications with Amazon EC2 F1 Instances - CMP402 - re:Invent 2017

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS re:INVENT
Accelerate Your C/C++ Applications
with Amazon EC2 F1 Instances
C M P 4 0 2
N o v e m b e r 3 0 , 2 0 1 7
Gadi Hutt – gadi@amazon.com
Kristopher King

What to Expect From This Session
Overview of FPGA Acceleration and Amazon EC2 F1
Fundamentals of Application Accelerations
The C/C++ OpenCL Cloud FPGA Development Flow
Hands on Workshop:
• Get started with the FPGA Developer AMI
• Learn how to run, profile and optimize hardware accelerators
• Build an FFmpeg plugin which transparently accelerates video encoding using the F1 FPGA platform

CPU: High speed, lower efficiency GPU/FPGA: High throughput, higher efficiency
GPUs and FPGAs provide massive parallelism and higher efficiency than CPUs for
certain categories of applications
More parallelism for higher throughout…
Accelerated Computing Concepts

module filter1 (clock, rst, strm_in, strm_out)
for (i=0; i<NUMUNITS; i=i+1)
always@(posedge clock)
integer i,j; //index for loops
tmp_kernel[j] = k[i*OFFSETX];
FPGA handles compute-
intensive, deeply
pipelined operations
CPU handles the rest
application
How Application Acceleration Works

CPU
Application
DDR-4
Attached
Memory
DDR-4
Attached
Memory
DDR-4
Attached
Memory
DDR-4
Attached
Memory
DDR-4
Attached
Memory
DDR-4
Attached
Memory
DDR-4
Attached
Memory
DDR4
Attached
Memory
PCIe
DDR
Controllers
Launch Instance
and Load AFI
Amazon FPGA
Image (AFI)
An F1 instance
can have any
number of AFIs
An AFI can be
loaded into the
FPGA in seconds
Amazon Machine
Image (AMI)
FPGA Acceleration Using F1
EC2
F1

F1 FPGA Instance Types on AWS
Up to 8 Xilinx UltraScale+ 16nm VU9P FPGA devices in a single instance
Each FPGAs has 4x DDR4 interfaces, with each interface accessing a
16GiB, 72-bit wide, ECC-protected memory, and PCI x16 Gen3 Connectivity
to the host CPU
Instance Size FPGAs DDR4
(GiB)
vCPUs Instance
Memory
(GiB)
NVMe
Instance
Storage (GB)
Network
Bandwidth
f1.2xlarge 1 4 x 16 8 122 1 x 470 Up to 10
Gbps
f1.16xlarge 8 32 x 16 64 976 4 x 940 25 Gbps

Cloud FPGA Accelerations
Genomics
Financial Analytics
Image and Video Processing
Big Data Analytics
Machine Learning
Security
Engineering simulations
Compression

Discover, Procure, Deploy, and Manage Software in the Cloud
AWS Marketplace

Cloud Scaling with EC2 F1

AWS Global Infrastructure
16 Regions – 42 Availability Zones – 74 Edge Locations
Region & Number of Availability Zones
AWS GovCloud (2) EU
Ireland (3)
US West Frankfurt (3)
Oregon (3) London (2)
Northern California (3)
Asia Pacific
US East Singapore (2)
N. Virginia (6), Ohio (3) Sydney (3), Tokyo (3),
Seoul (2), Mumbai (2)
Canada
Central (2) China
Beijing (2)
South America
São Paulo (3)
Announced Regions
China, France, Hong Kong, Sweden, Bahrain
AWS GovCloud (US-East)

AWS Global Infrastructure
16 Regions – 42 Availability Zones – 74 Edge Locations
F 1 F 1
F 1
Region & Number of Availability Zones
AWS GovCloud (2) EU
Ireland (3)
US West Frankfurt (3)
Oregon (3) London (2)
Northern California (3)
Asia Pacific
US East Singapore (2)
N. Virginia (6), Ohio (3) Sydney (3), Tokyo (3),
Seoul (2), Mumbai (2)
Canada
Central (2) China
Beijing (2)
South America
São Paulo (3)
Announced Regions
China, France, Hong Kong, Sweden, Bahrain
AWS GovCloud (US-East)
F1

The FPGA Developer AMI
Use Xilinx SDx to develop, simulate and deploy your custom FPGA logic
Xilinx SDx for custom logic development Virtual JTAG for interactive debugging
Developing Applications for F1

OpenCL C/C++ Development Flow

Alternative Development paths
From Our Partners

The C/C++/OpenCL Development Flow

Developing and Using AFIs

OpenCL Development Environment
• Develop, profile, debug and deploy
accelerated applications on F1
• Concurrent programming of the
host application and FPGA kernels
• Automatic hardware execution
flows
• Built-in debug, profiling and
performance analysis tools

The F1 Shell: Abstracting and Securing
FPGA I/Os
The F1 FPGA Shell allows for faster coding of accelerations
by removing the need to develop I/O related and health
monitoring of the FPGA hardware
• Exposes predefined interfaces to integrate the custom
acceleration kernels
• Standard based interfaces, qualified with the hardware
components
• Designed and optimized for ease of use and performance
Custom Kernels
DMA
DDR
AXI Interfaces
Shell
Xilinx VU9P FPGA

AWS F1 Platform Model
PCIe
Custom Kernels
DMA
DDR
AXI Interfaces
CPU Xilinx FPGA
Custom Application
Drivers
OpenCL Runtime
OpenCL API
User
Application
Code
AWS F1
Platform

Example Application: Video Encoding
PCIe
Custom Kernels
DMA
DDR
AXI Interfaces
Custom Application
Drivers
OpenCL Runtime
OpenCL API
User
Application
Code
AWS F1
Platform
FFMPEG + Plugin HW HEVC Encoder
CPU Xilinx FPGA

The Host-Application Execution Stack
A Master-slave model separates application code from
kernel logic implemented in the AFI
Host application submits work to FPGA kernels using
standard OpenCL API
OpenCL runtime and AWS drivers enable the
communication with the FPGA hardware
CPU
Custom Application
Drivers
OpenCL Runtime
OpenCL API

AFI Creation Flow Overview
Create
Kernels
Compile
Platform
Create
AFI
Create the encrypted
Amazon FPGA Image
Create SDAccel kernels from
C/C++, OpenCL, or RTL
Automatically generate
the FPGA binary

Creating Kernels from C/C++, OpenCL
Parallel Compiler generates high-performance HW kernels
from OpenCL, C, and C++
Advanced optimizations tuned for Xilinx FPGA devices
• Memory partitioning
• DSP block inferencing
• Loop unrolling, loop pipelining
Creates HW kernel with necessary AXI interfaces
Automatically generates PROJECT_NAME.xo file
SDAccel Compiler
.xo
SDAccel
kernel
C++
Open
CL
C

Compiling the Platform (1/2)
The compiler assembles the FPGA design
Automatically instantiates the kernels and F1 shell
Automatically generates DDR interfaces and interconnect
logic
Makes all the necessary connections
SDAccel Compiler (xocc)
Assembled FPGA Design
AWS F1 Shell
.xo .xo
Interconnect logic
kernelkernel

Compiling the Platform (2/2)
SDAccel runs synthesis and place&route on assembled
FPGA design
Generates FPGA binary (.xclbin)
Multiple iterations might be required to meet timing goals
For best results, Kernels should be designed with
recommendations from the UltraFast Design Methodology
Guide for the Vivado Design Suite
xclbin
SDAccel Compiler (xocc)
AWS F1 Shell
Interconnect logic
kernelkernel

Creating an Amazon FPGA Image
Xclbin is converted to an encrypted Amazon FPGA
Image (AFI)
AFIs are created and securely stored by an AWS
backend service
Distributable awsxclbin only contains the AFI id
AFI id is used at runtime to download the AFI from
the Vault into the FPGA
Application developers have no access to acceleration
RTL IP awsxclbin
xclbin
id AFI
id AFI
id AFI
Create AFI
AFI
AFI id
AWS AFI Storage

Developing the Host Application

Host Application Development Flow
Develop
Compile &
Execute
Profile &
Optimize
Use analysis tools to
optimize application
Setup OpenCL in
host application
Run host application
with FPGA kernels

010101010111010101010
101010101010101010100
001010101010101010101
010101010101110101010
101000111100101001111
000101010101010101010
000101000010101011000
101010100100100101010
110001100010101110001
101010100011010101110
101101111100011110000
010001010001111001101
010111101001011010101
001000000101010101110
010001001001010001100
010101101011100101101
010101010101011010100
101010110101010101010
101011100101010100101
010101010110101011001
010101010010101010101
User Application
Code
Developing Application Code
Application written in C/C++, compiled with GCC
OpenCL API used to communicate with FPGA
OpenCL runtime and AWS drivers enable the
communication with the FPGA hardware
Host Application can take many forms
• Standalone executable
• Plugin, shared lib, etc…
• Server for client-server system
FPGA Release
Kernel Execution
FPGA Setup
Kernel Execution
Kernel Execution

OpenCL Testing and Execution Modes
CPU Emulation Hardware Emulation Hardware Execution
Host application runs with a
C/C++ or OpenCL model of the
Kernels
Host application runs with a
simulated RTL model of the
Kernels
Host application runs with
actual FPGA implementation of
the Kernels
Confirm functional correctness
of the system
Test the host / kernel
integration, get performance
estimates
Confirm system runs correctly
and with desired performance
Fastest turnaround time Best debug capabilities Accurate performance results

Profiling and Optimization
EC2 F1 OpenCL provides comprehensive debug and analysis
tools to assess health and performance of the system
Profile Rule Checks highlight performance issues and provide
improvement recommendations
Visualize kernel execution and data transfer efficiency
• Host and device events displayed on a common
timeline
• OpenCL API call sequence
• Kernel execution sequence
• FPGA trace data including AXI transactions, kernel
start/stop, etc.
Xilinx FPGAHost CPU

Hands-On Workshop

Workshop outline
Get started with the FPGA Developer AMI
Connecting to your F1 instance and running hello world
Experience F1 acceleration running Ffmpeg and HEVC encoding
Developing, profiling, and optimizing F1 C/C++/OpenCL applications with SDAccel
Next steps

Workshop and FAQ
Start with introduction module:
https://github.com/awslabs/aws-fpga-app-
notes/blob/master/reInvent17_Developer_Workshop/README.md
FAQ:
Q: Why are precompiled Amazon FPGA images (AFIs) being used in the workshop?
A: To reduce the length of the workshop, we precompiled AFIs and made them public.
Q: Why does the workshop use F1.2xlarge instances?
A: Normally, a developer would use non-F1 instance for development. To avoid switching between
instances during the workshop, we will use F1.2xlarge only.
Q: Why do I need to exit the terminal between modules?
A: Exiting the terminal is a required step that clears the environment variables between modules.

Q&A and What’s Next?
FPGA Accelerated Computing Using Amazon EC2 F1 Instances
Thursday, 5:30 p.m. - Venetian, Level 3
Resources:
Github: https://github.com/aws/aws-fpga
Developer Forum: https://forums.aws.amazon.com/forum.jspa?forumID=243
FPGA Developer AMI:
https://aws.amazon.com/marketplace/pp/B06VVYBLZZ

Thank you!

Accelerate Your C/C++ Applications with Amazon EC2 F1 Instances - CMP402 - re:Invent 2017

Recomendados

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente (20)

Similar a Accelerate Your C/C++ Applications with Amazon EC2 F1 Instances - CMP402 - re:Invent 2017

Similar a Accelerate Your C/C++ Applications with Amazon EC2 F1 Instances - CMP402 - re:Invent 2017 (20)

Más de Amazon Web Services

Más de Amazon Web Services (20)

Accelerate Your C/C++ Applications with Amazon EC2 F1 Instances - CMP402 - re:Invent 2017