1. Creating Your Own PCI Express
System with FPGA Devices
Learning Zone @
Embedded World
© 2010 Altera Corporation - Public

2. PCI Express Overview
PCI Express (Peripheral CPU CPU
Component Interconnect PCI
Express) is a computer Graphics
Express North-bridge
(high
Memory
expansion standard BW, low
latency)
introduced by Intel in 2004 Serial ATA
HDD
− Officially abbreviated as PCIe PCI PCI
Express
(PCI-E is also commonly used) Mbyte down
devices South-bridge
(I/O bridge)
PCIe replaces PCI, PCI-X, USB 2.0
and AGP Gigabit
Ethernet*
LPC External
PCIe complements SERDES- Add ins
modules
LPC
based bus interface to the
CPU
© 2010 Altera Corporation - Public
ALTERA, ARRIA, CYCLONE, HARDCOPY, MAX, MEGACORE, NIOS, QUARTUS & STRATIX are Reg. U.S. Pat. & Tm. Off.
and Altera marks in and outside the U.S.
2

3. PCIe Functional Elements
Root complex CPU CPU
− Connects host CPU/memory
complex to PCIe hierarchy
− One or more downstream ports
(= rootports) Root Memory
complex
− Rootport discovers, initializes, and
enumerates PCIe topology
Downstream port
Switch PCIe link
Upstream port
− Assembly of logical PCIe-to-PCIe
bridges
− One upstream port directed towards Switch
root complex
− One or more downstream ports
Endpoint PCIe
PCIe Legacy Switch
− Legacy and native endpoints e.g. to PCI-X
endpoint endpoint
bridge
terminal point to PCI, USB,
InfiniBand, Fibre Channel, Ethernet…
PCI-X expansion slot
© 2010 Altera Corporation - Public
ALTERA, ARRIA, CYCLONE, HARDCOPY, MAX, MEGACORE, NIOS, QUARTUS & STRATIX are Reg. U.S. Pat. & Tm. Off.
and Altera marks in and outside the U.S.
3

4. PCIe Technology Timeline
Raw bit rate Link BW BW/lane/way
PCIe 1.x 2.5 GT/s 2 Gbps ~250 MBps Future
PCIe 2.0 5.0 GT/s 4 Gbps ~500 MBps FPGA
PCIe 3.0 8.0 GT/s 8 Gbps ~1 GBps Releases
Total Bandwidth / Gbps
100
90 • Gen3: 8 GT/s signaling
80 • Lower latencies, improved PM
• Enhanced software model
70
60
50 PCIe Gen2
40 @ 5 GT/s
30 PCIe Gen1
@ 2.5 GT/s
20
10 PCI/PCI-X
0
1999 2001 2003 2005 2007 2009 2011 2013
Note:
Link BW = 0.8 x (raw bit rate) in Gen1/Gen2 due to 8B/10B encoding
Dotted line is for projected numbers, Total bandwidth = Link Bandwithx8 Lanes
© 2010 Altera Corporation - Public
ALTERA, ARRIA, CYCLONE, HARDCOPY, MAX, MEGACORE, NIOS, QUARTUS & STRATIX are Reg. U.S. Pat. & Tm. Off.
and Altera marks in and outside the U.S.
4

5. PCIe Support in Altera Devices
Hard IP Gen1/2 Soft IP Gen1
Device
PCI Express PCI Express
Supported Supported
configuration configuration
Cyclone IV GX
x1, x2, x4 x1, x4
(PCIe Gen1 only) 1 per device
Arria II GX
x1, x4, x8 x1, x4, x8
(PCIe Gen1 only) 1 per device
Stratix IV GX x1, x4, x8 x1, x4, x8
2 to 4 per device
HardCopy IV GX x1, x4, x8 x1, x4, x8
1 to 2 per device
Stratix IV GT x1, x4, x8 x1, x4, x8
2 to 4 per device
Save up to 40K LEs and no License
Fees with Hard IP
© 2010 Altera Corporation - Public
ALTERA, ARRIA, CYCLONE, HARDCOPY, MAX, MEGACORE, NIOS, QUARTUS & STRATIX are Reg. U.S. Pat. & Tm. Off.
and Altera marks in and outside the U.S.
5

6. PCI Express Hard IP Block
Performs transaction, data link, PHYMAC layer and functionality
Supports
− PCI Express Gen 1 and Gen 2
− x1, x2, x4 & x8 lane configurations
− Root port and endpoint applications
Connects directly to embedded transceivers using internal PIPE interface
− Shared by two adjacent transceiver blocks
Enabled through the PCI Express Compiler Wizard
FPGA
PCI Express Hard IP Block
Embedded
Transceiver Block
PIPE Transceiver Block n
Transaction Data Link PHYMAC
Transceiver Block 2
Layer Layer Layer
Transceiver Block 1 To / from
Transceiver Block 0 Slot or cable
© 2010 Altera Corporation - Public
ALTERA, ARRIA, CYCLONE, HARDCOPY, MAX, MEGACORE, NIOS, QUARTUS & STRATIX are Reg. U.S. Pat. & Tm. Off.
and Altera marks in and outside the U.S.

7. PCI Express Hard IP Diagram
Configurable maximum
payload size
Transceiver Block PCI Express Hard IP Block PLD Fabric − 128, 256, 512, 1024*, or 2048*
bytes
Clock & Reset Selection
1 or 2* Virtual Channels (VC)
PMA PCS 16-Kbyte receive buffer per
VC
TL
FPGA Interface
PCI Express Interface Adapter 2-Kbyte transmit retry buffer
PIPE
Application
Transceiver Block Protocol Stack Layer 128-bit or 64-bit application
datapath width
Local Interrupt support (legacy, MSI
Mgmt IF & MSI-X)
PMA PCS (LMI)
Retry VC RX Advanced error reporting
Buffer Buffer (AER) support
Power management support
Local management interface
(LMI) to access configuration
registers
Status & debug interface
* Stratix IV GX/GT only
© 2010 Altera Corporation - Public
ALTERA, ARRIA, CYCLONE, HARDCOPY, MAX, MEGACORE, NIOS, QUARTUS & STRATIX are Reg. U.S. Pat. & Tm. Off.
and Altera marks in and outside the U.S.

8. PCI Express Solutions
© 2010 Altera Corporation - Public

9. Industrial PC or Controller
USB
Processor DSP
Graphics (Intel Atom) (Optional)
Hard Disk
PCIe RapidIO
DVI
Isolation
and RJ-45
Industrial
Motor Control Ethernet
Power Stage Isolation
PWM and RJ-45
Sensor Camera ADC/DAC OPTO I/O
© 2010 Altera Corporation - Public
ALTERA, ARRIA, CYCLONE, HARDCOPY, MAX, MEGACORE, NIOS, QUARTUS & STRATIX are Reg. U.S. Pat. & Tm. Off.
and Altera marks in and outside the U.S.

10. CNC EtherCAT Drives Demo
CNC application runs on 3S software and positions motors
via an EtherCAT Industrial Ethernet network connected to
an LCD for data display and I/O control options
© 2010 Altera Corporation - Public
ALTERA, ARRIA, CYCLONE, HARDCOPY, MAX, MEGACORE, NIOS, QUARTUS & STRATIX are Reg. U.S. Pat. & Tm. Off.
and Altera marks in and outside the U.S.

11. PCI Express Testing
© 2010 Altera Corporation - Public

12. PCIe Hard IP Testing
Altera PCIe Gen2 development card
Interoperability testing
− Compatibility/functionality
with chipsets
Core generator with
verifying payloads (built in
DMA engine)
− Test for PCIe compliance using
PCI-SIG tests
Performance testing
− Testing throughput of PCIe link
PCIe Gen2 motherboard
− Stress test using the chain DMA
architecture PCle Gen2 platforms
Test configurations Motherboard Chipset Processor
− Modes
AMD Asus M3A78-T 790GX/SB750 AMD790GX
PCI Express Gen1 x1, x4, x8
PCI Express Gen2 x1, x4, x8 Intel Asus P5Q-EN Intel-645 LGA775
Intel DX58S0 Tylersburg Core I7
© 2010 Altera Corporation - Public
ALTERA, ARRIA, CYCLONE, HARDCOPY, MAX, MEGACORE, NIOS, QUARTUS & STRATIX are Reg. U.S. Pat. & Tm. Off.
and Altera marks in and outside the U.S.
12

13. Preliminary Results of PCI Express Hard IP
Bandwidth Testing – Tylersburg Chipset
Simultaneous read/
HIP configuration DMA write DMA read
write
HIP:Gen1:x8:128 bits 1,691 MBps 1,695 MBps 1,683/1,449 MBps
HIP:Gen1:x8:64 bits 1,774 MBps 1,522 MBps 1,524/1,631 MBps
HIP:Gen2:x4:64 bits 1,772 MBps 1,528 MBps 1,528/1,634 MBps
HIP:Gen2:x4:128 bits 1,670 MBps 1,613 MBps 1,607/1,402 MBps
HIP:Gen2:x8:256 bits 3,400 MBps 3,300 MBps 3,173/2,155 MBps
Theoretical maximum bandwidth Demo Here
− Gen2 x8: 4,000 MBps
Stratix IV GX hard IP is close to theoretical maximum now
© 2010 Altera Corporation - Public
ALTERA, ARRIA, CYCLONE, HARDCOPY, MAX, MEGACORE, NIOS, QUARTUS & STRATIX are Reg. U.S. Pat. & Tm. Off.
and Altera marks in and outside the U.S.
13

14. PCI Express Development
Kits
© 2010 Altera Corporation - Public

15. PCI Express Development Kits
Stratix IV GX FPGA Arria II GX FPGA Cyclone IV GX Transceiver
Development Kit Development Kit Starter Kit
Applications Applications Applications
−Develop and test PCI −Develop and test PCI −Develop and test PCI
Express Gen2 (to x8 lane) Express Gen1 (to x4 lane) Express Gen1 (to x1 lane)
−Develop and test memory −Develop and test memory −Develop and test memory
subsystems subsystems subsystems
−System Development −System Development −FPGA prototyping & power
−Supported other protocols −Supported other protocols
measurement
via (HSMC) Daughter Cards via (HSMC) Daughter Cards −Testing transceiver signal
quality to 2.5Gbps
DK-DEV-4SGX230N/C2 $4,495 DK-DEV-2AGX125N $1,495 DK-START-4CGX15N $395.00
Now Now Coming
shipping shipping March 2010
© 2010 Altera Corporation - Public
ALTERA, ARRIA, CYCLONE, HARDCOPY, MAX, MEGACORE, NIOS, QUARTUS & STRATIX are Reg. U.S. Pat. & Tm. Off.
and Altera marks in and outside the U.S.

16. Altera PCI Express Solutions Summary
Industry-leading solutions
− Stratix IV GX FPGA is industry’s only shipping FPGA solution with
hard IP support for PCIe Gen2
− Arria II GX FPGA is the industry’s first low-cost 40-nm FPGA with hard IP
support for PCIe Gen1 x1, x4, x8
− Cyclone IV GX is the industry’s lowest-cost FPGA with hard IP support for
PCIe Gen1 x1, x4
Low-risk, hardware-verified solutions
− PCI-SIG compliance workshops
− Interoperability with multiple ASSP vendors
− 5 generations of transceiver-based FPGAs with PCI Express support
− Development kits/demo boards
© 2010 Altera Corporation - Public
ALTERA, ARRIA, CYCLONE, HARDCOPY, MAX, MEGACORE, NIOS, QUARTUS & STRATIX are Reg. U.S. Pat. & Tm. Off.
and Altera marks in and outside the U.S.
16

17. PCI Express Quickstart Guide
with SOPC Builder
PCI Express Endpoint using SOPC
Builder Lab
© 2010 Altera Corporation - Public

18. Example PCIe-SOPC Builder System
Memory DMADMA DMA
© 2010 Altera Corporation - Public
ALTERA, ARRIA, CYCLONE, HARDCOPY, MAX, MEGACORE, NIOS, QUARTUS & STRATIX are Reg. U.S. Pat. & Tm. Off.
and Altera marks in and outside the U.S.

19. PCIe Endpoint Design Example
© 2010 Altera Corporation - Public
ALTERA, ARRIA, CYCLONE, HARDCOPY, MAX, MEGACORE, NIOS, QUARTUS & STRATIX are Reg. U.S. Pat. & Tm. Off.
and Altera marks in and outside the U.S.

20. Backup
© 2010 Altera Corporation - Public

21. Auto Speed Negotiation
Initially link trains to L0 (normal operation) in 2.5G data rate
Speeds advertised in training sequence (TS) ordered sets
− Supported speeds by the other component noted
Speed change occurs when the link is in electrical idle
2.5 Gbps Detect
2.5 Gbps Polling.Active
2.5 Gbps
2.5 Gbps Polling.Config Recovery.RcvrLock 2.5 Gbps
5.0 Gbps
5.0 Gbps
2.5 Gbps
2.5 Gbps Config 5.0 Gbps Recovery.RcvrCfg Recovery.Speed
2.5 Gbps
2.5 Gbps L0 Recovery.Idle
5.0 Gbps
5.0 Gbps
Dedicated PCIe (PIPE) Clock Switch Circuitry in Stratix IV GX/GT and Hardcopy IV GX
© 2010 Altera Corporation - Public
ALTERA, ARRIA, CYCLONE, HARDCOPY, MAX, MEGACORE, NIOS, QUARTUS & STRATIX are Reg. U.S. Pat. & Tm. Off.
and Altera marks in and outside the U.S.
21

22. MSC Hpe IRP Platform
© 2010 Altera Corporation - Public
ALTERA, ARRIA, CYCLONE, HARDCOPY, MAX, MEGACORE, NIOS, QUARTUS & STRATIX are Reg. U.S. Pat. & Tm. Off.
and Altera marks in and outside the U.S.

23. PCI-SIG Industry Testing – PCI-SIG
PCI-SIG
FPGA/PHY Config. PCIe IP
qualification
Cyclone IV GX (Gen 1) x1, x4 Hard IP Q110
Arria II GX (Gen1) x1, x4, x8 Hard IP Q209
Stratix IV GX (Gen1 and Gen2) x1, x4, x8 Hard IP 1Q09
Arria GX (Gen1) x1,x4 Soft IP 2Q07
Stratix II GX (Gen1) x1, x4, x8 Soft IP 2Q06
Cyclone II/TI x1 (Gen1) x1 Soft IP 1Q06
Cyclone II/Philips x1 V (Gen1) x1 Soft IP 4Q05
Stratix GX (Gen1)
PCI-SIG qualification across all target lane configurations IP
x1, x4 Soft 1Q05
© 2010 Altera Corporation - Public
ALTERA, ARRIA, CYCLONE, HARDCOPY, MAX, MEGACORE, NIOS, QUARTUS & STRATIX are Reg. U.S. Pat. & Tm. Off.
and Altera marks in and outside the U.S.
23

24. PCI Express TX Eye Mask Test
Data
Summary
− The Stratix IV GX device TX eye passes the PCI Express Gen 2 mask
tests
© 2010 Altera Corporation - Public
ALTERA, ARRIA, CYCLONE, HARDCOPY, MAX, MEGACORE, NIOS, QUARTUS & STRATIX are Reg. U.S. Pat. & Tm. Off.
and Altera marks in and outside the U.S.
24

25. Development Kits
© 2010 Altera Corporation - Public

26. Stratix IV GX and HardCopy IV GX Shipping
Shipping
Development Kit now
now
PCI Express short form factor
Included daughtercards
− HSMC loop-back
− HSMC debug
Included software
− Quartus II Development Kit Edition
12-month license
− Example designs
Documentation
− User guide
− Reference manual
− Schematics
− Layout design files
© 2010 Altera Corporation - Public
ALTERA, ARRIA, CYCLONE, HARDCOPY, MAX, MEGACORE, NIOS, QUARTUS & STRATIX are Reg. U.S. Pat. & Tm. Off.
and Altera marks in and outside the U.S.

27. Arria II GX Development Kit Shipping
Shipping
now
now
PCI Express short form factor
Included daughtercards
− HSMC loop-back
− HSMC debug
Included software
− Quartus II Development Kit Edition
12-month license
− Example designs
Documentation
− User guide
− Reference manual
− Schematics
− Layout design files
© 2010 Altera Corporation - Public
ALTERA, ARRIA, CYCLONE, HARDCOPY, MAX, MEGACORE, NIOS, QUARTUS & STRATIX are Reg. U.S. Pat. & Tm. Off.
and Altera marks in and outside the U.S.
27