1. Hybrid Memory Cube
Developing Scalable and Resilient Memory Systems
Ryan Baxter
Senior Manager, Business Development
Server and Storage DRAM
rrbaxter@micron.com
©2013 Micron Technology, Inc. All rights reserved. Products are warranted only to meet Micron’s production data sheet specifications. Information, products, and/or specifications
are subject to change without notice. All information is provided on an ―AS IS‖ basis without warranties of any kind. Dates are estimates only. Drawings are not to scale. Micron and
the Micron logo are trademarks of Micron Technology, Inc. All other trademarks are the property of their respective owners.
©2013 Micron Technology, Inc.
|
1

2. Demand Drivers
▶ Insatiable need for bandwidth
▶ Impact of the cloud
▶ Global demand for mobility
▶ Big data analytics challenge
Annual Data Center IP Traffic
2012-2017 CAGR: 25%
EB/Yr
10,000
Within Data Center
Enterprise Computing
Data Center to Data Center
7,500
Data Center to User
5,000
High-Performance
Computing
2,500
SOCs &
Microservers
0
2012
2013
2014
2015
2016
2017
Acceleration &
Co-Processing
Source: Cisco Global Cloud Index 2013
©2013 Micron Technology, Inc.
|
2

3. HMC - A Revolutionary Shift
Increased
Bandwidth
Greater Power
Efficiency
Lower
TCO
Reduced System
Latency
Smaller, Scalable,
& Flexible
©2013 Micron Technology, Inc.
|
3

4. http://www.hybridmemorycube.org/about
Over 120 adopters to date!
Pace of adoption increasing—2013 adopters doubled from 2012
“…unprecedented levels of memory performance”
- Electronic News
“…like adding a turbocharger to your computer”
- datacenteracceleration.com
EE Times 40th Anniversary:
“one of the top ten technologies
expected to redefine the industry‖
©2013 Micron Technology, Inc.
|
4

5. System Memory Bandwidth
GB/s
DDR4
90
4 Memory Channels
284 Pins Per DIMM
Up to 85GB/s
12 Speed Bins
6+ yrs to Standardize
75
DDR3
3-4 Memory Channels
240 Pins Per DIMM
Up to 59.7GB/s
5 Speed Bins
DDR2
60
45
2 Memory Channels
Up to 10.7GB/s
3 Speed Bins
Bandwidth Per
Memory Channel
DDR
1-2 Memory Channels
Up to 6.4GB/s
85 page specification
<3 Yrs to Standardize
30
2012–2017 CAGR: 12.3%
15
0
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
Traditional Memory Designs Do Not Scale and Drive Exponential Complexity
©2013 Micron Technology, Inc.
|
5

6. High-Performance Memory Comparison
Single-Link HMC vs. DDR3L-1600 and DDR4-2133
What does it take to support 60 GB/s?
Requirements
Channel Complexity
TCO Valuation
90% simpler than DDR3L
88% simpler than DDR4
DDR3L
DDR4
HMC
0
0
Energy Efficiency
66% greener than DDR3L
55% greener than DDR4
3,000
0
Board Footprint
95% smaller than DDR3L
94% smaller than DDR4
250
20
40
0
300
750 pins
500
600
DDR3L
DDR4
HMC
6,000
2
9,000 mm
DDR3L
DDR4
HMC
60
pJ/b
DDR3L
Bandwidth
10.2X greater than DDR3L
8.5X greater than DDR4
DDR4
HMC
©2013 Micron Technology, Inc.
MB/
900 pin
|
6

7. Enabling Technologies
Abstracted Memory
Management
Memory Vaults vs. DRAM Arrays
• Significantly improves bandwidth, quality, and
reliability vs. traditional DRAM technologies
Logic Base Controller
• Reduces memory complexity and significantly
increases performance
• Allows memory to scale with CPU performance
February 12, 2014
Through-Silicon Via (TSV)
Assembly
Innovative Design & Process Flow
• Incorporates thousands of TSV sites per die to
reduce signal lengths and power
• Enables memory scalability through parallelism
Sophisticated Package Assembly
• Provides higher component density and
significantly improves signal integrity
©2013 Micron Technology, Inc.
|
7

8. HMC Architecture
Vault Control
Logic Base
Vault Control
Memory Control
Refresh
Controller
Crossbar Switch
Link Interface
Controller
Link Interface
Controller
Link Interface
Controller
Read
Buffer
Read Data
DRAM
Sequencer
Memory Control
Write
Buffer
Write Data
Vault Control
Request
Vault Control
TSV Repair
DRAM Repair
Crossbar Switch
Link Interface
Controller
Detail of Memory Interface
Processor
Links
3DI & TSV Technology
DRAM7
DRAM6
DRAM5
DRAM4
DRAM3
DRAM2
DRAM1
DRAM0
Logic Chip
Logic Base
• Multiple high-speed local buses for data movement
• Advanced memory controller functions
• DRAM control at the memory rather than at
distant host controller
• Reduced memory controller complexity
and increased efficiency
Vault Vaults are managed to maximize
overall device availability
• Optimized management of energy and
refresh
DRAM
Logic Base
February 12, 2014
• Self test, error detection, correction, and
repair in the logic base layer
Micron Confidential
|
©2013 Micron Technology, Inc.
|
8

9. HMC Architecture
Link Controller Interface
HMC-SR Options:
10 Gb/s, 12.5 Gb/s,
or 15 Gb/s
HMC
Host
TX
16 Lanes
16 Lanes
TX
8 or 16 Transmit Lanes
RX
RX
Example:
February 12, 2014
8 or 16 Receive Lanes
Micron Confidential
|
©2013 Micron Technology, Inc.
|
9

10. Packet-Based Communication
Protocol NOT affected by
any DRAM-related timings,
nor is it DRAM-specific!
▶ Packets comprised of 128-bit (16-byte) FLITs
 Packets include 1 to 9 FLITs, depending on command
▶ Host issues requests & HMC issues responses
▶ Each packet contains 64-bit header and 64-bit tail (1 FLIT)
▶ Multiple data transfer sizes supported (16B to 128B)
▶ Commands include reads, writes, atomics, error responses
 Simultaneous READs and WRITEs supported
Micron Confidential
|
©2013 Micron Technology, Inc.
|
10

11. Host Processor Memory Management
Simple memory requests
and responses; no DRAM
timings to manage
Functions moved to
HMC for management
Manufacturing Test
• Burn-in
• At-speed functional
Manage field maintenance
and self test
Manage all present and
future DRAM scaling and
process variation issues
Manage 100+ different
DRAM timing parameters
DRAM Layer
TSV
TSV
TSV
TSV
TSV
TSV
TSV
TSV
DRAM Layer
DRAM Layer
DRAM Layer
HOST
Re-drive Layer
Non-Managed DRAM
(DDR, WIO2, HBM, etc.)
Si Interposer
February 12, 2014
Micron Confidential
|
©2013 Micron Technology, Inc.
|
11

12. HMC Near Memory
▶ All links between host CPU
and HMC logic layer
▶ Maximum bandwidth per GB capacity:
HPC/Server – CPU/GPU
 Graphics
 Networking systems
 Test equipment

February 12, 2014
Micron Confidential
|
©2013 Micron Technology, Inc.
|
12

13. HMC ―Far‖ Memory
▶ Far Memory:
Some HMC links connect to host – some to
other cubes
 Scalable to meet system requirements
 Available in module form or soldered-down

▶ Future Products May Include:
Higher-speed electrical (SERDES)
 Optical interfaces
 Higher stack count for greater capacity
 Non-DRAM memory technologies

February 12, 2014
Micron Confidential
|
©2013 Micron Technology, Inc.
|
13

14. HMC Reliability
▶ Built-in RAS features
Logic stability
(DRAM controls in logic)
DRAM Array
Reliable handshake
(packet integrity verified
before memory access)
Logic / Interface
DRAM Array
DRAM Array
Logic / Interface
Logic / Interface
Vault data
ECC-protected
Host
Link retry
CRC protection
on link interface
Logic / Interface
DRAM Array
February 12, 2014
Address/command parity
for array transactions
Micron Confidential
|
©2013 Micron Technology, Inc.
|
14

15. HMC Standard Packages
Up to 1.28 Tb/s
memory bandwidth!
Standard BGA packaging solutions:
Cost-effective packaging using existing ecosystems
February 12, 2014
Micron Confidential
|
©2013 Micron Technology, Inc.
|
15

16. Hybrid Memory Cube
Micron Memory Innovation
We’ve combined fast logic process technology and advanced
DRAM designs to create an entirely new category we’re calling
Hybrid Memory Cube (HMC) technology. The end result is
a high-bandwidth, low-energy, high-density memory system
that’s unlike anything on the market today.
Unprecedented Performance
HMC will provide a revolutionary performance
shift that will enrich next-generation networking
and enable exaflop-scale supercomputing:
Reduced Power
Fraction of the energy per bit
Reduced Footprint
90% less space than today’s RDIMMs
Increased Bandwidth
15X the performance of DDR3*
* HMC SR-15G vs. DDR3-1333
February 12, 2014
Micron Confidential
|
©2013 Micron Technology, Inc.
|
16