SlideShare a Scribd company logo
1 of 30
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Pop-up Loft
Amazon DynamoDB and DAX
Rajeev Srinivasan
srrajeev@amazon.com
Sr Solutions Architect
Dynamo whitepaper
NoSQL foundations
0000 {“Texas”}
0001 {“Illinois”}
0002 {“Oregon”}
TXW
A
I
L
Key
Column
0000-0000-0000-0001
Game Heroes
Version 3.4
CRC ADE4
Key Value Graph Document Column-family
Dynamo:
Amazon’s
Highly Available
Key-value
Store
January 2012Fall 2007 June 2009
Meetup
235 2nd St
San
Francisco
What (some) customers store in NoSQL DBs
Market Orders Tokenization
(PHI, Credit Cards)
Chat MessagesUser Profiles
(Mobile)
IoT Sensor Data
(& device status!)
File MetadataSocial Media Feeds
DataXu’s Attribution Store
AWS
Direct
Connect
Amazon
DynamoDB
Amazon
RDS
AWS Data
Pipeline
AWS IAM
Amazon
SNS
Amazon
CloudWatch
EMR Job
Amazon
EC2
Amazon
S3 Bucket
1st Party Data
3rd Party Data
“Attribution" is the marketing term of art for allocating full or partial credit to
individual advertisements that eventually lead to a purchase or other desired
consumer interaction.
Highly available Consistent, single-digit
millisecond latency
at any scale
Fully managed
Secure
Integrates with AWS Lambda,
Amazon Redshift, and more
Amazon DynamoDB
Elastic is the new normal
Write Capacity Units
Read Capacity Units
ConsumedCapacityUnits
>200% increase from baseline
>300% increase from baseline
Time
Scaling high-velocity use cases with DynamoDB
Ad Tech Gaming MobileIoT Web
Partition Key
Mandatory
Key-value access pattern
Determines data distribution
Optional
Model 1:N relationships
Enables rich query capabilities
DynamoDB table
A1
(partition key)
A2
(sort key)
A3 A4 A7
A1
(partition key)
A2
(sort key)
A6 A4 A5
A1
(partition key)
A2
(sort key)
A1
(partition key)
A2
(sort key)
A3 A4 A5
Sort Key
Table
Items
DynamoDB LSIs
Can only be
defined as part
of initial table
creation
A1
(partition key)
A3
(sort key)
A2 A4 A5
A1
(partition key)
A4
(sort key)
A2 A3 A5
A1
(partition key)
A5
(sort key)
A2 A3 A4
• Alternate sort key
attribute
• Index is local to a
partition key
Local secondary indexes
RCUs/WCUs
provisioned
separately for GSIs
INCLUDE A2
ALL
KEYS_ONLY
A3
(partition key)
A1
(table key)
A2 A4 A7
A3
(partition key)
A1
(table key)
A3
(partition key)
A1
(table key)
A2
• Alternate partition
(+sort) key
• Index is across all
table partition keys
• Can be added or
removed anytime
• Eventually
consistent
A3
(partition key)
A1
(table key)
A2 A4 A7
A3
(partition key)
A1
(table key)
A2
A3
(partition key)
A1
(table key)
Global secondary indexes
DynamoDB GSIs
Data types
Type DynamoDB Type
String String
Integer, Float Number
Timestamp Number or String
Blob Binary
Boolean Bool
Null Null
List List
Set
Set of String,
Number, or Binary
Map Map
Table creation options
PartitionKey, Type:
SortKey, Type:
Provisioned Reads:
Provisioned Writes:
LSI Schema GSI Schema
AttributeName [S,N,B]
AttributeName [S,N,B]
1+
1+
Provisioned Reads: 1+
Provisioned Writes: 1+
TableNameOptionalRequired
CreateTable
String,
Number,
Binary ONLY
Per Second
Unique to
Account and
Region
Provisioned capacity
Provisioned capacity
Read Capacity Unit (RCU)
1 RCU returns 4KB of data for strongly
consistent reads, or double the data
at the same cost for eventually
consistent reads
Capacity is per second, rounded up to the
next whole number
Write Capacity Unit (WCU)
1 WCU writes 1KB of data, and each
item consumes 1 WCU minimum
Horizontal sharding
Host 1 Host 99 Host n
~Each new host brings compute, storage, and network bandwidth~
CustomerOrdersTable
CustomerOrdersTable
OrderId: 1
CustomerId: 1
ASIN: [B00X4WHP5E]
Partitioning
00
55
AA
FF
Hash(1) = 7B
CustomerOrdersTable
OrderId: 2
CustomerId: 4
ASIN: [B00OQVZDJM]
OrderId: 3
CustomerId: 3
ASIN: [B00U3FPN4U]
Partition A
33.33 % Keyspace
33.33 % Provisioned Capacity
Partition B
33.33 % Keyspace
33.33 % Provisioned Capacity
Partition C
33.33 % Keyspace
33.33 % Provisioned Capacity
Hash.MIN = 0
Hash.MAX = FF
Keyspace
Hash(2) = 48
Hash(3) = CD
CustomerOrdersTable
00
55
AA
FF
Partition A
33.33 % Keyspace
33.33 % Provisioned Capacity
Partition B
33.33 % Keyspace
33.33 % Provisioned Capacity
Partition C
33.33 % Keyspace
33.33 % Provisioned Capacity
Hash.MIN = 0
Hash.MAX = FF
Keyspace
Time
Partition A
33.33 % Keyspace
33.33 % Provisioned Capacity
Partition B
33.33 % Keyspace
33.33 % Provisioned Capacity
Partition D
Partition E
16.66 %
16.66 %
16.66 %
16.66 %
Partition split due to partition size
00
55
AA
FF
Partition A
33.33 % Keyspace
33.33 % Provisioned Capacity
Partition B
33.33 % Keyspace
33.33 % Provisioned Capacity
Partition C
33.33 % Keyspace
33.33 % Provisioned Capacity
Time
Partition A
Partition C
16.66 %
16.66 %
16.66 %
16.66 %
Partition split due to capacity increase
16.66 %
16.66 %
16.66 %
16.66 %
16.66 %
16.66 %
16.66 %
16.66 %
Partition B
Partition D
Partition E
Partition F
The desired size of a
partition is 10GB* and
when a partition surpasses
this it can split
*=subject to change
Split for partition size
The desired capacity of a
partition is expressed as:
3w + 1r < 3000 *
Where w = WCU & r = RCU
*=subject to change
Split for provisioned capacity
Partitioning
Partition A
1000 RCUs
100 WCUs
Partition C
1000 RCUs
100 WCUs
Host A Host C
Availability Zone A
Partition A
1000 RCUs
100 WCUs
Partition C
1000 RCUs
100 WCUs
Host E Host G
Availability Zone B
Partition A
1000 RCUs
100 WCUs
Partition C
1000 RCUs
100 WCUs
Host H Host J
Availability Zone C
CustomerOrdersTable
54:∞00:0 54:∞00:0 54:∞00:0
FF:∞AA:0 FF:∞AA:0 FF:∞AA:0
Data is replicated to
three Availability
Zones by design
3-way replication
OrderId: 1
CustomerId: 1
ASIN: [B00X4WHP5E]
Hash(1) = 7B
Partition B
1000 RCUs
100 WCUs
Host B Host F Host I
Partition B
1000 RCUs
100 WCUs
Partition B
1000 RCUs
100 WCUs
A9:∞55:0 A9:∞55:0 A9:∞55:0
Partitioning
DynamoDB Streams
Partition A
Partition B
Partition C
üOrdered stream of item
changes
üExactly once, strictly
ordered by key
üHighly durable, scalable
ü24-hour retention
üSub-second latency
üCompatible with Kinesis
Client Library
DynamoDB Streams
1
Shards have a lineage and
automatically close after time
or when the associated
DynamoDB partition splits
2
3
Updates
KCL
Worker
Amazon
Kinesis Client
Library
Application
KCL
Worker
KCL
Worker
GetRecords
Amazon DynamoDB
Table
DynamoDB Streams Stream
Shards
TTL job
Time-To-Live (TTL)
Amazon DynamoDB
Table
CustomerActiveOrder
OrderId: 1
CustomerId: 1
MyTTL: 1492641900
DynamoDB Streams
Amazon Kinesis
Amazon Redshift
An epoch timestamp marking when
an item can be deleted by a
background process, without
consuming any provisioned capacity
Time-To-Live
Removes data that is no longer relevant
Time-To-Live (TTL)
üTTL items
identifiable in
DynamoDB
Streams
üConfiguration protected by
AWS Identity and Access
Management (IAM), auditable
with AWS CloudTrail
üEventual deletion,
free to use
DynamoDB Auto Scaling
Specify: 1) Target capacity in percent 2) Upper and lower bound
DynamoDB Accelerator
Amazon
DynamoDB
msµs
DAXYour App
in VPC
Amazon DynamoDB Accelerator (DAX)
DynamoDB Accelerator (DAX)
Private IP, Client-side
Discovery
Supports AWS Java SDK on launch,
with more AWS SDKs to come
Cluster based, Multi-AZ Separate Query and
Item cache
DynamoDB in the VPC
Availability Zone #1 Availability Zone #2
Private Subnet Private Subnet
VPC endpoint
web
app
server
security
group
security
group
oMicroseconds latency in-memory cache
oMillions of requests per second
oFully managed, highly available
oRole-based access control
oNo IGW or VPC endpoint required
DAX
oDynamoDB in the VPC
oIAM resource policy
restricted
VPC Endpoints
AWS Lambda
security
group
security
group
DAX
web
app
server
DAX
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved
DynamoDB Global Tables ( G A )
F i r s t f u l l y m a n a g e d , m u l t i - m a s t e r , m u l t i - r e g i o n d a t a b a s e
Build high performance, globally distributed applications
Low latency reads and writes to locally available tables
Disaster proof with multi-region redundancy
Easy to set up and no application re-writes required
Globally dispersed users
Replica (N. America)
Replica (Europe)
Replica (Asia)
Global App
Global Table
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Amazon DynamoDB – Backup and Restore
F i r s t N o S Q L d a t a b a s e t o a u t o m a t e o n - d e m a n d a n d c o n t i n u o u s
b a c k u p s
Point in time restore
for short-term
retention and data
corruption protection
(coming soon)
Back up hundreds of
TB instantaneously
with NO performance
impact
On-demand
backups for long-
term data archival
and compliance
(GA)
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Amazon DynamoDB
E n c r y p t i o n @ r e s t ( c o m i n g s o o n )
Server-side
encryption
Supports
compliance
certifications
No application
code rewrites
Getting started?
Amazon DynamoDB
Local
Document SDKs DynamoDB as a
target for AWS
Data Migration Service
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Pop-up Loft
aws.amazon.com/activate
Everything and Anything Startups
Need to Get Started on AWS

More Related Content

What's hot

What's hot (20)

Amazon ElastiCache and Redis
Amazon ElastiCache and RedisAmazon ElastiCache and Redis
Amazon ElastiCache and Redis
 
Graph and Amazon Neptune
Graph and Amazon NeptuneGraph and Amazon Neptune
Graph and Amazon Neptune
 
Building High Performance Apps with In-memory Data
Building High Performance Apps with In-memory DataBuilding High Performance Apps with In-memory Data
Building High Performance Apps with In-memory Data
 
DynamoDB - What's new - DAT304 - re:Invent 2017
DynamoDB - What's new - DAT304 - re:Invent 2017DynamoDB - What's new - DAT304 - re:Invent 2017
DynamoDB - What's new - DAT304 - re:Invent 2017
 
Integrating Amazon Elasticsearch with your DevOps Tooling - AWS Online Tech T...
Integrating Amazon Elasticsearch with your DevOps Tooling - AWS Online Tech T...Integrating Amazon Elasticsearch with your DevOps Tooling - AWS Online Tech T...
Integrating Amazon Elasticsearch with your DevOps Tooling - AWS Online Tech T...
 
Technology Trends in Data Processing - DAT311 - re:Invent 2017
Technology Trends in Data Processing - DAT311 - re:Invent 2017Technology Trends in Data Processing - DAT311 - re:Invent 2017
Technology Trends in Data Processing - DAT311 - re:Invent 2017
 
Data Warehousing with Amazon Redshift
Data Warehousing with Amazon RedshiftData Warehousing with Amazon Redshift
Data Warehousing with Amazon Redshift
 
reModernize - Updating & Consolidating MySQL
reModernize - Updating & Consolidating MySQLreModernize - Updating & Consolidating MySQL
reModernize - Updating & Consolidating MySQL
 
Advanced Design Patterns for Amazon DynamoDB - DAT403 - re:Invent 2017
Advanced Design Patterns for Amazon DynamoDB - DAT403 - re:Invent 2017Advanced Design Patterns for Amazon DynamoDB - DAT403 - re:Invent 2017
Advanced Design Patterns for Amazon DynamoDB - DAT403 - re:Invent 2017
 
Loading Data into Redshift with Lab
Loading Data into Redshift with LabLoading Data into Redshift with Lab
Loading Data into Redshift with Lab
 
How Amazon.com Uses AWS Analytics
How Amazon.com Uses AWS AnalyticsHow Amazon.com Uses AWS Analytics
How Amazon.com Uses AWS Analytics
 
How to Choose The Right Database on AWS - Berlin Summit - 2019
How to Choose The Right Database on AWS - Berlin Summit - 2019How to Choose The Right Database on AWS - Berlin Summit - 2019
How to Choose The Right Database on AWS - Berlin Summit - 2019
 
Data Warehousing with Amazon Redshift
Data Warehousing with Amazon RedshiftData Warehousing with Amazon Redshift
Data Warehousing with Amazon Redshift
 
Introducing AWS DeepLens - AWS Online Tech Talks
Introducing AWS DeepLens - AWS Online Tech TalksIntroducing AWS DeepLens - AWS Online Tech Talks
Introducing AWS DeepLens - AWS Online Tech Talks
 
Data Warehousing and Data Lake Analytics, Together - AWS Online Tech Talks
Data Warehousing and Data Lake Analytics, Together - AWS Online Tech TalksData Warehousing and Data Lake Analytics, Together - AWS Online Tech Talks
Data Warehousing and Data Lake Analytics, Together - AWS Online Tech Talks
 
Data Warehousing with Amazon Redshift: Data Analytics Week at the SF Loft
Data Warehousing with Amazon Redshift: Data Analytics Week at the SF LoftData Warehousing with Amazon Redshift: Data Analytics Week at the SF Loft
Data Warehousing with Amazon Redshift: Data Analytics Week at the SF Loft
 
Amazon DynamoDB and Amazon DAX
Amazon DynamoDB and Amazon DAXAmazon DynamoDB and Amazon DAX
Amazon DynamoDB and Amazon DAX
 
Loading Data into Amazon Redshift
Loading Data into Amazon RedshiftLoading Data into Amazon Redshift
Loading Data into Amazon Redshift
 
Workshop on Advanced Design Patterns for Amazon DynamoDB - DAT405 - re:Invent...
Workshop on Advanced Design Patterns for Amazon DynamoDB - DAT405 - re:Invent...Workshop on Advanced Design Patterns for Amazon DynamoDB - DAT405 - re:Invent...
Workshop on Advanced Design Patterns for Amazon DynamoDB - DAT405 - re:Invent...
 
Data Warehouses and Data Lakes
Data Warehouses and Data LakesData Warehouses and Data Lakes
Data Warehouses and Data Lakes
 

Similar to Amazon DynamoDB and DAX

Similar to Amazon DynamoDB and DAX (20)

SRV404 Deep Dive on Amazon DynamoDB
SRV404 Deep Dive on Amazon DynamoDBSRV404 Deep Dive on Amazon DynamoDB
SRV404 Deep Dive on Amazon DynamoDB
 
SRV404 Deep Dive on Amazon DynamoDB
SRV404 Deep Dive on Amazon DynamoDBSRV404 Deep Dive on Amazon DynamoDB
SRV404 Deep Dive on Amazon DynamoDB
 
What's New with Amazon DynamoDB - SRV311 - Chicago AWS Summit
What's New with Amazon DynamoDB - SRV311 - Chicago AWS SummitWhat's New with Amazon DynamoDB - SRV311 - Chicago AWS Summit
What's New with Amazon DynamoDB - SRV311 - Chicago AWS Summit
 
What's New with Amazon DynamoDB - SRV311 - Atlanta AWS Summit
What's New with Amazon DynamoDB - SRV311 - Atlanta AWS SummitWhat's New with Amazon DynamoDB - SRV311 - Atlanta AWS Summit
What's New with Amazon DynamoDB - SRV311 - Atlanta AWS Summit
 
AWS SSA Webinar 33 - Getting started with databases on AWS Amazon DynamoDB
AWS SSA Webinar 33 - Getting started with databases on AWS Amazon DynamoDBAWS SSA Webinar 33 - Getting started with databases on AWS Amazon DynamoDB
AWS SSA Webinar 33 - Getting started with databases on AWS Amazon DynamoDB
 
SRV404 Deep Dive on Amazon DynamoDB
SRV404 Deep Dive on Amazon DynamoDBSRV404 Deep Dive on Amazon DynamoDB
SRV404 Deep Dive on Amazon DynamoDB
 
Deep Dive on Amazon DynamoDB
Deep Dive on Amazon DynamoDBDeep Dive on Amazon DynamoDB
Deep Dive on Amazon DynamoDB
 
Deep Dive on Amazon DynamoDB - AWS Online Tech Talks
Deep Dive on Amazon DynamoDB - AWS Online Tech TalksDeep Dive on Amazon DynamoDB - AWS Online Tech Talks
Deep Dive on Amazon DynamoDB - AWS Online Tech Talks
 
TechEd NZ 2014: Azure and Sharepoint
TechEd NZ 2014: Azure and SharepointTechEd NZ 2014: Azure and Sharepoint
TechEd NZ 2014: Azure and Sharepoint
 
Getting Started with Amazon Redshift
Getting Started with Amazon RedshiftGetting Started with Amazon Redshift
Getting Started with Amazon Redshift
 
Deploying your Data Warehouse on AWS
Deploying your Data Warehouse on AWSDeploying your Data Warehouse on AWS
Deploying your Data Warehouse on AWS
 
What’s New for Amazon DynamoDB - 2018 Q1 Update - AWS Online Tech Talks
What’s New for Amazon DynamoDB - 2018 Q1 Update - AWS Online Tech TalksWhat’s New for Amazon DynamoDB - 2018 Q1 Update - AWS Online Tech Talks
What’s New for Amazon DynamoDB - 2018 Q1 Update - AWS Online Tech Talks
 
Optimizing Apache HBase for Cloud Storage in Microsoft Azure HDInsight
Optimizing Apache HBase for Cloud Storage in Microsoft Azure HDInsightOptimizing Apache HBase for Cloud Storage in Microsoft Azure HDInsight
Optimizing Apache HBase for Cloud Storage in Microsoft Azure HDInsight
 
Getting Started with Amazon Redshift - AWS July 2016 Webinar Series
Getting Started with Amazon Redshift - AWS July 2016 Webinar SeriesGetting Started with Amazon Redshift - AWS July 2016 Webinar Series
Getting Started with Amazon Redshift - AWS July 2016 Webinar Series
 
Managed NoSQL databases
Managed NoSQL databasesManaged NoSQL databases
Managed NoSQL databases
 
DynamoDB & DAX: Database Week SF
DynamoDB & DAX: Database Week SFDynamoDB & DAX: Database Week SF
DynamoDB & DAX: Database Week SF
 
Managed NoSQL databases
Managed NoSQL databasesManaged NoSQL databases
Managed NoSQL databases
 
Getting Started with Amazon DynamoDB
Getting Started with Amazon DynamoDBGetting Started with Amazon DynamoDB
Getting Started with Amazon DynamoDB
 
Amazon Web Service Sales Role Play - Case Study
Amazon Web Service Sales Role Play - Case StudyAmazon Web Service Sales Role Play - Case Study
Amazon Web Service Sales Role Play - Case Study
 
DynamoDB & DAX: Database Week San Francisco
DynamoDB & DAX: Database Week San FranciscoDynamoDB & DAX: Database Week San Francisco
DynamoDB & DAX: Database Week San Francisco
 

More from Amazon Web Services

Tools for building your MVP on AWS
Tools for building your MVP on AWSTools for building your MVP on AWS
Tools for building your MVP on AWS
Amazon Web Services
 
How to Build a Winning Pitch Deck
How to Build a Winning Pitch DeckHow to Build a Winning Pitch Deck
How to Build a Winning Pitch Deck
Amazon Web Services
 
Building a web application without servers
Building a web application without serversBuilding a web application without servers
Building a web application without servers
Amazon Web Services
 
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
Amazon Web Services
 

More from Amazon Web Services (20)

Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
 
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
 
Esegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS FargateEsegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS Fargate
 
Costruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWSCostruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWS
 
Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot
 
Open banking as a service
Open banking as a serviceOpen banking as a service
Open banking as a service
 
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
 
OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...
 
Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows WorkloadsMicrosoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads
 
Computer Vision con AWS
Computer Vision con AWSComputer Vision con AWS
Computer Vision con AWS
 
Database Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatareDatabase Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatare
 
Crea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJSCrea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJS
 
API moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e webAPI moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e web
 
Database Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatareDatabase Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatare
 
Tools for building your MVP on AWS
Tools for building your MVP on AWSTools for building your MVP on AWS
Tools for building your MVP on AWS
 
How to Build a Winning Pitch Deck
How to Build a Winning Pitch DeckHow to Build a Winning Pitch Deck
How to Build a Winning Pitch Deck
 
Building a web application without servers
Building a web application without serversBuilding a web application without servers
Building a web application without servers
 
Fundraising Essentials
Fundraising EssentialsFundraising Essentials
Fundraising Essentials
 
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
 
Introduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container ServiceIntroduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container Service
 

Amazon DynamoDB and DAX

  • 1. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved Pop-up Loft Amazon DynamoDB and DAX Rajeev Srinivasan srrajeev@amazon.com Sr Solutions Architect
  • 3. NoSQL foundations 0000 {“Texas”} 0001 {“Illinois”} 0002 {“Oregon”} TXW A I L Key Column 0000-0000-0000-0001 Game Heroes Version 3.4 CRC ADE4 Key Value Graph Document Column-family Dynamo: Amazon’s Highly Available Key-value Store January 2012Fall 2007 June 2009 Meetup 235 2nd St San Francisco
  • 4. What (some) customers store in NoSQL DBs Market Orders Tokenization (PHI, Credit Cards) Chat MessagesUser Profiles (Mobile) IoT Sensor Data (& device status!) File MetadataSocial Media Feeds
  • 5. DataXu’s Attribution Store AWS Direct Connect Amazon DynamoDB Amazon RDS AWS Data Pipeline AWS IAM Amazon SNS Amazon CloudWatch EMR Job Amazon EC2 Amazon S3 Bucket 1st Party Data 3rd Party Data “Attribution" is the marketing term of art for allocating full or partial credit to individual advertisements that eventually lead to a purchase or other desired consumer interaction.
  • 6. Highly available Consistent, single-digit millisecond latency at any scale Fully managed Secure Integrates with AWS Lambda, Amazon Redshift, and more Amazon DynamoDB
  • 7. Elastic is the new normal Write Capacity Units Read Capacity Units ConsumedCapacityUnits >200% increase from baseline >300% increase from baseline Time
  • 8. Scaling high-velocity use cases with DynamoDB Ad Tech Gaming MobileIoT Web
  • 9. Partition Key Mandatory Key-value access pattern Determines data distribution Optional Model 1:N relationships Enables rich query capabilities DynamoDB table A1 (partition key) A2 (sort key) A3 A4 A7 A1 (partition key) A2 (sort key) A6 A4 A5 A1 (partition key) A2 (sort key) A1 (partition key) A2 (sort key) A3 A4 A5 Sort Key Table Items
  • 10. DynamoDB LSIs Can only be defined as part of initial table creation A1 (partition key) A3 (sort key) A2 A4 A5 A1 (partition key) A4 (sort key) A2 A3 A5 A1 (partition key) A5 (sort key) A2 A3 A4 • Alternate sort key attribute • Index is local to a partition key Local secondary indexes
  • 11. RCUs/WCUs provisioned separately for GSIs INCLUDE A2 ALL KEYS_ONLY A3 (partition key) A1 (table key) A2 A4 A7 A3 (partition key) A1 (table key) A3 (partition key) A1 (table key) A2 • Alternate partition (+sort) key • Index is across all table partition keys • Can be added or removed anytime • Eventually consistent A3 (partition key) A1 (table key) A2 A4 A7 A3 (partition key) A1 (table key) A2 A3 (partition key) A1 (table key) Global secondary indexes DynamoDB GSIs
  • 12. Data types Type DynamoDB Type String String Integer, Float Number Timestamp Number or String Blob Binary Boolean Bool Null Null List List Set Set of String, Number, or Binary Map Map
  • 13. Table creation options PartitionKey, Type: SortKey, Type: Provisioned Reads: Provisioned Writes: LSI Schema GSI Schema AttributeName [S,N,B] AttributeName [S,N,B] 1+ 1+ Provisioned Reads: 1+ Provisioned Writes: 1+ TableNameOptionalRequired CreateTable String, Number, Binary ONLY Per Second Unique to Account and Region
  • 14. Provisioned capacity Provisioned capacity Read Capacity Unit (RCU) 1 RCU returns 4KB of data for strongly consistent reads, or double the data at the same cost for eventually consistent reads Capacity is per second, rounded up to the next whole number Write Capacity Unit (WCU) 1 WCU writes 1KB of data, and each item consumes 1 WCU minimum
  • 15. Horizontal sharding Host 1 Host 99 Host n ~Each new host brings compute, storage, and network bandwidth~ CustomerOrdersTable
  • 16. CustomerOrdersTable OrderId: 1 CustomerId: 1 ASIN: [B00X4WHP5E] Partitioning 00 55 AA FF Hash(1) = 7B CustomerOrdersTable OrderId: 2 CustomerId: 4 ASIN: [B00OQVZDJM] OrderId: 3 CustomerId: 3 ASIN: [B00U3FPN4U] Partition A 33.33 % Keyspace 33.33 % Provisioned Capacity Partition B 33.33 % Keyspace 33.33 % Provisioned Capacity Partition C 33.33 % Keyspace 33.33 % Provisioned Capacity Hash.MIN = 0 Hash.MAX = FF Keyspace Hash(2) = 48 Hash(3) = CD
  • 17. CustomerOrdersTable 00 55 AA FF Partition A 33.33 % Keyspace 33.33 % Provisioned Capacity Partition B 33.33 % Keyspace 33.33 % Provisioned Capacity Partition C 33.33 % Keyspace 33.33 % Provisioned Capacity Hash.MIN = 0 Hash.MAX = FF Keyspace Time Partition A 33.33 % Keyspace 33.33 % Provisioned Capacity Partition B 33.33 % Keyspace 33.33 % Provisioned Capacity Partition D Partition E 16.66 % 16.66 % 16.66 % 16.66 % Partition split due to partition size 00 55 AA FF Partition A 33.33 % Keyspace 33.33 % Provisioned Capacity Partition B 33.33 % Keyspace 33.33 % Provisioned Capacity Partition C 33.33 % Keyspace 33.33 % Provisioned Capacity Time Partition A Partition C 16.66 % 16.66 % 16.66 % 16.66 % Partition split due to capacity increase 16.66 % 16.66 % 16.66 % 16.66 % 16.66 % 16.66 % 16.66 % 16.66 % Partition B Partition D Partition E Partition F The desired size of a partition is 10GB* and when a partition surpasses this it can split *=subject to change Split for partition size The desired capacity of a partition is expressed as: 3w + 1r < 3000 * Where w = WCU & r = RCU *=subject to change Split for provisioned capacity Partitioning
  • 18. Partition A 1000 RCUs 100 WCUs Partition C 1000 RCUs 100 WCUs Host A Host C Availability Zone A Partition A 1000 RCUs 100 WCUs Partition C 1000 RCUs 100 WCUs Host E Host G Availability Zone B Partition A 1000 RCUs 100 WCUs Partition C 1000 RCUs 100 WCUs Host H Host J Availability Zone C CustomerOrdersTable 54:∞00:0 54:∞00:0 54:∞00:0 FF:∞AA:0 FF:∞AA:0 FF:∞AA:0 Data is replicated to three Availability Zones by design 3-way replication OrderId: 1 CustomerId: 1 ASIN: [B00X4WHP5E] Hash(1) = 7B Partition B 1000 RCUs 100 WCUs Host B Host F Host I Partition B 1000 RCUs 100 WCUs Partition B 1000 RCUs 100 WCUs A9:∞55:0 A9:∞55:0 A9:∞55:0 Partitioning
  • 19. DynamoDB Streams Partition A Partition B Partition C üOrdered stream of item changes üExactly once, strictly ordered by key üHighly durable, scalable ü24-hour retention üSub-second latency üCompatible with Kinesis Client Library DynamoDB Streams 1 Shards have a lineage and automatically close after time or when the associated DynamoDB partition splits 2 3 Updates KCL Worker Amazon Kinesis Client Library Application KCL Worker KCL Worker GetRecords Amazon DynamoDB Table DynamoDB Streams Stream Shards
  • 20. TTL job Time-To-Live (TTL) Amazon DynamoDB Table CustomerActiveOrder OrderId: 1 CustomerId: 1 MyTTL: 1492641900 DynamoDB Streams Amazon Kinesis Amazon Redshift An epoch timestamp marking when an item can be deleted by a background process, without consuming any provisioned capacity Time-To-Live Removes data that is no longer relevant
  • 21. Time-To-Live (TTL) üTTL items identifiable in DynamoDB Streams üConfiguration protected by AWS Identity and Access Management (IAM), auditable with AWS CloudTrail üEventual deletion, free to use
  • 22. DynamoDB Auto Scaling Specify: 1) Target capacity in percent 2) Upper and lower bound
  • 23. DynamoDB Accelerator Amazon DynamoDB msµs DAXYour App in VPC Amazon DynamoDB Accelerator (DAX)
  • 24. DynamoDB Accelerator (DAX) Private IP, Client-side Discovery Supports AWS Java SDK on launch, with more AWS SDKs to come Cluster based, Multi-AZ Separate Query and Item cache
  • 25. DynamoDB in the VPC Availability Zone #1 Availability Zone #2 Private Subnet Private Subnet VPC endpoint web app server security group security group oMicroseconds latency in-memory cache oMillions of requests per second oFully managed, highly available oRole-based access control oNo IGW or VPC endpoint required DAX oDynamoDB in the VPC oIAM resource policy restricted VPC Endpoints AWS Lambda security group security group DAX web app server DAX
  • 26. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved DynamoDB Global Tables ( G A ) F i r s t f u l l y m a n a g e d , m u l t i - m a s t e r , m u l t i - r e g i o n d a t a b a s e Build high performance, globally distributed applications Low latency reads and writes to locally available tables Disaster proof with multi-region redundancy Easy to set up and no application re-writes required Globally dispersed users Replica (N. America) Replica (Europe) Replica (Asia) Global App Global Table
  • 27. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved Amazon DynamoDB – Backup and Restore F i r s t N o S Q L d a t a b a s e t o a u t o m a t e o n - d e m a n d a n d c o n t i n u o u s b a c k u p s Point in time restore for short-term retention and data corruption protection (coming soon) Back up hundreds of TB instantaneously with NO performance impact On-demand backups for long- term data archival and compliance (GA)
  • 28. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved Amazon DynamoDB E n c r y p t i o n @ r e s t ( c o m i n g s o o n ) Server-side encryption Supports compliance certifications No application code rewrites
  • 29. Getting started? Amazon DynamoDB Local Document SDKs DynamoDB as a target for AWS Data Migration Service
  • 30. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved Pop-up Loft aws.amazon.com/activate Everything and Anything Startups Need to Get Started on AWS