Amazon DynamoDB Workshop
•
3 recomendaciones•4,271 vistas
Amazon DynamoDB is a fully managed NoSQL database service that provides fast and predictable performance with seamless scalability. You can use Amazon DynamoDB to create a database table that can store and retrieve any amount of data, and serve any level of request traffic. Amazon DynamoDB automatically spreads the data and traffic for the table over a sufficient number of servers to handle the request capacity specified by the customer and the amount of data stored, while maintaining consistent and fast performance.
Recomendados
Más contenido relacionado
La actualidad más candente
La actualidad más candente (20)
Similar a Amazon DynamoDB Workshop
Similar a Amazon DynamoDB Workshop (20)
Más de Amazon Web Services
Más de Amazon Web Services (20)
Último
Último (20)
Amazon DynamoDB Workshop
- 2. RDBMS Poor Availability Limited Scalability High Cost
- 5. lack of strong opera=onal consistency every engineer needs to learn distributed systems complexity
- 6. Non-Relational Fast & Predictable Performance Seamless Scalability Easy Administration
- 7. App/Web Tier Client Tier Data Tier Search Cache Blob Store RDBMSNoSQL Data Warehouse
- 8. Data Tier Search Cache Blob Store RDBMSNoSQL Data Warehouse logging analy=cs key/value simple query rich search hot reads complex queries and transac=ons
- 9. Data Tier Amazon DynamoDB Amazon RDS Amazon ElastiCache Amazon S3 Amazon Redshift Amazon CloudSearch logging analy=cs key/value simple query rich search hot reads complex queries and transac=ons
- 10. database service automated operations predictable performance fast development always durable low latency cost effective =
- 13. WRITES Continuously replicated to 3 zones Quorum acknowledgment Persisted to disk (custom SSD) READS Strongly or eventually consistent No trade-off in latency
- 21. Images Table
- 22. Images Table User Image Date Link Bob aed4c 2013-10-01 s3://… Bob cf2e2 2013-09-05 s3://… Bob f93bae 2013-10-08 s3://… Alice ca61a 2013-09-12 s3://… Hash and range primary key schema (composite key)
- 23. Images Table User Image Date Link Bob aed4c 2013-10-01 s3://… Bob cf2e2 2013-09-05 s3://… Bob f93bae 2013-10-08 s3://… Alice ca61a 2013-09-12 s3://… Range: Index is sorted by range key, can do range queries Hash: Equality queries only
- 24. Images Table User Image Date Link Bob aed4c 2013-10-01 s3://… Bob cf2e2 2013-09-05 s3://… Bob f93bae 2013-10-08 s3://… Alice ca61a 2013-09-12 s3://… Bob Query for Bob’s Images Query FROM Images WHERE User=Bob
- 27. Images Table User Image Date Link Bob aed4c 2013-10-01 s3://… Bob cf2e2 2013-09-05 s3://… Bob f93bae 2013-10-08 s3://… Alice ca61a 2013-09-12 s3://… User Date Image Bob 2013-09-05 cf2e2 Bob 2013-10-01 aed4c Bob 2013-10-08 f93bae Alice 2013-09-12 ca61a Table ByDate Local Secondary Index Local secondary index on Date
- 28. Images Table User Image Date Link Bob aed4c 2013-10-01 s3://… Bob cf2e2 2013-09-05 s3://… Bob f93bae 2013-10-08 s3://… Alice ca61a 2013-09-12 s3://… User Date Image Bob 2013-09-05 cf2e2 Bob 2013-10-01 aed4c Bob 2013-10-08 f93bae Alice 2013-09-12 ca61a Table ByDate Local Secondary Index Query for Bob’s two most recent imagesQuery FROM Image-Index WHERE User=Bob, DESC, LIMIT 2
- 31. ImageTags Table Image User aed4c Alice aed4c Bob f93bae Bob
- 32. ImageTags Table Image User aed4c Alice aed4c Bob f93bae Bob Hash and range primary key schema
- 33. Image User aed4c Alice aed4c Bob f93bae Bob ImageTags Table Bob Query FROM ImageTags WHERE Image=aed4c
- 34. ImageTags Table Image User aed4c Alice aed4c Bob f93bae Alice f93bae Bob Bob Tag Alice in Image f93bae
- 36. ImageTags Table Image User aed4c Alice aed4c Bob f93bae Alice f93bae Bob Table
- 37. ImageTags Table Global secondary index on User, Image User Image Bob aed4c Bob f93bae Alice aed4c Alice f93bae ByUser Global Secondary Index Image User aed4c Alice aed4c Bob f93bae Alice f93bae Bob Table Alternate hash and range keys
- 38. ImageTags Table Alice Query for images tagged Alice User Image Bob aed4c Bob f93bae Alice aed4c Alice f93bae ByUser Global Secondary Index Image User aed4c Alice aed4c Bob f93bae Alice f93bae Bob Table Query FROM ImageTags-Index WHERE User=Alice
- 42. { Id : abecd, Players : [ Alice, Bob ], State : STARTED, Turn : Bob, Top-Right : O } Game Item
- 44. Amazon DynamoDB Alice Bob Update: Top-Right : O Turn : Bob
- 45. Amazon DynamoDB Alice Bob Update: Top-Left : X Turn : Alice
- 46. Alice Bob (1) Amazon DynamoDB Bob (2) Bob (3)
- 47. Alice Bob (1) Amazon DynamoDB Bob (2) Bob (3)
- 48. Alice Bob (1) Amazon DynamoDB Bob (2) Bob (3)
- 52. Bob (1) Bob (2) Read Version: 1 Turn: Bob Version: 1 Turn: Bob Time
- 53. Bob (1) Bob (2) Read Read Version: 1 Turn: Bob Version: 1 Turn: Bob Version: 1 Turn: Bob Time
- 54. Bob (1) Bob (2) Time Read Read Version: 1 Turn: Bob Version: 1 Turn: Bob Version: 1 Turn: Bob Version: 2 Turn: Alice Top-Right: X Version: 2 Turn: Alice Top-Left: X
- 55. Bob (1) Bob (2) Read Read Version: 1 Turn: Bob Version: 1 Turn: Bob Version: 1 Turn: Bob Version: 2 Turn: Alice Top-Right: X Version: 2 Turn: Alice Top-Left: X Version: 2 Turn: Alice Top-Right: X Put If V=1 Time
- 56. Bob (1) Bob (2) Read Read Version: 1 Turn: Bob Version: 1 Turn: Bob Version: 1 Turn: Bob Version: 2 Turn: Alice Top-Right: X Version: 2 Turn: Alice Top-Right: X Version: 2 Turn: Alice Top-Right: X Put If V=1 Put If V=1 Time
- 57. @DynamoDBTable( tableName = "TicTacToeGame" ) public class Game { private String id; // ... other attributes private Long version; @DynamoDBHashKey public String getId() { ... } @DynamoDBVersionAttribute public Long getVersion() { ... } }
- 58. @DynamoDBTable( tableName = "TicTacToeGame" ) public class Game { private String id; // ... other attributes private Long version; @DynamoDBHashKey public String getId() { ... } @DynamoDBVersionAttribute public Long getVersion() { ... } }
- 63. Need to scale for the sale Administrator (You) ProductCatalog Table
- 64. Administrator (You) Provision 5000 read capacity units ProductCatalog Table
- 65. Administrator (You) 2500 read capacity units (each) Provision 5000 read capacity units Partition 1 Partition 2 ProductCatalog Table
- 66. Administrator (You) ProductCatalog Table Provision 100,000 read capacity units Partition 1 2000 RCU Partition K 2000 RCU Partition M 2000 RCU Partition 50 2000 RCU
- 67. Product A Product B Shoppers Partition 1 2000 RCU Partition K 2000 RCU Partition M 2000 RCU Partition 50 2000 RCU ProductCatalog Table
- 68. Partition 1 2000 RCU Partition K 2000 RCU Partition M 2000 RCU Partition 50 2000 RCU Product A Product B Shoppers ProductCatalog Table
- 69. Partition 1 2000 RCU Partition K 2000 RCU Partition M 2000 RCU Partition 50 2000 RCU Product A Product B Shoppers ProductCatalog Table
- 70. Partition 1 2000 RCU Partition K 2000 RCU Partition M 2000 RCU Partition 50 2000 RCU Product A Product B Shoppers ProductCatalog Table
- 74. 0 400 800 1200 1600 CapacityUnits Time Provisioned Consumed “save up” unused capacity consume saved up capacity
- 75. 0 400 800 1200 1600 CapacityUnits Time Provisioned Consumed Attempted Burst capacity: 300 seconds (1200 × 300 = 3600 CU) Throttled requests
- 76. Partition 1 2000 RCU Partition K 2000 RCU Partition M 2000 RCU Partition 50 2000 RCU Product A Product B Shoppers ProductCatalog Table
- 78. RequestsPerSecond Item Primary Key Request Distribution Per Hash Key DynamoDB Requests
- 79. RequestsPerSecond Item Primary Key Request Distribution Per Hash Key DynamoDB Requests Cache Hits
- 80. Partition 1 Partition 2 ProductCatalog Table User DynamoDB User SELECT Id, Description, ... FROM ProductCatalog WHERE Id="POPULAR_PRODUCT"
- 82. Shard write-heavy items Design for retries
- 83. Candidate A Votes: 20 Candidate B Votes: 30 Voter Votes Table
- 84. Candidate A Votes: 21 Candidate B Votes: 30 Votes Table UPDATE Votes SET Votes = Votes + 1 WHERE Id=“Candidate A”Voter (aka atomic increment)
- 85. Need to scale for the election Administrator (You) Votes Table
- 86. Administrator (You) Provision 200,000 write capacity units Partition 1 1000 WCU Partition K 1000 WCU Partition M 1000 WCU Partition N 1000 WCU Votes Table
- 87. Candidate A Candidate B Voters Partition 1 1000 WCU Partition K 1000 WCU Partition M 1000 WCU Partition N 1000 WCU Votes Table
- 88. Partition 1 1000 WCU Partition K 1000 WCU Partition M 1000 WCU Partition N 1000 WCU Votes Table Candidate A Candidate B Voters
- 89. Candidate A Candidate A_2 Candidate A_1 Candidate A_3 Candidate A_4 Candidate A_7 Candidate A_6 Candidate A_8 Candidate A_5
- 90. Candidate A_2 Candidate B_1 Candidate B_2 Candidate B_3 Candidate B_5 Candidate B_4 Candidate B_7 Candidate B_6 Candidate A_1 Candidate A_3 Candidate A_4 Candidate A_7 Candidate B_8 Candidate A_6 Candidate A_8 Candidate A_5 Voter Votes Table
- 91. Candidate A_2 Candidate B_1 Candidate B_2 Candidate B_3 Candidate B_5 Candidate B_4 Candidate B_7 Candidate B_6 Candidate A_1 Candidate A_3 Candidate A_4 Candidate A_7 Candidate B_8 UpdateItem: “CandidateA_” + rand(0, 10) ADD 1 to Votes Candidate A_6 Candidate A_8 Candidate A_5 Voter Votes Table
- 92. Votes Table Candidate A_2 Candidate B_1 Candidate B_2 Candidate B_3 Candidate B_5 Candidate B_4 Candidate B_7 Candidate B_6 Candidate A_1 Candidate A_3 Candidate A_4 Candidate A_5 Candidate A_6 Candidate A_8 Candidate A_7 Candidate B_8 Periodic Process Candidate A Total: 2.5M 1. Sum 2. Store Voter
- 94. Substitute indexes for filters
- 96. GameId Date Host Opponent Status d9bl3 2014-10-02 David Alice DONE 72f49 2014-09-30 Alice Bob PENDING o2pnb 2014-10-08 Bob Carol IN_PROGRESS b932s 2014-10-03 Carol Bob PENDING ef9ca 2014-10-03 David Bob IN_PROGRESS Games Table
- 97. GameId Date Host Opponent Status d9bl3 2014-10-02 David Alice DONE 72f49 2014-09-30 Alice Bob PENDING o2pnb 2014-10-08 Bob Carol IN_PROGRESS b932s 2014-10-03 Carol Bob PENDING ef9ca 2014-10-03 David Bob IN_PROGRESS Games Table
- 100. Secondary Index Bob Opponent Status GameId Date Host Alice DONE d9bl3 2014-10-02 David Carol IN_PROGRESS o2pnb 2014-10-08 Bob Bob IN_PROGRESS ef9ca 2014-10-03 David Bob PENDING 72f49 2014-10-03 Alice Bob PENDING b932s 2014-09-30 Carol
- 101. Secondary Index Opponent Status GameId Date Host Alice DONE d9bl3 2014-10-02 David Carol IN_PROGRESS o2pnb 2014-10-08 Bob Bob IN_PROGRESS ef9ca 2014-10-03 David Bob PENDING 72f49 2014-10-03 Alice Bob PENDING b932s 2014-09-30 Carol Bob
- 102. Secondary Index Opponent Status GameId Date Host Alice DONE d9bl3 2014-10-02 David Carol IN_PROGRESS o2pnb 2014-10-08 Bob Bob IN_PROGRESS ef9ca 2014-10-03 David Bob PENDING 72f49 2014-10-03 Alice Bob PENDING b932s 2014-09-30 Carol Bob SELECT * FROM Game WHERE Opponent='Bob' AND Status='PENDING'
- 103. Bob SELECT * FROM Game WHERE Opponent='Bob' AND Status='PENDING' ORDER BY Date? Opponent Status GameId Date Host Alice DONE d9bl3 2014-10-02 David Carol IN_PROGRESS o2pnb 2014-10-08 Bob Bob IN_PROGRESS ef9ca 2014-10-03 David Bob PENDING 72f49 2014-10-03 Alice Bob PENDING b932s 2014-09-30 Carol Secondary Index
- 104. SELECT * FROM Game WHERE Opponent='Bob' AND Status='IN_PROGRESS' ORDER BY Date? (hash) (range) (?)
- 105. Secondary Index Opponent Date GameId Status Host Alice 2014-10-02 d9bl3 DONE David Carol 2014-10-08 o2pnb IN_PROGRESS Bob Bob 2014-09-30 72f49 PENDING Alice Bob 2014-10-03 b932s PENDING Carol Bob 2014-10-03 ef9ca IN_PROGRESS David Bob
- 106. Secondary Index Bob Opponent Date GameId Status Host Alice 2014-10-02 d9bl3 DONE David Carol 2014-10-08 o2pnb IN_PROGRESS Bob Bob 2014-09-30 72f49 PENDING Alice Bob 2014-10-03 b932s PENDING Carol Bob 2014-10-03 ef9ca IN_PROGRESS David SELECT * FROM Game WHERE Opponent='Bob' ORDER BY Date DESC FILTER ON Status='PENDING' (filtered out)
- 108. Bob
- 109. Bob
- 112. Secondary Index Opponent StatusDate GameId Host Alice DONE_2014-10-02 d9bl3 David Carol IN_PROGRESS_2014-10-08 o2pnb Bob Bob IN_PROGRESS_2014-10-03 ef9ca David Bob PENDING_2014-09-30 72f49 Alice Bob PENDING_2014-10-03 b932s Carol
- 113. Opponent StatusDate GameId Host Alice DONE_2014-10-02 d9bl3 David Carol IN_PROGRESS_2014-10-08 o2pnb Bob Bob IN_PROGRESS_2014-10-03 ef9ca David Bob PENDING_2014-09-30 72f49 Alice Bob PENDING_2014-10-03 b932s Carol Secondary Index Bob SELECT * FROM Game WHERE Opponent='Bob' AND StatusDate STARTS_WITH 'PENDING'
- 118. client Amazon DynamoDB Web Service Elastic Load Balancing URL Shortener App Target Website example.com website
- 119. client example.com website Web Service Elastic Load Balancing GET /u/d83 HTTP 1.1 301 Moved Permanently Location: http://www.example.com/maps/... Amazon DynamoDB
- 120. client example.com website Web Service Elastic Load Balancing GET /maps/Pike+Place... Amazon DynamoDB
- 121. ShortUrl RedirectTo CreatedBy CreateDate example.com/u/d83 <long url> David 2014-09-09 example.com/u/abc <long url> Alice 2013-08-17 example.com/u/123 <long url> 2013-12-21 example.com/u/ea2 <long url> Bob 2014-03-30 Url Table Table
- 122. ShortUrl RedirectTo CreatedBy CreateDate example.com/u/d83 <long url> David 2014-09-09 example.com/u/abc <long url> Alice 2013-08-17 example.com/u/123 <long url> 2013-12-21 example.com/u/ea2 <long url> Bob 2014-03-30 Url Table Item
- 123. ShortUrl RedirectTo CreatedBy CreateDate example.com/u/d83 <long url> David 2014-09-09 example.com/u/abc <long url> Alice 2013-08-17 example.com/u/123 <long url> 2013-12-21 example.com/u/ea2 <long url> Bob 2014-03-30 Url Table Attributes (schema-less)
- 124. ShortUrl RedirectTo CreatedBy CreateDate example.com/u/d83 <long url> David 2014-09-09 example.com/u/abc <long url> Alice 2013-08-17 example.com/u/123 <long url> 2013-12-21 example.com/u/ea2 <long url> Bob 2014-03-30 Url Table Primary Key (Hash Index)
- 125. Item (JSON Document)
- 126. String Set String Number Map Other types: • Binary • List • Null • Number Set • Binary Set Extra features: • Retrieve & Update parts of documents Boolean
- 127. 400 KB
- 129. dynamodb.putItem({ TableName: 'Url', Item: { ShortUrl: "short.example.com/4ua93", RedirectTo: "long.example.com/maps/Pike+Place...", IsLive: true, Tags: ["map", "directions"], Metadata: { CreatedBy: "David", CreateDate: "2014-19-21T08:49", TargetViews: 100 }, Stats: { LastVisited: "2014-10-30T10:31", PageViews: 326 } } }, function(err, data) { if (err) print(err); // an error occurred else print(data); // successful response }); Request Response (empty)
- 130. dynamodb.getItem({ TableName: 'Url', Key: { "ShortUrl": "short.example.com/4ua93" }, ConsistentRead: true }, function(err, data) { if (err) print(err); else print(data); }); Optional read-your-writes consistency
- 131. dynamodb.scan({ TableName: 'Url', ProjectionExpression: 'ShortUrl, ' + 'RedirectTo, Stats.LastVisited', Limit: 2 }).eachPage(function(err, data) { if (err) print(err); else if (data) print(data); }); SQL-like expressions Paginated (max 1 MB per page)
- 132. dynamodb.scan({ TableName: 'Url', FilterExpression: 'begins_with(RedirectTo, :domain) ' + 'AND contains(Tags, :tag)' + 'AND (' + ' Stats.PageViews > Metadata.TargetViews' + ' OR ' + ' Stats.StarCount > Metadata.TargetStars' + ')', ExpressionAttributeValues: { ':domain': 'long.example.com/', ':tag': 'directions' } }).eachPage(function(err, data) { if (err) print(err); else if (data) print(data); }); Nested conditions Compare existing attributes Filter Expressions and Improved Conditions!
- 133. Update when viewed
- 134. dynamodb.updateItem({ TableName: 'Url', Key: { "ShortUrl": "short.example.com/4ua93" }, UpdateExpression: 'SET Usage.Views = Usage.Views + :increment_by, ' + 'Usage.LastVisited = :now', ExpressionAttributeValues: { ':increment_by' : 1, ':now' : new Date().toISOString() }, ReturnValues: 'ALL_NEW', ConditionExpression: 'attribute_exists(ShortUrl)' }, function(err, data) { if (err) print(err); else print(data); }); Increment views by 1
- 135. dynamodb.updateItem({ TableName: 'Url', Key: { "ShortUrl": "short.example.com/4ua93" }, UpdateExpression: 'SET Usage.Views = Usage.Views + :increment_by, ' + 'Usage.LastVisited = :now', ExpressionAttributeValues: { ':increment_by' : 1, ':now' : new Date().toISOString() }, ReturnValues: 'ALL_NEW', ConditionExpression: 'attribute_exists(ShortUrl)' }, function(err, data) { if (err) print(err); else print(data); }); Writes are also free reads!
- 139. David Recipient Date Sender Message David 2014-10-02 Bob … … 48 more messages for David … David 2014-10-03 Alice … Alice 2014-09-28 Bob … Alice 2014-10-01 Carol … (Many more messages) MessagesTable
- 140. Recipient Date Sender Message David 2014-10-02 Bob … … 48 more messages for David … David 2014-10-03 Alice … Alice 2014-09-28 Bob … Alice 2014-10-01 Carol … (Many more messages) David MessagesTable SELECT * FROM Messages WHERE Recipient='David' LIMIT 50 ORDER BY Date DESC
- 141. Recipient Date Sender Message David 2014-10-02 Bob … … 48 more messages for David … David 2014-10-03 Alice … Alice 2014-09-28 Bob … Alice 2014-10-01 Carol … (Many more messages) David MessagesTable Cost of query? SELECT * FROM Messages WHERE Recipient='David' LIMIT 50 ORDER BY Date DESC
- 142. Recipient Date Sender Message David 2014-10-02 Bob … … 48 more messages for David … David 2014-10-03 Alice … Alice 2014-09-28 Bob … Alice 2014-10-01 Carol … (Many more messages) MessagesTable Average item size = 256 KB Large message bodies Attachments? David SELECT * FROM Messages WHERE Recipient='David' LIMIT 50 ORDER BY Date DESC
- 143. Recipient Date Sender Message David 2014-10-02 Bob … … 48 more messages for David … David 2014-10-03 Alice … Alice 2014-09-28 Bob … Alice 2014-10-01 Carol … (Many more messages) David MessagesTable 50 items × 256 KB each SELECT * FROM Messages WHERE Recipient='David' LIMIT 50 ORDER BY Date DESC
- 145. Recipient Date Sender Subject MsgId David 2014-10-02 Bob Hi!… afed David 2014-10-03 Alice RE: The… 3kf8 Alice 2014-09-28 Bob FW: Ok… 9d2b Alice 2014-10-01 Carol Hi!... ct7r Inbox Table Messages Table MsgId Body 9d2b … 3kf8 … ct7r … afed … (Bulk message data)(Message metadata, indexed by recipient)
- 146. Recipient Date Sender Subject MsgId David 2014-10-02 Bob Hi!… afed David 2014-10-03 Alice RE: The… 3kf8 Alice 2014-09-28 Bob FW: Ok… 9d2b Alice 2014-10-01 Carol Hi!... ct7r Inbox Table Messages Table MsgId Body 9d2b … 3kf8 … ct7r … afed … David 1. Query Inbox: 1 RCU (50 sequential items at 128 bytes)
- 147. Recipient Date Sender Subject MsgId David 2014-10-02 Bob Hi!… afed David 2014-10-03 Alice RE: The… 3kf8 Alice 2014-09-28 Bob FW: Ok… 9d2b Alice 2014-10-01 Carol Hi!... ct7r Inbox Table Messages Table MsgId Body 9d2b … 3kf8 … ct7r … afed … David 1. Query Inbox: 1 RCU 2. BatchGetItem Messages: 1600 RCU (50 separate items at 256 KB) (50 sequential items at 128 bytes)
- 148. Messages Table Inbox Table David PutItem { MsgId: 123, Body: ... } PutItem { Recipient: Steve, Date: 2014-10-23, MsgId: 123, ... }
- 149. Messages Table Inbox Table David PutItem { MsgId: 123, Body: ... } PutItem { Recipient: Steve, Date: 2014-10-23, MsgId: 123, ... }
- 150. David PutItem { MsgId: 123, Body: ..., Recipient: Steve, Sender: David, Date: 2014-10-23, ... } Inbox Global secondary index Messages Table