Se ha denunciado esta presentación.
Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.

DynamoDB & DAX

137 visualizaciones

Publicado el

Amazon DynamoDB is a fast and flexible NoSQL database service for all applications that need consistent, single-digit millisecond latency at any scale. It is a fully managed cloud database and supports both document and key-value store models. Its flexible data model, reliable performance, and automatic scaling of throughput capacity, makes it a great fit for mobile, web, gaming, ad tech, IoT, and many other applications. We’ll take a look at how DynamoDB works and how it can be accelerated by DAX, the DynamoDB Accelerator.

Speaker: Lex Crosett - Solutions Architect, AWS

  • Sé el primero en comentar

  • Sé el primero en recomendar esto

DynamoDB & DAX

  1. 1. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. DynamoDB and DAX Lex Crosett Enterprise Solutions Architect crosettl@amazon.com
  2. 2. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Characteristics of internet-scale apps Users 1 million+ Data volume TB, PB, EB Locality Global Performance Milliseconds, microseconds Request rate Millions Access Mobile, IoT, devices Scale Up and down Economics Pay as you go Developer access Instant API access
  3. 3. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Purpose-built databases for internet-scale apps The world’s largest e-commerce business, Amazon, runs on nonrelational cloud databases due to scale, performance, and maintenance benefits. — Werner Vogels CTO, Amazon
  4. 4. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon DynamoDB Fully managed nonrelational database for any scale Fast, consistent performance Virtually unlimited throughput Virtually unlimited storage Encryption at rest and transit Fine-grained access control PCI, HIPAA, FIPS140-2 eligible Maintenance-free Serverless Auto scaling Backup and restore Global tables
  5. 5. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Fully managed Maintenance-free Think of all the other things you could do if you didn’t have to manage: Security Operating system patching Database patching Access control Audit Encryption Compliance Durability Sustain server, rack, and datacenter outages Re-replicate data quickly upon hardware failure Manage backup and restore Availability High availability configuration Monitoring Cross-region replication Performance Performance tuning Indexing In-memory caching Scalability Capacity planning Host provisioning Host repair and retirement
  6. 6. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. No servers No administration AWS Lambda integration Fully managed Serverless
  7. 7. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. $$$ Savings Automated scaling policies Scales up when you need it Scales down when you don’t Scheduled auto scaling Fully managed Auto scaling
  8. 8. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Continuous backups for point-in-time recovery On-demand backups for long-term data archiving and compliance Back up PBs of data instantly with no performance impact Fully managed Backup and restore
  9. 9. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Fully managed Global tables Build high-performance, globally distributed applications Low latency reads and writes to locally available tables Multiregion redundancy and resiliency Easy to set up and no application rewrites required
  10. 10. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. High performance Millions of requests per second Trillions of items Petabytes of storage Single-digit millisecond read and write latencies
  11. 11. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Only 3 millisecond variance Consistent and low latency More than 1 million requests per second High request volume High performance
  12. 12. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Fully managed, highly available cache for DynamoDB Even faster— microsecond latency Scales to millions of requests per second API compatible Your applications DAX DynamoDB High performance Amazon DynamoDB Accelerator (DAX)
  13. 13. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Server-side encryption for fully managed, easy to set up table-level encryption Access DynamoDB via secure Amazon VPC endpoints Secure Encryption
  14. 14. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Fine-grained access control to allow or deny access to items and attributes based on the needs of your application AWS Identity and Access Management (IAM) policies to regulate access to items and attributes stored in tables Secure Access control
  15. 15. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Customer stories
  16. 16. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon DynamoDB 200,000+ customers
  17. 17. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Backup and restore on mobile app for 300 million users Migrated from Cassandra to DynamoDB • Almost 1 PB in DynamoDB, 130 million daily API requests • Consistent performance and 70% cost savings (TCO)“DynamoDB provided consistent high performance at a drastically lower cost than Cassandra.” —Seongkyu Kim Samsung
  18. 18. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. One of the 10 largest banks in the U.S. serving 45 million customers Migrated from mainframe to DynamoDB: • Serving millions of transactions with the new serverless architecture • Reduced latency: end-to-end requests <55ms • Unbound scale for app developers“New solution is so much faster...with an average response time of 55ms.” —Srini Uppalapati CapitalOne
  19. 19. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Profile, match, and swipe data 25 TB data, 20 billion matches, 190 countries Migrated from MongoDB to DynamoDB • 60% cost savings (TCO) “DynamoDB helps us achieve greater developer efficiency…at a lower cost.” —Jun-young Kwak Tinder
  20. 20. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. “Our internal customers saw processing delays decrease from 1 second to 100 milliseconds. It shows how DynamoDB can serve as the foundation of a highly efficient, mission-critical system.” —Mike Thomas Amazon Every time you buy something on Amazon.com, the internal Herd system supports 100s of millions of active workflows that need to start within a couple of seconds Migrated from Oracle to DynamoDB: • Workflow processing delays dropped from 1s to 100ms • Scaling and maintenance effort dropped 10x • Retired more than 300 Oracle hosts
  21. 21. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Partition Key Mandatory Key-value access pattern Determines data distribution Optional Model 1:N relationships Enables rich query capabilities DynamoDB table A1 (partition key) A2 (sort key) A3 A4 A7 A1 (partition key) A2 (sort key) A6 A4 A5 A1 (partition key) A2 (sort key) A1 (partition key) A2 (sort key) A3 A4 A5 Sort Key Table Items
  22. 22. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. DynamoDB LSIs Can only be defined as part of initial table creation A1 (partition key) A3 (sort key) A2 A4 A5 A1 (partition key) A4 (sort key) A2 A3 A5 A1 (partition key) A5 (sort key) A2 A3 A4 • Alternate sort key attribute • Index is local to a partition key Local secondary indexes
  23. 23. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. RCUs/WCUs provisioned separately for GSIs INCLUDE A2 ALL KEYS_ONLY A3 (partition key) A1 (table key) A2 A4 A7 A3 (partition key) A1 (table key) A3 (partition key) A1 (table key) A2 • Alternate partition (+sort) key • Index is across all table partition keys • Can be added or removed anytime • Eventually consistent A3 (partition key) A1 (table key) A2 A4 A7 A3 (partition key) A1 (table key) A2 A3 (partition key) A1 (table key) Global secondary indexes DynamoDB GSIs
  24. 24. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Data types Type DynamoDB Type String String Integer, Float Number Timestamp Number or String Blob Binary Boolean Bool Null Null List List Set Set of String, Number, or Binary Map Map
  25. 25. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Scaling Throughput Provision any amount of throughput to a table Size Add any number of items to a table • Max item size is 400 KB • Local secondary indexes (LSIs) limit the number of range keys due to 10 GB limit Scaling Achieved through partitioning
  26. 26. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Throughput Provisioned at the table level • Write capacity units (WCUs) are measured in 1 KB per second • Read capacity units (RCUs) are measured in 4 KB per second • RCUs measure strictly consistent reads • Eventually consistent reads cost 1/2 of consistent reads Read and write throughput limits are independent WCURCU
  27. 27. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. What bad NoSQL looks like… Partition Time Heat
  28. 28. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Getting the most out of DynamoDB throughput “To get the most out of DynamoDB throughput, create tables where the hash key element has a large number of distinct values, and values are requested fairly uniformly, as randomly as possible.” —DynamoDB Developer Guide Space: access is evenly spread over the key space Time: requests arrive evenly spaced in time
  29. 29. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Much better picture…
  30. 30. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 25% Requests in excess of the soft partition limit are throttled, after burst capacity is used up capacity soft limit capacity hard limit Adaptive Capacity kicks in, in response to throttling, allowing this partition to consume unused table capacity, above the partition soft limit partition 1 partition 2 partition 3 partition 4
  31. 31. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. SQL vs. NoSQL access pattern
  32. 32. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Hierarchical data structures as items Use composite sort key to define a hierarchy Highly selective result sets with sort queries Index anything, scales to any size Primary Key Attributes ProductID type Items 1 bookID title author genre publisher datePublished ISBN Ringworld Larry Niven Science Fiction Ballantine Oct-70 0-345-02046-4 2 albumID title artist genre label studio released producer Dark Side of the Moon Pink Floyd Progressive Rock Harvest Abbey Road 3/1/73 Pink Floyd 2 albumID:trackID title length music vocals Speak to Me 1:30 Mason Instrumental 2 albumID:trackID title length music vocals Breathe 2:43 Waters, Gilmour, Wright Gilmour 2 albumID:trackID title length music vocals On the Run 3:30 Gilmour, Waters Instrumental 3 movieID title genre writer producer Idiocracy Scifi Comedy Mike Judge 20th Century Fox 3 movieID:actorID name character image Luke Wilson Joe Bowers img2.jpg 3 movieID:actorID name character image Maya Rudolph Rita img3.jpg 3 movieID:actorID name character image Dax Shepard Frito Pendejo img1.jpg
  33. 33. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. … or as documents (JSON) JSON data types (M, L, BOOL, NULL) Document SDKs available Indexing only by using DynamoDB Streams or AWS Lambda 400 KB maximum item size (limits hierarchical data structure) Primary Key Attributes ProductID Items 1 id title author genre publisher datePublished ISBN bookID Ringworld Larry Niven Science Fiction Ballantine Oct-70 0-345-02046-4 2 id title artist genre Attributes albumID Dark Side of the Moon Pink Floyd Progressive Rock { label:"Harvest", studio: "Abbey Road", published: "3/1/73", producer: "Pink Floyd", tracks: [{title: "Speak to Me", length: "1:30", music: "Mason", vocals: "Instrumental"},{title: ”Breathe", length: ”2:43", music: ”Waters, Gilmour, Wright", vocals: ”Gilmour"},{title: ”On the Run", length: “3:30", music: ”Gilmour, Waters", vocals: "Instrumental"}]} 3 id title genre writer Attributes movieID Idiocracy Scifi Comedy Mike Judge { producer: "20th Century Fox", actors: [{ name: "Luke Wilson", dob: "9/21/71", character: "Joe Bowers", image: "img2.jpg"},{ name: "Maya Rudolph", dob: "7/27/72", character: "Rita", image: "img1.jpg"},{ name: "Dax Shepard", dob: "1/2/75", character: "Frito Pendejo", image: "img3.jpg"}]
  34. 34. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. DynamoDB Streams Partition A Partition B Partition C Ordered stream of item changes Exactly once, strictly ordered by key Highly durable, scalable 24-hour retention Sub-second latency Compatible with Kinesis Client Library DynamoDB Streams 1 Shards have a lineage and automatically close after time or when the associated DynamoDB partition splits 2 3 Updates KCL Worker Amazon Kinesis Client Library Application KCL Worker KCL Worker GetRecords Amazon DynamoDB Table DynamoDB Streams Stream Shards
  35. 35. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Stream Table Partition 1 Partition 2 Partition 3 Partition 4 Partition 5 Table Shard 1 Shard 2 Shard 3 Shard 4 KCL Worker KCL Worker KCL Worker KCL Worker Amazon Kinesis Client Library application DynamoDB client application Updates DynamoDB Streams and Amazon Kinesis Client Library
  36. 36. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. DynamoDB Streams and AWS Lambda
  37. 37. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. TTL job Time-To-Live (TTL) Amazon DynamoDB Table CustomerActiveOrder OrderId: 1 CustomerId: 1 MyTTL: 1492641900 DynamoDB Streams Amazon Kinesis Amazon Redshift An epoch timestamp marking when an item can be deleted by a background process, without consuming any provisioned capacity Time-To-Live Removes data that is no longer relevant
  38. 38. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. DynamoDB Accelerator Amazon DynamoDB msμs DAXYour App in VPC Amazon DynamoDB Accelerator (DAX)
  39. 39. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. DynamoDB Accelerator (DAX) Private IP, Client-side Discovery Supports AWS SDKs Cluster based, Multi-AZ Separate Query and Item cache
  40. 40. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. DynamoDB in the VPC Availability Zone #1 Availability Zone #2 Private Subnet Private Subnet VPC endpoint web app server security group security group oMicroseconds latency in-memory cache oMillions of requests per second oFully managed, highly available oRole-based access control oNo IGW or VPC endpoint required DAX oDynamoDB in the VPC oIAM resource policy restricted VPC Endpoints AWS Lambda security group security group DAX web app server DAX
  41. 41. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved DynamoDB Global Tables F i r s t f u l l y m a n a g e d , m u l t i - m a s t e r , m u l t i - r e g i o n d a t a b a s e Build high performance, globally distributed applications Low latency reads and writes to locally available tables Disaster proof with multi-region redundancy Easy to set up and no application re-writes required Globally dispersed users Replica (N. America) Replica (Europe) Replica (Asia) Global App Global Table
  42. 42. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved Amazon DynamoDB – Backup and Restore F i r s t N o S Q L d a t a b a s e t o a u t o m a t e o n - d e m a n d a n d c o n t i n u o u s b a c k u p s Point in time restore for short-term retention and data corruption protection Back up hundreds of TB instantaneously with NO performance impact On-demand backups for long- term data archival and compliance
  43. 43. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved Amazon DynamoDB E n c r y p t i o n @ r e s t Server-side encryption Supports compliance certifications No application code rewrites
  44. 44. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Get started Visit DynamoDB.com

×