6. Revenue Share Calculator Service Requirements
1. Automatic calculation for the complete previous month on the 1st of each
month for each customer
2. Manual calculation via a simple GUI
• For the current month so far
• Recalculation for the previous months
3. Calculation results as downloads, download link per email
4. Archive with calculated results for each customer
5. For internal use only!
7. Serverless decision for RevShare project
• spike workloads only at the beginning of a month
• service is idle more than 99% of the time
We use Function as a Service, to never pay for idle.
7
8. RS core
calculator
Turnover info
input data:
order data & calculation rules
output data:
RS calculation & input data
email with path to
download RS
calculation result
Central DB
Management
Server
extract customer, year
and month from the
file path
Key example: revsharetool-storage-demo/input-data/1000000/import/201806/revshare-final.json
RS calculator app
customer, year, month
12. Selecting memory size for a Lambda function
• Price 0,00001667 USD per GB-second
• CPU power allocation proportional to amount of memory
• more „GB“ will often just improve performance and not increase costs!
• cheaper GB setting may save costs, when lambda function
often has to wait
12
18. Lambda free tier
• 136170 sec. per month with the most powerful CPU setting
• = more than 1h per day
• with max 1 call every 2 sec.
18
19. Amazon API
Gateway
• Expose Lambda functions as RESTful web
service
• API Management functionality built in:
• authorization and access control for API calls with IAM
• request monitoring and analytics
• overload protection
• version management
• APIs as products (SaaS, AWS Marketplace)
• SDK generation
Amazon API Gateway
20. API Gateway for different AWS services
• Access different AWS
services in a consistent
manner
22. Cost influence of API gateway
22
0,408 1,032
1,656
2,28
2,904
3,528
4,152
4,776
3,908
4,532
5,156
5,78
6,404
7,028
7,652
8,276
100 MS 400MS 700MS 1000MS 1300MS 1600MS 1900MS 2200MS
duration per call
cost per million API calls in $ for a 128 MB
function
without API Gateway with API Gateway
33. Auto Scaling
Source: Yan Cui: "The problems with DynamoDB Auto Scaling and how it might be improved"
https://hackernoon.com/the-problems-with-dynamodb-auto-scaling-and-how-it-might-be-improved-a92029c8c10b
36. DynamoDB Pricing
DynamoDB works by provisioning throughput at the table level. The throughput
is set up as follows:
• Each write capacity unit (WCU) gives 1KB/s of write throughput
• Each read capacity unit (RCU) gives 4KB/s of read throughput
• Read and write throughput are independent
37. Partitioning Math
By Capacity = (Total RCU)/3000 + (Total WCU)/1000
By Size = Total size / 10 GB
Number of partitions = CEILING (MAX (Capacity, Size))
Max RCUs per partition=3000; Max WCUs per partition=1000
Throughput (RCU & WCU) is uniformly spread across partitions
In case of exceeding throughput per partition -
ProvisionedThroughputExceededException
38. Partitioning Math
Table Size=40 GB , RCUs=1500, WCUs=400
By Capacity = 1500/3000 + 400/1000=0.9
By Size = 40 GB / 10 GB =4
Number of partitions = CEILING (MAX (0.9,4)) =4
RCUs per partition=1500/4 =375
WCUs per partition=400/4 =100
Data per partition=40 GB/4 = 10GB
39. Example of ProvisionedThroughputExceededException
DynamoDB Table with 40 GB of data -> gives 4 partitions
Source: https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/bp-partition-key-design.html#bp-partition-key-
partitions-adaptive
40. DynamoDB
Exceeding Throughput
Causes:
• Uneven distribution of data due to the wrong choice of partition key
• Frequent accessing of the same key in a partition (the most popular item,
also known as a hot key)
• A request rate greater than the provisioned throughput
41. Solutions to the hot key problems
• Distribute your key accross partitions
• Avoid mixing cold and hot keys
• Split the hot key (key+ random (0,X))
Source: Deep Dive on DynamoDB https://www.youtube.com/watch?v=bCW3lhsJKfw
Source: Advanced Design Patterns for Amazon DynamoDB
https://www.youtube.com/watch?v=jzeKPKpucS0
42. New Feature : DynamoDB Adaptive Capacity
Source: https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/bp-partition-key-design.html#bp-partition-key-
partitions-adaptive
43. DynamoDB canonical use cases
• Key-value lookups on well-distributed records
• Avoiding complex queries (and joins)
• Limiting hot keys
44. Challenges with DynamoDB
DynamoDB is not the right choice for each solution
• Think of the access pattern in advance
• Difficult to change access patterns (partition&sort key) & table structure
afterwards
• Difficult to replace DynamoDB with another even (NoSQL) database in the future
45. Why NoSQL database DynamoDB?
Project use case and challenges:
• Fixed Scheme (argument against NoSQL DB)
• Small size of the table and the entities
• Learning curve for the new technology
• No aggregate functions was sum, avg, count and as expected no joins
