This document summarizes a company's journey to migrating their API from traditional servers to AWS Lambda. Some key points covered include: - The company decided to move to serverless to reduce costs and management overhead - They used the aws-serverless-express library to run their Express app on Lambda - This provided a standard development process while avoiding vendor lock-in - The document outlines several limitations to consider like package size limits and cold starts - It provides examples of strategies used to address issues like keeping Lambdas warm and moving static assets to S3

1. Journey with Express
to AWS Lambda
Marian Rusnak
JS Monthly, February 2018

2. 2

3. 3

4. AOL + Yahoo, owned by Verizon
Media and technology company
Reach over 2 billion mobile devices
Deliver over 1 billion hours of video each year
Over one trillion ad requests processed each month
HuffPost is the most shared site on Facebook
TechCrunch is the most followed tech publication on Twitter
4
Oath

5. How we landed on AWS Lambda
Technology stack
Limitations
5
Agenda

6. Company
decided to
move to 6

7. 7

8. New flexible system to support any type of ads
Improved development of templates
Integrated into other systems within the company
Low load expected (1000s a day)
8
New API

9. Lambda
9
Pay when it runs
No server management

10. 10

11. Pay when it runs (no EC2 instances with low utilization)
Function or a zip (or binary file for other languages)
No server management
Free tier (forever): 1M requests/month, 400 000 seconds (1 GB RAM)
11
AWS Lambda

12. 12

13. https://github.com/awslabs/aws-serverless-express
13
aws-serverless-express
// lambda.js
const awsServerlessExpress = require('aws-serverless-express');
const app = require('./app');
const server = awsServerlessExpress.createServer(app);
exports.handler = (event, context) => awsServerlessExpress.proxy(server, event, context);

14. Pros:
● Standard development
● One project
Cons:
● Larger function
14
aws-serverless-express

15. claudia generate-serverless-express-proxy --express-module app
15
Claudia.js
claudia create --handler lambda.handler --deploy-proxy-api --region us-east-1
Generate boilerplate
Create entities
claudia update
Deploy
https://claudiajs.com/tutorials/serverless-express.html

16. Database
16

17. 17

18. 25 GB for free
200 million requests free per month
Pay for storage, writes and reads
18
DynamoDB

19. Querying not flexible
1 MB limit on retrieved data
400 KB maximum item size
19
DynamoDB

20. It still works!
Available in RDS - no management
JSON data type in v5.7
20
MySQL

21. File Uploads
21

22. Stores files by keys - they form sort of directories
Relatively cheap
$0.023 per GB
Reads $0.0004 per 1,000 requests
Writes $0.005 per 1,000 requests
Transfers $0.09 per GB, first GB free
22
S3

23. API Gateway 10 MB maximum payload
Lambda 6 MB
23
File upload limit

24. GET /my-presign-endpoint
24
Pre-signed uploads to S3
{
“key”: “foo.jpg”
“url”: “s3.com/abc123/foo.jpg”
}

25. Local
development
25

26. s3rver - https://github.com/jamhall/s3rver
MySQL in Docker
Local mock server - pre-signed upload, other external services
26
Local

27. API takes long
to respond
27

28. Lambda container is shut down when idle for ~15 mins
Takes around around 10s to start (worse for other languages)
Job that makes request every 10 mins to keep Lambda alive
CloudWatch Events - every 5 mins
You cannot avoid random shutdowns
28
Lambda cold starts

29. Lambda is not good for serving apps with many subsequent requests
Swagger static assets moved to S3
This drastically improved response time
29
Static assets

30. Let’s secure
secrets
30

31. AWS Key Management Service
Decrypt env variables when Lambda starts, then keep in memory
31
KMS
kms.decrypt({CiphertextBlob: new Buffer('******', 'base64')}, (err, data) => {
console.log('Decrypt error:', err);
password = data.Plaintext.toString('ascii');
});

32. 30s API Gateway execution timeout
KMS decryption times out API requests
No problems recently
32
Timeout

33. Binary for video
processing
33

34. 50 MB deployment package limit
250 MB limit for deployment via S3
75 GB total limit
34
Package size limitations

35. HTTP 413
35

36. 8192 bytes
20K including headers
36
CloudFront URL size limit

37. Lambda shuts down when idle
30s API Gateway execution timeout
API Gateway 10 MB maximum payload, Lambda 6 MB
Deployment package up to 50 MB, 250 MB via S3, 75 GB total per region
CloudFront URL size limit of 8192 bytes, 20K including headers
DynamoDB 1 MB limit on retrieved data
DynamoDB 400 KB maximum item size
37
Summary of limitations

38. No server management when using AWS Lambda
Using Express and aws-serverless-express prevents vendor lock-in
Cost effective solution for non-heavy traffic API
Relational DB might often be better option than NoSQL
38
Result

39. https://www.oath.com/
https://aws.amazon.com/documentation/
https://github.com/awslabs/aws-serverless-express
https://claudiajs.com/
https://github.com/jamhall/s3rver
39
References