Presentation from the developer track at I Love APIs London 2016 featuring Matt McClean, Amazon Web Services. Developers have been jumping on the microservices bandwagon because of the obvious benefits of faster release cycles and innovation. However, microservices' downside is the increased server costs, operational costs, and performance costs. To reduce this complexity, Amazon Web Services created AWS Lambda - a compute platform that lets you build microservices with no provisioning and servers. Matt McClean, Solution Architect from AWS, presents how to use AWS Lambda to build your microservices. He covers various architectural patterns and anti-patterns for using AWS Lambda.

1. Serverless computing with AWS Lambda
Matt McClean
AWS Solutions Architect
1

2. What to expect from this session
• Evolution of Cloud Compute toward Serverless
• What is AWS Lambda?
• Building a serverless data processing architecture
• Real-time data processing architecture patterns
• Tips and tricks for using AWS Lambda
• Live demo of Apigee + AWS Lambda app

3. Evolution of Cloud Compute towards
Serverless
3
©2016 Apigee. All Rights Reserved.

4. Evolution of Compute – Public Cloud
Infrastructure
Instances
Application code

5. Evolution of Compute – Containers
Infrastructure
Instances
Application code
Containers

6. Evolution of Compute – Serverless
Application code

7. What is AWS Lambda?
7
©2016 Apigee. All Rights Reserved.

8. AWS Lambda
Serverless, event-driven compute service
Lambda = microservice without servers

9. AWS Lambda – Benefits
EVENT-DRIVEN SCALE
SERVERLESS
SUBSECOND BILLING

10. AWS Lambda – Capabilities
BRING YOUR OWN CODE SIMPLE RESOURCE MODEL
FLEXIBLE INVOCATION PATHS GRANULAR PERMISSIONS CONTROL

11. AWS Lambda – How it Works
AUTHORING STATELESS
DEPLOYMENT MONITORING & LOGGING

12. AWS Lambda – Partner Ecosystem

13. Building a Serverless Data Processing
Architecture
13
©2016 Apigee. All Rights Reserved.

14. SOURCE of
data
Fleet of servers poll for
changes
Listening to source Pushes to queue Pull off queue
Fleet of servers act as workers
to process the data
Auto-scale
worker nodes
to adjust with
load
S3 objects
Amazon Kinesis records
DynamoDB tables
SNS messages
IoT devices
…
Cross-AZ
replication for
high
availability
Load balancer
to distribute
traffic
Data Processing Architecture with Servers
Application code

15. SOURCE of
data
Attach a Lambda function
And that’s it!
Lambda does
Listening/polling
Queuing
Auto scaling
Redundancy
Load balancing
Data Processing Architecture Without Servers
Application code

16. Real-Time Data Processing Architecture
Patterns
16
©2016 Apigee. All Rights Reserved.

17. Sample Real-Time File Processing Workflow Using
Amazon S3 & AWS Lambda
Amazon S3 AWS Lambda
Amazon S3
New file
uploaded
Amazon
DynamoDB

18. Example – Video Clip Transcode
// Transcode video files uploaded to an S3 bucket
var async = require('async');
var fs = require('fs');
var exec = require('child_process').exec;
var aws = require('aws-sdk');
var s3 = new aws.S3();
// Lambda entry point
exports.handler = function(event, context) {
var inputVideo = '/tmp/myVideo.avi';
var outputVideo = '/tmp/myVideo.mp4';
var inputBucket = event.Records[0].s3.bucket.name;
var inputKey = event.Records[0].s3.object.key;
var outputBucket = inputBucket + '-out';
var outputKey = inputKey.substr(0,inputKey.length-4) + '.mp4';
// Runs the array of functions in series, each passing their results to the next
async.waterfall([
Set up variables
Serialize steps

19. Example – Video Clip Transcode
// Download the object from S3 into a buffer
function download(next) {
s3.getObject({Bucket: inputBucket, Key: inputKey}, next); },
// Write the buffer to a file
function write(response, next) {
fs.writeFile(inputVideo, response.Body, next); },
// Spawn a process to transcode
function transcode(next) {
exec("./ffmpeg -i " + inputVideo + " -vcodec mpeg4 -b:v 1200k “
+ outputVideo, next); },
// Read the file to a buffer
function read(next) {
fs.readFile(outputVideo, next); },
// Upload the buffer to an object in S3
function upload(objectBuffer, next) {
s3.putObject({Bucket: outputBucket, Key: outputKey, Body: objectBuffer},
context.done); },
], context.fail);
};
Get file from S3
Write to disk
ffmpeg transcode
Read from disk
Upload to S3

20. Sample Real-Time Stream Processing Workflow Using Amazon
Kinesis & AWS Lambda
Amazon Kinesis AWS Lambda
Amazon Redshift
Amazon SNS
New records
available

21. Sample Real-Time DB Triggers Workflow Using Amazon
DynamoDB & AWS Lambda
New data
available
Amazon DynamoDB AWS Lambda
Amazon
DynamoDB
Amazon Redshift

22. Sample Real-Time Message Handling Workflow Using
Amazon SNS & AWS Lambda
New message
published
Amazon SNS AWS Lambda
Amazon SNS
Amazon Kinesis

23. Sample CRUD Backend Workflow Using Amazon API
Gateway & AWS Lambda
New API
called
Amazon API Gateway AWS Lambda
Amazon S3
Amazon
DynamoDB

24. More Triggers for AWS Lambda
AWS
CloudFormation
Custom Resources
Amazon SES
Actions
Amazon Cognito
Sync Triggers
… and the list will
continue to grow!
AWS IoT ActionsAWS Lambda
Scheduled Events
Amazon Echo
Skills
Amazon SWF Tasks

25. Tips and Tricks for using AWS Lambda
25
©2016 Apigee. All Rights Reserved.

26. Using Lambda to Audit CloudTrail Activity
AWS
Lambda
Amazon S3Amazon CloudTrail
Amazon SNS
AWS IAM

27. Using Lambda for Automated Infrastructure Management
AWS
Lambda
Amazon
SNS
Amazon
CloudWatch Alarm
ec2
runInstance
ecs
startTask
beanstalk
updateApp
kinesis
splitShard
Any API call
https://aws.amazon.com/blogs/compute/scaling-amazon-ecs-services-automatically-using-amazon-cloudwatch-and-aws-lambda/
Amazon
CloudWatch Event

28. Using Lambda to Forward AWS Events to Slack
AWS
Lambda
Amazon
SNS
Amazon
CloudWatch Events
Auto Scaling
Slack
https://aws.amazon.com/blogs/aws/new-slack-integration-blueprints-for-aws-lambda/

29. Using Lambda to Deploy Lambda Functions
https://aws.amazon.com/blogs/compute/dynamic-github-actions-with-aws-lambda/
AWS
Lambda
Amazon
SNS
GitHub Repo
lambda
createFn ()

30. Using Lambda in your software delivery pipeline
AWS
Lambda
AWS
CodeCommit
AWS
CodePipeline
https://aws.amazon.com/about-aws/whats-new/2016/01/aws-codepipeline-
adds-support-for-triggering-aws-lambda-functions/

31. A Few Other Tips and Tricks for Extending AWS Lambda
Functionality
Ø Use /tmp space as cache
Ø Run arbitrary binaries
Ø Use Grunt and Jenkins plugins for deploying
Ø Build event-driven Amazon ECS
Ø Try out Serverless Framework

32. Reference Architecture
32

33. API Management covers the entire API Lifecycle
33
Design
Develop
Secure
Publish
Monitor
Analyze
Monetize
Scale
AWS API
Gateway

34. Demo
34

35. Visit
http://aws.amazon.com/lambda, the
AWS Compute blog, or the
Lambda forum to learn more and to
get started using Lambda.
More info

36. Thank you