Convert Your Code into a Microservice using AWS Lambda.

1. Convert Your Code into a Microservice using AWS Lambda
Vyom Nagrani
Sr. Product Manager
July 13th, 2015

2. Agenda
• What is AWS Lambda?
• Lambda common use cases
• How Lambda works
• Building a microservice on Lambda
• Hands-on exercises
• Q&A

3. What is AWS Lambda?

4. AWS Compute offerings
Lambda
Serverless compute
platform for stateless
code execution in
response to Triggers
ECS
Container
management service
for running Docker on
a managed cluster of
EC2
EC2
Virtual servers
in the Cloud

5. High performance at any
scale; Cost-effective and
efficient
No Infrastructure to manage
Pay only for what you use:
Lambda automatically
matches capacity to your
request rate. Purchase
compute in 100ms
increments.
Bring Your Own Code
“Productivity focused compute platform to build powerful, dynamic,
modular applications in the cloud”
Run code in a choice of standard
languages. Use threads,
processes, files and shell scripts
normally.
Focus on business logic, not
infrastructure. You upload code;
AWS Lambda handles everything
else.
Why Lambda?

6. How Lambda works
S3 bucket
notifications
DynamoDB
Streams
Kinesis
records
Cognito
sync
SNS
publish
Custom
triggers
CloudTrail
activity LambdaDynamoDB
Kinesis S3
Any custom
Invoked in response to triggers
- Changes in data
- Changes in state
Author in familiar language
using any libraries; Execute
only when needed,
automatic scale
Redshift
SNS
Access any service,
including your own
Any AWS
Such as…
“Lambda
functions”

7. Old way of orchestrating workflows
SOURCE of
data
Fleet of servers poll for
changes
Listening to source Pushes to queue Pull off queue
Fleet of servers act as
workers to process the
data
Auto-scale
worker nodes
to adjust with
load
S3 objects
SNS
messages
Kinesis
records
DDB Tables
IoT Devices
…
Cross AZ
replication,
Load
Balancer

8. New way of orchestrating workflows
SOURCE of
data
Attach a Lambda function
And that’s it!
Lambda does the
listening, the polling,
the queuing, the
autoscaling, and
spins up as many
workers as needed to
match the rate of
change of source
data

9. Lambda common use cases

10. 10
Typical usage scenarios
S3 + Lambda
Dynamic data ingestion
Image thumbnailing, Video
transcoding, File indexing, Log
processing, Content validation,
Aggregation and filtering
Kinesis + Lambda
Live stream processing
Application activity auditing,
transaction order processing,
Click stream analysis, IoT device
response, telemetry and metering
SNS + Lambda
Smart IT, Custom message
Automating IT alarm response,
Custom actions, IT auditing,
customizing broadcast
messages, text to email push
Direct call + Lambda
Serverless backend
Microservices, Mobile backends,
IoT backends
DynamoDB + Lambda
Database triggers
[In preview] Data validation, Data
filtering, Live notifications
Alexa + Lambda
Voice triggers
Build a custom Automated Voice
Response system in the cloud

11. Typical workflow for dynamic data ingestion using S3+Lambda
Notification
Amazon S3 AWS Lambda
processes
the object
Amazon S3
New object
uploaded
Amazon
DynamoDB

12. Customers running dynamic data ingestion and processing
using S3+Lambda
AWS
Lambda
Indexing
tables or
notifications
“I want to apply custom logic to
process content being uploaded to my
data store”.
• Watermarking / thumbnailing
• Transcoding
• Indexing and deduplication
• Aggregation and filtering
• Pre processing
• Content validation
Amazon S3
Bucket
Events
Processed
files

13. Typical workflow for real time stream processing using
Kinesis+Lambda
Amazon
Kinesis
AWS Lambda
processes
the stream
Amazon
CloudWatch Logs
Amazon SNS

14. Customers running real-time data stream processing on
Kinesis+Lambda
AWS
Lambda
Aggregate
statistics
Real-time
analytics
Kinesis Stream
“I want to apply custom logic to
process data being uploaded through
my Kinesis stream”.
• Client activity tracking
• Metrics generation
• Data cleansing
• Log filtering
• Indexing and searching
• Log routing
• Live alarms and notifications

15. Typical workflow for smart monitoring and automation using
SNS+Lambda
AWS
Lambda
Amazon
SNS
Amazon
CloudWatch Alarm

16. Typical workflow for custom messaging using SNS+Lambda
Amazon
SNS
Amazon SNSAWS
Lambda

17. How Lambda works

18. Lambda functions
Simple resource model
• Set memory to any size from 128MB to 1GB, in 64MB steps
• Receive an equivalent portion of other resources (disk, network, compute
power, etc.)
• Lambda tells you how much memory you used, so you can tune this setting.
Flexible invocation paths
• Lambda functions can be invoked “on demand” through CLI and Console
• Subscribe to one or many event sources
• Reuse the same Lambda function with multiple event sources
Granular permissions control (using IAM)
• Define what permissions the function has
• Uses IAM role (execution role) for granular permission control
• Recommended minimum permission – log to CloudWatch
• E.g. “read from <X> DDB table only in the context of <Y> function”

19. Writing and deploying Lambda Functions
• The Basics
– Node.js or Java
– AWS SDK comes built in and ready to use
– Lambda handles inbound traffic
– Use processes, threads, /tmp, sockets, …
– Bring your own libraries, even native ones
• Deployment options
– Author directly using the console
– Package code as a ZIP and upload through uploadFunction API
– 3rd Party plugins (Grunt, Jenkins, CloudBees, Codeship)
• Think stateless
– Use S3, DynamoDB, or other Internet storage for persistent data
– Don’t expect affinity to the infrastructure (you can’t “log in to the box”)

20. Monitoring and debugging
Lambda Functions
• Console Dashboard
• Lists all Lambda functions
• Easy editing of resources, event
sources and other settings
• At-a-glance metrics
• Metrics in CloudWatch
• Requests
• Errors
• Latency
• Throttles
• Logging in CloudWatch Logs

21. Invoking Lambda Functions
• Request Response: Call from mobile or web apps
– Wait for a response
– AWS SDK, AWS Mobile SDK, REST API, CLI
• Push Event: Incoming events from Amazon S3 or SNS
– One event per Lambda invocation
– Unordered model
– 3 tries (won’t retry buggy code indefinitely)
• Pull Event: Get DynamoDB changes or Amazon Kinesis records as events
– Ordered model with multiple records per event
– Unlimited retries (until data expires)

22. Under the covers - Invocation permissions
• Resource policies
– Used in the Push model
– Define resource policies attached to a
Lambda function
– E.g. “User X can invoke on function Y in the
context of bucket Z”
– Resource policies allow for cross
account access!
• IAM roles
– Used In the pull model
– Lambda derives permission from execution
role to read from particular Stream
– E.g. “User A has permissions to read from
Stream B in the context of Function C”

23. Building a microservice on Lambda

24. What are Microservices?
In short, the microservice architectural style is an approach to developing a single
application as a suite of small services, each running in its own process and
communicating with lightweight mechanisms, often an HTTP resource API. These
services are built around business capabilities and independently deployable
by fully automated deployment machinery. There is a bare minimum of
centralized management of these services, which may be written in different
programming languages and use different data storage technologies.
-- James Lewis and Martin Fowler
http://martinfowler.com/microservices

25. What are Microservices? (Cont’d)
Microservices
Componentization via
Smaller Services
Decentralized
Governance
Organized around
Business Capabilities
Smart endpoints and
dumb pipes
Infrastructure Automation
Decentralized Data
Management

26. Microservices and APIs
https://www.nginx.com/blog/building-microservices-using-an-api-gateway/

27. Sample Microservice on Lambda
https://aws.amazon.com/blogs/compute/the-squirrelbin-architecture-a-serverless-microservice-using-aws-lambda/

28. Hands-on exercises

30. Today’s exercises
1. Create and test your first Lambda function (Hello World)
2. Triggering Lambda from S3
3. Create a file deduplication Microservice
– Use code from https://github.com/vyomnagrani/awslambda/blob/master/demo-s3-file-dedup.js
4. Triggering Lambda from SNS
5. Create a message customization Microservice
– Use code from https://github.com/vyomnagrani/awslambda/blob/master/demo-sns-customize.js
6. Create a CRUD-backend Microservice with a public HTTP endpoint [Advanced users can jump directly to this one]
– Build off of https://github.com/vyomnagrani/awslambda/blob/master/demo-CRUD-backend.txt

31. Q&A
Visit http://aws.amazon.com/lambda, the AWS Compute blog, and
the Lambda forum to learn more and get started using Lambda.

32. aws.amazon.com/activate
Everything and Anything Startups
Need to Get Started on AWS