Visibility into Serverless Applications built using AWS Fargate (CON312-R1) - AWS re:Invent 2018

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Visibility into Serverless Applications
built using AWS Fargate
Uttara Sridhar
Senior Software Engineer
Amazon
C O N 3 1 2 - R
Karly Sindy
R&D Software Engineer
Catalytic Data Science

Agenda
Sample application on AWS Fargate
Visibility Aspects
Logging
Monitoring
Debugging
Catalytic’s parallelized bioinformatics workflows with Fargate

Scorekeep Application
Creates and manages
game sessions and users
Backend
Business logic
Stores state in DynamoDB
Frontend
Page elements built using Angular
nginx to serve pages and
route /api requests to backend

Serverless Containers
Why Containers?
Portable
Flexible
Fast
Efficient
Why Serverless?
No instance management
Faster time-to-market
Scalable
Pay for use

AWS Fargate
Simple, easy to use,
powerful consumption model
Resource
based pricing
No instances
to manage
Container
native APIFargate

Availability Zone #1 Availability Zone #2 Availability Zone #3
Scheduling and Orchestration
Cluster Manager Placement Engine
Amazon ECS supports AWS Fargate

Amazon ECS supports AWS Fargate
EC2 Instance
ECS
AMI
Docker
agent
ECS
agent
EC2 Instance
ECS
AMI
Docker
agent
ECS
agent
EC2 Instance
ECS
AMI
Docker
agent
ECS
agent
Scheduling and Orchestration
Cluster Manager Placement Engine

Benefits
MANAGED by AWS
No Amazon EC2 Instances to provision, scale or
manage.
ELASTIC
Scale up & down seamlessly. Pay only for what
you use.
INTEGRATED
with the AWS ecosystem: VPC Networking,
Amazon Elastic Load Balancing, AWS IAM
Permissions, Amazon CloudWatch and more.
Your Docker
Containers

Scorekeep Application Architecture
https://github.com/aws-samples/eb-java-scorekeep/tree/fargate
Frontend Server
Container
Angular + Nginx
API Server
Container
Java
Port
8080
Port
5000
Internet

Application Building Blocks
Compute Networking
* CPU units
* Memory
* Storage
Security
* Amazon VPC
* Subnets
* Internet, NAT Gateways
* Route tables
* Elastic Network Interface
* AWS IAM Permissions
* Security groups
* How to know the health of
my application?
* How to measure latencies,
error rate, call volume?
* How to automatically alert
application owners?
* How to troubleshoot?
Visibility

Scorekeep Application with AWS Fargate
Amazon ECS Task Definition:
{
"family": "scorekeep",
"cpu": "1 vCpu",
"memory": "2 gb",
"networkMode":"awsvpc",
"taskRoleArn": "arn:aws:…",
"executionRoleArn”: “arn:…”
"requiresCompatibilities":
[
"FARGATE"
],
"containerDefinitions": [
{
"name": "scorekeep-frontend",
"image":“xxx.dkr.ecr…frontend",
"cpu": 256,
"memoryReservation": 512,
"portMappings" : [
{ "containerPort": 8080 }
],
"logConfiguration": {
"logDriver": "awslogs",
"options": {
"awslogs-group": "scorekeep",
"awslogs-region": "us-east-1",
"awslogs-stream-prefix":
"scorekeep/frontend”
}
}
}
]}
{
"name": "scorekeep-api",
"image":“xxx.dkr.ecr…api",
"cpu": 768,
"portMappings" : [
],
"environment": […], #env var
"options": {
“awslogs-group": "scorekeep",
"scorekeep/api”
}
}
},

Amazon ECS Task Definition: Compute
{
"cpu": "1 vCpu",
"memory": "2 gb",
[
"FARGATE"
],
{
"cpu": 256,
"portMappings" : [
],
"options": {
}
}
}
]}
{
"cpu": 768,
"portMappings" : [
],
"options": {
"scorekeep/api”
}
}
},

Amazon ECS Task Definition: Networking
{
"cpu": "1 vCpu",
"memory": "2 gb",
[
"FARGATE"
],
{
"cpu": 256,
"portMappings" : [
],
"options": {
}
}
}
]}
{
"cpu": 768,
"portMappings" : [
],
"options": {
"scorekeep/api”
}
}
},

Amazon ECS Task Definition: Security
{
"cpu": "1 vCpu",
"memory": "2 gb",
[
"FARGATE"
],
{
"cpu": 256,
"portMappings" : [
],
"options": {
}
}
}
]}
{
"cpu": 768,
"portMappings" : [
],
"options": {
"scorekeep/api”
}
}
},

Amazon ECS Task Definition: Visibility
{
"cpu": "1 vCpu",
"memory": "2 gb",
[
"FARGATE"
],
{
"cpu": 256,
"portMappings" : [
],
"options": {
}
}
}
]}
{
"cpu": 768,
"portMappings" : [
],
"options": {
"scorekeep/api”
}
}
},

# Register Task Definition
$ aws ecs register-task-definition --cli-input-json file:///scorekeep-task-
definition.json
# Create Cluster
$ aws ecs create-cluster --cluster-name FargateDemoCluster
# Create Service
$ aws ecs create-service --service-name ScorekeepService
--cluster FargateDemoCluster --task-definition scorekeep:1
--launch-type FARGATE –desired-count 5
--network-configuration
# Networking
"awsvpcConfiguration={subnets=[subnet-xxx], securityGroups=[sg-xxx]}“
--load-balancer “[{"targetGroupArn":“arn:aws:xxx",
"containerName":“scorekeep-frontend",
"containerPort":8080}]”

Scorekeep Application
Frontend Server
Container
Angular + Nginx
API Server
Container
Java
Port
8080
Port
5000
Internet

Visibility Aspects
Logging Monitoring Debugging

Visibility Aspects
Monitoring DebuggingLogging

Logging
Critical component to our diagnostics and monitoring
Enforce best practices to logging
Provide as much relevant information
Avoid spamming
Use appropriate log levels: fatal, error, warning

Logging
Send application logs to a logging framework
AWS Fargate supports Amazon CloudWatch log driver
Track user activity and API usage
AWS CloudTrail
Tracing application requests
AWS X-Ray

Logging with Amazon CloudWatch
Use the awslogs driver
to send stdout from
your application to
Cloudwatch logs
Create a log group in
Cloudwatch
Configure the log
driver in your task
definition
Remember to add
permissions via the
Task Execution Role

Logging with AWS CloudTrail
{
"EventId": "89baa6bb-9380-4983-9670-5688359abbb9",
"EventName": "DescribeServices",
"EventTime": 1542301015.0,
"EventSource": "ecs.amazonaws.com",
"Username": "root",
"Resources": [],
"CloudTrailEvent":
"{"eventVersion":"1.04","userIdentity":{"type":"Root","principalId":”xxxx",
"arn":"arn:aws:iam::xxxx:root","accountId":”xxxx","accessKeyId":”xxxx","sess
ionContext":{"attributes":{"mfaAuthenticated":"false","creationDate":"2018-11-
15T07:26:14Z"}}},"eventTime":"2018-11-
15T16:56:55Z","eventSource":"ecs.amazonaws.com","eventName":"DescribeServices",
"awsRegion":"eu-west-
1","sourceIPAddress":"172.92.129.249","userAgent":"console.amazonaws.com","requ
estParameters":{"services":["scorekeep-service"],"cluster":"scorekeep-
cluster"},"responseElements":null,"requestID":"5b38e298-2b16-427c-9b05-
e60272ffa1a0","eventID":"89baa6bb-9380-4983-9670-
5688359abbb9","eventType":"AwsApiCall","recipientAccountId":”xxxx"}"
}

Tracing with AWS X-Ray
Create a Docker image
that runs the daemon.
Add managed policies
to your task role to
grant the daemon
permission to upload
trace data to X-Ray.
Run it as a side-car
container. Update
Amazon ECS Task
definition.
Invoke Update Service.
{
"name": "xray-daemon",
"image": ”xxxx.dkr.ecr.xxx.amazonaws.com/xray-daemon",
"cpu": 32,
"portMappings" : [
{
"containerPort": 2000,
"protocol": "udp"
}]
}

Visibility Aspects
DebuggingMonitoringLogging

Monitoring
Metrics
Measure availability, reliability
and performance
Instrument/profile your code
Helpful for troubleshooting

Monitoring: Metrics
Key Metrics
Application metrics: Server-side/client-side latency, error rates, timeouts, CPU/memory,
heartbeats
Load balancers metrics: request count, connection count, 5xxs, 4xxs
Dependency metrics: latency, error rates, timeouts, DB throughput
Amazon CloudWatch
Default Amazon ECS Metrics
Metric data sent to CloudWatch in 1-minute periods and recorded for a period of two weeks
Available metrics: CPUUtilization, MemoryUtilization
Available dimensions: ClusterName,ServiceName
Emit custom metrics, use Amazon CloudWatch metric filters

Monitoring: Metrics with Amazon CloudWatch

Monitoring
Metrics
and performance
Dashboards
Monitor your resources in a
single view
Operational meetings

Monitoring: Dashboarding with Amazon CloudWatch

Monitoring: with Datadog Autodiscovery and AWS Fargate

Monitoring
Metrics
and performance
Dashboards
single view
Alarms
Automated alerts to notify
threshold breaches
Paging on-call, email
notifications, or take autoscaling
actions

Monitoring
Metrics
and performance
Dashboards
single view
Alarms
threshold breaches
actions
State Change Events
Trace using Amazon CloudWatch
Events

Monitoring: with Amazon CloudWatch Events
{
"version": "0",
"id": "d5e4d7c2-8b04-d387-ded5-779de044ce5b",
"detail-type": "ECS Task State Change",
"source": "aws.ecs",
…
"resources": [
"arn:aws:ecs:eu-west-1:xxx:task/8b813908-0e5f-4bc0-a901-65144ce6d9ef"
],
"detail": {
"stoppedReason": "Scaling activity initiated by (deployment ecs-
svc/9223370498172059827)",
"launchType": "FARGATE",
"cpu": "1024",
"memory": "2048",
"desiredStatus": "STOPPED",
"group": "service:scorekeep-service",
"lastStatus": "STOPPED",
"containers": […]
…
}
}

Monitoring
Metrics
and performance
Dashboards
single view
Alarms
threshold breaches
actions
State Change Events
Trace using Amazon CloudWatch
Events
Health Checks

Monitoring: with Container Health Checks
- Define custom health check commands in the Amazon ECS Task
Definition

Debugging
Amazon ECS APIs and Console
• Task and container stopped reason
• Task StateChange events using Amazon CloudWatch
• Service events in describe-service payload

Debugging Task Limit Exceeded
Request limit increase
https://docs.aws.amazon.com/general/latest/gr/aws_service_limits.html

Debugging Container Pull Error
Amazon ECS Troubleshooting Guide
https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_cannot_pull_image.html

Debugging
Application Logs using Amazon CloudWatch

Debugging
Application Logs using Amazon CloudWatch
Task Metadata API

Debugging with Task Metadata
Query environmental data and statistics for running tasks from within the Task!
Enables monitoring tools like Datadog, etc
Endpoints available:
Task Level (for all containers)
169.254.170.2/v2/metadata – Metadata JSON for Task
169.254.170.2/v2/stats - Docker stats JSON for all containers in the Task
Container Level
169.254.170.2/v2/metadata/<container-id>
169.254.170.2/v2/stats/<container-id>

{
"Cluster": "default",
"TaskARN": "arn:aws:ecs:us-west-2:012345678910:task/9781c248-0edd-4cdb-9a93-
f63cb662a5d3",
"Family": "nginx",
"Revision": "5",
"DesiredStatus": "RUNNING",
"KnownStatus": "RUNNING",
"Containers": [
{
"DockerId": "731a0d6a3b4210e2448339bc7015aaa",
"Name": "~internal~ecs~pause",
"DockerName": "ecs-nginx-5-internalecspause-1234",

{
"Containers": [
{
"DockerId":
"731a0d6a3b4210e2448339bc7015aaa",
"Name": "~internal~ecs~pause",
"DockerName": "ecs-nginx-5-
internalecspause-1234",
"Image": "amazon/amazon-ecs-
pause:0.1.0",
"ImageID": "",
"Labels": {
"com.amazonaws.ecs.cluster":
"default",
"com.amazonaws.ecs.task-arn":
"com.amazonaws.ecs.task-definition-
family": "nginx",
"com.amazonaws.ecs.task-
definition-version": "5"
},
"DesiredStatus":
"RESOURCES_PROVISIONED",
"KnownStatus":
"RESOURCES_PROVISIONED",
"Limits": {
"CPU": 512,
"Memory": 512
}

Lessons learned
• AWS Fargate is a container compute engine. It is available as a launch type for
Amazon ECS to run containers without having to manage EC2 instances.
• Monitoring and Alarming is critical for understanding the state of your system
and knowing when something goes wrong.
• AWS ecosystem has a variety of tools dedicated to improving the visibility of
your application health using AWS Fargate.
• In this session, we learned how to use Amazon ECS with Fargate and use
Amazon CloudWatch, Amazon CloudTrail and AWS X-Ray to help us identify
and diagnose events.

Catalytic’s parallelized bioinformatics
workflows with Fargate
C O N 3 1 2 - R
Karly Sindy
R&D Software Developer
Catalytic Data Science

Magpie: A Multiplexed Automated Gene Editing Pipeline
prototype to production

Gene editing systems: site-specific
GTGCACCTGACTCCTGTGGAGAAGTCTGCCGTTACT
GTGCACCTGACTCCTGAGGAGAAGTCTGCCGTTACT
mutation
…TCCTGAGGAGA…
repaired
…TCCTGTGGAGA…
not repaired
collect —> NGS

Magpie - analysis of gene editing systems
CACCTGACTCCTGAGGAGAAGTCTGCCGTTACT
GTGCACCTGACTCCTGAGGAGAAGTCTGCCGTTA
GTGCACCTGACTCCTGAGGAGAAGTCTGCC
ACTCCTGTGGAGAAGTCTGCCGTTACT
GTGCACCTGACTCCTGAGGAGAAGTCTGC
ACCTGACTCCTGTGGAGAAGTCTGCCGTTA
GTGCACCTGACTCCTGTGGAGAAGTCTGCCGTTACT

Customer request
Faster!

Prototype: parallel runs

AWS Fargate Benefits
Parallel runs means parallel results.
Better for sporatic application load:
financially and in developer time.

Lessons learned
Available CLIs
Plan for limited task size and quota.
Reduce docker image size for quicker startup.
Create cleanup cron for runaway tasks.

Task definition: ecs-cli compose
task_definition:
ecs_network_mode: awsvpc
task_role_arn: $ecsTaskRoleArn
task_execution_role: $ecsTaskExecutionRole
task_size:
cpu_limit: 4096
mem_limit: 8GB
services:
magpie:
essential: true
run_params:
network_configuration:
awsvpc_configuration:
subnets:
ecs-params.yml snippet
version: ‘2’
services:
magpie:
image: <xxx.ecr.us-east-
1.amazon.com/magpie>
ports:
- "8000:8000"
logging:
driver: awslogs
options:
awslogs-group: magpie-log
awslogs-region: us-east-1
awslogs-stream-prefix: ecs
docker-compose.yml snippet

Run task: lambda
client.run_task(
cluster=ecs_cluster_name,
taskDefinition=task_definition_arn,
overrides=params,
launchType='FARGATE',
networkConfiguration={
'awsvpcConfiguration': {
'subnets': [
ecs_task_vpc_subnet_1,
ecs_task_vpc_subnet_2 ],
}
}
)
lambda function snippet
Cluster : demo-tasks
magpie:248772e-j879-4…
demo-tasks
magpie:24
family:magpie
x.x.x.x
x.x.x.x
x:x:x:x:x:x
magpie xxx.ecr.us-east-1.am...
demo-magpie-log
amplicon
magpie

Requirements for production
Fault Tolerance
Retry when hit task limit or other infra error.
Recall Past Processes
Dynamically workflow applications together.
Visibility
Visually track errors and progress with asynchronous workflows.

Production: Amazon Dynamo, AWS Step Functions, and AWS Fargate

Incorporating complex workflows

Visibility
Monitoring
Amazon DynamoDB, AWS Step Function,
Amazon ECS
$ aws dynamodb scan --table-name analytic-cdsqab-run
--filter-expression "(contains(runStatus,:runStatus))"
--expression-attribute-values '{":runStatus":{"S":"ERROR"}}'
--projection-expression "batchId"
demo-tasks >
demo >
Cluster : demo

Visibility
Debug
Amazon DynamoDB, AWS Step Functions
Monitoring
Amazon ECS

Visibility
Logs
Amazon CloudWatch
Debug
Amazon DynamoDB, AWS Step Functions
Monitoring
Amazon ECS

Wish List
Fire and forget
In the spirit of serverless, automatically infrastructure errors in AWS Fargate, AWS Lambdas,
Amazon ECS, DBs.
Mount volume
Amazon EFS, NFS, Amazon EBS
Task-level metrics
Amazon ECS console

Thank you!
Uttara Sridhar
uttara@amazon.com
Karly Sindy
Karly.Sindy@CatalyticDS.com

Visibility into Serverless Applications built using AWS Fargate (CON312-R1) - AWS re:Invent 2018

Recomendados

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente (20)

Similar a Visibility into Serverless Applications built using AWS Fargate (CON312-R1) - AWS re:Invent 2018

Similar a Visibility into Serverless Applications built using AWS Fargate (CON312-R1) - AWS re:Invent 2018 (20)

Más de Amazon Web Services

Más de Amazon Web Services (20)

Visibility into Serverless Applications built using AWS Fargate (CON312-R1) - AWS re:Invent 2018