Writing serverless functions is quite easy. And also combining some of them to build up a more complex system is not too complicated at all.
But what happens if these tiny functions are not working as expected? How to trace and monitor each function by itself and the system as a whole? Is there any change to debug? And what about testing? Do we always have to deploy to the cloud or are there any local solutions to emulate the productive environment? And by the way, how can we automate all of this?
Lots of questions are waiting to be answered.
Required audience experience
basic knoweldge of serverless computing
Objective of the talk
The audience will learn how to handle and “survive” serverless scenarios in real life. During the session I will demonstrate with the help of a demo application (aws lambda based) how to test, trace, monitor and debug servlerless functions.
3. ABOUT ME
Who am i?
• CIO New Technologies
• Enterprise & Mobile
• Author, Speaker, Coach & Mentor
• Snowboard & MTB Enthusiast
• Traveller between the worlds
Lars Röwekamp (a.k.a. @mobileLarson)
#WISSENTEILEN
LR
5. #WISSENTEILEN
“Run your business code
highly-available
in the cloud in response
to events and scale
without any servers to
manage.“*
*(AWS Lambda product description)
6. #WISSENTEILEN
“Run your business code
highly distributed
and event driven in a non
transparent environment
with no single
point of control.“*
*(my personal interpretation)
7. „Run Code, not Servers“, they said!
„It will be fun“, they said
12. #WISSENTEILEN
AWS Cloud
AWS Step Functions workflow: Store Image
Store raw Image
Store Image Information
1
2
Use-Case: Upload Image
upload image
with additional
information
13. #WISSENTEILEN
AWS Cloud
AWS Step Functions workflow: Store Image
Create ThumbnailStore raw Image
Store Image Information
1
2
Use-Case: Upload Image
upload image
with additional
information
14. #WISSENTEILEN
AWS Cloud
AWS Step Functions workflow: Store Image
Create Thumbnail
Inform Subscribers
Store raw Image
Store Image Information
1
2
Use-Case: Upload Image
upload image
with additional
information
15. #WISSENTEILEN
AWS Cloud
AWS Step Functions workflow: Store Image
Create Thumbnail
Inform Subscribers
Store raw Image
Store Image Information
1
2
Use-Case: Upload Image
upload image
with additional
information
16. #WISSENTEILEN
AWS Cloud
AWS Step Functions workflow: Store Image
Create Thumbnail
Inform Subscribers
Store raw Image
Store Image Information
1
2
Use-Case: Upload Image
upload image
with additional
information
17. #WISSENTEILEN
AWS Cloud
AWS Step Functions workflow: Store Image
Create Thumbnail
Inform Subscribers
Store raw Image
Store Image Information
1
2
Use-Case: Upload Image
upload image
with additional
information
18. #WISSENTEILEN
AWS Cloud
AWS Step Functions workflow: Store Image
Create Thumbnail
Inform Subscribers
Store raw Image
Store Image Information
1
2
Use-Case: Upload Image
upload image
with additional
information
34. #WISSENTEILEN
With well-planned monitoring, we should be able to
• anticipate disruptions
• quickly identfiy the source of problems
• trigger automated recovery processes
• trigger alarms
Real-Life Monitoring
36. #WISSENTEILEN
Well-planned monitoring should take various aspects into account
• reliability: components and communication
• usage: functional and non functional
• performance: duration, latentcy and timeouts
• security: access rights, attacts
• costs: current and trends
Real-Life Monitoring
38. 3
2
Tracing Metrics
4
4
Alerting
#WISSENTEILEN
The 4 Pillars of Monitoring
Represents state of an
application.
When things go wrong, we
need the logs to establish
what change in state
caused the error.
1
Logging
Logging
41. 3
2
Tracing Metrics
1
Logging
#WISSENTEILEN
The 4 Pillars of Monitoring
4
Alerting
The component of a
monitoring system that
performs actions based on
changes in metric values.
Mostly used to auto-heal
system or inform person in
charge.
Alerting3
2
Tracing Metrics
1
Logging
#WISSENTEILEN
The 4 Pillars of Monitoring
4
Alerting
The component of a
monitoring system that
performs actions based on
changes in metric values.
Mostly used to auto-heal
system or inform person in
charge.
Alerting3
2
Tracing Metrics
1
Logging
#WISSENTEILEN
The 4 Pillars of Monitoring
4
Alerting
The component of a
monitoring system that
performs actions based on
changes in metric values.
Mostly used to auto-heal
system or inform person in
charge.
Alerting
63. #WISSENTEILEN
What kind of „benchmarks“ do we want for our monitoring?
• collect rich set of system and application metrics
• metrics and logs should not add user-facing latency
• metrics and logs should appear in real-time
• metrics should be granular and correlated
Monitoring Best Practices
64. #WISSENTEILEN
#1 avoid user-facing latency
Monitoring Best Practices
AWS Cloud
My Lambda logs
log
stream
log
data
async
sync
Log Aggregator
log
data
1
very fast and cheap
2
3
time consuming and “expensive”
parse
log stream
65. #WISSENTEILEN
#2 collect rich set of system and application metrics
Monitoring Best Practices
AWS Cloud
My Lambda logs
log
stream
log
data
async
sync
Log Aggregator
metrics
custom
metrics
custom
metrics
log
data
2
3
1
very fast and cheap
parse
log stream
66. #WISSENTEILEN
#3 avoid unnecessary costs
Monitoring Best Practices
AWS Cloud
My Lambda logs
log
stream
log
data
async
sync
Log Aggregator
metrics
log & custom
metrics
custom
metrics
custom
metrics
log
data
archive
logs
1
2
67. #WISSENTEILEN
#4 correlate logs and metrics
Monitoring Best Practices
AWS Cloud
My Lambda logs
log
stream
log
data
async
sync
Log Aggregator
metrics
log & custom
metrics
custom
metrics
custom
metrics
log
data
archive
Logs
1
correlation
ID
68. #WISSENTEILEN
#5 use ENV vars to enable/disable debug logging at Edge server
Monitoring Best Practices
AWS Cloud
My Lambda logs
log
stream
log
data
async
sync
Log Aggregator
metrics
log & custom
metrics
custom
metrics
custom
metrics
log
data
archive
Logs
DEBUG
on/off
ENV var
1
2
74. #WISSENTEILEN
Testing, the serverless way
Unit-Tests for Serverless Functions
• seperation of logic and infrastructure
• mock dependencies
• jUnit
• testNG
75. #WISSENTEILEN
Testing, the serverless way
Integration Tests for Cloud Components
• dev/test components in the cloud*
• mock/fake cloud components
• S3rver
• dynamodb-local
• kinesalite
• node-lambda
• fake-sns
• elasticmq
(* $$$, a.k.a. kind of expensive)
76. #WISSENTEILEN
Testing, the serverless way
End-to-End test for User-Stories
• cloud test stages*
• local cloud emulation
• SAM local (Amazon)
• LocalStack (Atlassian)
• docker-lambda (LambCI.org)
(* $$$, a.k.a. kind of expensive)
78. #WISSENTEILEN
„Based on AWS SAM, SAM Local is an AWS CLI
tool that provides an environment for you to develop,
test, and analyze your serverless applications locally
before uploading them to the Lambda runtime.“
• start local gateway from a SAM template
• validate SAM template
• generate sample payload for various event sources
How to test via SAM local?
82. #WISSENTEILEN
localStack
• easy to use mocking framework*
• builds up local cloud environment**
• Base Edition (free) & Pro Edition ($$, coming soon)
• fast growing community
• 100k test runs in 08.2017 für 0$
*based an established mocking frameworks
** at this moment AWS „only“, others shall follow
How to test with LocalStack?
84. #WISSENTEILEN
What kind of „benchmarks“ do we want for our testing?
• test functional changes fast and cost efficient
• test basic integrational changes fast and cost efficient
• test integrational changes as „real“ as possible
• test use cases and user stories in total
Testing Best Practices
85. #WISSENTEILEN
#1 seperate business logic from infrastructure inside your functions
Testing Best Practices
AWS CloudOn-Premise
handler
logic
unit test candidate
e
i
u
u
integration test candidate
end-to-end test candidate
u
86. #WISSENTEILEN
#2 mock cloud infrastructure components of your functions
Testing Best Practices
AWS CloudOn-Premise
handler
logic
unit test candidate
e
i
u
u
integration test candidate
end-to-end test candidate
u
um
87. #WISSENTEILEN
#3 use local environment for functional testing (e.g. SAM local)
Testing Best Practices
AWS CloudOn-Premise
handler
logic
unit test candidate
e
i
u
u
integration test candidate
end-to-end test candidate
uvia SAM local
via SAM local
SAM
yaml
TEST
u
u
88. #WISSENTEILEN
#4 use local environment for trigger integration testing
Testing Best Practices
AWS CloudOn-Premise
handler
logic
unit test candidate
e
i
u
u
integration test candidate
end-to-end test candidate
uvia SAM local
via SAM local
SAM
yaml
TEST
u
u
i
i
89. #WISSENTEILEN
#5 use local „cloud“ components for basic integration testing
Testing Best Practices
AWS CloudOn-Premise
handler
logic
unit test candidate
e
i
u
u
integration test candidate
end-to-end test candidate
u
via DynamoDB local
via FakeS3 via SAM local
via SAM local
SAM
yaml
TEST
u
u
i
i
i
i
warning: local cloud
components can only proof
functional correctness but
not infrastructural
correctness like DLQs,
timeouts, throttling, SLAs, …
90. #WISSENTEILEN
#6 use temporary integration cloud for partial integration testing
Testing Best Practices
AWS CloudOn-Premise
handler
logic
unit test candidate
e
i
u
u
integration test candidate
end-to-end test candidate
uvia SAM local
via SAM local
SAM
yaml
TEST
u
u
via DynamoDB local
via FakeS3
i
i
Temorary Intregration #Dev1
ii
INT
i
i
91. #WISSENTEILEN
#7 use permanent integration cloud for end-to-end testing
Testing Best Practices
AWS CloudOn-Premise
handler
logic
unit test candidate
e
i
u
u
integration test candidate
end-to-end test candidate
uvia SAM local
via SAM local
SAM
yaml
TEST
u
u
via DynamoDB local
via FakeS3
i
i
Permament IntregrationINT
e
e
e
e
i
i
93. #WISSENTEILEN
Writing hundreds of serverless functions is easy, integrating them in
contrast is kind of hard. Therefore we must be able to …
• avoid problems in advance -> test on all levels
• anticipate disruptions -> collect and analyze metrics
• quickly identfiy the source of problems -> via logging and tracing
• trigger automated recovery processes -> set up CI/CD pipeline
• trigger alarms -> establish alerting system
Surviving Serverless in the Real-Life