Trends and development practices in Serverless architectures

Testing large-scale, serverless
and asynchronous systems
Matt Fellows
@matthewfellows
...without end-to-end integrated tests! #AWSAgility

2
Why Serverless?
https://aws.amazon.com/retail/data-analytics/

WHAT WE ARE KNOWN FOR
3
Microservices
Evolution Drivers for levelling up
● Increase time-to-value
● Increase utilisation / cost
reduction
● Raise abstraction; focus on
composition
● Scale product teams
● Massively distributed,
real-time apps
● API contracts (e.g. OAS)
● Ubiquity of cloud computing
● Rise of containers
● Unreliable networking /
components
● Rise of the Service Mesh
● Improved observability
Why Serverless? > Drivers

5
Data Pipelines
Trends > Data pipelines

7 https://aws.amazon.com/blogs/big-data/automating-analytic-workflows-on-aws/

8 https://www.slideshare.net/nateware/aws-gdc2015-gameanalyticsfinal

9
Trends > Data pipelines > Before
Systems of Record
Integration / Middleware
APIs
Presentation
Synchronous,
coupled invocation
Data locked here

10
Trends > Data pipelines > After
System of Record
Event
API
Local cache /
materialised view
API
3. Event processed into local
view of data
1. Change of system state
2. Event is shared

Observations
Architecture > Microservices
● Democratisation of data
○ “Everyone gets a data”
● Hollywood Principle
● Pipelines are the new batch
● Prerequisite for AI/ML
● Composition and enrichment
● Lambda to “glue” steps together

12
IT Ops. Automation
Trends > IT Ops Automation

Before...
● Ad-hoc
● Cron jobs running on instances
● Scripts > code
● Shared “Job” machines, with broad
security permissions and “all-the-tools”
● Lots of manual “glue” and orchestration

14
Decide
Alarm
Schedule
Event
subscription
Notify Pagerduty
Act
Isolate instance from
ASG
Ship logs to Splunk
Offsite backup
Orient
Security Analysis
Application Anomaly /
Canary Detection
Policy Violation
Chaos Engineering
Observe
Flow Logs
CloudTrail
CloudWatch Logs
Application Event
Configured Rule

New approach
● Scheduled Lambda to replace “cron” jobs
● Real-time cron - event-driven
● Function written and deployed as code -
first class citizens
Rise of the OODA loop

16
The new uService
Trends > Microservices

17
Trends > Microservices > Before
Instances
Team 1
Team 2
Team 3
Team ...

18
API Gateway
Trends > Microservices > Before
Services
Services
Team 1: Orders Team 2: Identity
Services
Team ...
Team 3:
Experience

19
Trends > Microservices > After > New Patterns
API
External
Events
API
Outside World
Microservice
Interface
API
Master Data
API
Inside bounded context
Event Emitters
API
Materialised view /
cache
Events and
Processing
Data
Event Handlers
Processing
Offline Processing

20
Provisioning
Amazon ML
Trends > Microservices > After > Example Team 1 Bounded Context
Place Order API
External
Events
Order Status API
Outside World
Internal Events
Microservice
Interface
Data, Events and
Processing
Suggestions API
Publish Event
to external
subscribers
Insights /
Predictions
Customer /
Other Event
Orders
Push API
Inside bounded context
CRM
Materialised view of
customer
Provisioning Event
External
Events
… API

21
Trends > Microservices > Evolution
Sizeofthing
Number of communicating things
Monolith
SOA
Serverless
Microservices
Fn
Luckily, built on the
same primitive!

Observations
● Microservices still a thing
○ ..but are now just the interface of the
iceberg
● Further decomposition of services into
smaller functions
● Event-driven + asynchronous >
synchronous invocation
● Lambda and managed services as “glue”
● Architectural composition
BUT - More things to manage

Summary
● Serverless as a higher-order system
○ Composition
● IT Operations Automation
○ Better discipline (OODA)
○ Everything as code
● Microservices still a thing, new designs
emerging
○ Event-driven and loosely coupled
○ “Iceberg” patterns

24
Much Compose! So speed!
Architecture > Functions + Serverless (Cloud Native)Architecture > Functions + Serverless (Cloud Native)

25
New challenges
Architecture > Functions + Serverless (Cloud Native)

26
If you can’t build a good monolith you
shouldn’t be doing microservices

...the µServices
ride
● Ability to write a modular monolith
● Strong grasp on your domain model
● Ability to design loosely coupled services
○ = state + transactions
● Ability to scale and decouple teams
● Ability to deploy continuously
● A strong “DevOps” culture
● Be comfortable with decentralised,
complex systems + uncertainty

28
If you can’t build good microservices
you shouldn’t be doing serverless

29
What practices do we need in this
new world?

31
Demo: AWS Meetup Lounge
CloudFront
API Gateway
Authentication
You!
awslounge.onegeek.com.au
Twitter Fetch
Twitter feed
Twitter Push
IoT Message
Broker
Sentiment
Analysis
Checkpoint
Amazon ML
#AWSAgility

32
Demo: AWS Meetup Lounge
Demo

34
Smaller functions = more things!
Challenges > Async

35
Options
● ...
Challenges
● Observability + Tracing
● Structured logging
● Shift responsibility into
platform
● Good acceptance tests
Challenges > Vendor lock-in

36
Vendor lock-in

37 https://twitter.com/alambert/status/958050456101642241

38
Options
● Multi-cloud = lowest
common denominator
● There is always a vendor
(even if it’s you!)
● You’ll never be completely
vendor agnostic
● Multi-cloud abstraction
● Avoid altogether
● Structure your code better
Challenges

39
Options
● Multi-cloud = lowest
common denominator
● There is always a vendor
(even if it’s you!)
● You’ll never be completely
vendor agnostic
● Multi-cloud abstraction
● Avoid altogether
● Structure your code better
Challenges

Lambda
Handler
Challenges > Vendor lock-in > Lambda Anti-pattern
{
"Records": [
{
"EventVersion": "1.0",
"EventSubscriptionArn": "somearn",
"EventSource": "aws:sns",
"Sns": {
...
}
}
]
}
{
"Success": true
}

MQ Handler
WebSocket
handler
MQ Adapter WS Adapter
Azure Function
Adapter
Google Cloud Fn
Adapter
Azure Function
Handler
Google Cloud Fn
Handler
Lambda
Handler
Lambda
Adapter
Service
Repository
Adapter
(Anti-corruption layer)
Ports
Business Logic
Collaborators ...
Challenges > Vendor lock-in > Ports and Adaptors

42
Challenges > Vendor lock-in > Ports and Adaptors > Sentiment Analysis Example
Twitter feed
Sentiment
Analysis
IoT Message
Broker

Challenges > Vendor lock-in > Ports and Adaptors > Lambda Port

Challenges > Vendor lock-in > Ports and Adaptors > SNS Adapter

Challenges > Vendor lock-in > Ports and Adaptors > Lambda Sentiment Service

Summary
● Separate protocol handling from
business logic
● Business logic shouldn’t change with
introduction of a new protocol
● Port, Adapter and Business Logic are
independently testable

47
How do you test locally?
Challenges > Testing Locally

48
Options
● Stub services (e.g. Localstack)
● Multiple handlers
● Stub API (e.g. Moto)
● Unit tests

49

50
Options
● Feature arms race
● Trustworthy?
● Integration style = slower
● All cloud providers?
● Stub services (e.g. Localstack)
● Stub API (e.g. Moto)
● Multiple handlers
● Unit tests
Challenges

51
Writing good unit tests
● Follow test pyramid
● Stub out AWS APIs
● Test your business logic
separately from any
serverless features (e.g.
lambda handler)
● Callback pyramid of death
● Tests are hard to write
● Your business logic knows
about AWS
● You have to stub multiple
services per test
● Large methods
● Confusing control flow
Challenges > Testing Locally > Unit Test
Code Smells

52
Challenges > Testing Locally > Unit Test > Twitter Fetch
Checkpoint
Twitter Fetch

Challenges > Testing Locally > Unit Test > Lambda Service
Look ma, no AWS
Code

55
Challenges > Testing Locally > Unit Test > Sentiment Analysis Example
Twitter feed
Sentiment
Analysis

Integration-style test
of handler
Challenges > Testing Locally > Unit Test > Lambda Handler

Summary
● Business logic should be tested
separately from port + adapter
● Majority of tests, should be fast unit
tests
● Stub AWS Services where required
● (Lambda) Handlers can be locally
integration tested with pre-canned JSON
events

58
Async - who is my consumer?
Challenges > Async

59
Async - delay in feedback
Challenges > Async

60
Options
● Local integration testing
● Cloud env. integration testing
● Contract testing
Challenges > Async

61
Options
● Can’t run complete local
environment
● Many moving parts
● Problems with integration
testing...
Challenges
Challenges > Async

62
Options
● Can’t run complete local
environment
● Many moving parts
● Problems with integration
testing...
Challenges
Challenges > Async

63
Contract tests
Challenges > Async

64
IoT Message
Broker
Sentiment
Analysis
Contract
{
'Positive': 0.234,
'Negative': 10.123,
'Neutral': 7.334,
'Mixed': 0
}
Consumer
Provider
Challenges > Async > Contract Tests

65
CloudFront
API Gateway
Authentication
You!
awslounge.onegeek.com.au
Twitter Fetch
Twitter feed
Twitter Push
IoT Message
Broker
Sentiment
Analysis
Checkpoint
Amazon ML
#awsmelb

66
Authentication Twitter Fetch
Twitter feed
Twitter Push
IoT Message
Broker
Sentiment
Analysis
Checkpoint
#awsmelb
Challenges > Async > Contract Tests > Contracts are everywhere!
Contract
Contract
Contract
Contract
Contract
Contract
Contract
Contract

67
Problems with automated e2e tests
● Slow
● Easy to break
● Hard to fix
● Scales badly across teams
● Lots of set up maintenance
● $$ potentially costly

68
C P
mock
Understanding Test Symmetry

69
Test symmetry
Solved problems New problems
● Hard to keep both sides in
sync
● Fast feedback
● Few dependencies
● No dedicated environment
● Reliable
● Easy to debug

70
C P
mock

71
C P
mock

PPCC
mock
contract

Bug turnaround -
minutes
Contracts tests

Know before you
commit
Contracts tests

Make changes with
speed and confidence
Contracts tests

Deploy independently
Contracts tests

Better API design
(Consumer)
Contracts tests

78
● Open source
● Multiple languages
○ Javascript
○ Go
○ Python
○ JVM
○ .NET
○ + more
pact.io
● HTTP contracts
● Message contracts
● … protocol agnostic!

Expectations on
Sentiment Analyser
made clear

80
Acceptance Tests vs Canary Release
Challenges > Async

Summary
81
Summary
Problems
● Serverless complexity > uServices
● Testing cloud native services
● Avoiding vendor lock-in
● Handling events and asynchronous Fn’s
● More Fn’s = more things!

Summary
82
Summary
Problems Solutions
● Microservices+
● Create cloud and protocol
anti-corruption layer
● Mock out AWS SDK in unit tests
● Use contract tests
● Reduce reliance on end-to-end
environments
● Balance ad-hoc exploratory +
acceptance tests
● Serverless complexity >
uServices
● Testing cloud native services
● Avoiding vendor lock-in
● Handling events +
asynchronous Fn’s
● More Fn’s = more things!

Testing large-scale, serverless
and asynchronous systems
Matt Fellows
@matthewfellows
...without end-to-end integrated tests!

Trends and development practices in Serverless architectures

Recomendados

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente (15)

Similar a Trends and development practices in Serverless architectures

Similar a Trends and development practices in Serverless architectures (20)

Más de DiUS

Más de DiUS (20)

Último

Último (20)

Trends and development practices in Serverless architectures