The document discusses the challenges of monitoring microservices architectures. It notes that microservices are deployed much more frequently than traditional applications, sometimes in seconds or milliseconds. This rapid deployment rate requires monitoring tools that can measure and report metrics at an equally fast pace. It also explains that microservices architectures involve large numbers of loosely coupled services distributed across multiple regions, zones, and instances. Monitoring tools must be able to handle the scale and complexity of these distributed systems. Other challenges discussed include visualizing request flows, understanding common failure scenarios, and testing monitoring systems at a large scale. The document advocates for using simulations to model microservices architectures and stress test monitoring tools to help address these challenges.

1. Monitoring Microservices:
A ChallengeAdrian Cockcroft @adrianco
Technology Fellow - Battery Ventures
March 2015

2. What does @adrianco do?
@adrianco
Technology Due
Diligence on Deals
Presentations at
Conferences
Presentations at
Companies
Technical Advice
for Portfolio
Companies
Program
Committee for
Conferences
Networking with
Interesting PeopleTinkering with
Technologies
Maintain Deep
Relationship with
Cloud Vendors

3. What does @adrianco do?
@adrianco
Technology Due
Diligence on Deals
Presentations at
Conferences
Presentations at
Companies
Technical Advice
for Portfolio
Companies
Program
Committee for
Conferences
Networking with
Interesting PeopleTinkering with
Technologies
Maintain Deep
Relationship with
Cloud Vendors

4. What does @adrianco do?
@adrianco
Technology Due
Diligence on Deals
Presentations at
Conferences
Presentations at
Companies
Technical Advice
for Portfolio
Companies
Program
Committee for
Conferences
Networking with
Interesting PeopleTinkering with
Technologies
Maintain Deep
Relationship with
Cloud Vendors

5. What does @adrianco do?
@adrianco
Technology Due
Diligence on Deals
Presentations at
Conferences
Presentations at
Companies
Technical Advice
for Portfolio
Companies
Program
Committee for
Conferences
Networking with
Interesting PeopleTinkering with
Technologies
Maintain Deep
Relationship with
Cloud Vendors

6. What does @adrianco do?
@adrianco
Technology Due
Diligence on Deals
Presentations at
Conferences
Presentations at
Companies
Technical Advice
for Portfolio
Companies
Program
Committee for
Conferences
Networking with
Interesting PeopleTinkering with
Technologies
Maintain Deep
Relationship with
Cloud Vendors

7. Microservices

8. A Microservice Definition
!
Loosely coupled service oriented
architecture with bounded contexts

9. A Microservice Definition
!
Loosely coupled service oriented
architecture with bounded contexts
If every service has to be
updated at the same time
it’s not loosely coupled

10. A Microservice Definition
!
Loosely coupled service oriented
architecture with bounded contexts
If every service has to be
updated at the same time
it’s not loosely coupled
If you have to know too much about surrounding
services you don’t have a bounded context. See the
Domain Driven Design book by Eric Evans.

11. Speed

12. Speeding Up Deployments
Datacenter Snowflakes
• Deploy in months
• Live for years

13. Speeding Up Deployments
Datacenter Snowflakes
• Deploy in months
• Live for years
Virtualized and Cloud
• Deploy in minutes
• Live for weeks

14. Speeding Up Deployments
Datacenter Snowflakes
• Deploy in months
• Live for years
Virtualized and Cloud
• Deploy in minutes
• Live for weeks
Container Deployments
• Deploy in seconds
• Live for minutes/hours

15. Speeding Up Deployments
Datacenter Snowflakes
• Deploy in months
• Live for years
Virtualized and Cloud
• Deploy in minutes
• Live for weeks
Container Deployments
• Deploy in seconds
• Live for minutes/hours
AWS Lambda Events
• Respond in milliseconds
• Live for seconds

16. Speeding Up Deployments
Measuring CPU usage once a minute makes no sense for containers…
Coping with rate of change is the first challenge for monitoring tools.
Datacenter Snowflakes
• Deploy in months
• Live for years
Virtualized and Cloud
• Deploy in minutes
• Live for weeks
Container Deployments
• Deploy in seconds
• Live for minutes/hours
AWS Lambda Events
• Respond in milliseconds
• Live for seconds

17. Scale

18. A Possible Hierarchy
Continents
Regions
Zones
Services
Versions
Containers
Instances
How Many?
3 to 5
2-4 per Continent
1-5 per Region
100’s per Zone
Many per Service
1000’s per Version
10,000’s
It’s much more challenging
than just a large number of
machines

19. Flow

20. Some tools can show
the request flow
across a few services

21. But interesting
architectures have a
lot of microservices!
Flow visualization is
a challenge.
See http://www.slideshare.net/LappleApple/gilt-from-monolith-ruby-app-to-micro-service-scala-service-architecture

22. Failures

23. ELB Load Balancer
Zuul API Proxy
Karyon
Business
Logic
Staash Data
Access Layer
Priam Cassandra
Datastore
Simple NetflixOSS
style microservices
architecture on three
AWS Availability Zones

24. ELB Load Balancer
Zuul API Proxy
Karyon
Business
Logic
Staash Data
Access Layer
Priam Cassandra
Datastore
Simple NetflixOSS
style microservices
architecture on three
AWS Availability Zones

25. ELB Load Balancer
Zuul API Proxy
Karyon
Business
Logic
Staash Data
Access Layer
Priam Cassandra
Datastore
Simple NetflixOSS
style microservices
architecture on three
AWS Availability Zones
Zone partition/failure
What should you do?
What should monitors show?

26. ELB Load Balancer
Zuul API Proxy
Karyon
Business
Logic
Staash Data
Access Layer
Priam Cassandra
Datastore
Simple NetflixOSS
style microservices
architecture on three
AWS Availability Zones
Zone partition/failure
What should you do?
What should monitors show?
By design, everything works
with 2 of 3 zones running.
This is not an outage, inform
but don’t touch anything!
Halt deployments perhaps?

27. ELB Load Balancer
Zuul API Proxy
Karyon
Business
Logic
Staash Data
Access Layer
Priam Cassandra
Datastore
Simple NetflixOSS
style microservices
architecture on three
AWS Availability Zones
Zone partition/failure
What should you do?
What should monitors show?
By design, everything works
with 2 of 3 zones running.
This is not an outage, inform
but don’t touch anything!
Halt deployments perhaps?
Challenge: understand and
communicate common
microservice failure patterns.

28. Testing

29. Testing monitoring tools at scale
gets expensive quickly…

30. Simulation

31. Simulated Microservices
Model and visualize microservices
Simulate interesting architectures
Generate large scale configurations
Eventually stress test real tools
!
See github.com/adrianco/spigo
Simulate Protocol Interactions in Go
Visualize with D3
ELB Load Balancer
Zuul API Proxy
Karyon
Business
Logic
Staash
Data
Access
Layer
Priam Cassandra
Datastore
Three
Availability
Zones

32. Code for an architecture
Create(cname, "priamCassandra", Regions, priamCassandracount, cname)
Create(sqname, "store", Regions, mysqlcount, sqname)
Create(evname, "store", Regions, evcachecount)
Create(tname, "staash", Regions, staashcount, sqname, evname, cname)
Create(pname, "karyon", Regions, phpcount, tname)
Create(nname, "karyon", Regions, nodecount, tname)
Create(zuname, "zuul", Regions, zuulcount, pname, nname)
Create(elbname, "elb", Regions, 0, zuname)
Create(dns, "denominator", 0, 0, elbname)

33. My challenge to you:
Build your architecture in Spigo.
Stress monitoring tools with it.
Fix monitoring for microservices!

34. Office Hours Lunch: 12:15pm, Ballroom A
Disclosure: some of the companies mentioned may be Battery Ventures Portfolio Companies
See www.battery.com for a list of portfolio investments
● Microservices Challenges
● Speed and Scale
● Flow and Failures
● Testing and Simulation
!
● Battery Ventures http://www.battery.com
● Adrian’s Tweets @adrianco and Blog http://perfcap.blogspot.com
● Slideshare http://slideshare.com/adriancockcroft