How many services do you have? 5, 10, 100? How do you even run large number of services? A micro service may be relatively simple. But services also mean distributed systems, which are inherently complex. 5 services are complex. A thousand services across many generations are at least 200 times as complex. How do we deal with such complexity?
This talk discusses service architecture at Internet scale, the need for larger transaction density, larger horizontal and vertical scale, more predictable latencies under stress, and the need for standardization and visibility. We’ll dive into how we build our latest generation service infrastructure based on Scala and Akka to serve the needs of such a large scale ecosystem.
Lastly, have the cake and eat it too. No, we’re not keeping all the goodies only to ourselves. They are all there for you in open source.
4. Cost/Benefits of Moving to Microservices
• Independence – faster PDLC
• Freedom of choice for service implementation
• Easy evolution of service & technology
• Coexisting services across generations
• Complexity & Latency
Gains
• Homogeneity
• Consistency of implementation across
• Timing & Determinism
Losses
Hmm. To be, or not to be… a service, that is...
8. Latency by Deployment Topology
• Avoid too many layers of services
• Keep state close to the edge
• The more hops, the higher and less deterministic the latency is
11. Services Need to Scale
• Scale horizontally with increasing workload
• More nodes, or…
• More pods with increasing workload
• Scale vertically – why?
• Keep the number of instances under control
• 125 nodes @16CPU easier to manage than 1000 nodes @2CPU
• Less load on network and switching infrastructure
• Potentially better utilization & cache hits
• Stateful systems: More limited horizontal scale
• Need critical mass for redundancy
20. Why Does it Matter?
Respond in a deterministic, timely manner. Controls determinism
Stays responsive in the face of failure – even cascading failures
Stays responsive under workload spikes
Basic building block for responsive, resilient, and elastic systems
Responsive
Resilient
Elastic
Message Driven
22. Circuit Breaker Keeps systems responsive under
failure
Avoids cascading failures
Especially with multi-generational
downstream services
Critical part to keeping your 1000
services alive
25. Standardization
• Monitoring
• Need to collect metrics, consistently
• Logging
• Correlation across services
• Uniformity in logs
• Security
• Need to apply standard security configuration
• Environment Resolution
• Staging, production, etc.
Consistency in the face of Heterogeneity
30. Service
• Akka Http/Spray Routes and Http Request Handler Actors
• Configured in squbs-meta.conf
• A service can be defined in a dependency artifact
31. Extension
• To start low level (non-actor) facilities needed for the environment
33. Cubes
Another deployment Topology
squbs: rhymes with cubes
Drop-in modules
Cubes can run in isolation as well as on
a flat classpath
Easy to compose/decompose/refactor
Cubes share the actor system
Provide better predictability
37. More Utilities
• Http Client
• Admin Console
• Actor Registry
• Perpetual Stream
• Persistence Buffer
• …
38. Summary
• Large number of services have benefits, but are more difficult
• Control your service topology for more determinism and lower latency
• Rule of thumb: No more than two hops of synchronous calls from
edge
• Reactive systems – ideal for services
• Responsive & resilient
• Standardization
• Walk like a duck, quack like a duck, and manage it like a duck
• squbs: Have the cake, and eat it too