3. Load balancers are important (again)
¨ Containers and Microservices
¨ Background on Load Balancing
¨ Teaching old load balancers new tricks
¨ Emerging patterns for Load Balancing in
microservices
¨ Future directions
4. Information Technology is rapidly
accelerating
Time
Web
Digital
Social
Mobile
Things
Smart Things
RateofChange
7. ¨ Collaboration between Dev
and Ops teams
¨ Tooling and automation to
squeeze commit-to-deploy
time
Addressing the Gap:
DevOps
8. Agile and DevOps
1. Frequent Iterations
2. All phases all the time
3. Risk-based
1. Shorten time from commit
to production
2. “You build it, You run it”
3. Measure everything
4. Experimentation and
learning
Agile DevOps
9. Agile + DevOps Revolution
10+ Deploys per Day
Flickr, 2009
50 million Deploys Per
Year
Amazon Web Services,
2015
10. Image by BMW Werk Leipzig - http://bmw-werk-leipzig.de, CC BY-SA 2.0 de, https://commons.wikimedia.org/w/index.php?curid=11928438
Key
Mechanisms:
Automation
17. Load Balancing: Traditional vs. Cloud Native
Static Applications, well
defined topology
Dynamic Microservices,
changing topology
W W W
A A A A A
W
M
M
TrafficismostlyN-S
M
M
M
M
M
M
M
M
M
Traffic is mix
of N-S and E-W
25. A brief history of Load balancing
Internet
W W
High Availability
Internet
W W WW
HA + Load Balancing
Internet
Webserver
26. A brief history of Load balancing
Internet
W WW
App 1 App 2
W WW
foo.com/app1 foo.com/app2
L7 routing
Internet
W WW
SSL Traffic
Unencrypted Traffic
Internet
W WW
App Firewall
30. NetScaler CPX
Containerized
NetScaler for
Developers /
Cloud native
deployments
¨ Same hardened code, same features*
as bigger form factors
¨ Unified control plane with Ingress and
other LB (NetScaler MAS)
¨ Unified monitoring, logging and
analytics
* Same as NetScaler Standard Edition minus VPN
31. NetScaler CPX Express
¨ “Developer” Edition*
¨ No license required
¨ Available for download from Docker App Store in Q3
¨ Drop-in replaceable with licensed version
¨ Get it here:
– http://microloadbalancer.com
*Not for production use
40. Client-side LB
¤ Embedded into calling
application
n e.g., Netflix Ribbon
(with Eureka)
¤ Run as side-cars
(alongside each
application or one-
per-host) Eureka
M M
M
M
M
M
41. Client-side LB
¤ Run as side-cars, or
one-per-host
Service
Discovery
M
M
M
M
M
M
Clients
Service
LB
LB
42. Client-side LB Examples
¨ Netflix Prana - Hystrix as side car (deprecated)
¨ Linkerd from Buoyant.io (based on Twitter Finagle)
¨ AirBnB Smartstack/Synapse (uses HAProxy)
¨ Uber Hyperbahn
¤ Like Finagle, switched from HAProxy
¨ Kube Proxy
¤ Initially user-space, now based on IPTables
¨ Docker Swarm LB
¤ Based on IPVS
¨ Mesos LB
¤ Based on IPTables
43. Zero-Downtime Deploys
¨ Canary deploys
¨ Traffic can be selected
based on source
(country), headers
M M M M
Version N
M M M M M
Version N+1
10%90%
CD
Pipeline
45. Zero downtime deploys with NetScaler
HostH1
HostH2
HostH3
Ingress NetScaler
HostH4
ContainerClusterCluster API
Container
Events Reconfig
uration
Controller
NetScaler CPXJenkins
46. Advanced Resilience – Circuit Breaker
¨ Circuit Breaker is a
pattern that prevents
cascading failures due
to unresponsive
downstream services
¨ A load balancer can be
configured to ignore a
slow backend
Credit: http://martinfowler.com/bliki/CircuitBreaker.html
49. Wrap-up
¨ Load balancing is
different for containers /
microservices
¨ Integration with container
cluster managers is
needed
¨ Consider using the same
LB technology for ingress
and intra-cluster
¨ Emerging patterns
solidify the importance
of the load balancer
¨ LB in the wire brings
¤ Simplicity
¤ Resilience
¤ Future proofing