Running serverless at scale
•
1 recomendación•701 vistas
In this talk, we will discuss some of the things to consider when running a serverless architecture at scale. How to think about and manage cold starts, and strategy for scaling out the app beyond the basic limits, and more.
Recomendados
Recomendados
Más contenido relacionado
La actualidad más candente
La actualidad más candente (20)
Similar a Running serverless at scale
Similar a Running serverless at scale (20)
Más de Yan Cui
Más de Yan Cui (20)
Último
Último (20)
Running serverless at scale
- 1. running serverless at SC AL E
- 2. Yan Cui http://theburningmonk.com @theburningmonk Principal Engineer @ Independent Consultant
- 5. available in Austria, Switzerland, Germany, Japan, Canada, Italy, US, Spain and Brazil
- 6. available on 30+ platforms
- 8. follow @dazneng for updates about the engineering team We’re hiring! Visit engineering.dazn.com to learn more. WE’RE HIRING!
- 9. AWS user since 2009
- 10. AWS user since 2009
- 11. What do you mean by ‘serverless’?
- 12. “Serverless”
- 14. Gojko Adzic It is serverless the same way WiFi is wireless. http://bit.ly/2yQgwwb
- 15. Serverless means… don’t pay for it if no-one uses it don’t need to worry about scaling don’t need to provision and manage servers
- 16. “Function-as-a-Service” AWS Lambda Azure Functions Google Cloud Functions Auth0 Webtask Spotinst Functions Kubeless IBM Cloud Functions
- 17. AWS Lambda
- 18. AWS Lambda API Gateway IOT SNS Kinesis CloudWatch
- 22. Simon Wardley Serverless will fundamentally change how we build business around technology and how you code.
- 23. Why serverless?
- 24. more Scalable
- 25. 1,000 concurrent executions (soft limit) 500 increase per minute (hard-ish limit)
- 26. 1,000 concurrent executions (soft limit) 500 increase per minute (hard-ish limit) AUTO-APPROVED RAISE TO 3000
- 27. 1,000 concurrent executions (soft limit) 500 increase per minute (hard-ish limit)
- 31. Cheaper (don’t pay for idle servers)
- 32. Resilience (built-in redundancy and multi-AZ)
- 34. Secure
- 37. protection from OS attacks Amazon automatically apply latest patches to host VMs
- 40. Deploy
- 50. the DevOps forcethe DevOps force is strong with serverlessis strong with serverless
- 51. idea production choose language + framework master language + framework figure out deployment configure AMI configure ELB configure autoscaling capacity planning over-provision for launch are we doing microservices? configure CI/CD
- 52. idea production choose language + framework master language + framework figure out deployment configure AMI configure ELB configure autoscaling capacity planning over-provision for launch are we doing microservices? configure CI/CD
- 53. idea production greater Velocity from idea to product
- 55. less ops responsibility on your shoulders
- 59. 1,000 concurrent executions (soft limit) 500 increase per minute (hard-ish limit)
- 63. serverless is not right for every use case (yet)
- 65. Lambda VPC
- 66. there are no silver bullets
- 68. 0 Containers “it works on my machine!” “production ready!”days
- 69. 0 Containers “it works on my machine!” “production ready!”days Serverless 0 “it works!” “production ready!” days
- 70. 0 Containers “it works on my machine!” “production ready!”days Serverless 0 “it works!” “production ready!” days v2! v3! v4! v5! v6!
- 71. EC2 docker us-east-1a us-east-1b us-east-1a us-east-1b Theory
- 72. Reality
- 73. Reality
- 74. scale-to-zero
- 75. serverful serverless us-east-1a us-east-1b us-east-1a us-east-1bscaled to zero!
- 76. scaling limits VPC long-running cold starts performance
- 77. scaling limits VPC long-running cold starts performance
- 78. scaling limits VPC long-running cold starts performance
- 82. scaling limits VPC long-running cold starts performance
- 84. scaling limits VPC long-running cold starts performance
- 86. scaling limits VPC long-running cold starts performance
- 87. What can we do now?
- 89. 1,000 concurrent executions (soft limit) 500 increase per minute (hard-ish limit)
- 90. 1,000 concurrent executions (soft limit) 500 increase per minute (hard-ish limit)
- 92. 1,000 concurrent executions (soft limit) 500 increase per minute (hard-ish limit) contact your AWS TAM about raising this
- 108. CloudFront API Gateway Lambda
- 109. cache as close to the user as possible!
- 110. CloudFront API Gateway Lambda cache
- 111. CloudFront API Gateway Lambda cache cache
- 112. CloudFront API Gateway Lambda cache cache cache take advantage of container reuse
- 114. CloudFront Lambda@Edge executed in the nearest AWS region
- 115. CloudFront Lambda@Edge executed in the nearest AWS region traffic is distributed across many regions
- 116. CloudFront Lambda@Edge executed in the nearest AWS region traffic is distributed across many regions fewer hops, shorter round- trip = faster response
- 117. avoid services that require persistent connections
- 118. DynamoDB on-demand tables can scale to up to 40K TPS DynamoDB
- 120. RDS ๏ set client-side pool size to 1 ๏ set server-side max connection high ๏ set server-side connection timeout to ~10 mins (to clean phantom connections) ๏ use read-replicas
- 121. RDS ๏ set client-side pool size to 1 ๏ set server-side max connection high ๏ set server-side connection timeout to ~10 mins (to clean phantom connections) ๏ use read-replicas Serverless Aurora is not ready for production-use (yet)
- 122. RDS new service HTTPs socket pooling, etc.
- 127. function-less
- 128. CloudFront API Gateway Lambda 500/min limit on scaling out
- 129. CloudFront API Gateway Lambda
- 130. CloudFront API Gateway DynamoDB
- 132. engineer the problem away instead of solving it high 5!
- 133. What about cold starts?
- 136. “cold starts only happen to the first request”
- 137. “cold starts only happen to the first request” WRONG!!!
- 138. function invocationconcurrent execution i.e. a container
- 139. function invocationconcurrent execution i.e. a container class instance method call
- 140. Lambda scales the number of concurrent executions based on traffic
- 141. existing “containers” are reused where possible
- 142. time invocation
- 156. FREQUENCY DURATION
- 157. FREQUENCY DURATION dictated by user traffic, out of your control
- 159. “Lambda is not a suitable solution for me because of cold starts”
- 160. “what is your latency requirement?”
- 161. cold starts that don’t add to user-facing latency is generally not worth worrying about
- 162. Node.js functions, no VPC, 1GB, averages ~500ms cold start with production workload
- 163. Node.js functions, no VPC, 1GB, averages ~500ms cold start with production workload (good enough for most web applications)
- 164. sporadic spikes latency existed before Lambda
- 165. GC pauses…
- 166. overloaded servers…
- 167. slow downstream, databases, etc.
- 168. networking issues…
- 169. cold starts is generally not an issue if you have a steady traffic pattern
- 170. time req/s
- 173. minimise the duration of cold starts so they fall within acceptable latency range
- 174. use Node.js, Python or Golang
- 175. trim dependencies
- 176. don’t require the full AWS-SDK if you only need one client
- 178. https://theburningmonk.com/2019/03/just-how-expensive-is-the-full-aws-sdk/ full AWS-SDK vs. DynamoDB only
- 179. https://theburningmonk.com/2019/03/just-how-expensive-is-the-full-aws-sdk/ full AWS-SDK vs. DynamoDB only webpack!!
- 181. avoid VPCs unless you need to access VPC-protected resources
- 185. horizontal scalability all the way
- 186. lambda warmer doesn’t work
- 187. make cold start durations acceptable
- 188. “not everything at Netflix runs at Netflix scale”