Real world serverless - architecture, patterns and lessons learned
•
5 recomendaciones•580 vistas
Serverless computing is quite the rage today. It enables companies to move to a platform with basically limitless capacity and it allows teams to embrace DevOps right from the start. So, this is literally the silver bullet we have waited for and everybody should embrace it. Right? Well, it depends… In this talk we will look at different scenarios and approaches to building and designing complex Serverless applications on AWS beyond your basic “Hello World”. We will cover architectures and patterns that help in adopting Serverless, for example how to design your Serverless landscape without getting lost in event-driven-chaos. Covering the whole development process, we will look at real world problems like testing, debugging, monitoring and securing your Serverless applications. Furthermore, we will use some of the essential tools that make working with AWS Lambda easier, such using Claudia.JS. Finally, we will discuss the drawbacks and misconceptions, like potential vendor lock-in and the illusion of NoOps. If you find Serverless computing interesting, then this talk will show you how to adapt this architectural style without getting burned too much.
Recomendados
Recomendados
Más contenido relacionado
La actualidad más candente
La actualidad más candente (20)
Similar a Real world serverless - architecture, patterns and lessons learned
Similar a Real world serverless - architecture, patterns and lessons learned (20)
Más de David Schmitz
Más de David Schmitz (13)
Último
Último (20)
Real world serverless - architecture, patterns and lessons learned
- 1. REAL WORLD SERVERLESS Going Lambda without being burned too much
- 6. @Koenighotze
- 9. @Koenighotze –Me “Some of the things we learned, while moving an application (partially) to AWS Lambda.”
- 14. @Koenighotze Some basics Tools and stuff Architecture Security Housekeeping Operations
- 15. @Koenighotze
- 18. @Koenighotze
- 20. @Koenighotze
- 21. @Koenighotze PLATFORM AS A SERVICE
- 22. @Koenighotze
- 24. @Koenighotze
- 26. @Koenighotze DESIGNING DISTRIBUTED SYSTEMS IS DIFFICULT
- 27. @Koenighotze HOW TO DRAW AN OWL?
- 28. @Koenighotze 1. Draw some circles 2. Draw the rest of the damn owl
- 29. @Koenighotze
- 30. @Koenighotze
- 31. @Koenighotze
- 32. @Koenighotze
- 35. @Koenighotze Serverless computing allows you to build and run applications and services without thinking about servers. Serverless applications don't require you to provision, scale, and manage any servers… Building serverless applications means that your developers can focus on their core product instead of worrying about managing and operating servers or runtimes…
- 36. @Koenighotze Serverless computing allows you to build and run applications and services without thinking about servers. Serverless applications don't require you to provision, scale, and manage any servers… Building serverless applications means that your developers can focus on their core product instead of worrying about managing and operating servers or runtimes…
- 37. @Koenighotze FOCUS ON WHAT MATTERS
- 38. @Koenighotze
- 39. @Koenighotze
- 45. @Koenighotze WHAT IS A “LAMBDA FUNCTION”?
- 47. @Koenighotze
- 48. @Koenighotze
- 50. @Koenighotze
- 51. @Koenighotze
- 52. @Koenighotze S3
- 53. @Koenighotze
- 54. @Koenighotze
- 55. @Koenighotze
- 56. @Koenighotze
- 58. @Koenighotze
- 60. @Koenighotze
- 61. @Koenighotze
- 65. @Koenighotze
- 66. @Koenighotze AWS LAMBDA MAY NEED NO SPECIAL FRAMEWORK
- 71. @Koenighotze Handler create Order forward Order cancel Order Vendor agnostic Vendor specific Anticorruptionlayer Pure domain logic
- 72. @Koenighotze YOU ABSOLUTELY NEED TOOLING FOR DEPLOYMENT
- 74. @Koenighotze
- 75. @Koenighotze
- 77. @Koenighotze $ zip index.zip index.js node_modules $ aws lambda create-function --function-name HelloWorld --runtime nodejs6.10 --role arn:aws:iam::…:… --handler index.handler --zip-file fileb://index.zip
- 79. @Koenighotze $ claudia create --region eu-west-1 --deploy-proxy-api --handler index.handler --name HelloWorld
- 82. @Koenighotze
- 84. @Koenighotze AWSTemplateFormatVersion: '2010-09-09' Transform: AWS::Serverless-2016-10-31 Description: Hello World Resources: HelloWorld: Type: AWS::Serverless::Function Properties: Timeout: 5 Runtime: nodejs6.10 Handler: index.handler CodeUri: .
- 85. @Koenighotze AWSTemplateFormatVersion: '2010-09-09' Transform: AWS::Serverless-2016-10-31 Description: Hello World Resources: HelloWorld: Type: AWS::Serverless::Function Properties: Timeout: 5 Runtime: nodejs6.10 Handler: index.handler CodeUri: .
- 86. @Koenighotze AWSTemplateFormatVersion: '2010-09-09' Transform: AWS::Serverless-2016-10-31 Description: Hello World Resources: HelloWorld: Type: AWS::Serverless::Function Properties: Timeout: 5 Runtime: nodejs6.10 Handler: index.handler CodeUri: .
- 87. @Koenighotze $ aws cloudformation package --template-file sam.yaml --s3-bucket somebucket > build/packaged.yaml $ aws cloudformation deploy --template-file build/packaged.yaml --stack-name voxxed-demo --capabilities CAPABILITY_NAMED_IAM
- 88. @Koenighotze $ aws cloudformation package --template-file sam.yaml --s3-bucket somebucket > build/packaged.yaml $ aws cloudformation deploy --template-file build/packaged.yaml --stack-name voxxed-demo --capabilities CAPABILITY_NAMED_IAM
- 89. @Koenighotze $ aws cloudformation package --template-file sam.yaml --s3-bucket somebucket > build/packaged.yaml $ aws cloudformation deploy --template-file build/packaged.yaml --stack-name voxxed-demo --capabilities CAPABILITY_NAMED_IAM
- 90. @Koenighotze $ aws cloudformation package --template-file sam.yaml --s3-bucket somebucket > build/packaged.yaml $ aws cloudformation deploy --template-file build/packaged.yaml --stack-name voxxed-demo --capabilities CAPABILITY_NAMED_IAM
- 91. @Koenighotze $ aws cloudformation package --template-file sam.yaml --s3-bucket somebucket > build/packaged.yaml $ aws cloudformation deploy --template-file build/packaged.yaml --stack-name voxxed-demo --capabilities CAPABILITY_NAMED_IAM
- 92. @Koenighotze ALL TOOLS HAVE DIFFERENT DRAWBACKS
- 94. @Koenighotze CLEANLINESS IS NEXT TO GODLINESS
- 95. @Koenighotze
- 97. @Koenighotze INTEGRATION TEST RIGHT FROM THE START
- 99. @Koenighotze
- 100. @Koenighotze
- 101. @Koenighotze YOUR TESTING TOOLS SHOULD BE AS MATURE AS YOUR PRODUCTION TOOLS
- 102. @Koenighotze TESTING LAMBDA FOR REAL IS JUST TOO EASY
- 103. @Koenighotze aws lambda invoke --function-name ReceiptUpload --region eu-central-1 --payload file://expensive_receipt.json out.json
- 104. @Koenighotze aws lambda invoke --function-name ReceiptUpload --region eu-central-1 --payload file://expensive_receipt.json out.json
- 106. @Koenighotze STAGING IS A THING
- 107. @Koenighotze
- 108. @Koenighotze Parameters: … Stage: Type: String Default: DEV ForwardBilling: Type: AWS::Serverless::Function Properties: FunctionName: !Sub ForwardBilling-${Stage}
- 109. @Koenighotze Parameters: … Stage: Type: String Default: DEV ForwardBilling: Type: AWS::Serverless::Function Properties: FunctionName: !Sub ForwardBilling-${Stage}
- 110. @Koenighotze aws cloudformation deploy --template-file … --stack-name … --parameter-overrides Stage=PROD
- 111. @Koenighotze aws cloudformation deploy --template-file … --stack-name … --parameter-overrides Stage=PROD
- 113. @Koenighotze
- 114. @Koenighotze
- 115. @Koenighotze aws cloudformation deploy … --parameter-overrides Stage=PROD --profile ACCOUNT_A
- 116. @Koenighotze aws cloudformation deploy … --parameter-overrides Stage=PROD --profile ACCOUNT_A
- 118. @Koenighotze
- 119. @Koenighotze HOW DO YOU REASON ABOUT CLOUD ARCHITECTURES?
- 120. @Koenighotze HOW TO KNOW WHAT TO CLEANUP?
- 121. @Koenighotze PYTHON LIST_LAMBDAS.PY —INACTIVE-DAYS-FILTER 10 https:!//github.com/epsagon/list-lambdas
- 122. @Koenighotze
- 123. @Koenighotze
- 124. @Koenighotze
- 125. @Koenighotze REDUCE ATTACK SURFACE REDUCE COGNITIVE CLUTTER STAY IN CONTROL
- 127. @Koenighotze COLD START HAPPENS ONCE, RIGHT?
- 128. @Koenighotze Users
- 134. @Koenighotze
- 135. @Koenighotze COLD START WILL HAPPEN ONCE FOR EACH CONCURRENT EXECUTION OF YOUR FUNCTION
- 136. @Koenighotze WHAT CAN YOU DO?
- 137. @Koenighotze
- 138. @Koenighotze NO SLA
- 139. @Koenighotze NO HARD DETAILS JUST GUIDELINES
- 141. @Koenighotze
- 144. @Koenighotze
- 145. @Koenighotze /AWS/LAMBDA/…REPORT REQUESTID: DURATION: 3208.99 MS BILLED DURATION: 3300 MS MEMORY SIZE: 128 MB MAX MEMORY USED: 59 MB
- 146. @Koenighotze /AWS/LAMBDA/…REPORT REQUESTID: DURATION: 3208.99 MS BILLED DURATION: 3300 MS MEMORY SIZE: 128 MB MAX MEMORY USED: 59 MB
- 149. @Koenighotze
- 150. @Koenighotze AWS WILL KILL EVEN WARM LAMBDAS AFTER A CERTAIN TIME FRAME
- 154. @Koenighotze DESIGN WITH LATENCY IN MIND
- 156. @Koenighotze SERVERLESS IS GOOD FOR SECURITY
- 157. @Koenighotze NO LONG LIVED SERVERS OR MANUAL PATCH SCHEDULES
- 158. @Koenighotze …THAT YOU NEED TO HANDLE
- 160. @Koenighotze HOW DO YOU DEPLOY A WEB APPLICATION FIREWALL… …WITHOUT A WEB …WITHOUT AN APPLICATION …WITHOUT A SERVER
- 161. @Koenighotze NEW OR ADAPTED SECURE CODING PATTERNS NEEDED
- 163. @Koenighotze
- 164. @Koenighotze IMAGINE EVERYBODY WANTS TO HARM YOU
- 165. @Koenighotze
- 167. @Koenighotze
- 169. @Koenighotze CLASSIC LAYERED CAKE ARCHITECTURE DOES NOT APPLY TO SERVERLESS
- 171. @Koenighotze const Joi = require('joi') const eventSchema = { 'lambdaName': Joi.string().required(), 'invokationCount': Joi.number().required(), 'dryRun': Joi.boolean().default(true) } exports.handler = (event, context, callback) => { const { error, value } = Joi.validate(event, eventSchema) if (error) { return callback(new Error('Cannot parse event')) } ...doSomethingWithValue() }
- 172. @Koenighotze const Joi = require('joi') const eventSchema = { 'lambdaName': Joi.string().required(), 'invokationCount': Joi.number().required(), 'dryRun': Joi.boolean().default(true) } exports.handler = (event, context, callback) => { const { error, value } = Joi.validate(event, eventSchema) if (error) { return callback(new Error('Cannot parse event')) } ...doSomethingWithValue() }
- 173. @Koenighotze const Joi = require('joi') const eventSchema = { 'lambdaName': Joi.string().required(), 'invokationCount': Joi.number().required(), 'dryRun': Joi.boolean().default(true) } exports.handler = (event, context, callback) => { const { error, value } = Joi.validate(event, eventSchema) if (error) { return callback(new Error('Cannot parse event')) } ...doSomethingWithValue() }
- 175. @Koenighotze
- 176. @Koenighotze SECURITY IS LIKE CSS NO ONE DELETES RULES YOU ONLY ADD RULES
- 179. @Koenighotze "Statement": [{ "Effect": "Allow", "Action": [ “s3:*", ], "Resource": [ “arn:aws:s3:::*” ] }, …
- 180. @Koenighotze "Statement": [{ "Effect": "Allow", "Action": [ "s3:*", ], "Resource": [ "arn:aws:s3:::*" ] }, …
- 181. @Koenighotze "Statement": [{ "Effect": "Allow", "Action": [ “s3:*", ], "Resource": [ “arn:aws:s3:::senacor/fis/orders/*” ] }, …
- 182. @Koenighotze "Statement": [{ "Effect": "Allow", "Action": [ "s3:GetObject" ], "Resource": [ “arn:aws:s3:::senacor/fis/orders/*” ] }, …
- 183. @Koenighotze "Statement": [{ "Effect": "Allow", "Action": [ "s3:GetObject" ], "Resource": [ “arn:aws:s3:::senacor/fis/orders/*” ] }, …
- 184. @Koenighotze
- 186. @Koenighotze
- 187. @Koenighotze NO EXCUSES WITH PROPER TOOLING
- 188. @Koenighotze Resources: ForwardBillingRole: Type: "AWS::IAM::Role" Properties: RoleName: !Sub "ForwardBillingRole-${Stage}" AssumeRolePolicyDocument: ... ManagedPolicyArns: ... Policies: - PolicyName: !Sub "StoreBillingInS3Policy-${Stage}" PolicyDocument: ... ForwardBilling: Type: AWS::Serverless::Function Properties: Role: !GetAtt ForwardBillingRole.Arn
- 189. @Koenighotze Resources: ForwardBillingRole: Type: "AWS::IAM::Role" Properties: RoleName: !Sub "ForwardBillingRole-${Stage}" AssumeRolePolicyDocument: ... ManagedPolicyArns: ... Policies: - PolicyName: !Sub "StoreBillingInS3Policy-${Stage}" PolicyDocument: ... ForwardBilling: Type: AWS::Serverless::Function Properties: Role: !GetAtt ForwardBillingRole.Arn
- 190. @Koenighotze Resources: ForwardBillingRole: Type: "AWS::IAM::Role" Properties: RoleName: !Sub "ForwardBillingRole-${Stage}" AssumeRolePolicyDocument: ... ManagedPolicyArns: ... Policies: - PolicyName: !Sub "StoreBillingInS3Policy-${Stage}" PolicyDocument: ... ForwardBilling: Type: AWS::Serverless::Function Properties: Role: !GetAtt ForwardBillingRole.Arn
- 191. @Koenighotze Resources: ForwardBillingRole: Type: "AWS::IAM::Role" Properties: RoleName: !Sub "ForwardBillingRole-${Stage}" AssumeRolePolicyDocument: ... ManagedPolicyArns: ... Policies: - PolicyName: !Sub "StoreBillingInS3Policy-${Stage}" PolicyDocument: ... ForwardBilling: Type: AWS::Serverless::Function Properties: Role: !GetAtt ForwardBillingRole.Arn
- 193. @Koenighotze YOUR FUNCTION MAY BE SMALL, BUT YOUR DEPENDENCIES MAY NOT
- 194. @Koenighotze $ du -s src/ 8.0K src/ $ ls -lh lambda.zip 7.6M Mar 1 07:13 lambda.zip
- 195. @Koenighotze $ du -s src/ 8.0K src/ $ ls -lh lambda.zip 7.6M Mar 1 07:13 lambda.zip
- 196. @Koenighotze
- 197. @Koenighotze
- 198. @Koenighotze
- 199. @Koenighotze
- 200. @Koenighotze SECURITY MUST BE PART OF DEVELOPMENT RIGHT FROM THE START
- 201. @Koenighotze YOU BUILD IT YOU RUN IT
- 202. @Koenighotze YOU BUILD IT YOU SECURE IT AND THEN RUN IT
- 205. @Koenighotze BEWARE OF KILLING YOUR VINTAGE BACKENDS
- 208. @Koenighotze “Billings are submitted using our legacy API”
- 209. @Koenighotze
- 210. @Koenighotze …AND AT THE END OF A MONTH…
- 211. @Koenighotze
- 212. @KoenighotzeGrigoriy Myasoyedov.1897 Please stop this lambda non-sense But the HYPE!
- 214. @Koenighotze aws lambda put-function-concurrency --function-name <NAME> --reserved-concurrent-executions <N>
- 215. @Koenighotze aws lambda put-function-concurrency --function-name <NAME> --reserved-concurrent-executions <N>
- 216. @Koenighotze KEEP AN EYE ON METRIC FOR CONCURRENCY FAILURES
- 217. @Koenighotze BE AWARE BOTTLENECKS TEND TO MOVE UPWARDS
- 219. @Koenighotze ATTACK OF SELF DENIAL
- 221. @Koenighotze BE AWARE OF PER ACCOUNT LIMITS
- 222. @Koenighotze
- 223. @Koenighotze
- 224. @Koenighotze
- 234. @Koenighotze ONE LAMBDA WITH DISPATCH MULTI LAMBDA - SHARED ZIP MULTI LAMBDA - SHARED NOTHING
- 236. @Koenighotze create Order forward Order cancel Order Order ZIP
- 237. @Koenighotze create Order forward Order cancel Order Order ZIP
- 240. @Koenighotze create Order forward Order cancel Order Order ZIP
- 241. @Koenighotze
- 244. @Koenighotze create Order forward Order cancel Order Order ZIP
- 245. @Koenighotze Create Order ZIP Forward Order ZIP Cancel Order ZIP cancel Orderforward Ordercreate Order
- 246. @Koenighotze cancel Orderforward Ordercreate Order
- 248. @Koenighotze
- 249. @Koenighotze
- 250. @Koenighotze
- 251. @Koenighotze SUMMARY
- 253. @Koenighotze DESIGNING DISTRIBUTED SYSTEMS IS STILL HARD
- 255. @Koenighotze YOU NEED A CLOUD-SAVVY OPERATIONS TEAM MEMBER
- 257. @Koenighotze
- 258. @Koenighotze
- 259. @Koenighotze
- 261. @Koenighotze NOT A SILVER BULLET
- 262. @Koenighotze SOMETIMES A SIMPLE VM IS ENOUGH
- 264. @Koenighotze LESS SERVERS THAT WE NEED TO TAKE CARE OF