Se ha denunciado esta presentación.
Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.

How Amazon.com Uses AWS Management Tools - DEV340 - re:Invent 2017

553 visualizaciones

Publicado el

Amazon.com enables all of its developers to be productive on AWS by operating across tens-of-thousands of team-owned AWS accounts, all while raising the bar on security, visibility and operational control. Amazon has been able to achieve these seemingly conflicting ideals by automating setup and management of these accounts at scale using AWS Management Tools such as CloudFormation, Config, CloudTrail, CloudWatch and EC2 Systems Manager. In this session, discover more about how Amazon.com built ASAP using AWS Management tools, and understand some of the decisions they made as their usage of AWS evolved over time. You will learn about the design, architecture and implementation that Amazon.com went through as part of this effort.

  • Sé el primero en comentar

How Amazon.com Uses AWS Management Tools - DEV340 - re:Invent 2017

  1. 1. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. How Amazon.com Uses AWS Management Tools M i k e B u r k e , P r i n c i p a l T e c h . P r o g r a m M a n a g e r , A m a z o n . c o m P r a s h a n t P r a h l a d , S r . M g r P r o d u c t M a n a g e m e n t , A W S D E V 3 4 0 N o v e m b e r 2 9 , 2 0 1 7 Guardrails, not Gates
  2. 2. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. What can you expect from this session • Management Tools? • Your governance philosophy first, tools will follow • Amazon.com’s journey • Role of management tools in journey • You too can implement similar mechanisms
  3. 3. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Getting situated in the Cloud Common questions: • What AWS Account structure should I use? • How much governance should I put in place? • How do I keep up with my company’s AWS usage? (and have a life) • Do I have adequate metadata for verification/analysis?
  4. 4. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. The challenge - Define - Discover - Monitor - Manage - Report - Respond - Agility - Innovation Governance Developmentspeed
  5. 5. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS enables you to do both - Define - Discover - Monitor - Manage - Report - Respond - Agility - Innovation Governance Developmentspeed With AWS you can programmatically: • Define provisioning and configuration of resources • Continuously discover new resources and changes to existing resources • Monitor resources and operations for compliance • Manage, report on, and respond to changes to your resources
  6. 6. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.  CloudFormation  Service Catalog  CloudTrail  Config  CloudWatch  Trusted Advisor  EC2 Systems Manager  Parameter Store // State Manager // Inventory // Maintenance Windows // Patch Manager // Run Command Introducing AWS Management Tools
  7. 7. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Discover Manage Monitor Report Respond Define AWS CloudFormation AWS Service Catalog Amazon EC2 Systems Manager AWS Config AWS CloudTrail Amazon CloudWatch Trusted Advisor AWS Management Tools: You choose
  8. 8. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Management Tools: CliffsNotes 1.Tools that automate lifecycle management of AWS resources, especially as you scale usage 2.No need to trade off visibility/control against agility 3.Provide better visibility and control than customers experienced previously 4.Primitives that can match changes to customers governing philosophy
  9. 9. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Customers who use AWS Management Tools
  10. 10. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS at Amazon
  11. 11. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Behind The Scenes Exclusive
  12. 12. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Behind The Scenes Exclusive
  13. 13. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. How We Roll Two-Pizza Teams +
  14. 14. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. How We Roll Two-Pizza Teams + Single Threaded Ownership +
  15. 15. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. How We Roll Two-Pizza Teams + Single Threaded Ownership + Bias For Action +
  16. 16. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. How We Roll Two-Pizza Teams + Single Threaded Ownership + Bias For Action + Continuous Deployment =
  17. 17. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. How We Roll Two-Pizza Teams + Single Threaded Ownership + Bias For Action + Continuous Deployment = A LOT OF CHANGE
  18. 18. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. A Brief History
  19. 19. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Initial Approach On-Premise Network us-east-1 eu-west-1 us-west-2 … Shared Account
  20. 20. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. On-Premise Network Team Accounts Today us-east-1 eu-west-1 us-west-2 … Shared Account
  21. 21. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Today • Prime Day 2016 “Amazon retail team increased the size of their EC2 fleet, adding capacity that was equal to all of AWS and Amazon.com back in 2009” • Prime Day 2017 • DynamoDB: 3.34 trillion requests, peaking at 12.9 million per second • AWS Config: over 14 million configuration items tracked • AWS CloudTrail: 50 billion events, 419 billion API calls • CloudFormation: Nearly 31,000 stacks created for Prime Day https://aws.amazon.com/blogs/aws/prime-day-2017-powered-by-aws
  22. 22. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Two Models, Different Challenges Team AccountsShared Account
  23. 23. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Access Control
  24. 24. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Access Control Amazon RDS Instance
  25. 25. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Access Control Amazon RDS Instance
  26. 26. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Access Control Upgrade DB Engine Version Amazon RDS Instance
  27. 27. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Access Control Amazon RDS Instance Delete Instance
  28. 28. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Traditional Approach – Central Team
  29. 29. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Traditional Approach – Central Team
  30. 30. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Alternative 1 – Build A Wrapper
  31. 31. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Alternative 2 – Resource Permissions "Action":[ ”rds:*” ], "Effect":"Allow", "Resource":"*", "Condition":{ "StringEquals":{ ”rds:db-tag/team-name": [”finance"] } }
  32. 32. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Many Accounts
  33. 33. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Initial Approach - Restricting IAM • Take away root credentials • All IAM Roles are created through a management layer • AuthZ through an identity broker
  34. 34. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Restricting IAM - Identity Broker 1. Authenticate + Authorize 2. Assume Role 3. STS Token 4. URL 5. Launch Console Identity Broker I am: Bob Roberts I want to: Manage-RDS On: AWS Account 1234367
  35. 35. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Restricting IAM – Lessons • Yet Another Shim! • Prevents some automation • Not granular enough
  36. 36. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Getting Out Of The Way
  37. 37. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ASAP SNS Topic AWS ConfigAWS Cloudtrail AWS Account Amazon Cloudwatch Events
  38. 38. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ASAP SNS Topic SQS Notification Queue Event Dispatcher Rule Evaluators AWS ConfigAWS Cloudtrail ASAP AWS Account Describe state Amazon Cloudwatch Events
  39. 39. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ASAP SNS Topic SQS Notification Queue Event Dispatcher Rule Evaluators Reactor SNS Topics AWS ConfigAWS Cloudtrail ASAP AWS Account Describe state Reactors Amazon Cloudwatch Events
  40. 40. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. • Things that changed • Baselines and drift detection • Change notification with rich data AWS Config AWS Account SNS Topic AWS Config
  41. 41. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon CloudWatch Events • Things that happened • Not just user-initiated events • Scheduled and custom events SNS Topic AWS Account Amazon Cloudwatch Events
  42. 42. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. • User and application activity • Every API call captured and logged • Broad coverage of AWS features • With Cloudwatch Events – no log crawls AWS Cloudtrail SNS Topic AWS Account AWS Cloudtrail Amazon Cloudwatch Events
  43. 43. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ASAP IRL AWS Account Amazon RDS Instance
  44. 44. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ASAP IRL AWS Account AWS Cloudtrail [{ ... "arn:aws:iam::123456789012:user/Mike", "eventTime": "2017-11-10T21:22:54Z", "eventSource": ”rds.amazonaws.com", "eventName": ”CreateDbInstance", "awsRegion": "us-east-2", "requestParameters": { ”dbInstanceId": ”mine-all-mine”, “MultiAZ” : “false” } ... Amazon RDS Instance
  45. 45. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ASAP IRL AWS Account AWS Cloudtrail Amazon Cloudwatch Events Amazon RDS Instance
  46. 46. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ASAP IRL AWS Account AWS Cloudtrail SNS Topic Amazon Cloudwatch Events Amazon RDS Instance
  47. 47. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ASAP IRL AWS Account AWS Cloudtrail SNS Topic ASAP SQS Notification Queue Event Dispatcher Rule Evaluators Amazon Cloudwatch Events Amazon RDS Instance
  48. 48. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ASAP IRL AWS Account Reactor AWS Cloudtrail SNS Topic ASAP SQS Notification Queue Event Dispatcher Rule Evaluators Reactor SNS TopicsAmazon Cloudwatch Events Amazon RDS Instance
  49. 49. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ASAP IRL AWS Account Reactor AWS Cloudtrail SNS Topic ASAP SQS Notification Queue Event Dispatcher Rule Evaluators Reactor SNS Topics Ticket / Notification Amazon Cloudwatch Events Amazon RDS Instance
  50. 50. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ASAP in Action SNS Topic SQS Notification Queue Event Dispatcher Rule Evaluators Reactor SNS TopicsAmazon Cloudwatch Events AWS Cloudtrail ASAP AWS Account Reactor Amazon RDS Instance Ticket / Notification
  51. 51. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. What do we get? • Scale • Ability to assess the broader context • Catch problems early • Rich feedback and teaching tools • Ability to automatically escalate, mitigate or reverse (careful!)
  52. 52. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Other considerations • Anti-entropy checks • Tamper detection • Exceptions • Regular sweeps
  53. 53. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Looking ahead • Not just security • Simplify fixes • AWS CloudFormation StackSets and AWS Config Rules
  54. 54. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Avoiding Problems
  55. 55. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS CloudFormation • Be secure by default • ‘Known Good’ configurations: • VPC • Static Website via S3 + CloudFront • Amazon Aurora Clusters • Standard resources: • Cloudwatch Log Groups • Default IAM roles
  56. 56. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon EC2 Systems Manager (SSM) • Automated patching • Patch Group – what instances get patched? • Patch Baseline – what patches are applied? {PatchFilterGroup= {PatchFilters=[ {Key=PRODUCT,Values=[AmazonLinux2017.09]}, {Key=CLASSIFICATION,Values=[Security]}]}, ApproveAfterDays=0}
  57. 57. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Takeaways • Shared accounts have scaling limits • AWS enables people to move faster .. try to stay out of their way. • Everything changes when everything is an API. • Set people off on the right path, help them stay on it.
  58. 58. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Takeaways • Shared accounts have scaling limits • AWS enables people to move faster .. try to stay out of their way. • Everything changes when everything is an API. • Set people off on the right path, help them stay on it. Governance vs. Agility doesn’t have to be a binary decision!
  59. 59. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Actions • Pick your governance philosophy early and know it will evolve • Select management tools that help you implement your governance philosophy • Adopt services when you’re ready and be comfortable to deprecate what you have built • You can build ASAP too!
  60. 60. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Management Tools: CliffsNotes 1.Tools that automate lifecycle management of AWS resources, especially as you scale usage 2.No need to trade off visibility/control against agility 3.Provide better visibility and control than customers experienced previously 4.Primitives that can match changes to customers governing philosophy
  61. 61. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Easy to try Management Tools 1. On the internet: https://aws.amazon.com/products/management/ 2. AWS Management Tools blog post : https://aws.amazon.com/blogs/mt/ 3. You can learn a lot just by turning on AWS Config Rules: https://github.com/awslabs/aws-config-rules 4. Use the console!
  62. 62. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Thank you! P l e a s e s u b m i t y o u r s u r v e y

×