The "Container services on AWS. Comparing Amazon ECS, AWS Fargate and Amazon EKS" session from the Technical Track of the AWS Startup Day Kyiv 2018

1. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Container services on AWS
Vasily Pantyukhin, AWS Solutions Architect

2. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
2

3. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
3

4. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
4

5. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
5
63%
https://www.cncf.io/blog/2018/08/29/cncf-survey-use-of-cloud-native-technologies-in-production-has-grown-over-200-percent/

6. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
6
Amazon ECS (2014)
AWS Fargate (2017)
Amazon EKS (2018)
Amazon ECR (2015)

7. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
7

8. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

9. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
9
EC2 instance EC2 instance EC2 instance EC2 instance EC2 instance

10. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
10
Availability Zone #1 Availability Zone #2 Availability Zone #3
Cluster

11. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
11
Availability Zone #1 Availability Zone #2 Availability Zone #3
Scheduling and Orchestration
Cluster Manager Placement Engine

12. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
12
ECS instance
ECS
AMI
Docker
agent
ECS
agent

13. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
task
task
task

14. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
ECS instance
ECS
AMI
Docker
agent
ECS
agent
task task
task task
task definition
JSON
• Image
• CPU
• RAM
• …

15. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
EC2
task task
task
EC2
task task
balancerservice
• Health check
• Auto-scaling
• Load
Balancing
• Service
discovery
• …
service definition
JSON

16. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Task
• коротко-живущие задачи
Service
• долго-живущие приложения

17. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
17
Масштабирование

18. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
TASK
TASK
TASK
TASK
TASK
TASK
Target Groups
/checkout
/catalog
/homepage EC2
EC2
Load Balancer
EC2 instance
Auto Scaling Group

19. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Target Groups
/checkout
/catalog
/homepage EC2
EC2
Load Balancer
EC2 instance
Auto Scaling Group
Amazon
CloudWatch
RequestCount
(per Target Group)
Service Scheduler
TASK
TASK
TASK
TASK
TASK
TASK
TASK
TASK
TASK

20. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Target Groups
/checkout
/catalog
/homepage EC2
EC2
Load Balancer
EC2 instance
Auto Scaling Group
TASK
TASK
TASK
TASK
TASK
TASK
TASK
TASK
TASK
MemoryUtilization
(per ECS Cluster)
Amazon
CloudWatch

21. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Target Groups
/checkout
/catalog
/homepage EC2
EC2
Load Balancer
EC2 instance
Auto Scaling Group
TASK
TASK
TASK
TASK
TASK
TASK
TASK
TASK
TASK
MemoryUtilization
(per ECS Cluster)
Amazon
CloudWatch
Service Scheduler
EC2

22. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Service Scale-out
EC2 EC2
ECS
EC2 EC2 EC2
Cluster Scale-out
CloudWatch
Alarm
tasks
instances

23. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
23
Ресурсы

24. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
CPU / RAM в task definition
CPU
soft от 128 CPU units (.125 vCPU) до 10240 (10 vCPU)
RAM
hard и soft от 4 MB до максимума

25. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
25
Сеть

26. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
26
bridge
EC2 instance
80
80172.31.0.101:6000
172.31.0.101:5000
ENI
172.31.0.101
EC2 instance
80
80172.31.0.102:6000
172.31.0.102:5000
ENI
172.31.0.102
Load Balancer
183.0.0.101:80

27. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
27
awsvpc
EC2 instance
80
80172.31.0.111:80
172.31.0.101:80
ENI
172.31.0.101
Load Balancer
183.0.0.101:80
ENI
172.31.0.111
EC2 instance
80
80172.31.0.112:80
172.31.0.102:80
ENI
172.31.0.102
ENI
172.31.0.112
loopback
loopback

28. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
28

29. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
ECS
AMI
Docker
agent
ECS
agent
Scheduling and Orchestration
Cluster Manager Placement Engine
ECS
AMI
Docker
agent
ECS
agent
ECS
AMI
Docker
agent
ECS
agent

30. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Scheduling and Orchestration
Cluster Manager Placement Engine

31. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Task
ECS scheduling and orchestration
ECS
AMI
Docker
agent
ECS
agent
Cluster
manager
Placement
engine
Task
ECS
AMI
Docker
agent
ECS
agent

32. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
ECS
AMI
Docker
agent
ECS
agent
Cluster
manager
Placement
engine
ECS
AMI
Docker
agent
ECS
agent
Task Task
ECS scheduling and orchestration

33. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

34. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
34
Масштабирование

35. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Service Scale-out
ECS
CloudWatch
Alarm
tasks

36. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
36
Ресурсы

37. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
CPU / RAM
CPU RAM
256 (.25 vCPU) 512 MB, 1 GB, 2 GB
512 (.5 vCPU) 1 GB, 2 GB, 3 GB, 4 GB
1024 (1 vCPU) 2 GB, 3 GB, 4 GB, 5 GB, 6 GB, 7 GB, 8 GB
2048 (2 vCPU) между 4 GB b 16 GB c 1-GB инкрементом
4096 (4 vCPU) между 8 GB и 30 GB с 1-GB инкрементом

38. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
38
Сеть

39. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
39
awsvpc
EC2 instance
80
80172.31.0.111:80
172.31.0.101:80
ENI
172.31.0.101
Load Balancer
183.0.0.101:80
ENI
172.31.0.111
EC2 instance
80
80172.31.0.112:80
172.31.0.102:80
ENI
172.31.0.102
ENI
172.31.0.112
Fargate task
Fargate task
Fargate task
Fargate task

40. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
40

41. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
• нужно управлять EC2
instances
• управление проще

42. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
• минимально 512MB RAM
• гибкие конфигурации
ресурсов tasks
• фиксированные
конфигурации CPU/RAM

43. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
• только за ресурсы EC2,
остальное бесплатно
• по количеству vCPU и
RAM, использованных с
начала docker pull до
прекращения выполнения
task

44. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Разница в цене при
различном уровне
загрузки CPU/RAM
m5.xlarge:

45. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Разница в цене при
различном уровне
загрузки CPU/RAM
c5.2xlarge c 50%
резервированием:

46. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
• выгодно при высокой
средней утилизации
ресурсов
• выгодно при разовых
запусках

47. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
• есть лимиты на
количество task на
instance
• не нужно следить за
лимитами
• гибкие варианты
настройки сети
• только режим awsvpc
• не работает с CLB,
ALB Target Type = IP

48. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
üвысоко-
утилизированные
системы контейнеров
üпакетные задачи по
расписанию или
одноразово
üWindows или
persistent storage
üнагрузка с резкими
пиками CPU/RAM
üмикро-микро-сервисы
с RAM < 512MB
üв регионах, где еще
нет Fargate
üминимизация усилий
по развертыванию и
сопровождению

49. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
49

50. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
50
51%
https://www.cncf.io/blog/2018/08/29/cncf-survey-use-of-cloud-native-technologies-in-production-has-grown-over-200-percent/

51. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
“Дайте мне Kubernetes”
• что бы не переучиваться
• те же версии
• полностью совместимый“

52. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Master node MasternodeMaster node
Worker nodes Worker nodes Worker nodes

53. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Worker nodes Worker nodes Worker nodes
Etcd
Controller
Etcd
Controller
Etcd
Controller

54. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Worker nodes Worker nodes Worker nodes
Etcd
Controller
Etcd
Controller
Etcd
Controller
Workers управляется
вами
Master управляется
AWS
Availability Zone #1 Availability Zone #2 Availability Zone #3
Masters в HA
https://github.com/awslabs/amazon-eks-ami

55. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
mycluster.eks.amazonaws.comKubectl
Availability Zone #1 Availability Zone #2 Availability Zone #3

56. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
56
EC2
CNI
plugin
Dockerkubelet

57. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
ECS instances worker nodes≈

58. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
pod
pod
pod

59. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
EC2
pod pod
pod pod
pod template
YAML
• Image
• CPU
• RAM
• …
CNI
plugin
Dockerkubelet

60. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
EC2
pod pod
pod
EC2
pod pod
balancerReplicaSet
Deployment
Service
DaemonSet
StatefulSet
• Health check
• Auto-scaling
• Load
Balancing
• Service
discovery
• …
manifests
YAML

61. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
≈service
Replica Set
Deployment
Service
…

62. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
62
Масштабирование

63. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
K8s Horizontal Pod Autoscaler
EC2 EC2
EKS
EC2 EC2 EC2
K8s Cluster Autoscaler использует AWS Auto-Scaling Groups
pods
worker nodes

64. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
64
Сеть

65. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Nginx Pod
Java Pod
ENI
Secondary IPs:
10.0.0.1
10.0.0.2
Veth IP: 10.0.0.1
Veth IP: 10.0.0.2
Nginx Pod
Java Pod
ENI
Veth IP: 10.0.0.20
Veth IP: 10.0.0.22
Secondary IPs:
10.0.0.20
10.0.0.22
VPC Subnet – 10.0.0.0/24
Instance 1 Instance 2
https://github.com/aws/amazon-vpc-cni-k8s
ENI
ENI

66. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
66
Безопасность

67. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
https://github.com/projectcalico
Kubernetes Network
Policies контролирует
правила сетевой
безопасности
Calico API
сетевых политик
Open source
>100 контрибуторов
Коммерческая
поддержка от Tigera

68. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Open source IAM Authenticator
3) авторизация AWS Identity в RBAC
K8s API
1) послать AWS Identity
2) проверка AWS Identity
4) K8S action allowed/denied AWS Auth
Kubectl
https://github.com/heptiolabs/kubernetes-aws-authenticator

69. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
69

70. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
• очень простой • более гибкий
• быстро начать и легко
поддерживать
• развитая экосистема

71. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
• ECS - все регионы
• С. Вирджиния, Орегон,
Ирландия
• Fargate - 3 в Европе, 3 в
Америке, 3 в Азии

72. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
• CLB, ALB и NLB
• CLB и NLB через
Service type
LoadBalancer
• ALB Ingress в beta
https://github.com/kubernetes-sigs/aws-alb-ingress-controller
• менее эффективная
балансировка через
proxy, потребляющая
сетевые ресурсы
• балансировка напрямую

73. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
• режим awsvpc позволяет
выделять отдельные
ENI для каждого task
• ENI разделяется между
pods
• гибкое управление
доступом через
отдельные Security Groups
• управление доступом
через внешние плагины

74. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
• интеграция с AWS IAM
”из коробки”
• IAM только для
аутентификации

75. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
• 0,20$ в час = 144$ в
месяц
+ стоимость EC2
• ECS только за ресурсы
EC2, остальное
бесплатно
• Fargate по количеству
vCPU и RAM

76. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
üбыстро начать или
минимизировать
затраты на поддержку
üK8S уже используется
üнужна гибкость и есть
кому ее реализовать
üмаксимальная интеграция
с сервисами AWS
üмаксимальный open
source
üработа в регионах. где EKS
еще не поддерживается
üнужны последние
версии K8S

77. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
77
Спасибо !