ARC304_From One to Many Evolving VPC Design

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS re:INVENT
F ROM ONE TO MANY: E VOL VING VPC DE SIG N
A n d r o s k i S p i c e r : S o l u t i o n s A r c h i t e c t
A R C 3 0 4
N o v e m b e r 2 8 , 2 0 1 7

SIMPLICITY
A M A Z O N V P C A R C H I T E C T U R E

Subnet
I TRUST YOU’VE HEARD OF
Route Table
Elastic Network Interface
Amazon VPC
Internet Gateway
Virtual Private
Gateway
VPN Connection
Network ACL
Security group
Enhanced Networking
VPC Peering
AWS Direct Connect
Availability Zone
VPC Endpoints
Amazon VPC
Region

RELATED SESSIONS
NET201—Creating Your Virtual Data Center: VPC
Fundamentals and Connectivity Options
NET301—Extending Datacenters to the Cloud:
Connectivity Options and Considerations for Hybrid
Environments

ONE VPC ….
Subnet
Network ACL
Availability Zone - A
Region
Subnet
Network ACL
Availability Zone - B

us-east-2
VPC
VPC
VPC
NA
HQ
Chicago
DX
London
DX
ap-northeast-1
VPC
VPC
VPC
VPC
EU
HQ
us-west-2
VPC
VPC
VPC
eu-east-2
VPC
VPC
VPC
VPC
VPC
VPC
Tokyo
DX
Amazon Public
Services
GLOBALLY
PRIVATE VIF
PUBLIC VIF
Shared
Services
VPC
Shared
Services
VPC
Shared
Services
VPC
DX GATEWAY
T O
MANY
APAC
HQ
INTER REGION
VPC PEER
VPC PEER

VPC DESIGN
A R C 3 0 4

VPC DESIGN
/16
Choose a CIDR
• CIDR fixed on VPC
creation
• /16 down to /28
• Should You Go Big?
Region

VPC IPv4 SPACE DESIGN
• Don’t overlap IP space
• Consider connectivity to corporate networks
• Plan for expansion to additional Availability Zones or
regions
Subnet
Availability Zone A
IPv4
Region

VPC IPv6 SPACE DESIGN
• Optionally enable IPv6 on VPC
• /56 of Amazon’s Global Unicast Address (GUA) per VPC
• /64 CIDR block per subnet
• IPv6 completely independent from IPv4
• Enabled per subnet or per instance (per ENI)
• Supported by Security Groups, Route Tables, NACLs, VPC
Peering, IGW, DX, Flow Logs, and DNS Resolution
Subnet
Availability Zone A
IPv6IPv4
Region

VPC ID : abc-de-fg-7
Availability Zone A
Subnet
Secondary CIDR :
10.2.0.0/16
Secondary CIDR :
10.3.0.0/16
Secondary CIDR :
10.4.0.0/16
Secondary CIDR :
10.5.0.0/16
Primary CIDR :
10.1.0.0/28
VPC RESIZING

Secondary CIDR :
10.2.0.0/16
Secondary CIDR :
10.3.0.0/16
Secondary CIDR :
10.4.0.0/16
Secondary CIDR :
10.5.0.0/16
Primary CIDR :
10.1.0.0/28
Route Table
Main Route Table
Destination Target
10.1.0.0/28 Local
10.2.0.0/16 Local
10.3.0.0/16 Local
10.4.0.0/16 Local
10.5.0.0/16 Local
US-WEST-2
INTER CIDR
ROUTING

Secondary CIDR :
10.2.0.0/16
Secondary CIDR :
10.3.0.0/16
Secondary CIDR :
10.4.0.0/16
Secondary CIDR :
10.5.0.0/16
Primary CIDR :
10.1.0.0/28
Route Table
US-WEST-2
CONSIDER!
• CIDR Block/s cannot overlap
• Existing CIDR Blocks cannot
change
• CIDR block must not be the same
or larger than the CIDR range of a
route in any of the VPC route
tables
Primary CIDR
10.3.0.0/16
VPC
Peering

Secondary CIDR :
10.2.0.0/16
Secondary CIDR :
10.4.0.0/16
Primary CIDR :
10.1.0.0/28
Route Table
US-WEST-2
CONSIDER!
• Secondary CIDR Blocks can be
added and removed
• Primary CIDR Block cannot be
resized or removed
Cannot Change!

Secondary CIDR :
10.2.0.0/16
Secondary CIDR :
10.4.0.0/16
Primary CIDR :
10.1.0.0/28
Route Table
US-WEST-2
CONSIDER!
• Primary CIDR Range Dictates which
other RFC1918 Ranges can be used
• For example, if you use 10.0.0.0/8,
then your additional CIDRs must be
from the RFC1918 10. space
Secondary CIDR :
192.168.0.0/16
Secondary CIDR :
172.16.0.0/16

SUBNET CREATION
Availability Zone A
• Even distribution of IP
space across AZs
• Use at least 2 AZs
• Subnets are AZ specific
• How big? How many?
Subnet
Availability Zone B
Subnet
Availability Zone C
Subnet
/16

SUBNET CREATION
Availability Zone A
Subnet
Subnet
Subnet
Subnet
Subnet
Subnet
Subnet
Subnet
Subnet
Subnet
Subnet
Subnet Subnet Subnet Subnet Subnet Subnet
Subnet Subnet Subnet Subnet Subnet
Subnet Subnet Subnet Subnet
Subnet
Subnet
Subnet
Subnet
Subnet
Subnet
Subnet
Subnet
Subnet
Subnet
Subnet
Subnet
Subnet
Subnet
Subnet
Subnet
/16

VPC SUBNET DESIGN
• T r a d i t i o n a l s w i t c h i n g l i m i t a t i o n s d o n o t a p p l y
• C o n s i d e r l a r g e , m i x e d - u s e s u b n e t s
• U s e s e c u r i t y g r o u p s t o e n f o r c e i s o l a t i o n
• U s e t a g s f o r g r o u p i n g r e s o u r c e s
• U s e s u b n e t s a s c o n t a i n e r s f o r r o u t i n g p o l i c y

IPv4 VPC SUBNET DESIGN
/16
Availability Zone A
Hybrid Subnet
Public subnet
Private subnet
Availability Zone B
Public subnet
Private subnet
Availability Zone C
Public subnet
Private subnet
/22 /22 /22
/20
/20
/20
/20
/20
/20
4091 IPs
1019 IPs
4091 IPs
Hybrid Subnet Hybrid Subnet

WHAT ABOUT IPV6 DESIGN
Availability Zone A
Public subnet
Private subnet
Availability Zone B
Public subnet
Private subnet
Availability Zone C
Public subnet
Private subnet
/64
/54
/64 /64
/64 /64 /64
18 QUINTILLION
18 QUINTILLION
18 SEXTILLION

LOCAL ROUTING POLICY
Availability Zone A
Hybrid subnet
Public subnet
Private subnet
.1
.1
.1
Main Route Table
Destination Target
10.1.0.0/16 Local
10.2.0.0/16 Local
Primary VPC CIDR 10.1.0.0/16
Availability Zone B
Hybrid subnet
Public subnet
Private subnet
.1
.1
.1
Availability Zone B
Public subnet
Private subnet
.1
.1
SECONDARY CIDR 10.2.0.0/16
US-WEST-2

Availability Zone A
Hybrid subnet
Public subnet
Private subnet
.1
.1
.1
Main Route Table
Destination Target
10.1.0.0/16 Local
10.2.0.0/16 Local
0.0.0.0/0 igw-a1234567
Availability Zone B
Hybrid subnet
Public subnet
Private subnet
.1
.1
.1
Availability Zone B
Public subnet
Private subnet
.1
.1
US-WEST-2
INTERNET
AMAZON
PUBLIC SERVICES
INTERNET GATEWAY

Availability Zone A
Private subnet
Public subnet
Private subnet
.1
.1
.1
VPC CIDR 2001:db8:1234:1a00::/56
Availability Zone B
Private subnet
Public subnet
Private subnet
.1
.1
.1
US-WEST-2
INTERNET
AMAZON
PUBLIC SERVICESPublic Route Table
Destination Target
10.1.0.0/16 Local
2001:db8:1234:1a00::/56 Local
0.0.0.0/0 IGW
::/0 igw-a1234567
INTERNET GATEWAY Global Unicast Address (GUA)

Availability Zone A
Private subnet
Public subnet
Private subnet
.1
.1
.1
VPC CIDR 2001:db8:1234:1a00::/56
Availability Zone B
Private subnet
Public subnet
Private subnet
.1
.1
.1
US-WEST-2
INTERNET
AMAZON
PUBLIC SERVICES
Public Route Table
Destination Target
10.1.0.0/16 Local
2001:db8:1234:1a00::/56 Local
0.0.0.0/0 IGW
::/0 eigw-0ab0
INTERNET GATEWAY
Global Unicast Address (GUA)
EGRESS ONLY

Availability Zone A
Hybrid subnet
Public subnet
Private subnet
.1
.1
.1
Main Route Table
Destination Target
10.1.0.0/16 Local
10.2.0.0/16 Local
172.16.0.0/16 vgw-a1234567
Availability Zone B
Hybrid subnet
Public subnet
Private subnet
.1
.1
.1
Availability Zone B
Public subnet
Private subnet
.1
.1
US-WEST-2
NORTHER VIRGINIA
CORPORATE
DATA CENTER
REMOTE OFFICE BUILDING
VIRTUAL GATEWAY (VGW)
172.16.0.0/16
AMAZON
DIRECT CONNECT

ROUTING IN THE HYBRID SUBNET
Availability Zone A
Public subnet
Private Route Table
Destination Target
10.1.0.0/16 Local
0.0.0.0/0 ???
Corp CIDR VGW
Private subnet
CORPORATE DATA CENTER
Availability Zone A
Public subnet
Private subnet
Hybrid subnetHybrid subnet

Availability Zone A
Public subnet
Private Route Table
Destination Target
10.1.0.0/16 Local
0.0.0.0/0 NAT instance
Corp CIDR VGW
Private subnet
Availability Zone A
Public subnet
Private subnet
INTERNET
GATEWAY
INTERNET
NAT INSTANCE
Hybrid subnet Hybrid subnet

Availability Zone A
Public subnet
Private Route Table
Destination Target
10.1.0.0/16 Local
0.0.0.0/0 NAT instance
Corp CIDR VGW
Private subnet
Availability Zone A
Public subnet
Private subnet
INTERNET
GATEWAY
INTERNET
NAT INSTANCE
NAT INSTANCE

DESIGNING SCALABLE NAT
A R C 3 0 4

EVOLVING DESIGN REQUIREMENTS
• Public subnets for resources reachable from Internet
• Hybrid subnets with egress-only access to public network
• Scalable, highly available NAT
• One AWS account
• One VPC
• One region

DEPLOY A NAT GATEWAY
Availability Zone A
Public subnet
Private Route Table
Destination Target
10.1.0.0/16 Local
0.0.0.0/0 NAT-GW-1
Corp CIDR VGW
Private subnet
CORPORATE
DATA CENTER
Availability Zone A
Public subnet
Private subnet
INTERNET
GATEWAY
INTERNET
NAT GATEWAY
• Still need IGW
• Separate subnets
• Requires EIP
• AZ specific
• Burst to 10 Gbps

Hybrid subnet
ROUTING IN THE PRIVATE SUBNET
Availability Zone A
Hybrid subnet
Public subnet
Private Route Table
Destination Target
10.1.0.0/16 Local
0.0.0.0/0 NAT-GW-2
Corp CIDR VGW
Private subnet
Availability Zone A
Public subnet
Private subnet
INTERNET
GATEWAY
INTERNET
NAT GATEWAY
NAT GATEWAY

ROUTING IN THE PRIVATE SUBNET
Availability Zone A
Public subnet
Private Route Table
Destination Target
10.1.0.0/16 Local
0.0.0.0/0 NAT-GW-2
Corp CIDR VGW
Private subnet
CORPORTAE DATA CENTER
Availability Zone A
Public subnet
Private subnet
INTERNET
GATEWAY
INTERNET
NAT GATEWAY
NAT GATEWAY

NAT GATEWAY: SECURING ACCESS
1
NAT Gateway ENI:
Network ACL
Public subnet
NAT
Gateway
Amazon Network ACLs still apply

Use routing policy to
control access to NAT
Gateway
Private subnet
Public subnet
Private subnet
NAT Enabled
no-NAT
no-NAT Private Route Table
Destination Target
10.1.0.0/16 Local
NAT Enabled Route Table
Destination Target
10.1.0.0/16 Local
0.0.0.0/0 NAT Gateway
NAT
Gateway
2

Outbound Rules
Type Protocol Port Range Destination
All traffic All 0 - 65535 0.0.0.0/0
Use security groups to restrict
outbound access for instances
Default VPC security group:
3

Outbound Rules
All traffic All 0 - 65535 10.2.0.0/16
Outbound Rules
All traffic All 0 - 65535 0.0.0.0/0
Use security groups to restrict
outbound access for instances
Default VPC security group:
NAT Enabled VPC security group:
3

NAT GATEWAY PROs & CONs
• Drop in replacement for NAT instance
• Fully managed
• Highly available and fault tolerant
• Scalable to 10 Gbps bursts per gateway
• Supports VPC Flow Logs
• No higher level functions like IPS, UTM, URL
Filtering, packet inspection, etc.
• Cannot associate security group to gateway

TOO MANY APPS INTERACT WITH
WHAT ABOUT INGRESS CONTROL
Customer
network
Amazon S3 Amazon DynamoDB Amazon Kinesis

INGRESS CONTROL
A R C 3 0 4

AMAZON ELASTIC LOAD BALANCING
Elastic Load Balancing supports three types of load balancers:
Application Load Balancer
Network Load Balancer
Classic Load Balancer
Layer 7 HTTP/HTTPS
WS/WSS
Layer 4 TCP
Layer 4 TCP , SSL,
HTTP,HTTPS

AMAZON NETWORK LOAD BALANCER (NLB)
Network Load Balancer
• Handle volatile traffic patterns
• Designed to load balance millions of requests per second
• Offers extremely low latencies for latency-sensitive applications
• Preserve Source IP address
• Static IP support
• Elastic IP support
• Supports Long Lived TCP Sessions
• Zonality

Target Group A
Availability Zone A
PRIVATE SUBNET
PUBLIC SUBNET
Availability Zone B
PRIVATE SUBNET
PUBLIC SUBNET
55.70.200.20 57.10.40.20
Target Group A
Instance ID Instance ID Instance ID Instance ID
IP Address IP Address IP Address IP Address
EIPEIP

Target Group A
Availability Zone A
PRIVATE SUBNET
PUBLIC SUBNET
Availability Zone B
PRIVATE SUBNET
PUBLIC SUBNET
55.70.200.20 57.10.40.20
Target Group B
Target Group
Instance ID Instance ID IP Address IP Address
Listener
Rule
TCP
443

Target Group A
Availability Zone A
PRIVATE SUBNET
PUBLIC SUBNET
Availability Zone B
PRIVATE SUBNET
PUBLIC SUBNET
55.70.200.20 57.10.40.20
Target Group A
Task Task
Target Groups are not
limited to Amazon EC2
OR
Your VPC/s
TaskTask

Target Group A
Availability Zone A
PRIVATE SUBNET
PUBLIC SUBNET
Availability Zone B
PRIVATE SUBNET
PUBLIC SUBNET
55.70.200.20 57.10.40.20
Target Group A
10.1.1.45 10.1.1.47
Customer
network
10.20.2.150
10.20.2.14
10.20.2.15
Target Group A
10.1.1.45 10.1.1.45

PRIVATE SUBNET
PUBLIC SUBNET
PRIVATE SUBNET
PUBLIC SUBNET
55.70.200.20 57.10.40.20
Target
Group
10.1.1.45 10.1.1.47 10.1.2.45 10.1.2.45
Route Table
Destination Target
10.1.0.0/16 Local
0.0.0.0 eni-7a795854
Amazon NLB
SQUID SQUID SQUID SQUID
10.1.3.0/24
HYBRID SUBNET
Internal
Apps
Internal
Apps
Internal
Apps
Internal
Apps
Internal
Apps
Internal
AppsSource Destination Check
Disabled by Default
IGW
INTERNET

EGRESS CONTROL
A R C 3 0 4

• VPN connectivity to private-only VPC
• No egress in the VPC to public networks
• Private IP access to Amazon S3
• Content-specific access controls
• One AWS account
• One VPC
• One region

YOU REALLY DON’T WANT TO DO THIS
Private subnet Private subnet
Virtual Private
Gateway
VPN
connection
Intranet
app
Intranet
app
Amazon
S3
Internet
Customer
border router
Customer VPN
Internet
Customer
network
Amazon
DynamoDB
Availability Zone A Availability Zone B
Amazon Kinesis

AWS Oregon (us-west-2) Region
CIDR : 10.0.0.0/16
Private Subnet: 10.0.1.0/24
VPC Endpoint Network Interface
Private IP : 10.0.1.7
EC2
Private IP 10.0.1.12
Private IP :10.0.2.120 VPC Endpoint Network Interface
Amazon S3Amazon DynamoDB
VPN
connection
Customer
network
Amazon Kinesis

AMAZON VPC ENDPOINTS
Interface Gateway
Virtual Devices Horizontally Scaled Redundant & Highly Available
VPC Components that Provide
Access to AWS Services
N E W

GATEWAY VPC ENDPOINT

GATEWAY ENDPOINTS
Private subnet
Public subnet
NAT
Gateway
Amazon
S3
Amazon
DynamoDB
Route Table
Destination Target
10.1.0.0/16 Local
0.0.0.0 NAT Gateway

GATEWAY ENDPOINTS
Target for a specified route in your route table
Route Table
Destination Target
10.1.0.0/16 Local
Corp CIDR VGW
Prefix List for S3 us-west-2 VPC-Endpoint
Prefix List for DynamoDB-
us-west-2
VPC-Endpoint
Amazon S3
Amazon DynamoDB
EC2 EC2 EC2

GATEWAY ENDPOINTS
Target for a specified route in your route table
Route Table
Destination Target
10.1.0.0/16 Local
Corp CIDR VGW
Prefix List for S3 us-west-2 VPC-Endpoint
Prefix List for DynamoDB-
us-west-2
VPC-Endpoint
Amazon S3
Amazon DynamoDB
aws ec2 create-vpc-endpoint
--vpc-id vpc-40f18d25
--service-name com.amazonaws.us-west-2.s3
--route-table-ids rtb-2ae6a24f rtb-61c78704
--vpc-id vpc-40f18d25
--service-name com.amazonaws.us-west-2.dynamodb
aws ec2 describe-vpc-endpoint-services
{ "ServiceNames": [
"com.amazonaws.us-east-1.s3",
"com.amazonaws.us-east-1.dynamodb"
] }

GATEWAY ENDPOINTS
Private subnet
Route Table
Destination Target
10.1.0.0/16 Local
0.0.0.0 NAT Gateway
Prefix List for S3 us-west-2 VPCE
Public subnet
NAT
Gateway
Amazon
S3
Amazon
DynamoDB
aws ec2 describe-prefix-lists
PREFIXLISTS pl-68a54001 com.amazonaws.us-west-2.s3
CIDRS 54.231.160.0/19
CIDRS 52.218.128.0/18

GATEWAY ENDPOINT SECURITY
… and use them in your outbound security group rules!

CONTROLLING ACCESS S3 VIA IAM POLICY
Private subnet
AWS Identity & Access
Management (IAM) policy on VPCE:
{
"Statement": [
{
"Sid": "vpce-restrict-to-backup-bucket",
"Principal": "*",
"Action": [
"s3:GetObject",
"s3:PutObject”
],
"Effect": "Allow",
"Resource": ["arn:aws:s3:::backups-reinvent",
"arn:aws:s3:::backups-reinvent/*"]
}
]
}
Backups bucket?

{
"Statement": [
{
"Sid": "bucket-restrict-to-specific-vpce",
"Principal": "*",
"Action": "s3:*",
"Effect": "Deny",
"Resource": ["arn:aws:s3:::backups-reinvent",
"arn:aws:s3:::backups-reinvent/*"],
"Condition": {
"StringNotEquals": {
"aws:sourceVpce": "vpce-bc42a4e5”
}
}
}
]
}
CONTROLLING ACCESS VIA VPC ENDPOINT
Private subnet
S3 bucket policy:
From
vpce-bc42a4e5?

CONTROLLING ACCESS DYNAMODB
Private subnet
AWS Identity & Access Management (IAM) policy on VPCE:
Accessing DynamoDB Through
a Specific VPCE Access
{ "Version": "2012-10-17",
"Statement": [
{ "Sid": "AccessFromSpecificEndpoint",
"Action": "dynamodb:*",
"Effect": "Deny",
"Resource": "arn:aws:dynamodb:region:account-id:table/*",
"Condition":
{ "StringNotEquals" : { "aws:sourceVpce": "vpce-11aa22bb" } }
}
]
}
VPCE : vpce-11aa22bb ?

{ "Statement": [ {
"Sid": "AccessToSpecificTable",
"Principal": "*",
"Action": [
"dynamodb:Batch*",
"dynamodb:Delete*",
"dynamodb:DescribeTable",
"dynamodb:GetItem",
"dynamodb:PutItem",
"dynamodb:Update*"
],
"Effect": "Allow",
"Resource": "arn:aws:dynamodb:us-east1:123456789012:table/StockTable"
}
]
}
CONTROLLING ACCESS DYNAMODB
Private subnet
VPCE Policy Granting Access to a Specific DDB Table
Table : StockTable ?

CONTROLLING VPC ACCESS TO S3
Recap on security layers:
1. Route table association
2. VPCE policy
3. Bucket policy
4. Security groups with prefix list
Private subnet
1.
2.
3.
4.

INTERFACE VPC ENDPOINT
N E W
P O W E R E D B Y A M A Z O N P R I V A T E L I N K S

INTERFACE VPC ENDPOINTS
A N E L A S T I C N E T W O R K I N T E R F A C E W I T H A
P R I V A T E I P A D D R E S S T H A T S E R V E S A S A N E N T R Y
P O I N T F O R T R A F F I C D E S T I N E D T O A S U P P O R T E D
A W S S E R V I C E
AWS Public Services
Amazon EC2 (API) & EC2 SSM
Amazon Elastic Load Balancing
Amazon Kinesis
Amazon Service Catalog
1 0 . 1 . 1 0 . 5 0

HOW IT WORKS AWS Public Services
Amazon EC2 (API) & EC2 SSM
Amazon Kinesis
Amazon Service Catalog
1 0 . 1 . 1 0 . 5 0
S U B N E T - 1 0 . 1 . 1 0 . 4 5
E C 2 F L E E T
H O T I N G A P P L I C A T I O N
Availability Zone A
M A K E S A R E Q U E S T
T O E L B E N D P O I N T
N A M E
k i n e s i s . u s - e a s t -
1 . a m a z o n a w s . c o m
R E S O L V E S T O T H E
P R I V A T E I P O F T H E E N I
Amazon Elastic Load Balancing

I NTERFACE V PC ENDPOI NTS
NO ROUTES IN YOUR ROUTE TABLE
NO IAM POLICY FOR ENDPOINT
NOT ACCESSIBLE VIA (VGW) VPN
ONE SUBNET PER AZ PER I-ENDPOINT
SUPPORTS TCP ONLY
--vpc-id vpc-ec43eb89
--vpc-endpoint-type Interface
--service-name com.amazonaws.us-east-1.elasticloadbalancing
--subnet-id subnet-abababab subnet-catbatratsat
--security-group-id sg-1a2b3c4d

C R E A T I N G
I N T E R F A C E V P C
E N D P O I N T S
--vpc-id vpc-ec43eb89
--vpc-endpoint-type Interface
--service-name com.amazonaws.us-east-1.elasticloadbalancing
--subnet-id subnet-abababab subnet-catbatratsat
--security-group-id sg-1a2b3c4d
S u b n e t
Availability Zone A
S u b n e t
Availability Zone B
S u b n e t
Availability Zone C
v p c - i d v p c - e c 4 3 e b 8 9
Amazon EC2 (API) & EC2 SSMAmazon Kinesis AWS Service CatalogElastic Load Balancing
aws ec2 describe-vpc-endpoints

ACCESS
T HR OUGH
I NT ER FAC E V PC
ENDPOI NTS
I F C R E A T E D I N O R E G O N ,
W H I C H H A S T H R E E A Z S
ENDPOINT-SPECIFIC REGIONAL DNS HOSTNAME
vpce-0fe5b17a0707d6abc-29p5708s.kinesis.us-west-2.vpce.amazonaws.com
ENDPOINT-SPECIFIC ZONAL DNS HOSTNAME
vpce-0fe5b17a0707d6abc-29p5708s-us-west-2a.kinesis.us-west-2.vpce.amazonaws.com
vpce-0fe5b17a0707d6abc-29p5708s-us-west-2b.kinesis.us-west-2.vpce.amazonaws.com
vpce-0fe5b17a0707d6abc-29p5708s-us-west-2c.kinesis.us-west-2.vpce.amazonaws.com
DEFAULT PUBLIC DNS HOSTNAME
kinesis.us-west-2.amazonaws.com
PRIVATE IP ADDRESS OF THE ENDPOINT NETWORK INTERFACE
10.1.10.50 10.1.20.50 10.1.30.50
SUBMIT REQUESTS TO THE SUPPORTED SERVICE
VIA AN ENDPOINT URL

AWS Oregon (us-west-2) Region
CIDR : 10.0.0.0/16
Amazon Kinesis
EC2
Private IP :
10.0.1.12
10.0.2.7
Private IP :
10.0.2.120
VPCE-
2222.KINESIS.AMAZON.COM
PRIVATE CONNECTION OVER
AWS NETWORK
CONSIDER
VPCE-
2222.KINESIS.AMAZON.COM
CUSTOMER
NETWORK
CONNECTING ENDPOINTS IN ANOTHER REGION
CONNECTING TO ENDPOINTS ACROSS A VPN
SERVICE PROVIDER TRAFFIC ORIGINATION
ADVERTISING WITH CUSTOMER DNS NAME

S E C U R I N G A C C E S S T O A M A Z O N I N T E R F A C E V P C E N D P O I N T S
S u b n e t 1 0 . 0 . 1 . 0 / 2 4
Availability Zone A
S u b n e t : 1 0 . 0 . 2 . 0 / 2 4
Availability Zone B
S u b n e t : 1 0 . 0 . 3 . 0 / 2 4
Availability Zone C
S e c u r i t y G r o u p
V P C C I D R : 1 0 . 0 . 0 . 0 / 1 6

USE CASES FOR INTERFACE VPC ENDPOINTS
• Endpoint consumers can establish private connectivity to Amazon services
• Customers can share internal services between VPCs, both within a single AWS
account and between AWS accounts
• Partners can deliver services to their customers’ VPCs, or on-premises
networks via Direct Connect

A. YOU ARE UNABLE TO CONNECT TO ENDPOINTS IN ANOTHER REGION
B. ENDPOINTS CANNOT BE ACCESSED ACROSS A VPN THAT USES AMAZON VGW
C. TRAFFIC CANNOT BE ORIGINATED BY SERVICE PROVIDERS
D. TCP TRAFFIC ONLY
THINGS TO NOTE

WHERE WE ARE Public-facing web
apps
Internal-only apps What’s next?
VPN
connection
Customer
network
Customer Gateway (CGW)

ONE VPC
A R C 3 0 4
TWO VPC THREE
VPC
MORE!

SO WHY NOT ONE BIG VPC?
Subnet
Availability Zone A
IPv4
Subnet
Availability Zone B
IPv4
Subnet
Availability Zone C
IPv4
Subnet
Availability Zone A
IPv4
Subnet
Availability Zone B
IPv4
Subnet
Availability Zone C
IPv4

PROD DEV
LOGGING &
MONITORING
CONSIDERATION FOR 1 OR MANY VPCs

PCI APPS HIIPAA
NON-REGULATED
APPS

LEGAL MARKETING SALES

PROD DR

LEGAL
SALES
FINANCE
App, S3 Access, ELB,
VPC Flow Logs, etc…
AWS
CloudTrail
AWS
Config
VPC
Flow Logs
Audit Logging &Analytics VPC
Amazon
RedshiftAmazon EMR
S3
Data Lake
VPC Endpoints
CONSIDERATIONS…
ACCOUNT #1111111
ACCOUNT #2222222
ACCOUNT #7777777
ACCOUNT #00000001

SHARED SERVICES VPC DESIGN
A R C 3 0 4
VPC
VPC
VPC
VPC
VPC

DESIGN
Region
VPCVPC
Customer
network
Public
apps
Internal
apps

HA VPN
TO
VPC
VPC
HA VPN Pair
Availability Zone A
iBGP
eBGP
Customer CIDRs or Default Route
eBGP
AWS ASN 7224
Re-advertise VPC CIDR via IGP
VGW
VPC CIDR
Customer ASN (Public or Private)
CGW1 CGW2
VPN1
Tun1
VPN1
Tun2
Availability Zone A
VPN2
Tun1
VPN2
Tun2
Reuse your CGW Public IP
to connect to more VPCs
Customer
network
MED
MED
REGION

VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
• DNS
• Directory
• Logging
• Monitoring
• Security
Shared services
Customer
network
REGION

• Centralize network connectivity to and from cloud
• Centralize management, security, and common services
• Account owners in control of own VPC resources
• Many AWS accounts
• Many VPCs
• One region

HUB & SPOKE
WITH PEERING
VPC
• DNS
• Directory
• Logging
• Monitoring
• SecurityShared services
VPC
Shared
services
VPC
VPC
Customer
network
Spoke VPC
Spoke VPC
Spoke VPCSpoke VPC
Spoke VPC
Spoke VPC
REGION
VPC
VPC
VPC
VPC
VPC
VPC

VPC PEERING
Customer
network
VPC
Hub VPC
Private subnet
VPC
Spoke VPC
Public subnet
10.2.0.0/1610.1.0.0/16
Private subnet
Shared services
10.2.22.0/24
10.1.11.0/24
REGION
Private Route Table
Destination Target
10.1.0.0/16 Local
10.2.0.0/16 PCX-1
Private Route Table
Destination Target
10.2.0.0/16 Local
10.1.11.0/24 PCX-1

EDGE-TO-EDGE ROUTING
VPC
Hub VPC
Private subnet
VPC
Spoke VPC
Public subnet
10.2.0.0/1610.1.0.0/16
Private subnet
Shared services
10.2.22.0/24
10.1.11.0/24
172.16.0.0/16
Customer
network
Private Route Table
Destination Target
10.1.0.0/16 Local
10.2.0.0/16 PCX-1
Private Route Table
Destination Target
10.2.0.0/16 Local
10.1.11.0/24 PCX-1
172.16.0.0/16 PCX-1
REGION

EDGE-TO-EDGE
ROUTING VIA
PROXY
VPC
Hub VPC
Private subnet
VPC
Spoke VPC
Proxy
subnets
10.2.0.0/1610.1.0.0/16
PCX-1 10.2.22.0/24
Internal
Network
Load
Balancer
Proxy
fleet
Internet
Public
services
S3
VPC
Customer
network
Proxy Route Table
Destination Target
10.1.0.0/16 local
10.2.0.0/16 PCX-1
172.16.0.0/16 VGW
Private Route Table
Destination Target
10.2.0.0/16 Local
10.1.0.0/16 PCX-1
Proxy Route Table
Destination Target
10.1.0.0/16 Local
10.2.0.0/16 PCX-1
172.16.0.0/16 VGW
0.0.0.0/0 IGW
S3 Prefix List VPCE

PROXY IN PRACTICE
Customer network
Availability Zone A
Private subnet
Public subnet
Private subnet
Network
Load
Balancer
Shared
services
Internet
VPC
Auto Scaling
proxy
fleet
Public
servicesS3
PCX-1
Availability Zone B
Private subnet
Public subnet
Private subnet
Network
Load
Balancer
Shared
services
Auto Scaling
proxy
fleet
Spoke VPC
VPC
Private subnet
Hub VPC

Availability Zone A
Private subnet
Public subnet
Private subnet
Network
Load
Balancer
Shared
services
VPC
Auto Scaling
proxy
Fleet
Public
servicesS3
PCX-1
Availability Zone B
Private subnet
Public subnet
Private subnet
Network
Load
Balancer
Bastion
host
Auto Scaling
proxy
fleet
Spoke VPC
VPC
Private subnet
Hub VPC
Customer
network
PROXY IN PRACTICE

SHARED SERVICES HUB: TO DO LIST
• Use IAM to restrict spoke AWS accounts from altering network
• Create a NetOps IAM role in all accounts:
https://aws.amazon.com/blogs/security/how-to-assign-permissions-using-new-aws-
managed-policies-for-job-functions/
• Enable AWS CloudTrail, AWS Config, and VPC Flow Logs for all
accounts
• Integrate CloudTrail with CloudWatch Logs and create alarms:
https://aws.amazon.com/blogs/aws/cloudtrail-integration-with-cloudwatch-now-
available-in-four-more-regions

PRO & CON: SHARED SERVICES HUB & SPOKE
• Minimizes on-premises network change
• Reduces latency, cost of cloud applications
accessing common services
• Provides spoke accounts control over own
resources
• But controls and secures egress traffic from
spokes
• Security groups work across peers
• Cost and management of central proxy
layer
• Not a transparent proxy
• Configuring end devices to use proxy
• Restricted to HTTP/S
• No transitive networking
• Peering data transfer cost

Customer
network
Availability Zone A
Private subnet
VPC
Availability Zone B
Private subnet
AWS Lambda
Amazon API Gateway
Elastic
Network
Interface
VPCVPC
VPC
VPC
Prod hub
VPC
Internet
Hybrid Serverless
Amazon
Aurora Replica
Mobile Application VPC

Legacy
Apps
Customer
network
Availability Zone A
Private subnet
VPC
Availability Zone B
Private subnet
AWS Lambda
Amazon API Gateway
Elastic
Network
Interface
VPCVPC
VPC
VPC
Prod hub
VPC
Internet
Amazon
Aurora
Replica
Mobile Application VPC
Hybrid Serverless

BRINGING IT ALL BACK
A R C 3 0 4

us-east-2
VPC
VPC
VPC
VPC
Transit VPC
VPC
us-west-2
VPC
VPC
VPC
eu-west-1
VPC
VPC
VPC
VPC
Transit VPC
VPC
AWS Network
Backbone
Provider
MPLS
Network
Branch Branch
NA
HQ
VPC
VPC
VPC
VPC
VPC
VPC
Chicago DX
AP
HQ
London DX
ap-northeast-1
VPC
VPC
VPC
VPC
Transit VPC
VPC
EU
HQ
Tokyo DX
You could
do this

us-east-2
VPC
VPC
VPC
NA
HQ
Chicago
DX
London
DX
ap-northeast-1
VPC
VPC
VPC
VPC
EU
HQ
us-west-2
VPC
VPC
VPC
eu-east-2
VPC
VPC
VPC
VPC
VPC
VPC
Tokyo
DX
Amazon Public
Services
GLOBALLY
PRIVATE VIF
PUBLIC VIF
Shared
Services
VPC
Shared
Services
VPC
Shared
Services
VPC
DX GATEWAY
T O
MANY
APAC
HQ
INTER-REGION
VPC PEER
VPC PEER

AMAZON DIRECT CONNECT GATEWAY
N E W

AMAZON DIRECT CONNECT GATEWAY
• # of DX Gateways (Global) : Default: 200
• # of VIF attachments per DX Gateway : Default: 30
• # of VGW associations per DX Gateway : Default: 10
LIMITS
• “Global” Object
• Logical grouping of VGW/VPC attachments and private
virtual Interfaces
• VGWs and VIFs can be in any region
• Provides connectivity between each VIF and all attached
VPCs

VPC
VPC
VPC
VPC
Private VIFs &
AWS Direct
Connect
(DX)
Customer
network
AWS Direct Connect
location
Private virtual interface (VIF) to VGW
• 1 PVI per VGW
• 1 BGP ASN
• 1 802.1Q VLAN Tag
• 1 BGP MD5 key
Private fiber connection
One or multiple
50-500 Mbps,
1 Gbps or 10 Gbps pipes
Region
PRIVATE VIF
BGP SESSION WITH EACH VGW

VPC
US-EAST-1
DX Connection
VPC
VPC
PRIVATEVIF
VPC
DX Location in Oregon
US-EAST-1
DX Gateway
VPC
VPC
PRIVATE VIF
VPC
EU-WEST-1
VPC
EU-WEST-1
• 1 PVIF Configuration needed to reach multiple VPCs
• No limitation on PVIF Creation
• BGP Session between customer network and GW
• 1 PVIF per VPC VGW
• PVIF limited by Bandwidth
• BGP Session between VGW & Customer Network
Associated

Amazon
Direct Connect Gateway
MED
VPC
ROUTES
AS_PATH_PREPEND
ONLY
CORP NETWORK ROUTES
172.16.0.0/16
PRIVATE VIF
ALL VPC ROUTES
CORPORATE
NETWORK
Main Route Table
Destination Target
10.30.0.0/16 Local
172.16.0.0/16 Local
10.30.0.0/1610.20.0.0/1610.10.0.0/16
Main Route Table
Destination Target
10.10.0.0/16 Local
172.16.0.0/16 Local
Main Route Table
Destination Target
10.20.0.0/16 Local
172.16.0.0/16 Local
10.10.0.0/16
10.20.0.0/16
10.30.0.0/16
172.16.0.0/16
10.10.0.0/16

VPC
Private Route Table
Destination Target
10.1.0.0/16 Local
172.160.0/16 VGW
create-direct-connect-gateway
--direct-connect-gateway-name abcd-GW
--amazon-side-asn 65555
STEP 1.
Create a Direct Connect Gateway
STEP 2.
Associate VPCs with Your Direct Connect GW
STEP 3.
Build Private VIFs to Your Gateway
associate
Name: My-DX-GW
BGP ASN: 65500
Private VIF
Provider
MPLS Network
Corporate HQ
172.160.0/16
10.1.0.0/16
create-direct-connect-gateway-
association
--direct-connect-gateway-id abcd1234-d
--virtual-gateway-id vgw-1234567
BGP ASN:
65505

VPC
VPC
VPC
VPC
LONDON HODX LOCATION
LONDON
One or multiple 50–500 Mbps,
US-EAST-1
AP-SOUTH-1
VPC
EU-WEST-1
VPC
VPC
NORTHERN VA,
USA DX LOCATION
NORTHERN VA
PRIVATE VIF
PRIVATE VIF

LONDON HO
VPC
VPC
VPC
VPC
VPC
LONDON HODX LOCATION
LONDON
One or multiple 50–500 Mbps,
US-EAST-1
DX Gateway
AP-SOUTH-1
VPC
EU-WEST-1
VPC
VPC
NORTHERN VA,
USA DX LOCATION
NORTHERN VA
PRIVATE VIF
PRIVATE VIF

VPC
US-WEST-2 / OREGON
VPC
VPC
EU-WEST-1
PRIVATE VIF
Oregon DX Location
DX Gateway
VPCE-
2222.FOO.AMAZON.COM
EC2 EC2 EC2
VPC
VPC
ACCOUNT-B
ACCOUNT-A
Customer
network
ACCOUNT-A
ACCOUNT-A
North American Office
US-WEST-2 / OREGON
INTER-REGION VPC PEERING
CONNECTION

VPC
US-WEST-2 / OREGON
VPC
VPC
EU-WEST-1
PRIVATE VIF
DX Gateway
VPCE-
2222.FOO.AMAZON.COM
EC2 EC2 EC2
VPC
VPC
ACCOUNT-B
ACCOUNT-A
Customer
network
ACCOUNT-A
ACCOUNT-A
North American Office
US-WEST-2 / OREGON
INTER-REGION VPC PEERING
CONNECTION
PRIVATE VIF
ACCOUNT-B
Oregon DX Location

• Leverages Amazon Global Network Backbone
• Multiple VIF attachments to a gateway
• Multiple VGW/VPC attachments to a gateway
• VIFs and VGWs can be in any region
• Single account at launch
• VIF, DX Gateway, and VGW must have same account
• VPC CIDRs cannot overlap
• A VGW can only be associated to a single DX Gateway
• CloudHub is not supported
• VPN Failover is supported
Pro & Con: Direct Connect GW

GREAT! BUT I ONLY HAVE VPNs
BETWEEN MY DC & AWS
W H A T A R E M Y O P T I O N S F O R S I M P L I F I C A T I O N ?

VPC MASS TRANSIT
A R C 3 0 4

DESIGN REQUIREMENTS
• Centralize and minimize network connections
• Allow end-to-end routing from cloud to existing networks
• Minimal operational overhead
• Leverage AWS network
• Many AWS accounts
• Many VPCs
• Many regions

Availability Zone A
Public subnet
VPC
Availability Zone B
Public subnet
Region
EC2 VPN EC2 VPN
TRANSIT VPC

Availability Zone A
Public subnet
VPC
Availability Zone B
Public subnet
Region
VPC Spoke VPC
Transit VPC
VPC
Spoke VPC
VPC
Spoke VPC
TRANSIT
VPC
EC2 VPN EC2 VPN

VPC
VPC
VPC
VPC
VPC
VPC
VPC
Transit VPC
Customer
network
Spoke VPC
Spoke VPC
Spoke VPCSpoke VPC
Spoke VPC
Spoke VPC
Branches
Transit
VPC
Region

https://aws.amazon.com/answers/networking/transit-vpc/
TRANSIT
VPC

NET402—Deep Dive:
AWS Direct Connect
and VPNs

EQUINIX
Oregon
Customer
network
us-east-1
us-east-2
us-west-1
us-west-2
DI R EC T C ONNEC T I NT ER -R EG I ON C ONNEC T I V I T Y I N US A

EQUINIX
Oregon
DI RECT CONNECT GLOB AL I NTER -REGI ON CONNECTI V I TY
?????? ?
G O L A B L D I R E C T C O N N E C T I V I T Y
• I S I T P O S S I B L E ?
• H O W D O Y O U A C H I E V E I T ?

AMAZON DIRECT CONNECT GLOBAL PUBLIC VIFS
N E W
C R E A T E P U B L I C V I F S T O P U B L I C A W S S E R V I C E S G L O B A L L Y

D X
L O C A T I O N
P U B L I C V I F
N. Virginia
Ohio
N. California
SAO PAULO
Frankfurt
IrelandLondon
A M E R I C A
E U R O P E
A S I A
Singapore SydneyMumbaiTokyo Seoul
Oregon
S . A M E R I C A
Montreal
Customer North American Office
B G P A S N
6 5 5 1 5
O h i o R o u t e s A d v e r t i s e d

• Be selective in your public network announcements
• Filtering public prefix announcements if necessary
• Authoritative AWS public IP list available:
https://ip-ranges.amazonaws.com/ip-ranges.json
• For notification of IP changes, subscribe to SNS topic:
arn:aws:sns:us-east-1:806199016981:AmazonIpSpaceChanged
AWS DIRECT CONNECT PUBLIC INTERFACE

I N
CONCLUSION

CUSTOMER HOSTED SERVICES
VIA PRIVATE LINKS
USE
DIRECT CONNECT GATEWAY
INTRA & INTER REGION
VPC PEER LINKS
DIRECT CONNECT PUBLIC VIFS
TRANSIT VPC

Thank you!
A R C 3 0 4
Y O U A R E A W E S O M E !

ARC304_From One to Many Evolving VPC Design

Recomendados

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente (20)

Similar a ARC304_From One to Many Evolving VPC Design

Similar a ARC304_From One to Many Evolving VPC Design (20)

Más de Amazon Web Services

Más de Amazon Web Services (20)

ARC304_From One to Many Evolving VPC Design