BayLISA: MidoNet Overlay Based Network Virtualization for IaaS Clouds

MidoNet: Overlay-based Virtual
Networking for IaaS Clouds
March 21, 2013

Adam Johnson
General Manager, Midokura
@adjohn

Requirements

uplink

Provider Virtual
Router (L3)

Tenant/Project A Tenant/Project B

Tenant B
Tenant A
Virtual Router
Virtual Router

Network A1 Network A2 Network B1
TenantB ofﬁce
Virtual L2 Virtual L2 Virtual L2
Switch A1 Switch A2 Switch B1 Tenant B
VPN Router

VM1 VM3 VM5 VM2 VM4 VM6
Ofﬁce
Network

Copyright ©2012 Midokura All rights reserved

2

Requirements

uplink

Isolated tenant Provider Virtual
network (virtual Router (L3)

data center)


Tenant B
Tenant A
Virtual Router
Virtual Router

TenantB ofﬁce
VPN Router

Ofﬁce
Network


3

Requirements

uplink

Provider Virtual
Router (L3)

L3 isolation (similar to
VPC and VRF)


Tenant B
Tenant A
Virtual Router
Virtual Router

TenantB ofﬁce
VPN Router

Ofﬁce
Network


4

Requirements

uplink

Provider Virtual
Router (L3)


Tenant B
Tenant A
Virtual Router
Virtual Router

TenantB ofﬁce
VPN Router

Ofﬁce
Network

Isolated L2 networks

5

Requirements

uplink

Provider Virtual
Redundant, optimized
Router (L3) and fault-tolerant
paths to the Internet
(e.g. via BGP)


Tenant B
Tenant A
Virtual Router
Virtual Router

TenantB ofﬁce
VPN Router

Ofﬁce
Network


6

Requirements

uplink

Provider Virtual

Fault-tolerant
Router (L3)

devices and links


Tenant B
Tenant A
Virtual Router
Virtual Router

TenantB ofﬁce
VPN Router

Ofﬁce
Network


7

Requirements

uplink

Provider Virtual
Router (L3)

NAT, LB, and
Firewalls
Filtering


Tenant B
Tenant A
Virtual Router
Virtual Router

TenantB ofﬁce
VPN Router

Ofﬁce
Network


8

Requirements

uplink

Provider Virtual
Router (L3)


Tenant B
L3 (and L2)
VPNs

Tenant A
Virtual Router
Virtual Router

TenantB ofﬁce
VPN Router

Ofﬁce
Network


9

Requirements

Solid integration with uplink Minimize ARP broadcasts by
leading open CMS: exploiting CMS config

OpenStack, CloudStack

Provider Virtual DHCP, DNS and other
RESTful API for CMS Router (L3)
services

integration and direct
tenant access


Tenant B
Tenant A
Virtual Router
Virtual Router

TenantB office
VPN Router

Office
Network


10

Requirements: recap

l  Multi-tenancy l Stateful NAT
l  Scalable, fault-tolerant u  Port masquerading
devices (or device- u  DNAT
agnostic network l ACLs

services). l Stateful (L4) Firewalls

l  L2 isolation u  Security Groups
l LB health checks
l  L3 routing isolation
l VPNs at L2 and L3
u  VPC
u  Like VRF (virtual u  IPSec
routing and fwd-ing) l REST API

l  BGP gateway l Integration with CMS

l  Scalable control plane u  OpenStack
u  ARP, DHCP, ICMP u  CloudStack
l  Floating IP

11

How to build it?

1. Virtualized physical devices
2. Centrally controlled OpenFlow-based hop-by-
hop switching fabric
3. Edge to edge overlays


12

1 Virtualized physical devices

VLAN
VLAN1
VLAN2

l  4096 limit on number of unique tags
l  Large spanning trees terminating on many hosts
l  High churn in switch control planes due to MAC learning
l Each VM is separate virtual MAC!
l  Need MLAG for L2Copyright ©2012 Midokura(vendor speciﬁc)
multi-path All rights reserved

13


VLAN (more)
VLAN1
VLAN2

l  L2 isolation
l  What about L3 and Internet access?
l  Use VRF or virtual appliances?


14


VRF

VRF VRF
VRF

Core Product
VLAN 20
Sales
VLAN 10 VLAN 99
VLAN21
VLAN11
VLAN22
VLAN12

出典：http://infrastructureadventures.com/tag/vrf-lite/

l  Not scalable to cloud scale
l  Expensive hardware
l  Not fault tolerant (HSRP?)
l  L2 and L3 isolation. What about NAT, LB, FW?


15

2 OpenFlow hop-by-hop switch fabric

OpenFlow Controller
(Cluster)

OpenFlow Switches

l  Fabric extends to the compute host software switch?
• State in each switch is proportional to the virtual network
state
• Need to update all switches in path when provisioning new
virtual devices or updating them.
• Not scalable, slowCopyright ©2012 Midokura All rights reserved

and non-atomic switch updates. 16

2 OpenFlow hop-by-hop switch fabric (more)

OpenFlow Controller
(Cluster)

OpenFlow Switches

l  Flow rules for VM ﬂows (microﬂows)?
l  Flow rules for virtual device simulation?


17

3 Edge-to-Edge Overlays

Edge

VM Edge Edge

Edge Edge VM
Edge

Use scalable IGP (iBGP, OSPF) to
build multi-path underlay


18


IP encapsulation
provides isolation

Edge

VM Edge Edge

Edge Edge VM
Edge


19

Virtual network
processing at ingress
host, decoupled from
Edge
physical network

VM Edge Edge

Edge Edge VM
Edge


20


Edge

VM Edge Edge

Edge Edge VM
Edge

Virtual network
changes don't aﬀect
underlay state

21


• Packet processing on x86 CPUs (at edge)
•  Intel DPDK facilitates packet processing
•  Number of cores in servers increasing fast

• Clos Networks (for underlay)
•  Spine and Leaf architecture with IP
•  Economical and high E-W bandwidth
• Merchant silicon (cheap IP switches)
•  Broadcom, Intel (Fulcrum Micro), Marvell
•  ODMs (Quanta, Accton) starting to sell directly
•  Switches are becoming just like Linux servers


22

Overlays are the right approach!

But not suﬃcient...
We still need a scalable control plane.


23

MidoNet SDN Solution

Logical Topology
vPort Virtual
Tenant A Switch A1
Virtual vPort
Router

vPort Provider Virtual
Virtual Switch A2
Router vPort

Tenant B
vPort Virtual Virtual
Router Switch B1
vPort


24


Logical Topology
vPort Virtual
Tenant A Switch A1
Virtual vPort
Router

vPort Provider Virtual
Virtual Switch A2
Router vPort

Tenant B
vPort Virtual Virtual
Router Switch B1
vPort

VM
MN MN VM
BGP BGP
Multi To ISP1
Homing

VM
Internet BGP Private IP Network MN VM
MN
To ISP2 Tunnel
BGP
To ISP3

VM
MN MN VM

MN MN MN

Network State Database

Physical Topology

25


Distributed State

MidoNet REST API

Dashboard


26


Lazy state
propagation Distributed State

Host A Host B

MidoNet MidoNet
VM1 Ctrl VM2 Ctrl

Linux Kernel + OVS KMOD Linux Kernel + OVS KMOD

HW HW


27


VM sends ﬁrst
packet; table Distributed State
miss; NetLink
upcall to MidoNet

Host A Host B

MidoNet MidoNet
VM1 Ctrl VM2 Ctrl


HW HW


28


MidoNet agent locally
processes packet (virtual Distributed State
layer simulation); installs
local ﬂow (drop/mod/
fwd)
Host A Host B

MidoNet MidoNet
VM1 Ctrl VM2 Ctrl


HW HW


29


Packet tunneled to
Distributed State peer host; decap;
kﬂow table miss;
Netlink notiﬁes
peer
MidoNet agent
Host A Host B

MidoNet MidoNet
VM1 Ctrl VM2 Ctrl


HW HW


30


Distributed State MN agent maps
tun-key to kernel
datapath port#;
installs fwd ﬂow
rule
Host A Host B

MidoNet MidoNet
VM1 Ctrl VM2 Ctrl


HW HW


31


Subsequent packets Distributed State
matched by ﬂow rules
at both ingress and
egress hosts

Host A Host B

MidoNet MidoNet
VM1 Ctrl VM2 Ctrl


HW HW


32


•  Distributed and scalable control plane
Ø Handle all control packets at local MidoNet agent
adjacent to VM
•  Scalable and fault tolerant central database
Ø Stores virtual network conﬁguration
Ø Dynamic network state
² MAC learning, ARP cache, etc
Ø Cached at edges on demand
•  All packet modiﬁcations at ingress
Ø One virtual hop
² No travel through middle boxes
Ø Drop at ingress


33


• Scalable edge gateway interface to external
networks
•  Multihomed BGP to ISP
• 
REST API and GUI
• 
Integration with popular open source cloud stacks
•  OpenStack
•  Removes SPOF of network node
•  Scalable and fault tolerant NAT for ﬂoating IP
•  Implements security groups eﬃciently
•  CloudStack


34


Deep OpenStack Integration
• Quantum Plugin
•  L2 isolation, of course
• Also…
•  L3 isolation (without VM / appliance)
•  Security groups (stateful ﬁrewall)
•  Floating IP (NAT)
•  Load balancing (L4)


35

OpenStack Integration

Horizon MidoNet
Nova API
Web GUI Manager
(Web GUI) INTERNET

Quantum
Plugin

RabbitMQ - Passing
Keystone
Queue MidoNet Edge
Authentication MidoNet Edge
MidoNet API MidoNet Edge
Agent
Agent
Agent

Compute Host
API MidoNet
MidoNet
MidoNet
Distributed
Instance A1

Distributed
Instance B1

Nova Quantum Distributed
State
Compute State
State
MidoNet Plugin
MidoNet
VIF Driver

Libvirt
driver

MidoNet
Datapath Agent



Future Directions

• Scalable L7 virtual appliances
• Content aware load balancer
• MPLS VPN termination
Ø Interconnect with carrier backbones
• multiple data center federation
Ø Virtual L2 between sites
• LISP
Ø Global IP mobility between sites


37


Conclusions

• IaaS clouds require new networking model
• Edge to edge overlays are the right
approach
• Servers are good enough at packet
processing
Ø  Can use them for edge gateways
• Multipath IP network fabric is cheap and
easy to build


38

Questions?

info@midokura.com

We’re hiring
http://midokura.com/careers/


Backup Slides


40


Packet from VM, VPN,
or external BGP peer
enters kernel Tunnel
datapath

MN

Packet
Encapsulated

Drop/Block


41


Tunnel

MN

Packet
Encapsulated

Drop/Block

One ﬂow rule reﬂecting the
outcome of the virtual layer
simulation AND the
mapping of egress vport to
peer host decides to drop or

fwd
42

Spine and Leaf Network Architecture

e.g Force10 Z9000

Spine L3 Switch L3 Switch L3 Switch L3 Switch x4

IBGP and
ECMP
4x40G
Leaf L3 Switch L3 Switch L3 Switch x32

48x10G

1536 x 10G
e.g Arista 7050T


43

BayLISA: MidoNet Overlay Based Network Virtualization for IaaS Clouds

Recomendados

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente (19)

Similar a BayLISA: MidoNet Overlay Based Network Virtualization for IaaS Clouds

Similar a BayLISA: MidoNet Overlay Based Network Virtualization for IaaS Clouds (20)

Último

Último (20)

BayLISA: MidoNet Overlay Based Network Virtualization for IaaS Clouds