3. What is virtualization?
Virtualization is the basic act of decoupling an infrastructure service
from the physical assets on which that service operates.
The infrastructure service exists entirely in a software abstraction
layer:
• Lifecycle – started/stopped anytime
• Identity – being independent to physical world
• Location – could deployed anywhere
• Configuration – being simpler to manage
7. Virtual Network
• Abstracted network view for an user
• Decoupled from physical infrastructure
• Composed as a set of logical network resources
• Provide isolation by:
• Address space - remove the threat of address conflict
• Performance - virtual networking more predictable for users
• Management - mimic usage of non-virtualized network
• Security – don’t allow tenant’s users (and their traffic) to
access and interrupt the work of other tenants
• Configuration independence and elasticity
• Easier to deploy and manage network services and
underlying network resources
®Cisco
8. Virtual network element lifecycle
Instantiated
create
Located
bind to
interfaces
Running
run
stop
Terminated
destroy
unbind
destroy
• Router
• Switch
• Gateway
• Firewall
• Load balancer
re-bind
(migrate)
9. Objects of network virtualization
• Device virtualization
• Virtualize physical devices
(nodes) in the network
• Data Plane virtualization
• Control Plane virtualization
• Management Plane
virtualization
• Data path virtualization
• Virtualize communication
path between network access
points
• Links virtualization
9
Router
Switch
Data Path
®Yeh-Ching Chung
10. Network Virtualization advantages
• Infrastructure utilization
• Infrastructure is shared between many different users or purposes
• Reduces infrastructure & energy cost
• Scalability
• Easy to extend resources in need
• Administrator can dynamically create or delete virtual network resources
• Agility
• Enables automation of network services establishment
• Network services can be orchestrated together with other IT infrastructure
• Resilience
• Virtual network will automatically redirect packets by redundant links
• In case of disaster, the virtual network can be easily recreated on new physical infrastructure
• Security
• Increased data traffic isolation and user segmentation
• Virtual network should work with firewall software
10
13. Resource sharing
Example: VLAN (Virtual Local Area Network)
• Device virtualization
• Divide physical switch into
multiple logical switches
14
• Virtualization is implemented within
switch management software
• VLAN can be a group of ports
• VLAN can be group of MAC addresses
• VLAN can be a specific upper layer
protocol
• VLAN can be a group of IP addresses
• VLAN can be a group of authenticated
users
• A network chip (frame forwarding
silicon) is shared by all virtual
switches
• Network chip must support VLAN
framing and processing
ETH Data
ETH Data
ETH Data
ETH Data
access
®Yeh-Ching Chung
14. Infrastructure sharing
Example: VLAN (Virtual Local Area Network)
15
• Link virtualization
• Divide physical link into
multiple logical links
SWITCH #1 SWITCH #2
ETH 1 Data ETH 2 Data ETH 1 Data
1 2 3 4 5 5
• Link virtualization is done by network
protocol (new Ethernet header 802.1Q)
• Ethernet frame contains new fields
• Link bandwidth is shared between VLANs
trunk
• Virtual links can be isolated one from each
other by setting rate limitation per vlan
15. Infrastructure sharing
Example: VLAN (Virtual Local Area Network)
16
# Create VLAN:
set vlans employee-vlan vlan-id 200
# Add ports to VLAN in access mode:
set interfaces ge-0/0/1 unit 0 family ethernet-switching port-mode access vlan members employee-vlan
set interfaces ge-0/0/2 unit 0 family ethernet-switching port-mode access vlan members employee-vlan
set interfaces ge-0/0/3 unit 0 family ethernet-switching port-mode access vlan members employee-vlan
commit
# Remove ports from VLAN:
delete interfaces ge-0/0/1 unit 0 family ethernet-switching vlan members employee-vlan
delete interfaces ge-0/0/2 unit 0 family ethernet-switching vlan members employee-vlan
delete interfaces ge-0/0/3 unit 0 family ethernet-switching vlan members employee-vlan
# Delete VLAN:
delete vlans employee-vlan
commit
Juniper JUNOS commands:
Instantiated
Located
Running
Terminated
SWITCH #1
1 2 3 4 5
access
®Juniper
16. Infrastructure sharing
Example: VLAN (Virtual Local Area Network)
SWITCH #1
1 2 3 4 5
trunk
access
# Create VLANs:
set vlans employee-vlan vlan-id 100
set vlans production-vlan vlan-id 200
set vlans research-vlan vlan-id 300
# Set VLANs on access ports (1GbE):
set interfaces ge-0/0/1 unit 0 family ethernet-switching port-mode access vlan members employee-vlan
set interfaces ge-0/0/2 unit 0 family ethernet-switching port-mode access vlan members production-vlan
set interfaces ge-0/0/3 unit 0 family ethernet-switching port-mode access vlan members research-vlan
# Set VLAN on trunk port (10GbE):
set interfaces xe-0/0/5 unit 0 family ethernet-switching port-mode trunk vlan members [employee-vlan production-vlan research-vlan]
# Create policer and filters limiting bandwidth to 1Gbps:
set firewall policer 1G if-exceeding bandwidth-limit 1g
set firewall policer 1G if-exceeding burst-size-limit 10m
set firewall policer 1G then discard
set firewall family ethernet-switching filter 1Gfilter term 1 then policer 1G
set firewall family ethernet-switching filter 1Gfilter term 1 then accept
# Apply 1Gbps filter to all VLANs:
set vlans employee-vlan filter input 1Gfilter
set vlans production-vlan filter input 1Gfilter
set vlans research-vlan filter input 1Gfilter
commit
®JuniperJuniper JUNOS commands:
17. Infrastructure sharing
Example: VLAN (Virtual Local Area Network)
18
VLANs are used in enterprises for:
• Grouping devices by organizational/location issues
• logical separation between groups in the organization
• VLAN for each building or each floor of a building
• Grouping devices for security
• It is often a good practice to put servers and key infrastructure in their
own VLAN, isolating them from the general broadcast traffic and
enabling greater protection,
• Any sensitive data (financial, research) should have its own VLAN
• Forming Demilitarized Zone containing an organization’s services
offered in Internet
• Grouping devices by traffic types
• VoIP quality is improved by isolating VoIP devices to their own VLAN.
• Other traffic types may also warrant their own VLAN:
• Network management traffic
• IP multicast traffic such as video
• File and print services
• Email & Internet browsing
• Database access
19. Infrastructure sharing
Example: VLAN (Virtual Local Area Network)
20
Configuring VLANs in hosts (Ubuntu):
# Enabling VLANs:
sudo apt-get install vlan
sudo modprobe 8021q
# Adding VLAN 102 to the interface eth0
sudo vconfig add eth0 102
> Added VLAN with VID == 102 to IF -:eth0:-
sudo ifconfig eth0.102 10.0.0.1/24
# Checking network interface
ifconfig eth0.102
> eth0.102 Link encap:Ethernet HWaddr 5c:f3:fc:e8:53:0a
> inet addr:10.0.0.1 Bcast:10.0.0.255 Mask:255.255.255.0
# Removing VLAN 102 from the interface eth0
sudo vconfig rem eth0.102
> Removed VLAN -:eth0.102:-
®IBM
20. Infrastructure sharing
Example: VLAN (Virtual Local Area Network)
• VLAN (Ethernet) networking has fundamental problem:
• It is OSI Layer 2 („Data link”) technology:
• Initially defined as the layer that allows adjacent network devices to
exchange frames
• MAC addresses added only because of coax cabling in the past
• IEEE always wanted to keep everything backward compatible
• Only OSI Layer 3 („Network”) should provide end-to-end packet delivery
across the network
• Nobody wants to change the device drivers in every host/switch
deployed in the global network so we are still using frame format from
40 year old technology
21
®Ivan Pepelnjak
®Computer Desktop Encyclopedia
21. Infrastructure sharing
Example: VLAN (Virtual Local Area Network)
• VLAN (Ethernet) networking has fundamental problems:
• Requires Control Plane protocol:
• Any Spanning Tree Protocol (STP) protocol doesn’t solve all existing problem
• Many broken implementation and incompatibilities
• Flooding of broadcast frames
• Every broadcast frame flooded throughout a L2 domain must be processed
by every host participating in that domain
• Every virtualization hypervisor host has to processes every broadcast frame
generated anywhere (regardless of whether its VMs belong to the VLAN
generating the flood or not)
• Once you get a loop in a bridged network your network is toast
• The whole Layer 2 network is a single failure domain
• Lack of addressing hierarchy
• Modern switches support up to 1K 20K 100K 400K MAC addresses
22
22. Infrastructure sharing
Example: VLAN (Virtual Local Area Network)
• VLAN advantages
• Cheap in terms of protocol
overhead:
• VLAN tag is only additional
4 bytes of the frame header
• Supported by most of the
network devices
• VLAN disadvantages
• Not scalable
• Only 4096 virtual networks in
802.1Q (vlan_id is 12-bit field)
• Only 1000 hosts in a virtual
network
• 802.1ad doesn’t solve all problems
• Management can become
complex
• To be configured on each device
• VLAN swapping required if
somewhere VLAN tag already used
• Broadcast storms in case of
switching loops affects all VLANs
23
23. Infrastructure sharing
Example: DWDM (Dense wavelength division multiplexing)
24
• Link virtualization
• Divide physical link into
multiple logical links
• Virtualization is implemented
within physical layer
• Each logical link is represented by a
specific wavelength („color” of the
light)
• Initial motivation was to multiply
bandwidth of a single optical fiber
• DWDM allows only for point-to-
point connections
®Cellco
24. Infrastructure sharing
Example: DWDM (Dense wavelength division multiplexing)
25
• Node virtualization
• Each wavelength („lambda”) can be
processed independently from
other lambdas
• ROADM device can be logically
represented as a set of virtual
optical switches
• Single virtual optical switch is
controlling „switching” of a single
lambda
25. Infrastructure sharing
Example: DWDM (Dense wavelength division multiplexing)
26
• Network virtualization
• Each lambda in the ring can be a virtual
network
• Lambda can be terminated on any pair of
optical transponders (add/drop ports)
Many challenges related to exposing optical layer to
virtual network user:
• Optical transmission impairments can lead to
infeasible lightpaths
• A lightpath set-up/tear-down needs to be done
sequentially in order to avoid undesirable optical
power fluctuations
• Multi-degree ROADMs are not blocking-free
• Wavelength continuity required to limit expensive
wavelength conversions
®CNMP
26. Infrastructure sharing
Example: DWDM
27
• Optical (DWDM) network virtualization is actual R&D topic
• Reasons for optical network virtualization:
• Cloud data centers are interconnected over national or international optical
networks (several 10GE links per site)
• Most of the inter-data center connections are statically provisioned and
dimensioned for peak load
• network assets to be underutilized for most of the time
• leaves data center owners with huge interconnection costs
• User (like Cloud providers) would like to use on-demand optical bandwidth
increase for specific period of time
• bulk data transfers between sites
• low-latency, high transfer speed
27. 28
• Virtualization on fundamental
level
• All nodes and links are exposed
• Direct hardware representation
• Users needs to control and
understand optical layer
• Virtualization on abstracted
level
• Network abstracted as one large
optical switch with all client ports
• Users see switch as a black box
• Optical layer is hidden
Infrastructure sharing and abstraction
Example: DWDM (Dense wavelength division multiplexing)
Physical network
Virtual networks
Physical network
Virtual networks
®ADVA
®ADVA
28. 29
Infrastructure sharing
Example: VRF (Virtual Routing and Forwarding)
• Device virtualization
• Divide physical router into
multiple logical routers • Memory (where routing and
forwarding tables are stored) of
frame forwarding silicon is divided
between VRF
• Router interfaces are bind to
specific VRF(s)
• Each VRF contains one routing and
one forwarding table
• No virtualization of the router
management:
• One CLI
• One config file
®infrastructureadventures
29. 30
Infrastructure sharing
Example: VRF (Virtual Routing and Forwarding)
• Link virtualization
• We need to use some
other network technology
to share a link between
many VRFs traffics
• We can use:
• VLANs
• MPLS
• GRE tunnels
• IP-in-IP
®infrastructureadventures
30. 31
Infrastructure sharing
Example: VRF (Virtual Routing and Forwarding)
• Who is using VRF?
• Datacenter Providers use it to share their resource between different
customers
• ISPs (Internet Service Providers) don’t need more than one router device to
connect a few customers VPN (Virtual Private Networks)
• Enterprises to segment their internal networks
®ayyappanworld
32. 33
Virtualization technique:
Tunneling
• Tunnel is a connection across a network which ships protocol frames at payload
that normally wouldn't forwarded by network because of breaking of the
classical network layering
• Intermediate nodes of tunnel don’t see encapsulated frames (it is just data)
• Encapsulated frames could be encrypted (SSL/TLS, SSH, IPsec)
• Connecting distance sites:
• Tunnels via global Internet
• Tunnels via WAN networks
®Cisco
33. 34
Virtualization technique:
Tunneling
• Tunneling encapsulation examples:
Ethernet IP
header
GRE
header
GRE Data
Ethernet Ethernet Data
IP in IP
Ethernet in IP (VXLAN)
Ethernet in IP (GRE)
Ethernet MPLS
header
MPLS Data
IP header IP Data
IP in MPLS
Tunnels via Internet
Tunnel via MPLS network
(popular service offered by
core/ISP networks)
Ethernet IP
header
UDP
header
VXLAN Data
Ethernet Ethernet Data
VXLAN
header
34. 35
Virtualization technique:
MPLS Tunneling
ETH MPLS
10 Data
ETH MPLS
20 Data
ETH MPLS
13 Data
ETH Data
ETH Data
LSP (Label Switched Path) – it is MPLS tunnel
MPLS benefits over IP networks:
• Improved route look up time by
using MPLS labels to forward traffic
• Increased network throughput
• Control over how traffic moves
through the network (traffic
engineering)
• Supports many connectivity
services: point-to-point, point-to-
multipoint, L2VPN, L3VPN, any
transport over MPLS, fast
restoration, protections, etc.
• Can coexist with classical IP routing
MPLS is most popular transport
technology in Network Providers
Networks.
®unknown
35. 36
Virtualization technique:
Abstracting as Overlay Network
Overlay networking:
• A virtual network that is built on top of an existing physical network (underlay network)
• Edge nodes of physical network become nodes of overlay network
• Tunnels between edge nodes become logical links of overlay network
• Virtual networking like yet another network application (like E-mail, Web, Skype)
• Many virtual networks can coexist independently over the same physical network
(Underlay Network)
®unknown
36. 37
Virtualization technique:
Abstracting as Overlay Network
Overlay networks are used by Enterprises
• VPN (Virtual Private Network) solutions: L2 VPNs and L3 VPNs
• extends a private network across a public network, such as the Internet
• Using Internet/MPLS tunneling protocols (the tunnel's termination point on the
customer/network edge)
• The levels of security provided
®Wikipedia
37. 38
Virtualization technique:
Abstracting as Overlay Network
Overlay networks are used by Enterprises
• L2 VPN:
• MPLS-based L2 VPN (Point-to-point)
• Provider MPLS network emulating „a cable” connecting two sites
• VPLS (Point-to-multipoint)
• Provider MPLS network emulating „a switch” connecting many sites
Site 2
Site 1
Site 3
Large Provider
MPLS Network
Site 1
Site 2
Site 3
Site 4
VPLS
Learning
switch
38. 39
Virtualization technique:
Abstracting as Overlay Network
Overlay networks are used by
Enterprises
• L3 VPN:
• IP over GRE:
• Many IP over GRE tunnels across Provider IP
network
• MPLS-based L3 VPN
• Provider MPLS network emulating „a router”
connecting many sites
VRF
VRF
VRF
VRF
VRF
VRF
VRF
VRFVRF
®Joe Keegan
39. 40
Virtualization technique:
Abstracting as Overlay Network
Overlay network are used by Clouds
• VXLAN (Virtual Extensible LAN) – Ethernet over IP
• 16 millions logical networks (Layer 2 networks)
• VNID (VxLAN segment identifier): 24 bits
• Ethernet broadcast domain tunneled across IP network
• Ethernet broadcast/multicast implemented using IP multicast
• 50-bytes overhead (requires jumbo frames and higher MTU)
• Virtual Machines don’t aware of VXLAN usage
• Hypervisor hosts appear as simple IP hosts to the transport network
Ethernet IP
header
UDP
header
VXLAN Data
Ethernet Ethernet Data
VXLAN
header
40. 41
Virtualization technique:
Abstracting as Overlay Network: VXLAN
VTEP – Virtual Tunnel End-Point
VNID - VxLAN segment identifier
Virtual Machines Virtual Machines
Hypervisor
host
Hypervisor
host
S1-S4 VMs use Ethernet
MAC for frame addressing
It is de facto Ethernet (VMs) over IP (network).
®Yves Louis
41. 42
Virtualization technique:
Abstracting as Overlay Network: MPLS over GRE
• MPLS Label (LBL) is used to distinguish tenants (virtual networks)
• GRE used to pass MPLS frames over IP network
It is de facto IP (VMs) over IP (network).
http://www.opencontrail.org/
®Juniper
42. Virtualization technique:
Abstracting as Overlay Network
• Overlay advantages
• Full address isolation between
virtual network and physical
underlay infrastructure
• Independence from type of
underlay network and its topology:
• Use existing IP networks and global
Internet
• With additional encapsulation ISP
MPLS networks can be also used
• No changes in underlay network –
all virtualization complexity at
edges of network (follows original
Internet design)
• Network resilience is provided by
underlay network
• Fair scalability
• Support easy VM migration
(including policy, security and
VLANs)
• Overlay disadvantages
• Requires jumbo frames everywhere:
• Wrong MTU causes problems difficult to be
correctly identified and localized
• Encapsulation introduce CPU and latency
overheads (up to 60%) due to missing
checksum and TCP segmentation offloading
• Requires non-oversubscribed physical
underlay network:
• IP network provide no throughput isolation of
virtual networks
• Control Plane bottleneck still exists
• Gateways between virtual network and
other network may need to pass high
volumes of traffic
• Some value-added features in existing
networks cannot be leveraged due to
encapsulation
• Traffic engineering in IP core not possible
• Currently a lot of solutions and protocols for
creating overlays (compatibility problems) 43
44. 45
Virtualization technique:
Abstracting network node type
OpenFlow switches
• OpenFlow switch can become any of
classical network elements:
• Router
• Switch
• Gateway
• Firewall
• Load balancer
• Freedom of choosing virtual nodes
type and functionality
Virtual Network
45. 46
Virtualization technique:
Network slicing
Ingress
port
Eth src Eth dst
Ether
type
VLAN id
VLAN
priority
IP src IP dst
IP
proto
IP ToS
bits
TCP/UDP
src ports
TCP/UDP
dst ports
Possible only in OpenFlow networks:
• Defined with notion of flowspace (the set of all possible
header values defined by the OpenFlow tuple)
• The slice (virtual network) is any subset of OpenFlow
flowspace:
• To a slice belongs all frames with specific values of header
fields
• Network segmentation on any network protocol or
combination of network protocols (we can emulate VLAN,
MPLS, IP segmentation and any other technique)
• OpenFlow controller can set flow entries within a slice
• Very flexible approach for network sharing
OpenFlow 1.0 tuple:
®ON.Lab
46. 47
Virtualization technique:
Control isolation
FlowVisor
Controller
(slice A)
OpenFlow switches
Controller
(slice B)
Controller
(slice A)
Controller
(slice B)
Slice topology directly
reflects the physical
network topology and is
a subset of it
Each slice associated to a controller
Isolation of slices enforced
by FlowVisor (a proxy for
OpenFlow messages)
VLAN 50
VLAN 30 & IP 10.0.0.1/16
https://github.com/OPENNETWORKINGLAB/flowvisor
®ON.Lab
47. 48
Virtualization technique:
Topology abstraction
Topology abstraction:
• Virtual network topology can
be different than physical
topology
• Controller can see
simplified topology
• Collapse multi-hop path
into one-hop link
• Hosts (endpoints) could be
part of virtual network or not
Demo:
VM
OpenVirteX – A Network Hypervisor that
supports Topology, Address Space, and
Control Isolation
Network OS Network OS Network OS
Physical Network
www.openvirtex.org
®ON.Lab
48. 49
Virtualization technique:
Topology abstraction
• Virtual switch: collapse
ports dispersed over
network into a switch
• Use separate controller
for each virtual switch
• Allow OpenVirteX admin
to traffic engineering
within virtual switch
virtual
physical
...
...
virtual switch
edge ports
core ports
VM
OpenVirteX
Controller
®ON.Lab
49. 50
Virtualization technique:
Addressing isolation
OpenVirteX
Controller
(slice A)
OpenFlow switches
Controller
(slice B)
• Inside the network, frames have physical IP addresses
replaced with virtual IP address, containing encoded
tenant id (tenant id may be also encoded in MAC
addresses):
• First switch forwarding traffic flow must rewrite physical IP/MAC
to virtual IP/MAC
• Last switch forwarding traffic flow must rewrite virtual IP/MAC to
physical IP/MAC
• OpenFlow hardware switches must support IP/MAC rewriting
operations in the edge (but edge is software virtual switch in the
most of the cases)
• Endpoint (IP and MAC pair) can be part of only one tenant
• Each virtual network has a full flowspace available
• Address isolation happen also in fields
remapping/rewriting in OpenFlow messages to
switches
Physical IP/MAC
Virtual nodes and interfaces
Virtual IP/MAC
Virtual IP/MAC
Virtual IP/MAC
Physical IP/MAC
Physical IP/MAC
isolation
50. Virtualization technique:
OpenFlow-based virtualization
• OpenVirteX advantages
• Virtualization is pure Network
Control Plane feature:
• Only IP/MAC rewriting
functionality required in the
data plane
• No overhead in
CPU/latency/protocol
• Full address and control
isolation
• Any grade of topology
simplification possible:
• Traffic engineering possible
both within virtual network and
physical network
• Simple network control which
could be extremely granular
• OpenVirteX disadvantages
• Requires OpenFlow devices everywhere:
• Virtualized data traffic cannot be passed
through IP network/Internet (so overlay
must be used anyway)
• If OpenFlow device is used as edge node
then IP/MAC rewriting is required in the
hardware
• Inherits all OpenFlow disadvantages:
• Scalability problems still not solved
• OpenFlow hardware limits (number of
flows, flow installation time)
• No solutions for core network
• Incompatibility of OpenFlow versions
51
51. 52
Virtualization techniques: Summary
Virtualization aspects DWDM VLAN VRF Overlay OpenFlow (OpenVirteX)
Link sharing Lambda - pure physical
phenomenon
VLAN header in the frame -
(utilize VLAN, MPLS or
overlay)
- Performed almost fully in the
Network Control Plane (frame
addresses rewriting required)
Node sharing Performed by node
management
Performed by node
management
Multiple routing and
switching tables in
forwarding chip
-
(when router required than
utilizing VRF)
Performed almost fully in the
Network Control Plane (frame
addresses rewriting required)
Topology abstraction Virtual network as a single
node
- - Tunnels as abstract links or
switches
Ports collapsing and multi-hop
links
Address isolation - - - Encapsulation on edges Address translation on edges
Control isolation - - -
(partially happen for logical
router systems)
- Multiple Network controllers
having access to network
resources with policy enforcement
Performance isolation Very good Can be applied for data plane
if proper filters available in
the device
Quite good in data plane,
weak in control plane
Depends on underlay
technology (no isolation in IP
network)
Possible both in data and control
plane
Where used Core networks Enterprises, R&D networks,
Clouds, Access networks
Access networks,
Enterprises, Clouds
Clouds, Enterprises R&D networks, Clouds
53. 54
Virtualization technique:
Software forwarding
• Any frame forwarding done by the network hardware can be implemented in the
software
• Pure software forwarding solutions are more elastic:
• You don’t have to buy costly hardware – you need only a cheap server
• Much easier to introduce new functionalities and innovate the networking gears
• Open source networking!
• You can run as many software forwarding entities as you need and where you need
• Reusing server virtualization (virtual machines, docker containers) and orchestration (puppet,
fabric, chef, ansible) for deploying new network forwarding instances
• Software forwarding becoming faster because of:
• Better CPUs and NICs (Network Interface Card) every year
• Great tuning of packet processing in Linux (example: Intel DPDK network drivers and libraries –
100% more speed, Netmap, PF_RING, NAPI, Receive Side Scaling)
• Network ASIC accelerators, Direct Cache Access, Intel Flow Director inside CPUs and NICs:
• CPU becoming close to NPU (Network Processor Unit – programmable chips in network devices)
• Frame forwarding to correct VMs done in NICs not CPU
54. 55
Virtualization technique:
Software forwarding
• Linux switch performance:
• 2013: Open vSwitch and Linux bridge: 1Gbps
• 2014: Open vSwitch and Linux bridge (with DPDK) throughput: 13 Gbps
• 2015: 6WINGATE Open vSwitch throughput: 195 Gbps
• Modern hardware switch: 960 Gbps (interfaces: 96x10GbE and 8x40GbE) connects
48 servers (960Gbps/48 = 20Gbps per server)
• Incoming ASIC chips: 3.2 Tbps
• Server network cards: 2x 1/10GbE (future: 25/50/100GbE)
• Linux switch and VMs in a single server (share server performance):
• If Linux switch cannot forward all traffic this means that too many VMs deployed in a
server:
• Orchestrator may migrate some VMs to other servers
55. 56
Software forwarding:
Example: Linux bridge
• Historic intro about bridge device:
• Bridge devices were used in old time in Ethernet
coaxial networks (10 Mbps) to limit Ethernet
collision domains
• A bridge device connects few Ethernet segments
• Frame forwarding was done fully in software so
bridges equipped with few ports (2-4)
• Switch was evolution of the bridge:
• Fast hardware frame switching
• Much more ports
• Twisted pair cable used instead of coaxial cable
• 100 Mbps speed
• Today „bridging” means the same as „switching”
®Computer Desktop Encyclopedia
®Computer Desktop Encyclopedia
®Wondertek
56. Kernel
57
Software forwarding:
Example: Linux bridge
• Software Implementation of the network switch
• Connects physical and logical (virtual) network interfaces available in Linux
• Works in Linux Kernel
• Visible as logical network device in the Linux
Logical NIC
Logical NIC
Logical NIC
Physical NIC
Physical NIC
Linux server
Linux
bridge
57. 58
Software forwarding:
Example: Linux bridge
# Enabling Linux bridge in Debian:
apt-get install bridge-utils
# Create bridge:
brctl addbr br0
# Flush configuration from interfaces to be bridged:
ifconfig eth0 0
ifconfig eth1 0
# Add two prepared interfaces to the bridge:
brctl addif br0 interface eth0 eth1
# Put up the bridge:
ifconfig br0 up
# Optionally assign IP address to the bridge:
ifconfig br0 192.168.100.5 netmask 255.255.255.0
Linux server
Linux
bridge
eth0
eth1
Configuring bridge (Debian):
# Showing all bridges:
$ brctl show
bridge name bridge id STP enabled interfaces
br0 8000.00004c9f0bd2 no eth0
eth1
58. Kernel
59
Software interface:
Example: Linux TAP/TUN
• TUN and TAP are kernel virtual network interfaces:
• TAP simulates an Ethernet device and it operates with Ethernet frames
• TUN simulates a IP layer device and it operates with raw IP packets
Linux server
User space
Virtual NIC
TAP back-end program
char
device
Any application
# Create a TAP device in Python:
from pytun import TunTapDevice, IFF_TAP
tap = TunTapDevice(name=’tap0’‚flags=IFF_TAP)
# Set MAC and MTU of virtual network interface:
tap.hwaddr = 'x00x11x22x33x44x55'
tap.mtu = 1500
# Bring network interface up:
tap.up()
# Read Ethernet frame from TAP device; frame was sent by an
application via socket opened on virtual interface
buf = tap.read(tap.mtu)
# Write Ethernet frame to TAP device; frame will be received by
an application
tap.write(buf)
Open socket
TAP back-end program in Python:
TAP
read
write
59. Hypervisor
Virtual NIC
60
Software interface:
Example: Linux TAP/TUN
• TAPs are used by virtualization hypervisors (Xen, KVM, etc) to create virtual NICs
inside Virtual Machines
Linux server
Linux
bridge
eth0
char
device
Virtual Machine
10.0.0.1
Virtual NIC
Virtual Machine
10.0.0.2
Virtual NICVirtual NIC
char
device
TAP
TAP
10.0.0.254
60. 61
Software forwarding:
Virtual switch
VMware networking:
• Virtual Switch is a software switch
that provides networking for Virtual
Machines
• Virtual Switch is commonly
considered as part of hypervisor
• Server virtualization hypervisors
allows for complex networking use-
cases by the instantiation of many
parallel software switches:
• Interconnecting VMs with private IP
addressing (no access to Internet)
• Usage of public IP addresses by VMs,
accessible from the Internet
• NAT-based access to Internet from
VMs
®WMware
61. 62
Software link:
Example: Linux veth
• veth is pure software link (Linux virtual link)
• veth is composed of a pair of virtual network interfaces connected back-to-back together
• Ethernet frame sent to one end of the veth pair is received by the other end of the veth
pair
Kernel
Linux server
User space
Virtual NIC
Any application
Open socket
VETH
Virtual NIC
Any application
Open socket
# Create a veth pair of interfaces:
ip link add dev veth0 type veth peer name veth1
# Set IP addresses on veth interfaces:
ip addr add 10.0.0.1/24 dev veth0
ip addr add 10.0.0.2/24 dev veth1
# Bring network interfaces up:
ip link set dev veth0 up
ip link set dev veth1 up
veth0
veth1
veth creation in Linux:
62. 63
Software link:
Example: Linux veth
• veth can be used to create complex networks inside Linux server:
• Used by Cloud systems (e.g OpenStack)
• Used by network simulation/testing tools (e.g.: Mininet – OpenFlow network simulation)
Linux server
software
switch
software
switch
software
router
veth0
veth1
veth2
veth3
eth0
eth1
eth2
OpenFlow
switch
OpenFlow
switch
OpenFlow
switch
OpenFlow
switch
VM VM
veth
veth veth
veth
veth
veth veth
veth
veth
tap tap
Linux server
63. 64
Software forwarding:
Example: Open vSwitch (Open Virtual Switch)
• Open Source switch (Apache 2.0 license)
• Alternative to Linux bridge
• Much more functionalities
• Forwarding based on Ethernet, VLAN, IP, UDP, TCP
• OpenFlow, OVSDB, QoS, Monitoring
• Tunnel protocols (GRE, VXLAN, GENEVE, LISP, IPsec)
• Heavily used in production environments:
• default OpenStack and OpenNebula virtual switch
• Specially designed to make it easier to manage
VM network configuration and monitor state
spread across many physical hosts in dynamic
virtualized environments
• Available for POSIX systems, Windows, FreeBSD,
embedded systems
http://openvswitch.org/
®Open vSwitch
64. 65
Software forwarding:
Many other software switches
• Developed by server virtualization
vendors:
• Microsoft Hyper-V switch
• VMware vSwitch
• Developed by network vendors:
• Cisco Application Virtual Switch
• Juniper OpenContrail vRouter
• NEC ProgrammableFlow Virtual Switch
65. 66
Software forwarding:
Software routing
• Linux router:
• Routing tables in the kernel:
• Perform packet routing (data
plane)
• Configurable by hand:
• In shell: ip route
• Programmable by NETLINK
socket
• Routing control plane
established by user program
handling routing protocols
(RIP, OSPF, IS-IS, BGP, …):
• Open Source: Quagga, XORP
Kernel
Linux server
User space
Routing
Protocols
Suite
(Quagga,
XORP, …)
NETLINK
Kernel
Routing
Tables
OSPF
OSPF
Data Packets
control
routing
Similar software routing possible in BSD, Solaris, Windows.
Data Packets
66. Hypervisor
67
Software forwarding :
Virtual routers
• Whole routing system
deployed as Virtual Machine:
• Handles both data packets and
routing messages
• Additional functionalities:
• Firewall, VPN, switching
• VM appliances provided by
router vendors (look&feel like
hardware routers):
• Juniper vMX
• Brocade Vyatta vRouter
• Cisco Cloud Services Router
• HP Virtual Services Router
Linux server
Virtual
switch
Virtual Machine
Routing
software
Virtual Router Appliance
Virtual
switch
68. 69
Software processing:
Network Function Virtualization (NFV)
Classical Network Appliance Approach
BRAS
FirewallDPI
CDN
Tester/QoE
monitor
WAN
Acceleration
Message
Router
Radio/Fixed Access
Network Nodes
Carrier
Grade NAT
Session Border
Controller
PE RouterSGSN/GGSN
• Fragmented, purpose-built hardware.
• Physical install per appliance per site.
• Hardware development large barrier to entry for
new vendors, constraining innovation & competition.
Network Functions Virtualisation Approach
High volume Ethernet switches
High volume standard servers
High volume standard storage
Orchestrated,
automatic & remote install.
Competitive&
Innovative
OpenEcosystem
Independent
Software Vendors
®ETSI NFV
69. 70
Software processing:
Network Function Virtualization (NFV)
Network Functions are:
• Routing
• Firewalling
• Load balancing
• Network Address Translation (NAT)
• Access Gateway
• WAN acceleration
• QoE monitoring
• Deep packet inspection (DPI)
• Broadband Remote Accessing (BRAS)
• Session Boarder Controlling
• …
Network Functions in NFV:
• Provided in the form of Virtual
Machine Appliances
• Deployed on demand on
virtualization servers
Hyper
visor
Linux server
App
VM
App
VM
70. IP
network
71
Software processing:
Network Function Virtualization (NFV)
Web
server
hypervisor
Web
server
Virtual
switch
Virtual switch
Virtual switch
App
server
hypervisor
App
server
Virtual
switch
Virtual switch
Virtual switch
hypervisor
Virtual
switch
Virtual switch
Outside VXLANVXLAN
Virtual switch Virtual switch Virtual switch
SERVERSERVERSERVER
Classical multi-tier application architecture
NFV-based multi-tier
application architecture
Virtual Firewall VM
Virtual Load Balancer VM
VXLAN
STORAGE SERVERS (DB)
SWITCH
®Ivan Pepelnjak
71. Software processing:
Network Function Virtualization (NFV)
• NFV advantages
• Flexibility to easily,
dynamically provision and
instantiate new services in
various locations (i.e. no
need for new equipment
install)
• More service differentiation &
customization
• Easy scalability
• Higher innovation cycle in
the networking
• Usage of software
methodology and tooling for
making networking
• NFV disadvantages
• Higher network latency
• Now NFV rather not possible for
network core
• Still dedicated network ASIC is much faster
than CPU
• Still unclear whether the NFV technology
will ever offer the performance
necessary to replace proprietary
hardware:
• Sometimes NFV is 50-times slower when
doing network intensive tasks (i.e.:
processing a lot of small network frames)
• Unclear also if and when it will be cheaper
72