SlideShare una empresa de Scribd logo
1 de 38
Descargar para leer sin conexión
0Copyright©2015 NTT corp. All Rights Reserved.
Lagopus:
SDN software switch
Yoshihiro Nakajima
NTT Labs
Feb 26th 2015
1Copyright©2015 NTT corp. All Rights Reserved.
What is Lagopus (雷鳥属)?
 Lagopus is a small genus of birds in
the grouse subfamily, commonly known as ptarmigans.
All living in tundra or cold upland areas.
Reference: http://en.wikipedia.org/wiki/Lagopus
2Copyright©2015 NTT corp. All Rights Reserved.
Agenda
1. Background & motivation
2. Lagopus SDN switch
I. vSwitch
II. 40GbE FPGA NIC
III.DPDK extension
IV. OSS community
3. What’s next?
3Copyright©2015 NTT corp. All Rights Reserved.
Trend shift in networking
 Closed (Vender lock-in)
 Yearly dev cycle
 Waterfall dev
 Standardization
 Protocol
 Special purpose HW / appliance
 Distributed cntrl
 Custom ASIC / FPGA
 Open (lock-in free)
 Monthly dev cycle
 Agile dev
 DE fact standard
 API
 Commodity HW/ Server
 Logically centralized cntrl
 Merchant Chip
4Copyright©2015 NTT corp. All Rights Reserved.
What is Software-Defined Networking?
4
Innovate services and applications
in software development speed.
Reference: http://opennetsummit.org/talks/ONS2012/pitt-mon-ons.pdf
Decouple control plane and data plane
Free control plane out of the box
(OpenFlow is one of APIs)
Logically centralized view
Hide and abstract complexity of networks,
provide entire view of the network
Programmability via abstraction layer
Enables flexible and rapid service/application
development
SDN conceptual model
5Copyright©2015 NTT corp. All Rights Reserved.
OpenFlow vs conventional NW node
OpenFlow controller
OpenFlow switch
Data-plane
Flow Table
Control plane
(Routing /
swithching)
Data-plane
(ASIC, FPGA)
Control plane
(routing/ switching)
Flow match action
Flow match action counter
OpenFlow Protocol
Flexible flow match pattern
(Port #, VLAN ID, MAC addr,
IP addr, TCP port #)
Action (frame processing)
(output port, drop,
modify/pop/push header)
Flow statistics
(# of packet, byte size,
flow duration,… )
counter
Conventional NW node OpenFlow
Flow
Table
#2
Flow
Table
#3
OpenFlow switch agent
6Copyright©2015 NTT corp. All Rights Reserved.
Why SDN?
6
Differentiate services (Innovation)
• Increase user experience
• Provide unique service which is not in the
market
Time-to-Market(Velocity)
•Not to depend on vendors’ feature roadmap
•Develop necessary feature when you need
Cost-efficiency
•Reduce OPEX by automated workflows
•Reduce CAPEX by COTS hardware
7Copyright©2015 NTT corp. All Rights Reserved.
7
Evaluate the benefits of SDN
by implementing
our control plane and switch
8Copyright©2015 NTT corp. All Rights Reserved.
Lagopus switch
• Lagopus overview
• Xilinx FPGA 40GbE NIC
• DPDK extension
• OSS Community
9Copyright©2015 NTT corp. All Rights Reserved.
Goal of Lagopus project
 Provide NFV/SDN-aware switch software stack
 OpenFlow switch agent
 100Gbps-capable software dataplane and I/O libs
 Extensible switch configuration data store
 Expand software-based packet processing to
carrier networks
 Hardware acceleration, processing offload
10Copyright©2015 NTT corp. All Rights Reserved.
# of packet to be proceeded for 10Gbps
with 1 CPU core
0
2,000,000
4,000,000
6,000,000
8,000,000
10,000,000
12,000,000
14,000,000
16,000,000
0 256 512 768 1024 1280
#ofpacketsperseconds
Packet size (Byte)
Short packet 64Byte
14.88 MPPS, 67.2 ns
• 2Ghz: 134 clocks
• 3Ghz: 201 clocks
Computer packet 1KByte
1.2MPPS, 835 ns
• 2Ghz: 1670 clocks
• 3Ghz: 2505 clocks
11Copyright©2015 NTT corp. All Rights Reserved.
# of CPU cycle for typical packet
processing (2Ghz CPU)
0 1000 2000 3000 4000 5000 6000
L2 forwarding
IP routing
L2-L4 classification
TCP termination
Simple OF processing
DPI
# of required CPU cycles
10Gbps 1Gbps
12Copyright©2015 NTT corp. All Rights Reserved.
Lagopus switch
13Copyright©2015 NTT corp. All Rights Reserved.
Lessons learned from existing vswitch
 Not enough performance 
 Packet processing speed < 1Gbps 
 10K flow entry add > two hours 
 Flow management processing is too heavy 
 Develop & extension difficulties 
 Many abstraction layer cause confuse for me 
• Interface abstraction, switch abstraction, protocol
abstraction Packet abstraction… 
 Many packet processing codes exists for the same
processing 
• Userspace, Kernelspace,…
 Invisible flow entries cause chaos debugging 
14Copyright©2015 NTT corp. All Rights Reserved.
vSwitch requirement from user side
 Commodity x86 PC server and NIC support
 Gateway function to connect different NW
domains (MPLS, Access NW)
 10Gbps-wire rate with >= 1M flow entries
 Flow-aware control
 Multi flow table configuration
 High performance flow entry operation
 Userland-based dataplane
 Easy software upgrade and deploy
 Strong management plane supports
 Co-operable upper management systems
15Copyright©2015 NTT corp. All Rights Reserved.
Approach for vSwitch development
 Simple is better for everything
 Packet processing for speed
 Protocol processing for implement
 Straight forward approach
 Full scratch (No use of existing vSwitch code)
 No OpenFlow 1.0 support
 User land packet processing as much as
possible
 Keep kernel code small
 Every component & algorithm can be replaced
 Flow lookup, library, protocol handling
16Copyright©2015 NTT corp. All Rights Reserved.
Lagopus switch design
 Simple modular-based design
 Unified configuration data store
 Multiple agent support
 Support multiple data-planes
 Dataplane abstraction layer
 Multi-threaded dataplane
 Flexible parallel pipeline
• Multi-core CPU-aware design
• Intel DPDK and lock-free libraries
for I/O
17Copyright©2015 NTT corp. All Rights Reserved.
high-performance user-space
packet processing with Intel DPDK
NIC
skb_buf
Ethernet Driver API
Socket API
vswitch
packet
buffer
Dataplane
1. Interrupt
& DMA
2. system call
(read)
User
space
Kernel
space
Driver
4. DMA
3. system call (write)
NIC
Ethernet Driver API
Socket API
vswitch
packet
buffer
agentagent
1. DMA
Write
2. DMA
READ
DPDK
dataplane
Userspace
packet processing
(Event-based)
DPDK apps
(polling-based)
18Copyright©2015 NTT corp. All Rights Reserved.
Packet processing using multi core CPUs
 Exploit many core CPUs
 Reduce data copy & move (reference access)
 Simple packet classifier for parallel processing in I/O RX
 Decouple I/O processing and flow processing
 Improve D-cache efficiency
 Explicit thread assign to CPU core
NIC 1
RX
NIC 2
RX
I/O RX
CPU0
I/O RX
CPU1
NIC 1
TX
NIC 2
TX
I/O TX
CPU6
I/O TX
CPU7
Flow lookup
packet processing
CPU2
Flow lookup
packet processing
CPU4
Flow lookup
packet processing
CPU3
Flow lookup
packet processing
CPU5
NIC 3
RX
NIC 4
RX
NIC 3
TX
NIC 4
TX
NIC RX buffer
Ring buffer
Ring buffer NIC TX buffer
19Copyright©2015 NTT corp. All Rights Reserved.
Implementation techniques for speed
 Reduce overhead
 Polling-based packet receiving
 Reduction of # of access in I/O and memory
 Huge DTLB for $ miss of memory controller
 Bypass in packet processing
 Direct data copy from NIC buffer to CPU $
 Kernel stack bypass of network
 Fast flow lookup and flow $ mechanism
 Reduce memory access and leverage locality
 Packet batching
 Novel flow lookup algorithm
 Thread local storage and lockless-queue, RCU
 Explicit CPU and thread assignment
 Hardware-topology aware resource assignment
20Copyright©2015 NTT corp. All Rights Reserved.
Performance Evaluation
 Summary
 Throughput: 10Gbps wire-rate
 Flow entries: > 1M flow entires
> 4000 flow entries mods /sec
 Evaluation models
 WAN-DC gateway
• MPLS-VLAN mapping
 L2 switch
• Mac address switching
21Copyright©2015 NTT corp. All Rights Reserved.
WAN-DC Gateway
0
1
2
3
4
5
6
7
8
9
10
0 200 400 600 800 1000 1200 1400 1600
Throughput(Gbps)
Packet size (byte)
10 flow rules
100 flow rules
1k flow rules
10k flow rules
100k flow rules
1M flow rules
Throughput vs packet size, 1 flow, flow-cache
0
1
2
3
4
5
6
7
8
9
10
1 10 100 1000 10000 100000 1000000
Throughput(Gbps)
flows
10k flow rules
100k flow rules
1M flow rules
Throughput vs flows, 1518 bytes packet
22Copyright©2015 NTT corp. All Rights Reserved.
L2 switch performance (Mbps)
10GbE x 2 (RFC2889 test)
0
1000
2000
3000
4000
5000
6000
7000
8000
9000
10000
LINC OVS
(netdev)
OVS
(kernel)
Lagopus
(software)
Mbps
72
128
256
512
1024
1280
1518
Intel xeon E5-2660 (8cores, 16threads),
Intel X520-DA2, DDR3-1600 64GB
Packet size
23Copyright©2015 NTT corp. All Rights Reserved.
OpenFlow 1.3 Conformance Status
Type Action Set field Match Group Meter Total
# of test scenario
(mandatory,
optional)
56
(3 , 53)
161
(0 , 161)
714
(108 , 606)
15
(3 , 12)
36
(0 , 36)
991
(114 , 877)
Lagopus
2014.11.09
56
(3, 56)
161
(0, 161)
714
(108, 606)
15
(3, 12)
34
(0, 34)
980
(114, 866)
OVS (kernel)
2014.08.08
34
(3, 31)
96
(0, 96)
534
(108, 426)
6
(3, 3)
0
(0, 0)
670
(114, 556)
OVS (netdev)
2014.11.05
34
(3, 31)
102
(0, 102)
467
(93, 374)
8
(3, 5)
0
(0, 0)
611
(99, 556)
IVS
2015.02.11
17
(3, 14)
46
(0, 46)
323
(108, 229)
3
(0, 2)
0
(0, 0)
402
(111, 291)
ofswitch
2015.01.08
50
(3, 47)
100
(0, 100)
708
(108, 600)
15
(3, 12)
30
(0, 30)
962
(114, 848)
LINC
2015.01.29
24
(3, 21)
68
(0, 68)
428
(108, 320)
3
(3, 0)
4
(0, 4)
523
(114, 409)
Trema
2014.11.28
50
(3, 47)
159
(0 , 159)
708
(108, 600)
15
(3, 12)
34
(0, 34)
966
(114, 854)
http://osrg.github.io/ryu/certification.html
24Copyright©2015 NTT corp. All Rights Reserved.
40GbE FPGA NIC
25Copyright©2015 NTT corp. All Rights Reserved.
Lessoned learned from development
 vSwitch needs hardware acceleration for NFV
 Focus CPU cores for upper-layer packet processing
 Encryption, Packet scheduler are CPU heavy processing
 vSwitch has several improvement points
 vSwitch cannot leverage multicore CPU power
 Software-based packet scheduler is too heavy
 vSwitch cannot protect from ingress DoS attack
Performance
Flexiblity Availability
Existing SDN
hardware switch
+ HW
accelerator
High-performance & scalability with HW
advance features with HW
26Copyright©2015 NTT corp. All Rights Reserved.
Lagopus FPGA NIC
HW
SW
x86 (Xeon) + DPDK + Frontend API
VNF1@VM
QSFP+
QSFP+
40G TRX
[LOOK-UP]
40G TRX
vSwitch
VNF2@VM
PCIe Gen3 x8 PCIe Gen3 x8
VNF3@VM
Inter-DC (WAN)
Intra-DC (East-West)
[SR-IOV DEMUX]
[LABELING]
[PARSE]
[TAP-CTRL]
・・・
QoS
[TM]
FPGA
[DISPATCH]
DMA Queues x32 (duplex)
Frontend
Xilinx 40G/80G QoS NIC (XNIC)
IA server@DC
TrafficEscalation
Flow Director function for many core
Packet Mirroring function(NFV tap)
27Copyright©2015 NTT corp. All Rights Reserved.
Throughput
0.00
5.00
10.00
15.00
20.00
25.00
30.00
35.00
40.00
64 68 256 512 1024 1512
Throughput(Gpbs)
Frame Size (Bytes)
Throughput on Each Frame Size
Xeon(R) E5-2680 v2 @ 2.80GHz, 10C/20T, DPDK-1.7.0
(1) RX1, TX1, W2
(2) RX1, TX1, W6
(3) RX1, TX1, W2
(4) RX1, TX1, W6
(5) RX2, TX1, W5
Frame Size
(Bytes)
Line Rate
40G (Gbps)
Measured Results of
Xilinx NIC and Lagopus Interwork
RX --> TX Throughput (Gbps)
Flow Director = OFF Flow Director = ON
Core assignment (1) RX1, TX1, W2 (2) RX1, TX1, W6 (3) RX1, TX1, W2 (4) RX1, TX1, W6 (5) RX2, TX1, W5
64 30.47 2.05 4.96 4.20 5.00 8.15
68 30.90 2.17 5.62 4.41 5.69 8.53
256 37.10 8.18 21.68 16.70 21.88 30.32
512 38.49 16.26 38.46 33.50 38.46 38.47
1024 39.23 32.28 39.23 39.23 39.23 39.23
1512 39.47 39.46 39.47 39.46 39.47 39.46
28Copyright©2015 NTT corp. All Rights Reserved.
High performance virtual NIC
for DPDK-enabled NFV apps
29Copyright©2015 NTT corp. All Rights Reserved.
Concept
 DPDK is good for performance, but not for operation and
management
 Performance acceleration for DPDK apps and legacy apps
 Simultaneous restart support of DPDK apps and DPDK
vswitch
 Isolation for security and trust for guest VM and vswitch
 Trusted guest VM: zero-copy
 Non-trusted guest VM: one-copy
Guest1
QEMU
App
DPDK
Guest2
QEMU
App
DPDK
Virtio-net PMD Virtio-net PMD
Lagopus
vswitch
DPDK
PMD
Map memory in guest VM
to lagopus memory
Map memory in guest VM
to lagopus memory
virtio virtio
virtio
queue
virtio
queue
PMD
vNIC
PMD
vNIC
30Copyright©2015 NTT corp. All Rights Reserved.
DPDK VNIC extension (virtq-pmd)
 4.8 times faster than existing vhost pmd
 Support of simultaneous restart of VM gust and
host DPDK apps
0
500
1000
1500
2000
2500
3000
3500
0 500 1000 1500 2000
Required#ofCPUClocks
Packet size (bytes)
Transmission overhead per
packet
copy overhead hole PMD
virtq PMD vhost app
Kernel driver
0.00
10.00
20.00
30.00
40.00
50.00
60.00
70.00
0 500 1000 1500 2000
Gbps
COPY OFF COPY ON
Released as OSS on github
31Copyright©2015 NTT corp. All Rights Reserved.
Expand SDN OSS community
32Copyright©2015 NTT corp. All Rights Reserved.
Activity
 Hands-on in Tokyo & Taiwan
 Total over 400 engineers enjoyed
 United states?
Tokyo hands-on Taiwan hands-on
33Copyright©2015 NTT corp. All Rights Reserved.
PoC in SDN Japan 2014
 Location-aware
bandwidth-controlled
Internet WiFi with Ryu
and Lagopus
 Good audience can
connect to the Internet
with guaranteed BW
 Front area sheet, good
connection.
 Back area sheet, poor
connection
34Copyright©2015 NTT corp. All Rights Reserved.
What’s next?
35Copyright©2015 NTT corp. All Rights Reserved.
M-plane needs innovation!
 No sufficient abstraction in M-plane
 Proprietary CLI, script
 Invisibility leads to inefficiency
 Still legacy (SNMP is neither SIMPLE nor realtime)
plan
Provision
& control
monitor
Analyze
&
visualize
Plan NW based on
NW abstraction model
Provisioning on
NW abstraction model
Feedback to
NW abstraction model
Think NW based-on
NW abstraction model
36Copyright©2015 NTT corp. All Rights Reserved.
Y2013 Y2014 Y2015
Control plane
Management
plane
Data plane
Lagopus development
OF1.3 agent
High-performance
software dataplane
with DPDK
(static pipeline
configuration)
Switch config/mngmt
Db prototype
OF-CONFIG,
CLI, SNMP
prototypes
Flexible reconfigurable
pipeline in dataplane
Tunnel protocol
(VxLAN, GRE,
MPLS pseudo-wire)
40Gbps-capable
Soft dataplane
Extensible switch resource
Management/configuration
datastore
(CLI, OF-CONFIG, SNMP)
OF1.5 agent
Legacy protocol ctrl support
M-plane extension
(Openconfig)
37Copyright©2015 NTT corp. All Rights Reserved.
Thanks!
Web: http://lagopus.github.io/
Twitter: lagopusvswitch

Más contenido relacionado

La actualidad más candente

DPDK Summit 2015 - NTT - Yoshihiro Nakajima
DPDK Summit 2015 - NTT - Yoshihiro NakajimaDPDK Summit 2015 - NTT - Yoshihiro Nakajima
DPDK Summit 2015 - NTT - Yoshihiro NakajimaJim St. Leger
 
High Performance Networking Leveraging the DPDK and Growing Community
High Performance Networking Leveraging the DPDK and Growing CommunityHigh Performance Networking Leveraging the DPDK and Growing Community
High Performance Networking Leveraging the DPDK and Growing Community6WIND
 
DPDK in Containers Hands-on Lab
DPDK in Containers Hands-on LabDPDK in Containers Hands-on Lab
DPDK in Containers Hands-on LabMichelle Holley
 
DPDK Summit - 08 Sept 2014 - 6WIND - High Perf Networking Leveraging the DPDK...
DPDK Summit - 08 Sept 2014 - 6WIND - High Perf Networking Leveraging the DPDK...DPDK Summit - 08 Sept 2014 - 6WIND - High Perf Networking Leveraging the DPDK...
DPDK Summit - 08 Sept 2014 - 6WIND - High Perf Networking Leveraging the DPDK...Jim St. Leger
 
Recent advance in netmap/VALE(mSwitch)
Recent advance in netmap/VALE(mSwitch)Recent advance in netmap/VALE(mSwitch)
Recent advance in netmap/VALE(mSwitch)micchie
 
OVS and DPDK - T.F. Herbert, K. Traynor, M. Gray
OVS and DPDK - T.F. Herbert, K. Traynor, M. GrayOVS and DPDK - T.F. Herbert, K. Traynor, M. Gray
OVS and DPDK - T.F. Herbert, K. Traynor, M. Grayharryvanhaaren
 
Enable DPDK and SR-IOV for containerized virtual network functions with zun
Enable DPDK and SR-IOV for containerized virtual network functions with zunEnable DPDK and SR-IOV for containerized virtual network functions with zun
Enable DPDK and SR-IOV for containerized virtual network functions with zunheut2008
 
Introduction to DPDK
Introduction to DPDKIntroduction to DPDK
Introduction to DPDKKernel TLV
 
FD.io Vector Packet Processing (VPP)
FD.io Vector Packet Processing (VPP)FD.io Vector Packet Processing (VPP)
FD.io Vector Packet Processing (VPP)Kirill Tsym
 
The n00bs guide to ovs dpdk
The n00bs guide to ovs dpdkThe n00bs guide to ovs dpdk
The n00bs guide to ovs dpdkmarkdgray
 
DPDK Summit 2015 - Aspera - Charles Shiflett
DPDK Summit 2015 - Aspera - Charles ShiflettDPDK Summit 2015 - Aspera - Charles Shiflett
DPDK Summit 2015 - Aspera - Charles ShiflettJim St. Leger
 
Packet Framework - Cristian Dumitrescu
Packet Framework - Cristian DumitrescuPacket Framework - Cristian Dumitrescu
Packet Framework - Cristian Dumitrescuharryvanhaaren
 
Performance challenges in software networking
Performance challenges in software networkingPerformance challenges in software networking
Performance challenges in software networkingStephen Hemminger
 
Accelerating Neutron with Intel DPDK
Accelerating Neutron with Intel DPDKAccelerating Neutron with Intel DPDK
Accelerating Neutron with Intel DPDKAlexander Shalimov
 

La actualidad más candente (20)

DPDK Summit 2015 - NTT - Yoshihiro Nakajima
DPDK Summit 2015 - NTT - Yoshihiro NakajimaDPDK Summit 2015 - NTT - Yoshihiro Nakajima
DPDK Summit 2015 - NTT - Yoshihiro Nakajima
 
High Performance Networking Leveraging the DPDK and Growing Community
High Performance Networking Leveraging the DPDK and Growing CommunityHigh Performance Networking Leveraging the DPDK and Growing Community
High Performance Networking Leveraging the DPDK and Growing Community
 
DPDK in Containers Hands-on Lab
DPDK in Containers Hands-on LabDPDK in Containers Hands-on Lab
DPDK in Containers Hands-on Lab
 
DPDK Summit - 08 Sept 2014 - 6WIND - High Perf Networking Leveraging the DPDK...
DPDK Summit - 08 Sept 2014 - 6WIND - High Perf Networking Leveraging the DPDK...DPDK Summit - 08 Sept 2014 - 6WIND - High Perf Networking Leveraging the DPDK...
DPDK Summit - 08 Sept 2014 - 6WIND - High Perf Networking Leveraging the DPDK...
 
100 M pps on PC.
100 M pps on PC.100 M pps on PC.
100 M pps on PC.
 
Dpdk performance
Dpdk performanceDpdk performance
Dpdk performance
 
Recent advance in netmap/VALE(mSwitch)
Recent advance in netmap/VALE(mSwitch)Recent advance in netmap/VALE(mSwitch)
Recent advance in netmap/VALE(mSwitch)
 
Understanding DPDK
Understanding DPDKUnderstanding DPDK
Understanding DPDK
 
OVS and DPDK - T.F. Herbert, K. Traynor, M. Gray
OVS and DPDK - T.F. Herbert, K. Traynor, M. GrayOVS and DPDK - T.F. Herbert, K. Traynor, M. Gray
OVS and DPDK - T.F. Herbert, K. Traynor, M. Gray
 
Userspace networking
Userspace networkingUserspace networking
Userspace networking
 
Enable DPDK and SR-IOV for containerized virtual network functions with zun
Enable DPDK and SR-IOV for containerized virtual network functions with zunEnable DPDK and SR-IOV for containerized virtual network functions with zun
Enable DPDK and SR-IOV for containerized virtual network functions with zun
 
Introduction to DPDK
Introduction to DPDKIntroduction to DPDK
Introduction to DPDK
 
FD.io Vector Packet Processing (VPP)
FD.io Vector Packet Processing (VPP)FD.io Vector Packet Processing (VPP)
FD.io Vector Packet Processing (VPP)
 
The n00bs guide to ovs dpdk
The n00bs guide to ovs dpdkThe n00bs guide to ovs dpdk
The n00bs guide to ovs dpdk
 
DPDK Summit 2015 - Aspera - Charles Shiflett
DPDK Summit 2015 - Aspera - Charles ShiflettDPDK Summit 2015 - Aspera - Charles Shiflett
DPDK Summit 2015 - Aspera - Charles Shiflett
 
DPDK In Depth
DPDK In DepthDPDK In Depth
DPDK In Depth
 
Packet Framework - Cristian Dumitrescu
Packet Framework - Cristian DumitrescuPacket Framework - Cristian Dumitrescu
Packet Framework - Cristian Dumitrescu
 
DPDK KNI interface
DPDK KNI interfaceDPDK KNI interface
DPDK KNI interface
 
Performance challenges in software networking
Performance challenges in software networkingPerformance challenges in software networking
Performance challenges in software networking
 
Accelerating Neutron with Intel DPDK
Accelerating Neutron with Intel DPDKAccelerating Neutron with Intel DPDK
Accelerating Neutron with Intel DPDK
 

Similar a Lagopus presentation on 14th Annual ON*VECTOR International Photonics Workshop

Software Stacks to enable SDN and NFV
Software Stacks to enable SDN and NFVSoftware Stacks to enable SDN and NFV
Software Stacks to enable SDN and NFVYoshihiro Nakajima
 
Approaching hyperconvergedopenstack
Approaching hyperconvergedopenstackApproaching hyperconvergedopenstack
Approaching hyperconvergedopenstackIkuo Kumagai
 
NFV Infrastructure Manager with High Performance Software Switch Lagopus
NFV Infrastructure Manager with High Performance Software Switch Lagopus NFV Infrastructure Manager with High Performance Software Switch Lagopus
NFV Infrastructure Manager with High Performance Software Switch Lagopus Hirofumi Ichihara
 
Ceph Day Berlin: Deploying Flash Storage for Ceph without Compromising Perfor...
Ceph Day Berlin: Deploying Flash Storage for Ceph without Compromising Perfor...Ceph Day Berlin: Deploying Flash Storage for Ceph without Compromising Perfor...
Ceph Day Berlin: Deploying Flash Storage for Ceph without Compromising Perfor...Ceph Community
 
Ryu SDN Framework
Ryu SDN FrameworkRyu SDN Framework
Ryu SDN FrameworkAPNIC
 
Krzysztof Mazepa - Netflow/cflow - ulubionym narzędziem operatorów SP
Krzysztof Mazepa - Netflow/cflow - ulubionym narzędziem operatorów SPKrzysztof Mazepa - Netflow/cflow - ulubionym narzędziem operatorów SP
Krzysztof Mazepa - Netflow/cflow - ulubionym narzędziem operatorów SPPROIDEA
 
Advanced Networking: The Critical Path for HPC, Cloud, Machine Learning and more
Advanced Networking: The Critical Path for HPC, Cloud, Machine Learning and moreAdvanced Networking: The Critical Path for HPC, Cloud, Machine Learning and more
Advanced Networking: The Critical Path for HPC, Cloud, Machine Learning and moreinside-BigData.com
 
Ceph Day Seoul - AFCeph: SKT Scale Out Storage Ceph
Ceph Day Seoul - AFCeph: SKT Scale Out Storage Ceph Ceph Day Seoul - AFCeph: SKT Scale Out Storage Ceph
Ceph Day Seoul - AFCeph: SKT Scale Out Storage Ceph Ceph Community
 
Inside Microsoft's FPGA-Based Configurable Cloud
Inside Microsoft's FPGA-Based Configurable CloudInside Microsoft's FPGA-Based Configurable Cloud
Inside Microsoft's FPGA-Based Configurable Cloudinside-BigData.com
 
FD.IO Vector Packet Processing
FD.IO Vector Packet ProcessingFD.IO Vector Packet Processing
FD.IO Vector Packet ProcessingKernel TLV
 
Dataplane networking acceleration with OpenDataplane / Максим Уваров (Linaro)
Dataplane networking acceleration with OpenDataplane / Максим Уваров (Linaro)Dataplane networking acceleration with OpenDataplane / Максим Уваров (Linaro)
Dataplane networking acceleration with OpenDataplane / Максим Уваров (Linaro)Ontico
 
G rpc talk with intel (3)
G rpc talk with intel (3)G rpc talk with intel (3)
G rpc talk with intel (3)Intel
 
Practical Guide to Run an IEEE 802.15.4 Network with 6LoWPAN Under Linux
Practical Guide to Run an IEEE 802.15.4 Network with 6LoWPAN Under LinuxPractical Guide to Run an IEEE 802.15.4 Network with 6LoWPAN Under Linux
Practical Guide to Run an IEEE 802.15.4 Network with 6LoWPAN Under LinuxSamsung Open Source Group
 
FOSDEM 2017 Trip Report
FOSDEM 2017 Trip ReportFOSDEM 2017 Trip Report
FOSDEM 2017 Trip ReportOCaml Labs
 
2014/09/02 Cisco UCS HPC @ ANL
2014/09/02 Cisco UCS HPC @ ANL2014/09/02 Cisco UCS HPC @ ANL
2014/09/02 Cisco UCS HPC @ ANLdgoodell
 
SDN/OpenFlow #lspe
SDN/OpenFlow #lspeSDN/OpenFlow #lspe
SDN/OpenFlow #lspeChris Westin
 
100G Networking Berlin.pdf
100G Networking Berlin.pdf100G Networking Berlin.pdf
100G Networking Berlin.pdfJunZhao68
 
Adding IEEE 802.15.4 and 6LoWPAN to an Embedded Linux Device
Adding IEEE 802.15.4 and 6LoWPAN to an Embedded Linux DeviceAdding IEEE 802.15.4 and 6LoWPAN to an Embedded Linux Device
Adding IEEE 802.15.4 and 6LoWPAN to an Embedded Linux DeviceSamsung Open Source Group
 

Similar a Lagopus presentation on 14th Annual ON*VECTOR International Photonics Workshop (20)

Software Stacks to enable SDN and NFV
Software Stacks to enable SDN and NFVSoftware Stacks to enable SDN and NFV
Software Stacks to enable SDN and NFV
 
Approaching hyperconvergedopenstack
Approaching hyperconvergedopenstackApproaching hyperconvergedopenstack
Approaching hyperconvergedopenstack
 
NFV Infrastructure Manager with High Performance Software Switch Lagopus
NFV Infrastructure Manager with High Performance Software Switch Lagopus NFV Infrastructure Manager with High Performance Software Switch Lagopus
NFV Infrastructure Manager with High Performance Software Switch Lagopus
 
Ceph Day Berlin: Deploying Flash Storage for Ceph without Compromising Perfor...
Ceph Day Berlin: Deploying Flash Storage for Ceph without Compromising Perfor...Ceph Day Berlin: Deploying Flash Storage for Ceph without Compromising Perfor...
Ceph Day Berlin: Deploying Flash Storage for Ceph without Compromising Perfor...
 
OpenFlow Tutorial
OpenFlow TutorialOpenFlow Tutorial
OpenFlow Tutorial
 
Ryu SDN Framework
Ryu SDN FrameworkRyu SDN Framework
Ryu SDN Framework
 
Krzysztof Mazepa - Netflow/cflow - ulubionym narzędziem operatorów SP
Krzysztof Mazepa - Netflow/cflow - ulubionym narzędziem operatorów SPKrzysztof Mazepa - Netflow/cflow - ulubionym narzędziem operatorów SP
Krzysztof Mazepa - Netflow/cflow - ulubionym narzędziem operatorów SP
 
Advanced Networking: The Critical Path for HPC, Cloud, Machine Learning and more
Advanced Networking: The Critical Path for HPC, Cloud, Machine Learning and moreAdvanced Networking: The Critical Path for HPC, Cloud, Machine Learning and more
Advanced Networking: The Critical Path for HPC, Cloud, Machine Learning and more
 
Ceph Day Seoul - AFCeph: SKT Scale Out Storage Ceph
Ceph Day Seoul - AFCeph: SKT Scale Out Storage Ceph Ceph Day Seoul - AFCeph: SKT Scale Out Storage Ceph
Ceph Day Seoul - AFCeph: SKT Scale Out Storage Ceph
 
Inside Microsoft's FPGA-Based Configurable Cloud
Inside Microsoft's FPGA-Based Configurable CloudInside Microsoft's FPGA-Based Configurable Cloud
Inside Microsoft's FPGA-Based Configurable Cloud
 
FD.IO Vector Packet Processing
FD.IO Vector Packet ProcessingFD.IO Vector Packet Processing
FD.IO Vector Packet Processing
 
Dataplane networking acceleration with OpenDataplane / Максим Уваров (Linaro)
Dataplane networking acceleration with OpenDataplane / Максим Уваров (Linaro)Dataplane networking acceleration with OpenDataplane / Максим Уваров (Linaro)
Dataplane networking acceleration with OpenDataplane / Максим Уваров (Linaro)
 
G rpc talk with intel (3)
G rpc talk with intel (3)G rpc talk with intel (3)
G rpc talk with intel (3)
 
Mellanox Approach to NFV & SDN
Mellanox Approach to NFV & SDNMellanox Approach to NFV & SDN
Mellanox Approach to NFV & SDN
 
Practical Guide to Run an IEEE 802.15.4 Network with 6LoWPAN Under Linux
Practical Guide to Run an IEEE 802.15.4 Network with 6LoWPAN Under LinuxPractical Guide to Run an IEEE 802.15.4 Network with 6LoWPAN Under Linux
Practical Guide to Run an IEEE 802.15.4 Network with 6LoWPAN Under Linux
 
FOSDEM 2017 Trip Report
FOSDEM 2017 Trip ReportFOSDEM 2017 Trip Report
FOSDEM 2017 Trip Report
 
2014/09/02 Cisco UCS HPC @ ANL
2014/09/02 Cisco UCS HPC @ ANL2014/09/02 Cisco UCS HPC @ ANL
2014/09/02 Cisco UCS HPC @ ANL
 
SDN/OpenFlow #lspe
SDN/OpenFlow #lspeSDN/OpenFlow #lspe
SDN/OpenFlow #lspe
 
100G Networking Berlin.pdf
100G Networking Berlin.pdf100G Networking Berlin.pdf
100G Networking Berlin.pdf
 
Adding IEEE 802.15.4 and 6LoWPAN to an Embedded Linux Device
Adding IEEE 802.15.4 and 6LoWPAN to an Embedded Linux DeviceAdding IEEE 802.15.4 and 6LoWPAN to an Embedded Linux Device
Adding IEEE 802.15.4 and 6LoWPAN to an Embedded Linux Device
 

Último

BATbern52 Swisscom's Journey into Data Mesh
BATbern52 Swisscom's Journey into Data MeshBATbern52 Swisscom's Journey into Data Mesh
BATbern52 Swisscom's Journey into Data MeshBATbern
 
CYBER SECURITY AND CYBER CRIME COMPLETE GUIDE.pLptx
CYBER SECURITY AND CYBER CRIME COMPLETE GUIDE.pLptxCYBER SECURITY AND CYBER CRIME COMPLETE GUIDE.pLptx
CYBER SECURITY AND CYBER CRIME COMPLETE GUIDE.pLptxBarakaMuyengi
 
Einstein Copilot Conversational AI for your CRM.pdf
Einstein Copilot Conversational AI for your CRM.pdfEinstein Copilot Conversational AI for your CRM.pdf
Einstein Copilot Conversational AI for your CRM.pdfCloudMetic
 
renewable energy renewable energy renewable energy renewable energy
renewable energy renewable energy renewable energy  renewable energyrenewable energy renewable energy renewable energy  renewable energy
renewable energy renewable energy renewable energy renewable energyjeyasrig
 
Steps to Successfully Hire Ionic Developers
Steps to Successfully Hire Ionic DevelopersSteps to Successfully Hire Ionic Developers
Steps to Successfully Hire Ionic Developersmichealwillson701
 
Large Scale Architecture -- The Unreasonable Effectiveness of Simplicity
Large Scale Architecture -- The Unreasonable Effectiveness of SimplicityLarge Scale Architecture -- The Unreasonable Effectiveness of Simplicity
Large Scale Architecture -- The Unreasonable Effectiveness of SimplicityRandy Shoup
 
Take Advantage of Mx Tracking Flight Scheduling Solutions to Streamline Your ...
Take Advantage of Mx Tracking Flight Scheduling Solutions to Streamline Your ...Take Advantage of Mx Tracking Flight Scheduling Solutions to Streamline Your ...
Take Advantage of Mx Tracking Flight Scheduling Solutions to Streamline Your ...MyFAA
 
Mobile App Development company Houston
Mobile  App  Development  company HoustonMobile  App  Development  company Houston
Mobile App Development company Houstonjennysmithusa549
 
Practical Advice for FDA’s 510(k) Requirements.pdf
Practical Advice for FDA’s 510(k) Requirements.pdfPractical Advice for FDA’s 510(k) Requirements.pdf
Practical Advice for FDA’s 510(k) Requirements.pdfICS
 
Mobile App Development process | Expert Tips
Mobile App Development process | Expert TipsMobile App Development process | Expert Tips
Mobile App Development process | Expert Tipsmichealwillson701
 
Revolutionize Your Field Service Management with FSM Grid
Revolutionize Your Field Service Management with FSM GridRevolutionize Your Field Service Management with FSM Grid
Revolutionize Your Field Service Management with FSM GridMathew Thomas
 
Enterprise Content Managements Solutions
Enterprise Content Managements SolutionsEnterprise Content Managements Solutions
Enterprise Content Managements SolutionsIQBG inc
 
MinionLabs_Mr. Gokul Srinivas_Young Entrepreneur
MinionLabs_Mr. Gokul Srinivas_Young EntrepreneurMinionLabs_Mr. Gokul Srinivas_Young Entrepreneur
MinionLabs_Mr. Gokul Srinivas_Young EntrepreneurPriyadarshini T
 
If your code could speak, what would it tell you? Let GitHub Copilot Chat hel...
If your code could speak, what would it tell you? Let GitHub Copilot Chat hel...If your code could speak, what would it tell you? Let GitHub Copilot Chat hel...
If your code could speak, what would it tell you? Let GitHub Copilot Chat hel...Maxim Salnikov
 
8 Steps to Build a LangChain RAG Chatbot.
8 Steps to Build a LangChain RAG Chatbot.8 Steps to Build a LangChain RAG Chatbot.
8 Steps to Build a LangChain RAG Chatbot.Ritesh Kanjee
 
Building Generative AI-infused apps: what's possible and how to start
Building Generative AI-infused apps: what's possible and how to startBuilding Generative AI-infused apps: what's possible and how to start
Building Generative AI-infused apps: what's possible and how to startMaxim Salnikov
 
Leveling Up your Branding and Mastering MERN: Fullstack WebDev
Leveling Up your Branding and Mastering MERN: Fullstack WebDevLeveling Up your Branding and Mastering MERN: Fullstack WebDev
Leveling Up your Branding and Mastering MERN: Fullstack WebDevpmgdscunsri
 
VuNet software organisation powerpoint deck
VuNet software organisation powerpoint deckVuNet software organisation powerpoint deck
VuNet software organisation powerpoint deckNaval Singh
 
BusinessGPT - SECURITY AND GOVERNANCE FOR GENERATIVE AI.pptx
BusinessGPT  - SECURITY AND GOVERNANCE  FOR GENERATIVE AI.pptxBusinessGPT  - SECURITY AND GOVERNANCE  FOR GENERATIVE AI.pptx
BusinessGPT - SECURITY AND GOVERNANCE FOR GENERATIVE AI.pptxAGATSoftware
 
Flutter the Future of Mobile App Development - 5 Crucial Reasons.pdf
Flutter the Future of Mobile App Development - 5 Crucial Reasons.pdfFlutter the Future of Mobile App Development - 5 Crucial Reasons.pdf
Flutter the Future of Mobile App Development - 5 Crucial Reasons.pdfMind IT Systems
 

Último (20)

BATbern52 Swisscom's Journey into Data Mesh
BATbern52 Swisscom's Journey into Data MeshBATbern52 Swisscom's Journey into Data Mesh
BATbern52 Swisscom's Journey into Data Mesh
 
CYBER SECURITY AND CYBER CRIME COMPLETE GUIDE.pLptx
CYBER SECURITY AND CYBER CRIME COMPLETE GUIDE.pLptxCYBER SECURITY AND CYBER CRIME COMPLETE GUIDE.pLptx
CYBER SECURITY AND CYBER CRIME COMPLETE GUIDE.pLptx
 
Einstein Copilot Conversational AI for your CRM.pdf
Einstein Copilot Conversational AI for your CRM.pdfEinstein Copilot Conversational AI for your CRM.pdf
Einstein Copilot Conversational AI for your CRM.pdf
 
renewable energy renewable energy renewable energy renewable energy
renewable energy renewable energy renewable energy  renewable energyrenewable energy renewable energy renewable energy  renewable energy
renewable energy renewable energy renewable energy renewable energy
 
Steps to Successfully Hire Ionic Developers
Steps to Successfully Hire Ionic DevelopersSteps to Successfully Hire Ionic Developers
Steps to Successfully Hire Ionic Developers
 
Large Scale Architecture -- The Unreasonable Effectiveness of Simplicity
Large Scale Architecture -- The Unreasonable Effectiveness of SimplicityLarge Scale Architecture -- The Unreasonable Effectiveness of Simplicity
Large Scale Architecture -- The Unreasonable Effectiveness of Simplicity
 
Take Advantage of Mx Tracking Flight Scheduling Solutions to Streamline Your ...
Take Advantage of Mx Tracking Flight Scheduling Solutions to Streamline Your ...Take Advantage of Mx Tracking Flight Scheduling Solutions to Streamline Your ...
Take Advantage of Mx Tracking Flight Scheduling Solutions to Streamline Your ...
 
Mobile App Development company Houston
Mobile  App  Development  company HoustonMobile  App  Development  company Houston
Mobile App Development company Houston
 
Practical Advice for FDA’s 510(k) Requirements.pdf
Practical Advice for FDA’s 510(k) Requirements.pdfPractical Advice for FDA’s 510(k) Requirements.pdf
Practical Advice for FDA’s 510(k) Requirements.pdf
 
Mobile App Development process | Expert Tips
Mobile App Development process | Expert TipsMobile App Development process | Expert Tips
Mobile App Development process | Expert Tips
 
Revolutionize Your Field Service Management with FSM Grid
Revolutionize Your Field Service Management with FSM GridRevolutionize Your Field Service Management with FSM Grid
Revolutionize Your Field Service Management with FSM Grid
 
Enterprise Content Managements Solutions
Enterprise Content Managements SolutionsEnterprise Content Managements Solutions
Enterprise Content Managements Solutions
 
MinionLabs_Mr. Gokul Srinivas_Young Entrepreneur
MinionLabs_Mr. Gokul Srinivas_Young EntrepreneurMinionLabs_Mr. Gokul Srinivas_Young Entrepreneur
MinionLabs_Mr. Gokul Srinivas_Young Entrepreneur
 
If your code could speak, what would it tell you? Let GitHub Copilot Chat hel...
If your code could speak, what would it tell you? Let GitHub Copilot Chat hel...If your code could speak, what would it tell you? Let GitHub Copilot Chat hel...
If your code could speak, what would it tell you? Let GitHub Copilot Chat hel...
 
8 Steps to Build a LangChain RAG Chatbot.
8 Steps to Build a LangChain RAG Chatbot.8 Steps to Build a LangChain RAG Chatbot.
8 Steps to Build a LangChain RAG Chatbot.
 
Building Generative AI-infused apps: what's possible and how to start
Building Generative AI-infused apps: what's possible and how to startBuilding Generative AI-infused apps: what's possible and how to start
Building Generative AI-infused apps: what's possible and how to start
 
Leveling Up your Branding and Mastering MERN: Fullstack WebDev
Leveling Up your Branding and Mastering MERN: Fullstack WebDevLeveling Up your Branding and Mastering MERN: Fullstack WebDev
Leveling Up your Branding and Mastering MERN: Fullstack WebDev
 
VuNet software organisation powerpoint deck
VuNet software organisation powerpoint deckVuNet software organisation powerpoint deck
VuNet software organisation powerpoint deck
 
BusinessGPT - SECURITY AND GOVERNANCE FOR GENERATIVE AI.pptx
BusinessGPT  - SECURITY AND GOVERNANCE  FOR GENERATIVE AI.pptxBusinessGPT  - SECURITY AND GOVERNANCE  FOR GENERATIVE AI.pptx
BusinessGPT - SECURITY AND GOVERNANCE FOR GENERATIVE AI.pptx
 
Flutter the Future of Mobile App Development - 5 Crucial Reasons.pdf
Flutter the Future of Mobile App Development - 5 Crucial Reasons.pdfFlutter the Future of Mobile App Development - 5 Crucial Reasons.pdf
Flutter the Future of Mobile App Development - 5 Crucial Reasons.pdf
 

Lagopus presentation on 14th Annual ON*VECTOR International Photonics Workshop

  • 1. 0Copyright©2015 NTT corp. All Rights Reserved. Lagopus: SDN software switch Yoshihiro Nakajima NTT Labs Feb 26th 2015
  • 2. 1Copyright©2015 NTT corp. All Rights Reserved. What is Lagopus (雷鳥属)?  Lagopus is a small genus of birds in the grouse subfamily, commonly known as ptarmigans. All living in tundra or cold upland areas. Reference: http://en.wikipedia.org/wiki/Lagopus
  • 3. 2Copyright©2015 NTT corp. All Rights Reserved. Agenda 1. Background & motivation 2. Lagopus SDN switch I. vSwitch II. 40GbE FPGA NIC III.DPDK extension IV. OSS community 3. What’s next?
  • 4. 3Copyright©2015 NTT corp. All Rights Reserved. Trend shift in networking  Closed (Vender lock-in)  Yearly dev cycle  Waterfall dev  Standardization  Protocol  Special purpose HW / appliance  Distributed cntrl  Custom ASIC / FPGA  Open (lock-in free)  Monthly dev cycle  Agile dev  DE fact standard  API  Commodity HW/ Server  Logically centralized cntrl  Merchant Chip
  • 5. 4Copyright©2015 NTT corp. All Rights Reserved. What is Software-Defined Networking? 4 Innovate services and applications in software development speed. Reference: http://opennetsummit.org/talks/ONS2012/pitt-mon-ons.pdf Decouple control plane and data plane Free control plane out of the box (OpenFlow is one of APIs) Logically centralized view Hide and abstract complexity of networks, provide entire view of the network Programmability via abstraction layer Enables flexible and rapid service/application development SDN conceptual model
  • 6. 5Copyright©2015 NTT corp. All Rights Reserved. OpenFlow vs conventional NW node OpenFlow controller OpenFlow switch Data-plane Flow Table Control plane (Routing / swithching) Data-plane (ASIC, FPGA) Control plane (routing/ switching) Flow match action Flow match action counter OpenFlow Protocol Flexible flow match pattern (Port #, VLAN ID, MAC addr, IP addr, TCP port #) Action (frame processing) (output port, drop, modify/pop/push header) Flow statistics (# of packet, byte size, flow duration,… ) counter Conventional NW node OpenFlow Flow Table #2 Flow Table #3 OpenFlow switch agent
  • 7. 6Copyright©2015 NTT corp. All Rights Reserved. Why SDN? 6 Differentiate services (Innovation) • Increase user experience • Provide unique service which is not in the market Time-to-Market(Velocity) •Not to depend on vendors’ feature roadmap •Develop necessary feature when you need Cost-efficiency •Reduce OPEX by automated workflows •Reduce CAPEX by COTS hardware
  • 8. 7Copyright©2015 NTT corp. All Rights Reserved. 7 Evaluate the benefits of SDN by implementing our control plane and switch
  • 9. 8Copyright©2015 NTT corp. All Rights Reserved. Lagopus switch • Lagopus overview • Xilinx FPGA 40GbE NIC • DPDK extension • OSS Community
  • 10. 9Copyright©2015 NTT corp. All Rights Reserved. Goal of Lagopus project  Provide NFV/SDN-aware switch software stack  OpenFlow switch agent  100Gbps-capable software dataplane and I/O libs  Extensible switch configuration data store  Expand software-based packet processing to carrier networks  Hardware acceleration, processing offload
  • 11. 10Copyright©2015 NTT corp. All Rights Reserved. # of packet to be proceeded for 10Gbps with 1 CPU core 0 2,000,000 4,000,000 6,000,000 8,000,000 10,000,000 12,000,000 14,000,000 16,000,000 0 256 512 768 1024 1280 #ofpacketsperseconds Packet size (Byte) Short packet 64Byte 14.88 MPPS, 67.2 ns • 2Ghz: 134 clocks • 3Ghz: 201 clocks Computer packet 1KByte 1.2MPPS, 835 ns • 2Ghz: 1670 clocks • 3Ghz: 2505 clocks
  • 12. 11Copyright©2015 NTT corp. All Rights Reserved. # of CPU cycle for typical packet processing (2Ghz CPU) 0 1000 2000 3000 4000 5000 6000 L2 forwarding IP routing L2-L4 classification TCP termination Simple OF processing DPI # of required CPU cycles 10Gbps 1Gbps
  • 13. 12Copyright©2015 NTT corp. All Rights Reserved. Lagopus switch
  • 14. 13Copyright©2015 NTT corp. All Rights Reserved. Lessons learned from existing vswitch  Not enough performance   Packet processing speed < 1Gbps   10K flow entry add > two hours   Flow management processing is too heavy   Develop & extension difficulties   Many abstraction layer cause confuse for me  • Interface abstraction, switch abstraction, protocol abstraction Packet abstraction…   Many packet processing codes exists for the same processing  • Userspace, Kernelspace,…  Invisible flow entries cause chaos debugging 
  • 15. 14Copyright©2015 NTT corp. All Rights Reserved. vSwitch requirement from user side  Commodity x86 PC server and NIC support  Gateway function to connect different NW domains (MPLS, Access NW)  10Gbps-wire rate with >= 1M flow entries  Flow-aware control  Multi flow table configuration  High performance flow entry operation  Userland-based dataplane  Easy software upgrade and deploy  Strong management plane supports  Co-operable upper management systems
  • 16. 15Copyright©2015 NTT corp. All Rights Reserved. Approach for vSwitch development  Simple is better for everything  Packet processing for speed  Protocol processing for implement  Straight forward approach  Full scratch (No use of existing vSwitch code)  No OpenFlow 1.0 support  User land packet processing as much as possible  Keep kernel code small  Every component & algorithm can be replaced  Flow lookup, library, protocol handling
  • 17. 16Copyright©2015 NTT corp. All Rights Reserved. Lagopus switch design  Simple modular-based design  Unified configuration data store  Multiple agent support  Support multiple data-planes  Dataplane abstraction layer  Multi-threaded dataplane  Flexible parallel pipeline • Multi-core CPU-aware design • Intel DPDK and lock-free libraries for I/O
  • 18. 17Copyright©2015 NTT corp. All Rights Reserved. high-performance user-space packet processing with Intel DPDK NIC skb_buf Ethernet Driver API Socket API vswitch packet buffer Dataplane 1. Interrupt & DMA 2. system call (read) User space Kernel space Driver 4. DMA 3. system call (write) NIC Ethernet Driver API Socket API vswitch packet buffer agentagent 1. DMA Write 2. DMA READ DPDK dataplane Userspace packet processing (Event-based) DPDK apps (polling-based)
  • 19. 18Copyright©2015 NTT corp. All Rights Reserved. Packet processing using multi core CPUs  Exploit many core CPUs  Reduce data copy & move (reference access)  Simple packet classifier for parallel processing in I/O RX  Decouple I/O processing and flow processing  Improve D-cache efficiency  Explicit thread assign to CPU core NIC 1 RX NIC 2 RX I/O RX CPU0 I/O RX CPU1 NIC 1 TX NIC 2 TX I/O TX CPU6 I/O TX CPU7 Flow lookup packet processing CPU2 Flow lookup packet processing CPU4 Flow lookup packet processing CPU3 Flow lookup packet processing CPU5 NIC 3 RX NIC 4 RX NIC 3 TX NIC 4 TX NIC RX buffer Ring buffer Ring buffer NIC TX buffer
  • 20. 19Copyright©2015 NTT corp. All Rights Reserved. Implementation techniques for speed  Reduce overhead  Polling-based packet receiving  Reduction of # of access in I/O and memory  Huge DTLB for $ miss of memory controller  Bypass in packet processing  Direct data copy from NIC buffer to CPU $  Kernel stack bypass of network  Fast flow lookup and flow $ mechanism  Reduce memory access and leverage locality  Packet batching  Novel flow lookup algorithm  Thread local storage and lockless-queue, RCU  Explicit CPU and thread assignment  Hardware-topology aware resource assignment
  • 21. 20Copyright©2015 NTT corp. All Rights Reserved. Performance Evaluation  Summary  Throughput: 10Gbps wire-rate  Flow entries: > 1M flow entires > 4000 flow entries mods /sec  Evaluation models  WAN-DC gateway • MPLS-VLAN mapping  L2 switch • Mac address switching
  • 22. 21Copyright©2015 NTT corp. All Rights Reserved. WAN-DC Gateway 0 1 2 3 4 5 6 7 8 9 10 0 200 400 600 800 1000 1200 1400 1600 Throughput(Gbps) Packet size (byte) 10 flow rules 100 flow rules 1k flow rules 10k flow rules 100k flow rules 1M flow rules Throughput vs packet size, 1 flow, flow-cache 0 1 2 3 4 5 6 7 8 9 10 1 10 100 1000 10000 100000 1000000 Throughput(Gbps) flows 10k flow rules 100k flow rules 1M flow rules Throughput vs flows, 1518 bytes packet
  • 23. 22Copyright©2015 NTT corp. All Rights Reserved. L2 switch performance (Mbps) 10GbE x 2 (RFC2889 test) 0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 LINC OVS (netdev) OVS (kernel) Lagopus (software) Mbps 72 128 256 512 1024 1280 1518 Intel xeon E5-2660 (8cores, 16threads), Intel X520-DA2, DDR3-1600 64GB Packet size
  • 24. 23Copyright©2015 NTT corp. All Rights Reserved. OpenFlow 1.3 Conformance Status Type Action Set field Match Group Meter Total # of test scenario (mandatory, optional) 56 (3 , 53) 161 (0 , 161) 714 (108 , 606) 15 (3 , 12) 36 (0 , 36) 991 (114 , 877) Lagopus 2014.11.09 56 (3, 56) 161 (0, 161) 714 (108, 606) 15 (3, 12) 34 (0, 34) 980 (114, 866) OVS (kernel) 2014.08.08 34 (3, 31) 96 (0, 96) 534 (108, 426) 6 (3, 3) 0 (0, 0) 670 (114, 556) OVS (netdev) 2014.11.05 34 (3, 31) 102 (0, 102) 467 (93, 374) 8 (3, 5) 0 (0, 0) 611 (99, 556) IVS 2015.02.11 17 (3, 14) 46 (0, 46) 323 (108, 229) 3 (0, 2) 0 (0, 0) 402 (111, 291) ofswitch 2015.01.08 50 (3, 47) 100 (0, 100) 708 (108, 600) 15 (3, 12) 30 (0, 30) 962 (114, 848) LINC 2015.01.29 24 (3, 21) 68 (0, 68) 428 (108, 320) 3 (3, 0) 4 (0, 4) 523 (114, 409) Trema 2014.11.28 50 (3, 47) 159 (0 , 159) 708 (108, 600) 15 (3, 12) 34 (0, 34) 966 (114, 854) http://osrg.github.io/ryu/certification.html
  • 25. 24Copyright©2015 NTT corp. All Rights Reserved. 40GbE FPGA NIC
  • 26. 25Copyright©2015 NTT corp. All Rights Reserved. Lessoned learned from development  vSwitch needs hardware acceleration for NFV  Focus CPU cores for upper-layer packet processing  Encryption, Packet scheduler are CPU heavy processing  vSwitch has several improvement points  vSwitch cannot leverage multicore CPU power  Software-based packet scheduler is too heavy  vSwitch cannot protect from ingress DoS attack Performance Flexiblity Availability Existing SDN hardware switch + HW accelerator High-performance & scalability with HW advance features with HW
  • 27. 26Copyright©2015 NTT corp. All Rights Reserved. Lagopus FPGA NIC HW SW x86 (Xeon) + DPDK + Frontend API VNF1@VM QSFP+ QSFP+ 40G TRX [LOOK-UP] 40G TRX vSwitch VNF2@VM PCIe Gen3 x8 PCIe Gen3 x8 VNF3@VM Inter-DC (WAN) Intra-DC (East-West) [SR-IOV DEMUX] [LABELING] [PARSE] [TAP-CTRL] ・・・ QoS [TM] FPGA [DISPATCH] DMA Queues x32 (duplex) Frontend Xilinx 40G/80G QoS NIC (XNIC) IA server@DC TrafficEscalation Flow Director function for many core Packet Mirroring function(NFV tap)
  • 28. 27Copyright©2015 NTT corp. All Rights Reserved. Throughput 0.00 5.00 10.00 15.00 20.00 25.00 30.00 35.00 40.00 64 68 256 512 1024 1512 Throughput(Gpbs) Frame Size (Bytes) Throughput on Each Frame Size Xeon(R) E5-2680 v2 @ 2.80GHz, 10C/20T, DPDK-1.7.0 (1) RX1, TX1, W2 (2) RX1, TX1, W6 (3) RX1, TX1, W2 (4) RX1, TX1, W6 (5) RX2, TX1, W5 Frame Size (Bytes) Line Rate 40G (Gbps) Measured Results of Xilinx NIC and Lagopus Interwork RX --> TX Throughput (Gbps) Flow Director = OFF Flow Director = ON Core assignment (1) RX1, TX1, W2 (2) RX1, TX1, W6 (3) RX1, TX1, W2 (4) RX1, TX1, W6 (5) RX2, TX1, W5 64 30.47 2.05 4.96 4.20 5.00 8.15 68 30.90 2.17 5.62 4.41 5.69 8.53 256 37.10 8.18 21.68 16.70 21.88 30.32 512 38.49 16.26 38.46 33.50 38.46 38.47 1024 39.23 32.28 39.23 39.23 39.23 39.23 1512 39.47 39.46 39.47 39.46 39.47 39.46
  • 29. 28Copyright©2015 NTT corp. All Rights Reserved. High performance virtual NIC for DPDK-enabled NFV apps
  • 30. 29Copyright©2015 NTT corp. All Rights Reserved. Concept  DPDK is good for performance, but not for operation and management  Performance acceleration for DPDK apps and legacy apps  Simultaneous restart support of DPDK apps and DPDK vswitch  Isolation for security and trust for guest VM and vswitch  Trusted guest VM: zero-copy  Non-trusted guest VM: one-copy Guest1 QEMU App DPDK Guest2 QEMU App DPDK Virtio-net PMD Virtio-net PMD Lagopus vswitch DPDK PMD Map memory in guest VM to lagopus memory Map memory in guest VM to lagopus memory virtio virtio virtio queue virtio queue PMD vNIC PMD vNIC
  • 31. 30Copyright©2015 NTT corp. All Rights Reserved. DPDK VNIC extension (virtq-pmd)  4.8 times faster than existing vhost pmd  Support of simultaneous restart of VM gust and host DPDK apps 0 500 1000 1500 2000 2500 3000 3500 0 500 1000 1500 2000 Required#ofCPUClocks Packet size (bytes) Transmission overhead per packet copy overhead hole PMD virtq PMD vhost app Kernel driver 0.00 10.00 20.00 30.00 40.00 50.00 60.00 70.00 0 500 1000 1500 2000 Gbps COPY OFF COPY ON Released as OSS on github
  • 32. 31Copyright©2015 NTT corp. All Rights Reserved. Expand SDN OSS community
  • 33. 32Copyright©2015 NTT corp. All Rights Reserved. Activity  Hands-on in Tokyo & Taiwan  Total over 400 engineers enjoyed  United states? Tokyo hands-on Taiwan hands-on
  • 34. 33Copyright©2015 NTT corp. All Rights Reserved. PoC in SDN Japan 2014  Location-aware bandwidth-controlled Internet WiFi with Ryu and Lagopus  Good audience can connect to the Internet with guaranteed BW  Front area sheet, good connection.  Back area sheet, poor connection
  • 35. 34Copyright©2015 NTT corp. All Rights Reserved. What’s next?
  • 36. 35Copyright©2015 NTT corp. All Rights Reserved. M-plane needs innovation!  No sufficient abstraction in M-plane  Proprietary CLI, script  Invisibility leads to inefficiency  Still legacy (SNMP is neither SIMPLE nor realtime) plan Provision & control monitor Analyze & visualize Plan NW based on NW abstraction model Provisioning on NW abstraction model Feedback to NW abstraction model Think NW based-on NW abstraction model
  • 37. 36Copyright©2015 NTT corp. All Rights Reserved. Y2013 Y2014 Y2015 Control plane Management plane Data plane Lagopus development OF1.3 agent High-performance software dataplane with DPDK (static pipeline configuration) Switch config/mngmt Db prototype OF-CONFIG, CLI, SNMP prototypes Flexible reconfigurable pipeline in dataplane Tunnel protocol (VxLAN, GRE, MPLS pseudo-wire) 40Gbps-capable Soft dataplane Extensible switch resource Management/configuration datastore (CLI, OF-CONFIG, SNMP) OF1.5 agent Legacy protocol ctrl support M-plane extension (Openconfig)
  • 38. 37Copyright©2015 NTT corp. All Rights Reserved. Thanks! Web: http://lagopus.github.io/ Twitter: lagopusvswitch