2. What is Dragonflow?
Full Implementation of OpenStack Neutron API
Lightweight Distributed SDN Controller with pluggable
database
Project mission
To Implement advanced networking services in a manner
that is efficient, elegant and resource-nimble
Page 2
3. Dragonflow Highlights
Page 3
• Integral part of OpenStack
• Fully Open Source
• Scale, Performance and Latency
• Lightweight and Simple
• Easily Extendable
• Distributed SDN Control Plane
• Sync Policy Level abstraction to the CN
4. Dragonflow - Distributed SDN
Neutron-Server
Dragonflow Plugin
DB
OVS
Dragonflow
DB
Driver
Compute Node
OVS
Dragonflow
DB
Driver
Compute Node
OVS
Dragonflow
DB
Driver
Compute Node
OVS
Dragonflow
DB
Driver
Compute Node
DB
VM VM
..
VM VM
..
VM VM
.. VM VM
..
5. Compute NodeCompute NodeCompute Node
Dragonflow
Network DB
OVS
Neutron
Server
OVSDB
OVSDB-Server
ETCD RethinkDBRAMCloud
Kernel Datapath Module
NIC
User Space
Kernel Space
Dragonflow DB Drivers
OVSDB ETCD RethinkDBRMC
Future
Dragonflow Plugin
Route
Core
API
SG
vswitchd
Container
VM Dragonflow Controller
Abstraction Layer
L2 App L3 App DHCP App
Fault
Detection
SG
LBaaS …FWaaS
Pluggable DB
Layer
NBDBDrivers
SB DB Drivers
smartNIC OVSDB
OVSDB
ETCD
RMC
RethinkDB
OpenFlow
Dragonflow – Under The Hood
6. Current Release Features (Liberty)
L2 core API, IPv4, IPv6
GRE/VxLAN/Geneve tunneling protocols
Distributed L3 Virtual Router
Hybrid proactive + reactive flow installation
North-South traffic is still centralized
Distributed DHCP
(with just 500 lines of code!)
Pluggable Distributed Database
ETCD, RethinkDB, RAMCloud, OVSDB
9. 1 VM Send DHCP_DISCOVER
2 Classify Flow as DHCP, Forward to Controller
3 DHCP App sends DHCP_OFFER back to VM
4 VM Send DHCP_REQUEST
5 Classify Flow as DHCP, Forward to Controller
6 DHCP App populates DHCP_OPTIONS from DB/CFG and send
DHCP_ACK
Dragonflow Distributed DHCP
VM DHCP SERVER
1
3
4
6
7
Compute Node
Dragonflow
VM
OVS
VM
1 2
br-int
qvoXXX qvoXXX
OpenFlow
1
4
2
5
7
Dragonflow Controller
Abstraction Layer
L2
App
L3
App
DHCP
App
SG
36
Pluggable DB
Layer
DB
10. Dragonflow Distributed DHCP
Match:
Broadcast +UDP +S_Port=68 +D_Port=67
Action:
Send to DHCP table
Service Table
DHCP Table
Match: in_port => Action:
Set metadata with port unique key
SEND TO CONTROLLER
(for every local port that its network has DHCP
enabled)
Default:
goto “L2 Lookup Table”
Compute Node
VM
OVS
br-int
qvoXXX
VM
qvoXXX
1 2
Dragonflow
Dragonflow Local Controller
Abstraction Layer
L2
App
L3
App
DHCP
App
SG
DB
OpenFlow
Ingress Port Security
Ingress Classification
Dispatch to Ports
12. Database Framework
Requirements
• HA + Scalability
• Different Environments have different requirements
• Performance, Latency, Scalability, etc.
Why Pluggable?
• Long time to productize
• Mature Open Source alternatives
• Allow us to focus on the networking services only
13. DB Driver API
Implementations
RAMCloud
ETCD
RethinkDB
Zookeeper
Dragonflow Pluggable Database
Compute NodeCompute NodeCompute Node
Dragonflow
Local
Controller
Pluggable
DB Layer
Applicative
DB Layer
Adapter
DB
Driver
API
Expose DB
Features
Neutron Server
Dragonflow
Neutron
Plugin
DB Operations
Database
Server
DB Adapter
DB Adapter
DB Adapter
14. Distributed
Database
DB Data 3
DB Data 2
DB Data 1
Full Distribution
Compute Node 1
Dragonflow
Local Cache
OVS
Compute Node N
Dragonflow
OVS
Local Cache
Dragonflow DB Drivers
OVSDB ETCD RethinkDBRMC
DB Data 3
DB Data 2
DB Data 1
DB Data 3
DB Data 2
DB Data 1
15. Distributed
Database
DB Data 3
DB Data 2
DB Data 1
Selective Proactive Distribution
Compute Node 1
Dragonflow
Local Cache
OVS
DB Data 1
Compute Node N
Dragonflow
OVS
Local Cache
DB Data 3
DB Data 2
Dragonflow DB Drivers
OVSDB ETCD RethinkDBRMC
18. Dragonflow Pipeline
Installed in every OVS
Service
Traffic
Classification
Ingress Processing
(NAT, BUM)
ARP DHCP
L2
Lookup
L3
Lookup
DVR
Egress
Dispatching outgoing
traffic to external
nodes or local ports
Ingress
Port
Security
(ARP spoofing , SG, …)
Egress
Port
Security
Egress
Processing
(NAT)
Fully Proactive
Has Reactive Flows to Controller
Security Groups
…
Outgoing from local
port Classification and
tagging
Dispatching Incoming
traffic from external
nodes to local ports
20. Roadmap
Additional DBs Drivers ZooKeeper, Redis …
Selective Proactive DB
Hierarchical Port Binding (SDN ToR) move to ML2
Pluggable Pub/Sub Mechanism
DB Consistency
Distributed DNAT
Security Group
Containers (Kuryr plugin and nested VM support)
Topology Service Injection / Service Chaining
Inter Cloud Connectivity (Border Gateway / L2GW)
…
21. Hierarchical Port Binding (SDN ToR)
move to ML2
Rack n
ToR
VLAN
Segmentation
Rack 1
ToR
Rack 2
ToR
Rack 3
ToR
Vxlan
Segmentation
22. Dargonflow Hierarchical Port Binding (SDN ToR)
Neutron Server
REST API
Neutron Core plugins
ML2
Cisco(Nexus,
N1Kv)
OVN
Morevendor
plugins
Type Drivers Mechanism Drivers
VLAN
GRE
VXLAN
ONOS
Dragonflow
TOR
Neutron Service plugins
DragonflowDB
Rack n
ToR
VLAN
Segmentation
Vxlan
Segmentation
Compute Node
Dragonflow
VM
OVS
VM
br-int
qvoXXX qvoXXX
OpenFlow
Dragonflow Controller
Abstraction Layer
Vlan
L2
App
L3
App
DHCP
App
SG
Pluggable DB
Layer
DBDB
ToRMach
Driver
OpenDayLight
23. Pluggable Pub/Sub Mechanism
Neutron-Server
Dragonflow Plugin
DB
OVS
Dragonflow
DB
Driver
Compute Node
OVS
Dragonflow
DB
Driver
Compute Node
OVS
Dragonflow
DB
Driver
Compute Node
OVS
Dragonflow
DB
Driver
Compute Node
DB
VM VM
..VM VM
..
VM VM
.. VM VM
..
Pub/Sub
if the DB internally supports
Pub sub then we use it
24. Pluggable Pub/Sub Mechanism
Neutron-Server
Dragonflow Plugin
DB
OVS
Dragonflow
DB
Driver
Compute Node
OVS
Dragonflow
DB
Driver
Compute Node
OVS
Dragonflow
DB
Driver
Compute Node
OVS
Dragonflow
DB
Driver
Compute Node
DB
VM VM
..
VM VM
..
VM VM
..
VM VM
..
Pub/Sub
Why do we need it ?
Not all DBs support pub-sub (e.g. RamCloud)
We need to be able to customize
Performance, Latency, Scalability, etc.
25. DB Consistency Common Problem to all SDN Solution
SDN Controller
North-bound Interface (REST?)
South-bound Interface (Openflow)
SDN Apps
SDN DB
Neutron
DB
Neutron-server
ML2-Core-Plugin
ML2.Drivers.Mechanism.XXX
Services-Plugin
Service
Network
Neutron API Nova API
CLI / Dashboard (Horizon) / Orchestration Tool (Heat)
HW Switch
Nova
Nova Compute
VM VM
Nova Compute
VM VM
Virtual Switch (OVS?) Virtual Switch (OVS?)
Neutron
Plugin Agent
Neutron
Plugin Agent
Vendor-specific API
Message Queue (AMQP)
Neutron-L3-Agent
Neutron-DHCP-Agent
LoadBalancer
Firewall
VPN
L3Services
TopologyMgr.
OverlayMgr.
Security
26. Dragonflow DB Consistency
Neutron-Server
Dragonflow Plugin
DB
OVS
Dragonflow
DB
Driver
Compute Node
OVS
Dragonflow
DB
Driver
Compute Node
DB
VM VM
..
VM VM
..
Neutron
DB
The Neutron DB is the master Database
Introduce a full-sync diff based mechanism
NDB DDB
Introduce a virtual transaction mechanism
NDB DDB
Key DB Requirement from multi production
environments
Optimized for Read, multiple read request in
very high volume from nova, Horizon …
Multi Neutron server API running on
different hosts
Neutron-Server
Dragonflow Plugin
DB
Neutron-Server
Dragonflow Plugin
DB
27. Join the project Dragonflow
• Documentation
https://wiki.openstack.org/wiki/Dragonflow
• Bugs & blueprints
https://launchpad.net/dragonflow
• DF IRC channel
#openstack-dragonflow
Weekly on Monday at 0900 UTC in #openstack-meeting-4 (IRC)
30. Security Groups Problems
• Data plane performance
• Additional Linux Bridge on the Path
• Control plane performance
• Rules needs to be re-compiled on port changes
• Many rules due to security group capabilities
• Iptable commands issued by CLI process
• RPC bulks
32. Security Groups Translations
Direction:Egress, Type:IPv4, IP Protocol:TCP, Port Range:Any, Remote IP
Prefix:0.0.0.0/0
match:ct_state=+new+trk,tcp,reg6=X
actions=ct(commit,zone=NXM_NX_REG6[0..15]),resubmit(,<next_table>)
Direction:Egress, Type:IPv4, IP Protocol:TCP, Port Range:Any, Remote
Security Group: Y
match:ct_state=+new+trk,tcp,reg6=X,reg5=Y,
actions=ct(commit,zone=NXM_NX_REG6[0..15]),resubmit(,<next_table>)
33. Distributed DNAT (Floating IP)
OVS
VM
Compute Node
Public
network
OVS
VM
Compute Node
Public
network
OVS
Network Node
Router
Namespace
35. Neutron and libnetwork
A Docker
Container
Network
Sandbox
Endpoint
A Docker Container
Network Sandbox
Endpoint
A Docker
Container
Network
Sandbox
Endpoint
Frontend
Network
Endpoint
Backend
Network
Tenant A Net1
192.168.1.0/0
Tenant A Net2
192.168.5.0/0
VM1
192.168.1.5
VM2
192.168.1.7
192.168.5.2
36. Kuryr Project Overview
• Open source
• Part of OpenStack Neutron’s big stadium
• Under OpenStack big tent from next release!!!
• Brings the Neutron networking model as a provider for the Docker
CNM
• Aims to support different Container Orchestration Engines
• E.g. Kubernetes, Mesos, Docker Swarm
• Weekly IRC meetings
• Working together with OpenStack community
• Neutron, Magnum, Kolla
38. Dragonflow and Kuryr plans
• Dragonflow to support containers networking use cases
• Nested containers inside VMs support
• Containers can leverage all of Dragonflow features
• Distributed DHCP
• Security and QoS
• Containers performance and fault management
• Port forwarding
• Dragonflow distributed load balancer
• DNS as a Service in Dragonflow
• Integration with Kubernetes
• Full Integration of Dragonflow and Kuryr
• Containerized image of Dragonflow
• VIF Binding to Dragonflow
• OVS, IPVLAN
43. Simple But Extendable
• Various special services and behavior's
• VPN
• QoS (DSCP tagging)
• Dynamic Routing
• Inter clouds connectivity
• And so much more…
• External applications
• Centralized “SDN” applications
• New distributed networking services
• Networking as a Service to NFV
45. Classic Service Chaining
• Chain of ports the traffic traverses
• Classifier for entry point
• Different types of chains
• Static or dynamic
• Different underlying technologies
• NSH
• MPLS
• App ports
• End points of various kinds
• VMs
• Containers
• User space applications
• Physical devices
47. Service Injection Hooks
Logical Router
Logical Switch Logical Switch
VM 1 VM 2 VM 3
DSCP
Marking
DPI
Distributed
Load
Balancing
48. Topology Service Injection
• Interact with base OpenFlow pipeline
• Leverage classification metadata
• Distributed network services
• Flow based
• Compatible with SDN Applications
• Can use OpenFlow
• Expose virtual topology
• Inject services in specific hooks
• Easily extendable
• No code modifications
49. Service Injection Example – IPS
Compute Node
VM 1 IPS
Table 0 Service
Chains
Table N…
IPS Manager
Data Path App
IPS recognizes infected VM
50. Service Injection Example – IPS
Compute Node
VM 1 IPS
Table 0 Service
Chains
Table N…
IPS Manager
Data Path App
IPS app manager installs
blocking flows for VM1
traffic (Quarantine)
51. Use Cases
• Security Appliance
• Send specific traffic for inspection
• Traffic Mirroring
• Implement TAP on various different locations in the path
• Applicative Load Balancing
• Receive first packets of a connection and wire connection in flows
• Tenants Differentiate service between clouds
• Inter Cloud connectivity
• Border Gateway / L2GW
52. Server Server
Detect Elephant Flows
0 1 … 64
Flow Table
Test 1
10.0.0.3
Test 2
10.0.0.4
0 1 … 64
Flow Table
Elephant
detector
Detect elephant flow:
10.0.0.3 10.0.0.4 TCP port 1234
Write flows to
tableDSCP=64
slow path
fast path
Collect
sFlow
stats
53. Dragonflow Inter Cloud Connectivity (Border Gateway)
CN
CN
CN
NN
CN
CN
CN
NN
Data Center B
GW-GW
Tunnel
Data Center A
Intra-Cloud
Tunnels
Intra-Cloud
Tunnels
Connecting
Bare-Metal
Servers as
before
192.168.10.2
192.168.10.3
192.168.10.8
Why is this a good thing?
Common Applicative DB Adapter Layer
Same layer is used by all clients
Dragonflow Neutron plugin
Dragonflow local controller
External/Internal applications
Expressed in terms of the schema model
Converts model to “Key / Value”
Calls the DB Driver API for DB Operations
Leverage DB advance features
Knows to receive and wait for DB changes
According to a pre defined generic API with the driver
Selective publish-subscribe
Each local controller sync only relevant data according to its local ports
Depends on the virtual topology
Local controller gets all local ports information
DB framework must support waiting for changes on specific entry column values
The plugin tags the related objects with a special column value
Reduce the sync load and change rate
Each local controller only gets the subset of the data that is relevant for it
Each local controller sync only relevant data according to its local ports
Depends on the virtual topology
Local controller gets all local ports information
DB framework must support waiting for changes on specific entry column values
The plugin tags the related objects with a special column value
Reduce the sync load and change rate
Each local controller only gets the subset of the data that is relevant for it
Each local controller sync only relevant data according to its local ports
Depends on the virtual topology
Local controller gets all local ports information
DB framework must support waiting for changes on specific entry column values
The plugin tags the related objects with a special column value
Reduce the sync load and change rate
Each local controller only gets the subset of the data that is relevant for it
<voice note: Here we'd explain the part about them being vendor specific makes that each Neutron vendor would have to make its own implementation of libnetwork or cni reinventing the wheel and without the ability to share the common parts./>
<voice note: Here we'd explain the part about them being vendor specific makes that each Neutron vendor would have to make its own implementation of libnetwork or cni reinventing the wheel and without the ability to share the common parts./>
<voice note: Here we'd explain the part about them being vendor specific makes that each Neutron vendor would have to make its own implementation of libnetwork or cni reinventing the wheel and without the ability to share the common parts./>
<voice note: Here we'd explain the part about them being vendor specific makes that each Neutron vendor would have to make its own implementation of libnetwork or cni reinventing the wheel and without the ability to share the common parts./>
<voice note: Here we'd explain the part about them being vendor specific makes that each Neutron vendor would have to make its own implementation of libnetwork or cni reinventing the wheel and without the ability to share the common parts./>
<voice note: Here we'd explain the part about them being vendor specific makes that each Neutron vendor would have to make its own implementation of libnetwork or cni reinventing the wheel and without the ability to share the common parts./>
<voice note: Here I'd stop to thank Neutron drivers for welcoming us into the big stadium/>
<voice note: Talk about how this may be straight away support or by the plugins for this platforms that we can incorporate in our repository/>
<voice note: Here tell the people to join us and contribute/>
<voice note: Here we'd explain the part about them being vendor specific makes that each Neutron vendor would have to make its own implementation of libnetwork or cni reinventing the wheel and without the ability to share the common parts./>
<voice note: Here we'd explain the part about them being vendor specific makes that each Neutron vendor would have to make its own implementation of libnetwork or cni reinventing the wheel and without the ability to share the common parts./>
<voice note: Here we'd explain the part about them being vendor specific makes that each Neutron vendor would have to make its own implementation of libnetwork or cni reinventing the wheel and without the ability to share the common parts./>
<voice note: Here we'd explain the part about them being vendor specific makes that each Neutron vendor would have to make its own implementation of libnetwork or cni reinventing the wheel and without the ability to share the common parts./>
<voice note: Here we'd explain the part about them being vendor specific makes that each Neutron vendor would have to make its own implementation of libnetwork or cni reinventing the wheel and without the ability to share the common parts./>
<voice note: Here we'd explain the part about them being vendor specific makes that each Neutron vendor would have to make its own implementation of libnetwork or cni reinventing the wheel and without the ability to share the common parts./>
<voice note: Here we'd explain the part about them being vendor specific makes that each Neutron vendor would have to make its own implementation of libnetwork or cni reinventing the wheel and without the ability to share the common parts./>
<voice note: Here we'd explain the part about them being vendor specific makes that each Neutron vendor would have to make its own implementation of libnetwork or cni reinventing the wheel and without the ability to share the common parts./>
<voice note: Here we'd explain the part about them being vendor specific makes that each Neutron vendor would have to make its own implementation of libnetwork or cni reinventing the wheel and without the ability to share the common parts./>
<voice note: Here we'd explain the part about them being vendor specific makes that each Neutron vendor would have to make its own implementation of libnetwork or cni reinventing the wheel and without the ability to share the common parts./>
<voice note: Here we'd explain the part about them being vendor specific makes that each Neutron vendor would have to make its own implementation of libnetwork or cni reinventing the wheel and without the ability to share the common parts./>