Comparison of existing cni plugins for kubernetes. We compare existing CNI plugins by different features.

1. Comparison of existing CNI
plugins for Kubernetes
Adam Hamsik
adam.hamsik@lablabs.io

2. Labyrinth Labs
Rock-solid infrastructure and DevOps
● Building rock-solid and secure foundations for all your digital operations. Our
mission is to let you focus on your business without ever needing to worry about
technical issues again.
● Making you ready for growing traffic, safe against new security vulnerabilities and
data-loss.
2

3. Kubernetes Architecture
● Master acts as the control plane for Kubernetes. Kubernetes master is
responsible for maintaining the desired state for cluster. The “master” refers to a
collection of processes managing and is responsible at a minimum for running
scheduler, cluster controller, API Server.
● Nodes acts as the “worker” of Kubernetes cluster. The nodes in a cluster are the
machines (VMs, physical servers, etc) that run your applications and cloud
workflows. The Kubernetes master controls each node.
3

4. Kubernetes Architecture vol. 2
4

5. Kubernetes Master
● Kube-apiserver
● Etcd
● Kube-controller-manager
● Cloud-controller-manager
● Kube-scheduler
● CNI Plugin Control Plane
5

6. Kubernetes Node
● Kubelet
● Kube-proxy
● Container runtime
● CNI plugin
6

7. Kubernetes Plugins
● CRI (Container Runtime Interface) is an interface used for communication
between kubelet and container runtime
○ Docker
○ Cri-o
○ rkt
● CSI (Container Storage Interface) is an unified interface between container
orchestration systems and storage vendors
○ EBS
○ NetApp
○ Ceph
● CNI (Container Networking Interface) is an interface between network namespace
and container runtime
7

8. What is a CNI Plugin ?
● Simplest interface between container runtime and network
implementation
● Originated at CoreOS as part of Rkt Container runtime
● CNCF project
● Repository: https://github.com/containernetworking/cni
● Specification: the API between runtimes and network plugins
8

9. Kubernetes CNI Requirements
● pods on a node can communicate with all pods on all nodes without NAT
● agents on a node (e.g. system daemons, kubelet) can communicate with all pods
on that node
● pods in the host network of a node can communicate with all pods on all nodes
without NAT
● containers within a Pod share their network namespaces
○ You can access other container services through localhost
9

10. Kubernetes CNI Implementation
● Containers in a pod exists within network namespace and share same IP -
○ this setup allow for intrapod communication over localhost
● Pod are given cluster unique IP for the duration of its lifecycle
● Services are given a persistent cluster unique IP that spans Pods lifecycle
● External connectivity is generally handled by an infrastructure (default GWs)
10

11. Kubernetes CNI Workflow
● Container runtime executes CNI plugin with given config
● CNI Plugin executes IPAM module to configure IP addresses on a configured
interface
11

12. Demo Time vol.1
● Show
● CNI Plugin executes IPAM module to configure IP addresses on a configured
interface
12

13. List of existing CNI Plugins
● Overlay Network plugins
○ Flannel
○ Weave-net
● Routed Network plugins
○ AWS-VPC
○ Kube-router
○ Calico
○ Canal
● IPAM modules
○ Dhcp
○ host-local
● Multi CNI plugins
○ Damn
○ Multus
○ CNI-Genie
13

14. Flannel - Detailed description
● Developed by CoreOS
● Relatively easy to install/configure
● Setups layer 3 IPv4 overlay network over multiple backends
○ VXLAN
○ Host-gw
○ UDP
○ IPIP(Experimental)
○ IPSec(Experimental)
● Doesn’t support Network policies
14

15. Flannel - Detailed description
15https://www.slideshare.net/lorispack/using-coreos-flannel-for-docker-networking

16. Calico - Detailed description
● Best overall choice for performance, flexibility and power
● Calico requires Layer 3 network which is using BGP protocol to route packets
between hosts and pods
● Network Policies can be created to manage network access between pods
16

17. Calico - Detailed description
17https://www.docker.com/blog/networking-in-docker-enterprise-edition-2-0/

18. Kube-router - Detailed description
● Turnkey solutions for:
○ Pod Networking
■ kube-router handles Pod networking efficiently with direct routing thanks to the BGP
protocol and the GoBGP Go library.
○ IPVS/LVS Service Proxy
■ Kube-router uses battle-tested Linux LVS/IPVS to provide a service proxy and provides rich
set of scheduling options and enables advanced use-cases like DSR
○ Network Load Balancer
■ Kube-router has the ability to advertise service VIP's to L3 fabric BGP peers. So you can do
network load balancing with ECMP.
18

19. AWS VPC CNI - Detailed description
● Plugin used only in AWS deployments offers
○ High throughput
○ High availability
○ Low latency
○ Minimal network jitter
● Allocates/Attaches new AWS Elastic Networking Interfaces (ENIs) to nodes
○ There is a maximum number of interfaces attachable to running instance
○ VPC flow logs
○ VPC routing policies
○ security groups
19

20. Questions ?
adam.hamsik@lablabs.io
www.lablabs.io
20

21. More information
● http://events17.linuxfoundation.org/sites/events/files/slides/Container%20Netwo
rking%20Deep%20Dive.pdf
● https://thenewstack.io/hackers-guide-kubernetes-networking/
● https://www.cncf.io/wp-content/uploads/2017/11/Introduction-to-CNI-2.pdf
● https://itnext.io/benchmark-results-of-kubernetes-network-plugins-cni-over-
10gbit-s-network-36475925a560
● https://www.altoros.com/blog/kubernetes-networking-writing-your-own-simple-
cni-plug-in-with-bash/
21