Slide ware on solutions to address some industry challenges in container networking.

1. ContainersNetworking-
ExperienceKitsoverview
Haideemcmahon|technicalsolutionsmarketingmanager
softwaredefineddatacenterSOLUTIONSGROUP

2. 2
DatacenterSolutionsGroup
© 2018 Intel Corporation. Intel, the Intel logo, Xeon and Xeon logos are trademarks of Intel Corporation in the U.S. and/or other countries. *Other names and brands may be
claimed as the property of others.
Intel technologies’ features and benefits depend on system configuration and may require enabled hardware, software or service activation. Learn more at intel.com, or from the
OEM or retailer.
All products, computer systems, dates, and figures specified are preliminary based on current expectations, and are subject to change without notice.
No license (express or implied, by estoppel or otherwise) to any intellectual property rights is granted by this document.
​Intel disclaims all express and implied warranties, including without limitation, the implied warranties of merchantability, fitness for a particular purpose, and non-infringement,
as well as any warranty arising from course of performance, course of dealing, or usage in trade.
Intel processors of the same SKU may vary in frequency or power as a result of natural variability in the production process.
For more complete information about performance and benchmark results, visit www.intel.com/benchmarks.
Intel does not control or audit third-party benchmark data or the web sites referenced in this document. You should visit the referenced web site and confirm whether referenced
data are accurate.
Optimization Notice: Intel's compilers may or may not optimize to the same degree for non-Intel microprocessors for optimizations that are not unique to Intel microprocessors.
These optimizations include SSE2, SSE3, and SSSE3 instruction sets and other optimizations. Intel does not guarantee the availability, functionality, or effectiveness of any
optimization on microprocessors not manufactured by Intel. Microprocessor-dependent optimizations in this product are intended for use with Intel microprocessors. Certain
optimizations not specific to Intel microarchitecture are reserved for Intel microprocessors. Please refer to the applicable product User and Reference Guides for more
information regarding the specific instruction sets covered by this notice. Notice Revision #20110804.
Intel technologies’ features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on
system configuration. No computer system can be absolutely secure.
Intel® Advanced Vector Extensions (Intel® AVX)* provides higher throughput to certain processor operations. Due to varying processor power characteristics, utilizing AVX
instructions may cause a) some parts to operate at less than the rated frequency and b) some parts with Intel® Turbo Boost Technology 2.0 to not achieve any or maximum turbo
frequencies. Performance varies depending on hardware, software, and system configuration and you can learn more at http://www.intel.com/go/turbo.
Intel® Hyper-Threading Technology available on select Intel® processors. Requires an Intel® HT Technology-enabled system. Your performance varies depending on the specific
hardware and software you use. Learn more by visiting http://www.intel.com/info/hyperthreading.
All SKUs, frequencies, features and performance estimates are PRELIMINARY and can change without notice
Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Software and workloads used in performance tests may have
been optimized for performance only on Intel microprocessors. Performance tests, such as SYSmark and MobileMark, are measured using specific computer systems,
components, software, operations and functions. Any change to any of those factors may cause the results to vary. You should consult other information and performance tests
to assist you in fully evaluating your contemplated purchases, including the performance of that product when combined with other products. For more complete information
visit http://www.intel.com/performance. Configurations: Based on Intel estimates.
NoticesandDisclaimers

3. 3
DatacenterSolutionsGroup
TheCloudNativePath
Sources: BT on NFV in 2012; HPE http://community.hpe.com/t5/Telecom-IQ/HP-s-4-Stages-of-NFV/ba-p/6797122
Note: The timelines provided in the diagram have shifted, however, the transformation phases defined by HP in
2012 are still valid
decoupled
virtualized
cloudified
decomposed
2014 2015 2016 2017
functionsoutcomes
Standardize,
Open platforms
Better utilization
Elasticity and speed
Composable services

4. 4
DatacenterSolutionsGroup
WhatcommunicationServiceProviders
aresayingaboutcontainers
Source: AT&T Container Strategy and OpenStack's Role in It, OpenStack Boston 2017 (bit.ly/2rfftRA)
today 2018-2019 2019+
Enterpriseitworkloads
Virtualnetworkfunction(vnf)
Openstackservices
Container Container Containervm vm vm
Container Container
Container
vm
vm
vm
Container

5. 5
DatacenterSolutionsGroup
VNFs
NFVi-Network
NFVOrchestration
Containersnetworkingdeploymentsconsiderations
MultipleDeploymentModels
vCMTS vIMS vEPC vCPE vSBC
SR-IOV
Hybrid Unified
Containers
BareMetal
VM ContainersVM
&

6. 6
DatacenterSolutionsGroup
VNFs
NFVi-Network
NFVOrchestration
vCMTS vIMS vEPC vCPE vSBC
SR-IOV
ContainerBareMetalDeploymentModel
Collaboratewithearlymovers,driveOpenSourcedevelopmentsandenabletheindustry
ContainersBareMetal
Hardware
Containerized Virtual Network Functions
vRNCvCPE
vEPC vFirewall
vSBCvCMTS
vIMS vRouter
Orchestration
Host OS Docker Engine

7. 7
DatacenterSolutionsGroup
AddresskeyChallengesincontainersBareMetal
* Open Source: Available on Intel github https://github.com/Intel-Corp | NFD at https://github.com/kubernetes-incubator/node-feature-discovery
Node Feature Discovery
CPU Manager for
Kubernetes
Native Huge page support
for Kubernetes
VHOST USER
SR-IOV
Open Source: CNI plug-in - V2.0 June ‘17
Upstream K8s: TBD
Open Source: CNI plug-in - V1.0 Sep ‘17
Open Source: CNI plug-in - V2.0 April ‘17
Open Source: Nov. ‘16
Upstream K8: Incubation Graduation TBD
Open Source: V1.2 April ‘17
Upstream K8: Phase 1 - V1.8 Sept ‘17
Upstream K8: V1.8 Sept ‘17
SOFTWAREAVAILABILITY*
Upstream collected: V5.7.2 June ‘17 ;
5.8.0
Challengesbeingaddressed Solution
Multiple network interfaces for VNFs
High performance Data Plane (E-W)
High performance Data Plane (N-S)
Ability to request/allocate platform capabilities
CPU Core-Pinning and isolation for K8s pods
Dynamic Huge Page allocation
Platform telemetry information
KubernetesNetworking
DataPlane Acceleration
EnhancePlatformAwareness(EPA)
Telemetry

8. 8
DatacenterSolutionsGroup
VNFs
NFVi-Network
NFVOrchestration
vCMTS vIMS vEPC vCPE vSBC
SR-IOV
ContainerUnifiedInfrastructureDeploymentModel
ContainersUnifiedInfrastructure
Hardware
Orchestration
Hypervisor
Guest OS
Docker Engine
Guest OS
Docker Engine
Guest OS
Docker Engine
VM VM VM
App App App App App App

9. 9
DatacenterSolutionsGroup
Sameasin
Container
BareMetal
Kuryr
IndustrychallengesincontainersUnifiedInfrastructure
Challengesbeingaddressed Solution
Removing Network performance penalties for container in VM
Support for high performance Data Plane (E-W)
Multiple network interfaces for VNFs
Support for CPU Core Pinning for Kuryr-K8s pods
CPU Manager for
Kubernetes
Kuryr-
Kubernetes
MASTER VM

10. 10
DatacenterSolutionsGroup
ContainerBareMetalExperienceKITS
FeatureBriefs FeatureApplicationNotes
DEMOS
SWScript
BenchmarkReport
WhatISIT?
A library of best-practice
development guidelines for Container
bare metal orchestration
Shortens the time-to-expertise
Addresses challenges in performance,
manageability, security and service
assurance

11. 11
DatacenterSolutionsGroup
ContainerBaremetalExperienceKits
Reference Architecture
Reference Architecture
Installation Scripts
Reference Architecture
User Guide
Platform Telemetry
Technical Application Note
Feature Brief
Demo
Enhance Platform Awareness
Feature Brief
White Paper
Technical Application Note
Benchmark Test Report
Demo
Kubernetes Networking
Feature Brief
Technical Application Note
Demo
All collateral can be found at: https://networkbuilders.intel.com/network-technologies/container-experience-kits

12. 12
DatacenterSolutionsGroup
EngagewithIntel
Open Source
POC
Experience Kits
Best Practice Guidelines
Software community Engagewithintel
Container
Capabilities
CONTAINER
NETWORKING
Intel is addressing key challenges to using containers for NFV use cases
Most of these have been open sourced already
Explore more information available on Intel’s Network Builders site
https://networkbuilders.intel.com/network-technologies/container-experience-kits
VNF

13. 13
DatacenterSolutionsGroup
Q&A
Thankyou

14. 14
DatacenterSolutionsGroup 14
ContainersNetworking-
Technologyoverview

15. 15
DatacenterSolutionsGroup
VNFs
NFVi-Network
NFVOrchestration
vCMTS vIMS vEPC vCPE vSBC
SR-IOV
ContainerBareMetalDeploymentModel
Collaboratewithearlymovers,driveOpenSourcedevelopmentsandenabletheindustry
Containers Bare Metal
Hardware
Containerized Virtual Network Functions
vRNCvCPE
vEPC vFirewall
vSBCvCMTS
vIMS vRouter
Orchestration
Host OS Docker Engine

16. 16
DatacenterSolutionsGroup
AddresskeyChallengesincontainersBareMetal
* Open Source: Available on Intel github https://github.com/Intel-Corp | NFD at https://github.com/kubernetes-incubator/node-feature-discovery
Node Feature Discovery
CPU Manager for
Kubernetes
Native Huge page support
for Kubernetes
VHOST USER
SR-IOV
Open Source: CNI plug-in - V2.0 June ‘17
Upstream K8s: TBD
Open Source: CNI plug-in - V1.0 Sep ‘17
Open Source: CNI plug-in - V2.0 April ‘17
Open Source: Nov. ‘16
Upstream K8: Incubation Graduation TBD
Open Source: V1.2 April ‘17
Upstream K8: Phase 1 - V1.8 Sept ‘17
Upstream K8: V1.8 Sept ‘17
SOFTWAREAVAILABILITY*
Upstream collected: V5.7.2 June ‘17 ;
5.8.0
Challengesbeingaddressed Solution
Multiple network interfaces for VNFs
High performance Data Plane (E-W)
High performance Data Plane (N-S)
Ability to request/allocate platform capabilities
CPU Core-Pinning and isolation for K8s pods
Dynamic Huge Page allocation
Platform telemetry information
KubernetesNetworking
DataPlane Acceleration
EnhancePlatformAwareness(EPA)
Telemetry

17. 17
DatacenterSolutionsGroup
MultipleNetworkInterfacesforVNFs
 Kubernetes support only one Network interface – “eth0”
 In NFV use cases, it is required to provide multiple network
interfaces to the virtualized operating environment of the VNF
eth0
Pod
eth1
eth2
eth0
interface
Pod
Container
ContainerContainer
Container
ContainerContainer
problem
Usecases
 Functional separation of control and data network planes link
aggregation/bonding for redundancy of the network
 Support for implementation of different network SLAs
 Network segregation and Security
reference
 Multus CNI – https://github.com/Intel-Corp/multus-cni
 Native Kubernetes - Mailing list with details on discussions:
https://groups.google.com/forum/#!forum/kubernetes-sig-network
NetworkControlFlowwithMultus PodNetworkInterfaceswithMultus
KUBELET
SR-
IOV
Logging
Firewall
net0 net1
eth0
FlannelLinuxBridge
Kubernetes Pod
SR-IOV
net1
SR-IOV
net0eth0
LINUX BRIDGE VF0 VF1

18. 18
DatacenterSolutionsGroup
VhostUserCNIPlugin
 No Container Networking with software acceleration
for NFV, particularly for East – West Traffic
problem
solution
 Virtio_user/ vhost_user performance better than VETH pairs
 Supports VPP as well as DPDK OVS
 Vhost_user CNI plugin enables K8s to leverage data plane
acceleration
reference
 https://github.com/intel/vhost-user-net-plugin (V1.0 Sep ’17)
NIC
eth0
OVS- DPDK/VPP
vhostuser
Kubernetes Pod
Container
VNF Application
DPDK
virtio_user

19. 19
DatacenterSolutionsGroup
DPDK–SR-IOVCNIPlugin
 Lack of support for physical platform resource isolation
 No guaranteed network IO performance
 No support for Data Plane Networking
problem
solution
 Allows SR-IOV support in Kubernetes via a CNI plugin
 Supports two modes of operation:
1. SR-IOV: SR-IOV VFs are allocated to pod network namespace
2. DPDK: SR-IOV VFs are bounded to DPDK drivers in user space
reference
 github.com/kubernetes-incubator/node-feature-discovery
Kernel
SR-IOV Enabled Network Interface
Kubernetes Pod
Container
VNF Application
VFVF VF
DPDK
uio_pci_generic/igb_uio/vfio-pci

20. 20
DatacenterSolutionsGroup
NodeFeatureDiscovery
 No way to identify hardware capabilities or configuration
 Inability for workload to request certain hardware feature
problem
solution
 Node Feature Discovery (NFD) brings Enhanced Platform
Awareness (EPA) in K8s
 NFD detects resources on each node in a Kubernetes cluster
and advertises those features
 NFD allows matching of workload to platform capabilities
reference
 github.com/Intel-Corp/sriov-cni
SR-IOV Network Features Single Root I/O Virtualization
AVX CPUID Features: Intel® Advances Vector Extensions 512 (Intel® AVX-512)
Turbo
Boost
Intel® Turbo Boost Technology processor accelerator
NodeFeatureDiscoveryLabelDetails
NODE 1
NFD
DISCOVERY
POD
NODE 2
NFD
DISCOVERY
POD
SR-IOV
AVX
Turbo
Boost
Application A
Application B
POD label:
Application B
Application A
POD label:
SR-IOV
AVX
MASTER
ETCD
NODE 1
NODE 2
NODEFEATUReDISCOVERYINK8s

21. 21
DatacenterSolutionsGroup
CPUManagerforKubernetes–CPUPinningandIsolation
 Kubernetes has no mechanism to support core pinning and
isolation
 Results in high priority workloads not achieving SLAs
problem
solution
 CPU-Manager-For-Kubernetes introduces core pinning and
isolation to K8s without requiring changes to the code base
 CMK guarantees high priority workloads are pinned to
exclusive cores
 Gives a performance boost to high priority applications
 Negates the noisy neighbor* scenario
reference
 https://github.com/Intel-Corp/CPU-Manager-for-Kubernetes
WITHOUTCMK:CPUPinningandIsolation
Core0
CPU0 CPU1
Target
Workload
Core1
CPU2 CPU3
Noisy
Neighbor
Workload
Core0
CPU0 CPU1
Target
Workload
Core1
CPU2 CPU3
Noisy
Neighbor
Workload
WITHCMK:CPUPinningandIsolation
Noisy
Neighbor
Workload
* Noisy Neighbor Workload: An application that causes other virtual applications that share
the infrastructure to suffer from uneven performance

22. 22
DatacenterSolutionsGroup
HugepageNativeSupportinKubernetes
 No resource management of Huge Pages in kubernetes
 Responsibility of the cluster operator to handle it manually
problem
solution
 Huge Pages introduced as first class resource in kubernetes
 Support for Huge Pages via hugetlbfs enabled through a memory backed volume plugin
 Inherent accounting of Huge Pages
 Automatic relinquishing of Huge Pages in case of unexpected process termination
reference
 Alpha support for pre-allocated hugepages
 Hugetlbfs support based on empty dir volume plugin

23. 23
DatacenterSolutionsGroup
PlatformTelemetrySystemsupportinKubernetes
Compute Network Storage
Intel Run Sure Technologies
Resource
Telemetry
Interfaces
Open Collection
BasePlatform
Intel Infrastructure Management Technologies
See: Platform Service Assurance site (not including containers specific data):
https://networkbuilders.intel.com/network-technologies/serviceassurance
Container
Container and
Platform Telemetry
Platform
Telemetry
Container
Telemetry
Container
Telemetry

24. 24
DatacenterSolutionsGroup
Kuryr-Kubernetes
• Using Kuryr, we can also enable
Neutron Advanced Networking
features like Security groups, NAT etc.
• We can use any Neutron Plugin with
this like OVS, Linux Bridge, Midonet,
VPP
Openstack Neutron
API
Kuryr plugin
kubelet
Container Network Interface(CNI)
Kubernetes Network plugin
Kuryr–Kubernetes
Networkingcontrolflowdiagram
Pod Network ns
container container
br-tap
OVS Bridge Port Midonet Port
Linux Bridge Port
Any Neutron
Plug-in Port Type
VETH pair
tap
Bridge
Kuryr-Kubernetes
Networkingdataflowdiagram
Example:ContainerNetworkingusingKuryr

25. 25
DatacenterSolutionsGroup
Socket App
Nova
DPDK Pod Non DPDK
Pod
BSD Sockets API
Layer 4
Layer 3
Layer 2
Network
Appliance
Virt
I/O
Virt
I/O
Neutron
Virtual
Machine
MasterVMForContainersENABLINGDPDKinNestedContainers
 One Virtual Machine to many Containers
 Target: 1k Containers per VM
 Container Data Plane performance using existing & well
supported abstractions
objectives
Usecase
 Elasticity and scalability of containerized VNF application in VM
benefits
 VT-x ring de-privileging to move the VM and Container into user
space, making it accessible to the user space vSwitch with just a
single copy
 Standard Virtio interface that supports both interrupt and poll
modes, VNF and Cloud based applications
 Standard Vhost shared memory interface between DPDK vSwitch
and VNF
solutions
 Enabling DPDK in containers using VIRTIO
 Using Kuryr–Kubernetes, orchestrate the Dataplane networking for
DPDK workload using the infrastructure vSwitch

26. 26
DatacenterSolutionsGroup
EngagewithIntel
Open Source
POC
Experience Kits
Best Practice Guidelines
Software community Engagewithintel
Container
Capabilities
CONTAINER
NETWORKING
Intel is addressing key challenges to using containers for NFV use cases
Many of these have been open sourced already
Explore more information available on Intel’s Network Builders site
https://networkbuilders.intel.com/network-technologies/container-experience-kits
VNF

27. 27
DatacenterSolutionsGroup
Thankyou