Más contenido relacionado La actualidad más candente (20) Similar a Building DataCenter networks with VXLAN BGP-EVPN (20) Building DataCenter networks with VXLAN BGP-EVPN2. Cisco Confidential© 2015 Cisco and/or its affiliates. All rights reserved. 2
Building DataCenter Networks
with VXLAN BGP-EVPN
Lukas Krattiger
Principal Technical Marketing Engineer (PTME)
May 2016
In collaboration with
@CCIE21921
3. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 3
Session Objectives
• Focus on Data Center Networks and
Fabrics with Overlays
• Closer Look on Packet Encapsulation
(VXLAN)
Encapsulation and Forwarding
Underlay – the Transport for the Overlay
• Closer Look on Packet Encapsulation
(BGP EVPN)
Control-Plane – Exchanging Information
Optimizing the Forwarding
4. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 4
Session Non-Objectives
• Deep-Dive into FabricPath
There are many Sessions and Recordings
• Comparison between different Orchestration
and Management Tools
• Automation Workflows or Services Catalogs
5. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 5Cisco ConfidentialCisco Confidential© 2016 Cisco and/or its affiliates. All rights reserved. 5
“We can NOT solve our Problems
with the same Thinking we used
when we Created them”
Albert Einstein
6. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 6
Agenda
• Introduction to Data Center Fabrics
• VXLAN with BGP EVPN
• Overview
• Underlay
• Control & Data Plane
• Multi-Tenancy
• “Stories” and Use-Cases
• Fabric Management & Automation
7. Cisco Confidential 7© 2015 Cisco and/or its affiliates. All rights reserved.
Introduction to Data Center
Fabrics
8. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 8
STP
VPC
MAN/WAN
FabricPath
MAN/WAN
FabricPath
/BGP
MAN/WAN
VXLAN
/EVPN
VXLAN
Data Center “Fabric” Journey (Standalone)
9. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 9
Data Center Fabric Properties
Extended Namespace
Scalable Layer-2 Domains
Integrated Route and Bridge
Multi-Tenancy
Hybrid Overlays
Inter-Pod connectivity
10. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 10
Overlay Based Data Center Fabrics
• Desirable Attributes:
Mobility
Segmentation
Scale
Automated & Programmable
Abstracted consumption models
Full Cross Sectional Bandwidth
Layer-2 + Layer-3 Connectivity
Physical + Virtual
RR RR
11. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 11
Overlay Based Data Center: Edge Devices
Network Overlays Hybrid OverlaysHost Overlays
• Virtual end-points only
• Single admin domain
• VXLAN, NVGRE, STT
• Physical and Virtual
• Resiliency + Scale
• X-Organizations/Federation
• Open Standards
• Router/Switch end-points
• Protocols for Resiliency/Loops
• Traditional VPNs
• VXLAN, OTV, VPLS, LISP, FP
V
V
V
V
V
V
12. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 12
Data Center Fabric Properties
• Any subnet, anywhere, rapidly
• Reduced Failure Domains
• Extensible Scale & Resiliency
• Profile Controlled Configuration
RR RR
Full Bi-Sectional Bandwidth (N Spines)
Any/All Leaf Distributed Default Gateways
Any/All Subnets on Any Leaf
13. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 13
Spine/Leaf Topologies
• High Bi-Sectional Bandwidth
• Wide ECMP: Unicast or Multicast
• Uniform Reachability, Deterministic Latency
• High Redundancy: Node/Link Failure
• Line rate, low latency, for all traffic
14. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 14
Variety of Fabric Sizes
• Fabric size: Hundreds to 10s of Thousands
of 10G ports
• Variety of Building Blocks:
Varying Size
Varying Capacity
Desired oversubscription
Modular and Fixed
• Scale Out Architecture
Add compute, service, external connectivity as
the demand grows
More Spine, More Bandwidth, More Resiliency
16. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 16
Agenda
• Introduction to Data Center Fabrics
• VXLAN with BGP EVPN
• Overview
• Underlay
• Control & Data Plane
• Multi-Tenancy
• “Stories” and Use-Cases
• Fabric Management & Automation
17. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 17
Overview
Classic Ethernet IEEE 802.1Q Frame
Format
• Traditionally VLAN is expressed over 12 bits
(802.1Q tag)
Limits the maximum number of segments in a
Data Center to 4096 VLANs
Classic Ethernet
Frame
Destination MAC (DMAC)
Source MAC (SMAC)
802.1Q
TPID
0x8100
(16
bits)
TCI
PCP
(3 bits)
CFI
(1 bits)
VID
(12
bits)
Ether Type (Etype)
Data (Payload)
CRC/FCS
4 bytes
DMAC SMAC 802.1Q Etype CRCPayload
VLAN ID
12 bits
TPID = Tag Protocol Identifier, TCI = Tag Control Information, PCP = Priority Code Point,
CFI = Canonical Format Indicator, VID = VLAN Identifier
18. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 18
Overview
Introducing VXLAN
• Traditionally VLAN is expressed over 12 bits
(802.1Q tag)
Limits the maximum number of segments in a
Data Center to 4096 VLANs
• VXLAN leverages the VNI field with a total
address space of 24 bits
Support of ~16M segments
• The VXLAN Network Identifier (VNI/VNID) is
part of the VXLAN Header Cisco DFA
Frame
VXLAN
Frame
Classical Ethernet Frame
CRC
(new)
VxLAN
(8)
UDP
(8)
IP
(20)
Original CE Frame50 bytes
Outer
MAC
(14)
VNI
DMAC SMAC 802.1Q Etype CRCPayload
DMAC SMAC
802.1Q
optional
Etype Payload
ags
8 bits 24 bits 8 bits24 bits
Reserved ReservedVNI
VNI
19. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 19
VXLAN Frame Format
• MAC-in-IP Encapsulation
Underlay
Outer IP Header
Outer MAC Header
UDP Header
VXLAN Header
Original Layer-2 Frame
Overlay
14 Bytes
(4 Bytes Optional)
Ether Type
0x0800
VLAN ID
Tag
VLAN Type
0x8100
Src. MAC Address
Dest. MAC Address 48
48
16
16
16
20 Bytes
Dest. IP
Source IP
Header
Checksum
Protocol 0x11 (UDP)
IP Header
Misc. Data
72
8
16
32
32
8 Bytes
Checksum 0x0000
UDP Length
VXLAN Port
Source
Port
16
16
16
16
8 Bytes
Reserved
VNI
Reserved
VXLAN Flags RRRRIRRR 8
24
24
8
Src VTEP MAC Address
Next-Hop MAC Address
Src and Dst addresses
of the VTEPs
Allows for 16M
possible Segments
UDP 4789
Hash of the inner L2/L3/L4 headers of
the original frame.
Enables entropy for ECMP Load
balancing in the Network.
50(54)BytesofOverhead
20. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 20
Data Center Fabric Properties
Extended Namespace
Scalable Layer-2 Domains
Integrated Route and Bridge
Multi-Tenancy
21. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 21
Overlay Services
• Layer 2
• Layer 3
• Layer 2 and Layer 3
Tunnel Encapsulation
Underlay Transport
Network
Control Plane
• Peer Discovery mechanism
• Route Learning and Distribution
– Local Learning
– Remote Learning
Data Plane
• Overlay Layer 2/Layer 3 Unicast traffic
• Overlay Broadcast, Unknown Unicast,
Multicast traffic (BUM traffic) forwarding
– Ingress Replication
– Multicast
Understanding Overlay Technologies
22. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 22
Why VXLAN?
VXLAN provides a Network with
Segmentation, IP Mobility, and Scale
• “Standards” based Overlay (RFC 7348)
• Leverages Layer-3 ECMP – all links forwarding
• Increased Name-Space to 16M identifier
• Integration of Physical and Virtual
• It’s SDN
23. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 23
IP Interface
IP Interface
Edge Device
Edge Device
Edge Device
Edge Device
Edge Device
Edge Device
Local LAN
Segment
Local LAN
Segment
Physical Servers
Virtual Servers
VXLAN Taxonomy (1)
24. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 24
Local LAN
Segment
Local LAN
Segment
Physical Servers
Virtual Servers
VTEP
VTEP
VTEP
VTEP
VTEP
VTEP
VXLAN Taxonomy (2)
VTEP: VXLAN Tunnel End-Point
VNI/VNID: VXLAN Network Identifier
25. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 25
Getting the Puzzle Together!
Driving
Standards based
Overlay-
Evolution with
VXLAN BGP
EVPN
26. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 26
What is VXLAN with BGP EVPN?
• Standards based Overlay (VXLAN) with Standards based Control-Plane
(BGP)
• Layer-2 MAC and Layer-3 IP information distribution by Control-Plane
(BGP)
• Forwarding decision based on Control-Plane (minimizes flooding)
• Integrated Routing/Bridging (IRB) for Optimized Forwarding in the Overlay
• Multi-Tenancy At Scale
27. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 27
EVPN – Ethernet VPN
Control-
Plane
EVPN MP-BGP - RFC 7432
Data-
Plane
Multi-Protocol Label
Switching (MPLS)
draft-ietf-l2vpn-evpn
Provider Backbone Bridges
(PBB)
draft-ietf-l2vpn-pbb-evpn
Network Virtualization
Overlay (NVO)
draft-ietf-bess-evpn-overlay
EVPN over NVO Tunnels (ie VXLAN) for Data Center Fabric
encapsulations
Provides Layer-2 and Layer-3 Overlays over simple IP
Networks
28. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 28
Cisco’s VXLAN related IETF RFCs & Drafts
ID Title Category
RFC 7348 Virtual eXtensible Local Area Network Data Plane
RFC 7432 BGP MPLS based Ethernet VPNs Control Plane
draft-ietf-bess-evpn-overlay A Network Virtualization Overlay Solution using EVPN Control Plane
draft-ietf-bess-evpn-inter-subnet-forwarding Integrated Routing and Bridging in EVPN Control Plane
draft-ietf-bess-l2vpn-evpn-prefix-advertisement IP Prefix Advertisement in E-VPN Control Plane
draft-tissa-nvo3-oam-fm NVO3 Fault Management / OAM Management Plane
29. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 29
VXLAN Evolution
• Multi-Protocol BGP (MP-BGP) based Control-Plane using EVPN NLRI
(Network Layer Reachability Information)
• Make Forwarding decisions at VTEPs for Layer-2 (MAC) and Layer-3
(IP); Integrated Route/Bridge (IRB)
• Reduce Flooding
• Reduce impact of ARP on the Network
• Standards Based (IETF draft)
Protocol Learning
• Workload MAC and IP
Addresses learnt by VXLAN
Edge Devices (NVEs)
• Advertises Layer-2 and
Layer-3 Address-to-VTEP
Association (Overlay
Control-Plane)
• Flood Prevention
• Optimized ARP forwarding
30. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 30
VXLAN Evolution
• Forward based on MAC or IP address learnt via Control-Plane (MP-
BGP EVPN)
• Make routing decisions at VTEPs
• Scale and Multipathing (ECMP)
• Leverage Layer-3 Gateway capabilities along with Protocol
Information
• LISP-ish / LISP-like approach for Host/IP Mobility
Location (VTEP), Identifier (MAC, IP of End-Host)
IP Services
• VXLAN Routing
• Distributed Anycast
Gateway (requires Overlay
Control-Plane)
• Multi-Tenancy
31. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 31
Getting the Puzzle Together!
• Optimized Networks with VXLAN
Overlay
(VXLAN)Integrated
Route/Bridge
Underlay
BGP
(EVPN)
32. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 32
http://blogs.cisco.com/datacenter/vxlanevpn-
standards-based-overlay-with-control-plane
http://packetpushers.net/show-233-cisco-nexus-using-bgp-
as-a-vxlan-control-plane-sponsored/
http://blogs.cisco.com/cin/network-like-its-1999-with-bgp-
evpn
http://www.slideshare.net/robboyd/techwisetv-
workshop-secrets-of-scalable-multitenancy
33. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 33
Agenda
• Introduction to Data Center Fabrics
• VXLAN with BGP EVPN
• Overview
• Underlay
• Control & Data Plane
• Multi-Tenancy
• “Stories” and Use-Cases
• Fabric Management & Automation
34. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 34
Deployment Considerations
• MTU and Overlays
• Unicast Routing Protocol and IP Addressing
• Multicast for BUM* Traffic Replication
*BUM: Broadcast, Unknown Unicast & Multicast
35. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 35
MTU and VXLAN
• VXLAN adds 50 Bytes (or 54 Bytes) to the
Original Ethernet Frame
• Avoid Fragmentation by adjusting the IP
Networks MTU
• Data Centers often require Jumbo MTU;
most Server NIC do support up to 9000
Bytes
• Using a MTU of 9216* Bytes accommodates
VXLAN Overhead plus Server max. MTU
Underlay
Outer IP Header
Outer MAC Header
UDP Header
VXLAN Header
Original Layer-2 Frame
Overlay
50(54)BytesofOverhead
*Cisco Nexus 5600/6000 switches only support 9192 Byte for Layer-3 Traffic
36. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 36
Building your IP Network – Interface Principles (1)
• Know your IP addressing and IP scale
requirements
• Separate VTEP from Routing Protocol from
RP* Loopback
• Best to use individual Aggregates for the
Underlay
Unicast Routing p2p** Links
Unicast Routing Loopbacks
VTEP (NVE) Loopback
Multicast Routing Loopback (RP)
• IPv4 only (today)
*RP: Rendezvous-Point (Multicast)
**p2p: Point-to-Point
p2p Links
10.1.1.2/30
Rendezvous-Point
Loopback
10.254.254.1Routing Loopback
10.10.10.203/32
p2p Links
10.1.1.1/30
p2p Agg: 10.1.1.0/24
RID Agg: 10.10.10.0/24
VTEP Agg: 10.200.200.0/24
RP Agg: 10.254.254.0/24
Routing Loopback
10.10.10.101/32
V
VTEP Loopback
10.200.200.101/32
V
V
V
V
V
37. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 37
Building your IP Network – Interface Principles (2)
• Routed Ports/Interfaces
Layer-3 Interfaces between Spine and Leaf (no
switchport)
For each Point-2-Point (P2P) connection,
minimum /31 required
Alternative, use IP Unnumbered (/32)
• Use Loopback as Source-Interface for VTEP
(NVE*)
*NVE: Network Virtualization Edge
VTEP: VXLAN Tunnel End-Point
V
V
V
V
V
V
38. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 38
Building your IP Network – Some Math
*RID: Router ID; Unicast Routing Loopback
Example from depicted topology:
4 Spine * 6 Leaf = 24 Point-2-Point (P2P) Links
24 Links * 2 (/31) + 10 RID* + 6 VTEP + 4 Spine
= 48 IP Addresses for P2P Links
= 20 IP Addresses for Loopback Interfaces
68 IP Addresses required == /25 Prefix
A More Realistic Scenario:
4 Spine * 40 Leaf = 160 Point-2-Point (P2P) Link
160 Links * 4 (/30) + 44 RID* + 80 VTEP + 4 Spine
= 640 IP Addresses for P2P Links
= 128 IP Addresses for Loopback Interface
768 IP Addresses required == /22 Prefix
V
V
V
V
V
V
39. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 39
IP Unnumbered– Simplifying the Math
*RID: Router ID; Unicast Routing Loopback
Example from depicted topology:
4 Spine + 6 Leaf = 10 Individual Devices
= 6 IP Addresses for Loopback Interface (Used for VTEP)
= 10 IP Address Loopback Interface (RID* & IP Unnumbered)
16 IP Addresses required == /28 Prefix
A More Realistic Scenario:
4 Spine + 40 Leaf = 44 Individual Devices
= 40 IP Addresses for Loopback Interface (Used for VTEP)
= 44 IP Addresses for Loopback Interface (RID* & IP Unnumbered)
84 IP Addresses required == /25 Prefix
V
V
V
V
V
V
40. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 40
Building your IP Network – Routing Protocols; OSPF
• OSPF – watch your Network type!
Network Type Point-2-Point (P2P)
Preferred (only LSA type-1)
No DR/BDR election
Suits well for routed interfaces/ports (optimal from a
LSA Database perspective)
Full SPF calculation on Link Change
Network Type Broadcast
Suboptimal from a LSA Database perspective (LSA
type-1 & 2)
DR/BDR election
Additional election and Database Overhead
V
V
V
V
V
V
41. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 41
Building your IP Network – Routing Protocols; IS-IS
• IS-IS – what was this CLNS?
•Independent of IP (CLNS)
•Well suited for routed interfaces/ports
•No SPF calculation on Link change; only if
Topology changes
•Fast Re-convergence
•Not everyone is familiar with it
*CLNS: Connection-Less Network Service
V
V
V
V
V
V
42. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 42
Building your IP Network – Routing Protocols; eBGP
• eBGP – Service Provider style
•Two Different Models
•Two-AS
•Multi-AS
•BGP is a Distance Vector
•AS* are used to calculate the Path (AS_Path)
•If Underlay is eBGP, your Overlay becomes
eBGP
*AS: Autonomous System
V
V
V
V
V
V
43. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 43
Building your IP Network – Routing Protocols; eBGP
• eBGP – TWO-AS, yes it works!
•Total of 8 eBGP Peering (with 4 Spine)
•eBGP peering for Underlay-Routing based on
physical interface
•4 Spines = 4 BGP Peering per Leaf
•Advertise all Infrastructure Loopbacks
•eBGP peering for Overlay-Routing (EVPN)
•Loopback to Loopback Peering
•4 Spines = 4 BGP Peering
•Requires some BGP config knobs
•Disable BGP AS-Path check
•Next-Hop needs to be Unchanged
•Retain all Routes on Spine (not a RR)
V
V
V
V
V
V
AS#65500
44. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 44
Building your IP Network – Routing Protocols; eBGP
• eBGP – Multi-AS
•Total of 8 eBGP Peering (with 4 Spine)
•eBGP peering for Underlay-Routing based on
physical interface
•4 Spines = 4 BGP Peering per Leaf
•Advertise all Infrastructure Loopbacks
•eBGP peering for Overlay-Routing (EVPN)
•Loopback to Loopback Peering
•4 Spines = 4 BGP Peering
•Requires some BGP config knobs
•Next-Hop needs to be Unchanged
•Retain all Routes on Spine (not a RR)
V
V
V
V
V
V
AS#65500
45. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 45
Multicast Enabled Underlay
May use PIM-ASM or PIM-BiDir (Different hardware has different capabilities)
• Spine and Aggregation Switches make good Rendezvous-Point (RP) Locations in Topologies
• Reserve a range of Multicast Groups (Destination Groups/DGroups) to service the Overlay
and optimize for diverse VNIs
• In Spine/Leaf topologies with lean Spine
Use multiple Rendezvous-Point across the multiple Spines
Map different VNIs to different Rendezvous-Point for simple load balancing measure
Use Redundant Rendezvous-Pint
• Design a Multicast Underlay for a Network Overlay, Host VTEPs will leverage this Network
Nexus 1000v Nexus 3000 Nexus 5600 Nexus 7000/F3 Nexus 9000
ASR 1000
CSR 1000
ASR 9000
Multicast Mode IGMP v2/v3 PIM ASM PIM BiDir PIM ASM / PIM BiDir PIM ASM PIM BiDir PIM ASM / PIM BiDir
46. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 46
Multicast Enabled Underlay – PIM ASM*
• PIM Sparse-Mode (ASM)
• Redundant Rendezvous-Point using PIM
Anycast-RP or MSDP
• Source-Tree or Unidirectional Shared-Tree
(Source-Tree shown)
•Shared-Tree will always use RP for forwarding
• 1 Source-Tree per Multicast-Group per
VTEP (each VTEP is Source & Receiver)
*ASM: Any-Source Multicast
V
V
V
V
V
V
Rendezvous-PointRP
RP
VTEP1 (S,G) Tree
VTEP2 (S,G) Tree
47. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 47
Multicast Enabled Underlay – BiDir-PIM*
• Bidirectional PIM (BiDir)
• Redundant Rendezvous-Point using
Phantom-RP
• Building Bi-Directional Shared-Tree
Uses shortest path between Source and
Receiver with RP as routing-vector
• 1 Shared-Tree per Multicast-Group
*BiDir-PIM: Bidirectional PIM
V
V
V
V
V
V
Rendezvous-PointRP
RP
VTEPs (*,G) Tree
48. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 48
To Remember - Multicast Enabled Underlay
• Multi-Destination Traffic (Broadcast, Unknown Unicast, etc.) needs to be
replicated to ALL VTEPs serving a given VNI
Each VTEP is Multicast Source & Receiver
• For a given VNI, all VTEPs act as a Sender and a Receiver
• Head-End Replication will depend on hardware scale/capability
• Resilient, efficient, and scalable Multicast Forwarding is highly desirable
Choose the right Multicast Routing Protocol for your need (type/mode)
Use redundant Multicast Rendezvous Points (Spine/Aggregation generally preferred)
99% percent of Overlay problems are in the Underlay (OTV experience)
49. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 49
Agenda
• Introduction to Data Center Fabrics
• VXLAN with BGP EVPN
• Overview
• Underlay
• Control & Data Plane
• Multi-Tenancy
• “Stories” and Use-Cases
• Fabric Management & Automation
50. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 50
Multiprotocol BGP (MP-BGP) Primer
• Multiprotocol BGP (MP-BGP)
• Extension to Border Gateway Protocol
(BGP) - RFC 4760
• VPN Address-Family:
Allows different types of address families (e.g.
VPNv4, VPNv6, L2VPN EVPN, MVPN)
Information transported across single BGP
peering
RR RR
V2
V1
V3
BGP Route-ReflectorRR
iBGP Peering*
*eBGP supported without BGP Route-Reflector
51. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 51
RR RR
V2
V1
V3
BGP Route-ReflectorRR
iBGP Peering
VRF Info
Name: VRF-A
RD: 3:10.0.0.1 (auto)
Imp Route-Target 65500:50000 (auto)
Exp Route-Target 65500:50000 (auto)
VRF Info
Name: VRF-A
RD: 15:10.0.0.2 (auto)
Imp Route-Target 65500:50000 (auto)
Exp Route-Target 65500:50000 (auto)
VRF Info
Name: VRF-A
RD: 62:10.0.0.3 (auto)
Imp Route-Target 65500:50000 (auto)
Exp Route-Target 65500:50000 (auto)
Multiprotocol BGP (MP-BGP) Primer
• VPN segmentation for tenant routing (Multi-
Tenancy)
Route Distinguisher (RD)
8-byte field of VRF parameters
value to make VPN prefix unique:
RD + VPN prefix
52. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 52
RR RR
V2
V1
V3
BGP Route-ReflectorRR
iBGP Peering
VRF Info
Name: VRF-A
RD: 3:10.0.0.1 (auto)
Imp Route-Target 65500:50000 (auto)
Exp Route-Target 65500:50000 (auto)
VRF Info
Name: VRF-A
RD: 15:10.0.0.2 (auto)
Imp Route-Target 65500:50000 (auto)
Exp Route-Target 65500:50000 (auto)
VRF Info
Name: VRF-A
RD: 62:10.0.0.3 (auto)
Imp Route-Target 65500:50000 (auto)
Exp Route-Target 65500:50000 (auto)vrf context VRF-A
vni 50000
rd auto
address-family ipv4 unicast
route-target both auto
route-target both auto evpn
address-family ipv6 unicast
route-target both auto
route-target both auto evpn
Multiprotocol BGP (MP-BGP) Primer
• Cisco’s VXLAN/EVPN does provide
automated Route Distinguisher (RD)
Automatic uses Type 1 format
4-byte IP Address (Router ID)
4-byte Value (VRF ID)
53. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 53
RR RR
V2
V1
V3
BGP Route-ReflectorRR
iBGP Peering
MAC_A / IP_A >> V1
Route-Type2
MAC_A / IP_A >>
LOCAL
Route-Type2
BGP Advertisement
VPN-EVPN: RD:[MAC_A][IP_A]
BGP Next-Hop: V1
Route Target: 65500:50000
Label (L3VNI): 50000
Host A
MAC_A / IP_A
Multiprotocol BGP (MP-BGP) Primer
• VPN Segmentation for tenant routing (Multi-
Tenancy)
• Selective distribute VPN routes - Route
Target (RT)
8-byte field of VRF parameter
unique value to define the import/export rules
for VPN prefix
54. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 54
RR RR
V2
V1
V3
BGP Route-ReflectorRR
iBGP Peering
MAC_A / IP_A >> V1
Route-Type2
MAC_A / IP_A >>
LOCAL
Route-Type2
BGP Advertisement
VPN-EVPN: RD:[MAC_A][IP_A]
BGP Next-Hop: V1
Route Target: 65500:50000
Label (L3VNI): 50000
Host A
MAC_A / IP_A
vrf context VRF-A
vni 50000
rd auto
address-family ipv4 unicast
route-target both auto
route-target both auto evpn
address-family ipv6 unicast
route-target both auto
route-target both auto evpn
Multiprotocol BGP (MP-BGP) Primer
• Cisco’s VXLAN/EVPN does provide
automated Route Target (RT)
8-byte Route Target (2 x 4-byte)
ASN : VNI
55. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 55
Overlay with Optimized Routing
Spine
RR RR
V
V
V
V
V
V
EVPN Control Plane -- Host and Subnet Route Distribution
BGP Update
• Host-MAC
• Host-IP
• Internal IP Subnet
• External Prefixes
RR
Route-Reflectors deployed for
scaling purposes (iBGP)
BGP Adjacencies
Border
56. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 56
Overlay with Optimized Routing
Spine
RR RR
V
V
V
V
V
V
EVPN Control Plane -- Host and Subnet Route Distribution
BGP Update
• Host-MAC
• Host-IP
• Internal IP Subnet
• External Prefixes
RR
Route-Reflectors deployed for
scaling purposes (iBGP)
BGP Adjacencies
Border
Scalable Multi-Tenancy with Multiprotocol
BGP
EVPN Address-Family: Host MAC+IP, internal/external IP Subnets
BGP enhanced for Fast Convergence at Large Scale
Extensions for Fast and Seamless Host
Mobility
Distributed Gateway with Traffic Flow
Symmetry
ARP Suppression
57. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 57
Host A
MAC_A / IP_A
RR RR
V2
V1
V3
BGP Route-ReflectorRR
iBGP Peering*
Route
Type
MAC, IP L2VNI
(“VLAN”)
L3VNI
(“VRF”)
NH Encap Seq
2 MAC_A, IP_A 30001 50001 IP_V1 8:VXLAN 0
Host Advertisement
• Host Attaches
Host “A” attaches to Edge Device (VTEP)
• VTEP V1 advertises Host “A” reachability
information
MAC and L2VNI [mandatory]
IP and L3VNI [optional]
depending on ARP
• Additional route attributes advertised
MPLS Label1 (L2VNI)
MPLS Label2 (L3VNI)
Extended Communities
58. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 58
V2# show bgp l2vpn evpn 192.168.1.73
BGP routing table information for VRF default, address family L2VPN EVPN
Route Distinguisher: 10.0.0.1:32868
BGP routing table entry for [2]:[0]:[0]:[48]:[0050.56a3.c2bb]:[32]:[192.168.1.73]/272,
version 4
Paths: (1 available, best #1)
Flags: (0x000202) on xmit-list, is not in l2rib/evpn, is locked
Advertised path-id 1
Path type: internal, path is valid, is best path, no labeled nexthop
AS-Path: NONE, path sourced internal to AS
10.0.0.1 (metric 3) from 10.0.0.111 (10.0.0.111)
Origin IGP, MED not set, localpref 100, weight 0
Received label 30001 50001
Extcommunity: RT:65501:30001 RT:65501:50001 ENCAP:8 Router MAC:5087.89d4.5495
Originator: 10.0.0.1 Cluster list: 10.0.0.111
Ethernet Segment
Identifier
Ethernet Tag
Identifier
MAC Address
Length
MAC Address IP Address Length IP Address
Route Type:
2 - MAC/IP
L3VNI
Route Target:
L2VNI (VLAN)
Route Target:
L3VNI (VRF)
Router MAC of
Remote VTEP
Overlay Encapsulation:
8 - VXLAN
Remote VTEP
IP Address
L2VNI
59. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 59
Virtual Switch
RR RR
Host A
MAC_A / IP_A
Host C
MAC_C / IP_C
Host Y
MAC_Y / IP_Y
Host B
MAC_B / IP_B
V1
V3
V2
VTEPs advertise End-Host reachability
information (MAC,IP) within MP-BGP1
1
1
1
MAC, IP L2VN
I
L3VN
I
NH
MAC_C, IP_C 30001 50001 local
MAC_Y, IP_Y 30002 50001 local
MAC, IP L2VN
I
L3VN
I
NH
MAC_B, IP_B 30001 50002 local
MAC, IP L2VN
I
L3VN
I
NH
MAC_A, IP_A 30001 50001 local
Protocol Learning & Distribution
60. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 60
Virtual Switch
RR RR
Host A
MAC_A / IP_A
Host C
MAC_C / IP_C
Host Y
MAC_Y / IP_Y
Host B
MAC_B / IP_B
V1
V3
V2
BGP Route-Reflector “reflects” Overlay related
reachability information to other VTEPs2
2
2
MAC, IP L2VN
I
L3VN
I
NH
MAC_C, IP_C 30001 50001 local
MAC_Y, IP_Y 30002 50001 local
MAC, IP L2VN
I
L3VN
I
NH
MAC_B, IP_B 30001 50001 local
MAC, IP L2VN
I
L3VN
I
NH
MAC_A, IP_A 30001 50001 local
2
Protocol Learning & Distribution
61. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 61
Virtual Switch
RR RR
Host A
MAC_A / IP_A
Host C
MAC_C / IP_C
Host Y
MAC_Y / IP_Y
Host B
MAC_B / IP_B
V1
V3
V2
VTEPs receive respective reachability information
and installs them related to route-policy into RIB/FIB
MAC, IP L2VN
I
L3VN
I
NH
MAC_C, IP_C 30001 50001 local
MAC_Y, IP_Y 30002 50001 local
MAC_A, IP_A 30001 50001 IP_V1
MAC_B, IP_B 30001 50001 IP_V2
MAC, IP L2VN
I
L3VN
I
NH
MAC_B, IP_B 30001 50001 local
MAC_A, IP_A 30001 50001 IP_V1
MAC_C, IP_C 30001 50001 IP_V3
MAC_Y, IP_Y 30002 50001 IP_V3
MAC, IP L2VN
I
L3VN
I
NH
MAC_A, IP_A 30001 50001 local
MAC_B, IP_B 30001 50001 IP_V2
MAC_C, IP_C 30001 50001 IP_V3
MAC_Y, IP_Y 30002 50001 IP_V3
3 3
3
3
Protocol Learning & Distribution
62. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 62
RR RR
BGP Route-ReflectorRR
iBGP Peering*
Route
Type
MAC, IP L3VNI
(“VRF”)
NH Encap
5 Subnet_A/24 50001 IP_V1 8:VXLAN
V2
V1
V3
Subnet Route Advertisement
• IP Prefix Redistribution
From “Direct” (connected), “Static” or
dynamically learned Routes
• VTEP V1 advertises local Subnet through
redistribution of “Direct” (connected) routes
IP Prefix, IP Prefix Length, and L3VNI
• Additional route attributes advertised
MPLS Label (L3VNI)
Extended Communities
63. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 63
RR RR
V2
V1
V3
BGP Route-ReflectorRR
iBGP Peering*
Route
Type
MAC, IP L3VNI
(“VRF”)
NH Encap
5 Subnet_A/24 50000 IP_V1 8:VXLAN
5 Subnet_A/24 50001 IP_V2 8:VXLAN
5 Subnet_A/24 50001 IP_V3 8:VXLAN
Route
Type
MAC, IP L3VNI
(“VRF”)
NH Encap
5 Subnet_A/24 50001 IP_V1 8:VXLAN
Subnet Route Advertisement
• If multiple VTEP announce same IP Prefix,
Equal Cost Multipath (ECMP) will apply
• VTEP V1 advertises local Subnet through
redistribution of “Direct” (connected) routes
IP Prefix, IP Prefix Length, and L3VNI
• Additional route attributes advertised
MPLS Label (L3VNI)
Extended Communities
64. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 64
RR RR
V2
V1
V3
BGP Route-ReflectorRR
iBGP Peering*
Route
Type
MAC, IP L3VNI
(“VRF”)
NH Encap
5 Subnet_X/24 50001 IP_V1 8:VXLAN
Subnet Route Advertisement
• IP Prefix Learning
via BGP with VRF-Lite (Inter-AS Option A)
via LISP on Nexus 7000/7700
via other routing protocol (static or dynamic)
• VTEP V1 participated in external Peering
(LISP, BGP, OSPF etc.) and advertises
learned IP Prefixes into the Fabric
IP Prefix
IP Prefix Length
L3VNI
65. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 65
V2# show bgp l2vpn evpn 192.168.2.0
BGP routing table information for VRF default, address family L2VPN EVPN
Route Distinguisher: 10.0.0.1:3
BGP routing table entry for [5]:[0]:[0]:[24]:[192.168.2.0]:[0.0.0.0]/224, version 3
Paths: (1 available, best #1)
Flags: (0x000002) on xmit-list, is not in l2rib/evpn, is locked
Advertised path-id 1
Path type: internal, path is valid, is best path, no labeled nexthop
AS-Path: NONE, path sourced internal to AS
10.0.0.1 (metric 3) from 10.0.0.111 (10.0.0.111)
Origin incomplete, MED 0, localpref 100, weight 0
Received label 50001
Extcommunity: RT:65501:50001 ENCAP:8 Router MAC:5087.89d4.5495
Originator: 10.0.0.1 Cluster list: 10.0.0.111
Ethernet Segment
Identifier
Ethernet Tag
Identifier
IP Prefix Length IP Prefix GW IP Address
Route Type:
5 – IP Prefix
L3VNI
Route Target:
L3VNI (VLAN)
Router MAC of
Remote VTEP
Overlay Encapsulation:
8 - VXLAN
Remote VTEP
IP Address
66. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 66
ARP Suppression
• VXLAN/EVPN
Host A
MAC_A / IP_A Host B
MAC_B / IP_B
Virtual Switch
Host C
MAC_C / IP_C
Host Y
MAC_Y / IP_Y
RR RR
V2
V1
V3
1 ARP Request sent for IP_B sent from Host A
MAC, IP VNI NH
MAC_A, IP_A 30001 IP_V1
MAC_B, IP_B 30001 IP_V2
MAC, IP VNI NH
MAC_A, IP_A 30001 IP_V1
MAC_C, IP_C 30001 IP_V3
MAC_Y, IP_Y 30002 IP_V3
2 V1 knows about IP_B and can respond.
No need for ARP forwarding across the Network
MAC, IP VNI NH
MAC_B, IP_B 30001 IP_V2
MAC_C, IP_C 30001 IP_V3
MAC_Y, IP_Y 30002 IP_V3ARP Request for IP_B
Src MAC: MAC_A
Dst MAC: FF:FF:FF:FF:FF:FF
1
2
ARP Response for IP_B
Src MAC: MAC_B
Dst MAC: MAC_A
67. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 67
ARP Handling on Lookup “Miss” (1)
• VXLAN/EVPN
Host A
MAC_A / IP_A Host B
MAC_B / IP_B
Virtual Switch
Host C
MAC_C / IP_C
Host Y
MAC_Y / IP_Y
RR RR
1 ARP Request sent for IP_B sent from Host A
MAC, IP VNI NH
MAC_A, IP_A 30001 IP_V1
2 Miss of IP_B. Forward ARP Request to all
Ports except source-port (ARP snooping)
MAC, IP VNI NH
MAC_C, IP_C 30001 IP_V3
MAC_Y, IP_Y 30002 IP_V3
ARP Request for IP_B
Src MAC: MAC_A
Dst MAC: FF:FF:FF:FF:FF:FF
1
Missing
“B”
2
2
V2
V1
V3
MAC, IP VNI NH
MAC_A, IP_A 30001 IP_V1
MAC_C, IP_C 30001 IP_V3
MAC_Y, IP_Y 30002 IP_V3
ARP Request for IP_B
Src MAC: MAC_A
Dst MAC: FF:FF:FF:FF:FF:FF
ARP Request for IP_B
Src MAC: MAC_A
Dst MAC: FF:FF:FF:FF:FF:FF
68. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 68
ARP Handling on Lookup “Miss” (2)
• VXLAN/EVPN
Host A
MAC_A / IP_A Host B
MAC_B / IP_B
Virtual Switch
Host C
MAC_C / IP_C
Host Y
MAC_Y / IP_Y
RR RR
3 ARP Response is sent to V2
MAC, IP VNI NH
MAC_A, IP_A 30000 V1
4 V2 will populate this information in the
control-plane (learn) and forward it subsequently
MAC, IP VNI NH
MAC_C, IP_C 30000 V3
MAC_Y, IP_Y 30001 V3
ARP Response from IP_B
Src MAC: MAC_B
Dst MAC: MAC_A
3
MAC, IP VNI NH
MAC_A, IP_A 30001 IP_V1
MAC_B, IP_B 30001 IP_V2
ARP Response for IP_B
Src MAC: MAC_B
Dst MAC: MAC_A
4
4
MAC, IP VNI NH
MAC_A, IP_A 30001 IP_V1
MAC_C, IP_C 30001 IP_V3
MAC_Y, IP_Y 30002 IP_V3
V2
V1
V3
MAC, IP VNI NH
MAC_C, IP_C 30001 IP_V3
MAC_Y, IP_Y 30002 IP_V3
MAC_B, IP_B 30001 IP_V2
69. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 69
Packet Forwarding (Bridge)
• VXLAN/EVPN
Host A
MAC_A / IP_A Host B
MAC_B / IP_B
RR RR
MAC, IP VNI NH
MAC_B, IP_B 30001 Local
MAC_A, IP_A 30001 IP_V1
MAC, IP VNI NH
MAC_A, IP_A 30001 Local
MAC_B, IP_B 30001 IP_V2
4
SIP: IP_A
DIP: IP_B
SMAC: MAC_A
DMAC: MAC_B
1
SIP: IP_A
DIP: IP_B
SMAC: MAC_A
DMAC: MAC_B
Underlay
SIP: IP_V1
DIP: IP_V2
SMAC: MAC_V1
DMAC: hop-by-hop
UDP
VXLAN VNID: 30001
SMAC: MAC_A
DMAC: MAC_B
SIP: IP_A
DIP: IP_B
Overlay
2
SIP: IP_V1
DIP: IP_V2
SMAC: hop-by-hop
DMAC: MAC_V2
Underlay
VXLAN VNID: 30001
SMAC: MAC_A
DMAC: MAC_B
SIP: IP_A
DIP: IP_B
UDP
Overlay
3
V2
V1
V3
70. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 70
Packet Forwarding (Route)
• VXLAN/EVPN
Host A
MAC_A / IP_A Host F
MAC_F, IP_F
RR RR
4
SIP: IP_A
DIP: IP_F
SMAC: MAC_A
DMAC: MAC_GW
1
SIP: IP_A
DIP: IP_F
SMAC: MAC_GW
DMAC: MAC_F
Underlay
SIP: IP_V1
DIP: IP_V2
SMAC: MAC_V1
DMAC: hop-by-hop
UDP
VXLAN VNID: 50001
SMAC: MAC_A
DMAC: MAC_GW
SIP: IP_A
DIP: IP_F
Overlay
2
SIP: IP_V1
DIP: IP_V2
SMAC: hop-by-hop
DMAC: MAC_V2
Underlay
VXLAN VNID: 50001
SMAC: MAC_GW
DMAC: MAC_F
SIP: IP_A
DIP: IP_F
UDP
Overlay
3
V2
V1
V3
MAC, IP VNI NH VRF
MAC_A, IP_A 30001 Local 50001
MAC_F, IP_F 30005 IP_V2 50001
MAC, IP VNI NH VRF
MAC_A, IP_A 30001 Local 50001
MAC_F, IP_F 30005 E1/4 50001
71. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 71
Packet Forwarding (Route) – Silent Host
• VXLAN/EVPN
Host A
MAC_A / IP_A Host F
MAC_F, IP_F
RR RR
4
SIP: IP_A
DIP: IP_F
SMAC: MAC_A
DMAC: MAC_GW
1
SIP: IP_A
DIP: IP_F
SMAC: MAC_GW
DMAC: MAC_F
Underlay
SIP: IP_V1
DIP: IP_V2
SMAC: MAC_V1
DMAC: hop-by-hop
UDP
VXLAN VNID: 50001
SMAC: MAC_A
DMAC: MAC_GW
SIP: IP_A
DIP: IP_F
Overlay
2
SIP: IP_V1
DIP: IP_V2
SMAC: hop-by-hop
DMAC: MAC_V2
Underlay
VXLAN VNID: 50001
SMAC: MAC_GW
DMAC: MAC_F
SIP: IP_A
DIP: IP_F
UDP
Overlay
3
V2
V1
V3
MAC, IP VNI NH VRF
MAC_A, IP_A 30000 Local 50001
Subnet F 30005 IP_V2 50001
MAC, IP VNI NH VRF
MAC_A, IP_A 30000 Local 50001
Subnet F 30005 E1/4 50001
72. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 72
Data Center Fabric Properties
Extended Namespace
Scalable Layer-2 Domains
Integrated Route and Bridge
Multi-Tenancy
73. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 73
Anycast – One-to-Nearest Association
• a network addressing and routing
methodology
• datagrams sent from a single sender to
the topologically nearest node
• group of potential receivers, all identified
by the same destination address
RR RR
✔
✖
✖
✔
74. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 74
Distributed IP Anycast Gateway
• Distributed Inter-VXLAN Routing at Access
Layer (Leaf)
All Leafs share same gateway IP and MAC
Address for a given Subnet
• Gateway is always active
no redundancy protocol, hello exchange etc.
• Distributed state - Smaller ARP tables
Only local attached End-Points (Servers)
RR RR
SVI 100, Gateway IP: 192.168.1.1, Gateway MAC: AG:AG:AG:AG:AG:AG
SVI 200, Gateway IP: 10.10.10.1, Gateway MAC: AG:AG:AG:AG:AG:AG
SVI 100
SVI 200
SVI 100
SVI 200
SVI 100
SVI 200
SVI 100
SVI 200
SVI 100
SVI 200
SVI 100
SVI 200
75. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 75
Distributed IP Anycast Gateway
Spine
RR RR
V
V
V
V
V
V
SVI 200
SVI 100
SVI 100
SVI 100, Gateway IP: 192.168.1.1
SVI 200, Gateway IP: 10.10.10.1
Host1
MAC: AA:AA:AA:AA:AA:AA
IP: 192.168.1.11
VLAN 100
VXLAN VNI 30001
Host3
MAC: CC:CC:CC:CC:CC:CC
IP: 192.168.1.33
VLAN 100
VXLAN VNI 30001
Host2
MAC: BB:BB:BB:BB:BB:BB
IP: 10.10.10.22
VLAN 200
VXLAN VNI 30002
bridge
route
route
76. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 76
Distributed IP Anycast Gateway
Spine
RR RR
V
V
V
V
V
V
SVI 200
SVI 100
SVI 100
SVI 100, Gateway IP: 192.168.1.1
SVI 200, Gateway IP: 10.10.10.1
Host1
MAC: AA:AA:AA:AA:AA:AA
IP: 192.168.1.11
VLAN 100
VXLAN VNI 30001
Host3
MAC: CC:CC:CC:CC:CC:CC
IP: 192.168.1.33
VLAN 100
VXLAN VNI 30001
Host2
MAC: BB:BB:BB:BB:BB:BB
IP: 10.10.10.22
VLAN 200
VXLAN VNI 30002
bridge
route
route
Any Subnet Routed Anywhere – Any VTEP can serve any Subnet
Integrated Route & Bridge (IRB) - Route whenever you can, Bridge when
needed
No Hairpinning – Optimized East/West and North/South
Routing
Seamless Mobility - All Leaf share same Gateway
MAC
Reduced Failure Domain – Layer-2/Layer-3 Boundary at
Leaf
Optimal Scalability – Route Distributed & closest to the
Host
77. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 77
Integrated Routing and Bridging (IRB)
VXLAN/EVPN based overlays follow two
slightly different Integrated Routing and
Bridging (IRB) semantics
• Asymmetric
Uses an “asymmetric path” from the Host
towards the egressing port of the VTEP vs. the
way back
• Symmetric*
Uses an “symmetric path” from the Host
towards the egressing port of the VTEP vs. the
way back
RR RR
*Implemented by Cisco’s VXLAN/EVPN
78. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 78
Consistent Configuration
• Logical Configuration (VLAN, VRF, VNI)
consistently instantiated on ALL Leafs
• Optimal for Consistency
• Every VLAN/VNI Everywhere
• Sub-Optimal for Scale
• Instantiates Resources (VLAN/VNI) even if
no End-Point uses it
RR RR
SVI 100
SVI 200
SVI 300
SVI 100
SVI 200
SVI 300
SVI 100
SVI 200
SVI 300
SVI 100
SVI 200
SVI 300
SVI 100
SVI 200
SVI 300
SVI 100
SVI 200
SVI 300
79. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 79
Scoped Configuration
• Logical Configuration (VLAN, VRF, VNI)
scoped to Leafs with respective connected
End-Points
• Optimal for Scale
• Instantiates Resources (VLAN/VNI)
where End-Points are connected
• Consistency with End-Points
• Configuration Consistency depends on End-
Points
RR RR
SVI 100
SVI 200
SVI 200
SVI 300
SVI 100
SVI 100
SVI 200
SVI 300
SVI 300
SVI 200
80. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 80
Asymmetric IRB
• Similar to todays Inter-VLAN routing
• Requires to follow a consistent configuration
of VLAN and L2VNI across all Switches
• Post routed traffic will leverage destination
Layer 2 Segment (L2VNI), same as for
bridged traffic
RR RR
SVI200
SVI300
SVI300
SVI200
81. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 81
Asymmetric IRB
RR RR
SVI300
SVI300
SVI200
✖
SVI200
82. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 82
Asymmetric IRB
Leaf
VV
Host1
MAC: AA:AA:AA:AA:AA:AA
IP: 192.168.1.11
VLAN 300
VXLAN VNI 30003
Host2
MAC: BB:BB:BB:BB:BB:BB
IP: 10.10.10.22
VLAN 200
VXLAN VNI 30002
Host3
MAC: CC:CC:CC:CC:CC:CC
IP: 10.10.10.33
VLAN 200
VXLAN VNI 30002
Host4
MAC: DD:DD:DD:DD:DD:DD
IP: 192.168.1.44
VLAN 300
VXLAN VNI 30003
SVI 300SVI 200SVI 300 SVI 200
L2VNI 30002
L2VNI 30001
83. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 83
Symmetric IRB
• Similar to Transit Routing Segments
• Scoped Configuration of VLAN/L2VNI; only
required where End-Points (Server) reside
• New VNI (L3VNI) introduced per virtual
routing and forwarding (VRF) context
• Routed traffic uses transit VNI (L3VNI), while
bridged traffic uses L2VNI
RR RR
SVI200
SVI300
SVI300
SVI200
84. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 84
Symmetric IRB
RR RR
SVI200
SVI300
SVI300
SVI200
85. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 85
Symmetric IRB
Leaf
VV
Host1
MAC: AA:AA:AA:AA:AA:AA
IP: 192.168.1.11
VLAN 300
VXLAN VNI 30003
Host2
MAC: BB:BB:BB:BB:BB:BB
IP: 10.10.10.22
VLAN 200
VXLAN VNI 30002
Host3
MAC: CC:CC:CC:CC:CC:CC
IP: 10.10.10.33
VLAN 200
VXLAN VNI 30002
Host4
MAC: DD:DD:DD:DD:DD:DD
IP: 192.168.1.44
VLAN 300
VXLAN VNI 30003
SVI 300SVI 200SVI 300 SVI 200
L3VNI 50001 (VRF)
86. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 86
Data Center Fabric Properties
Extended Namespace
Scalable Layer-2 Domains
Integrated Route and Bridge
Multi-Tenancy
87. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 87
Agenda
• Introduction to Data Center Fabrics
• VXLAN with BGP EVPN
• Overview
• Underlay
• Control & Data Plane
• Multi-Tenancy
• “Stories” and Use-Cases
• Fabric Management & Automation
88. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 88
• A mode of operation, where multiple independent instances (tenant)
operate in a shared environment.
• Each instance (i.e. VRF/VLAN) is logically isolated, but physically
integrated.
What is Multi-Tenancy
89. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 89
Multi-Tenancy at Layer-2
• Per-Switch VLAN-to-VNI mapping
• Per-Port VLAN Significance
Multi-Tenancy at Layer-3
• VRF-to-VNI mapping
• MP-BGP for scaling with VPNs
Where can we apply Multi-Tenancy
90. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 90
Layer-2 Multi-Tenancy
Spine
RR RR
V
V
V
V
V
V
VLAN 100
VLAN 100
Host1
MAC: AA:AA:AA:AA:AA:AA
IP: 192.168.1.11
VLAN 100
VXLAN VNI 30001
Host3
MAC: CC:CC:CC:CC:CC:CC
IP: 192.168.1.33
VLAN 100
VXLAN VNI 30001
bridge
91. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 91
Layer-2 Multi-Tenancy – Bridge Domains
Host1
MAC: AA:AA:AA:AA:AA:AA
IP: 192.168.1.11
VLAN 100
VXLAN VNI 30001
Host3
MAC: CC:CC:CC:CC:CC:CC
IP: 192.168.1.33
VLAN 100
VXLAN VNI 30001
Leaf
VV
VLAN 100 VLAN 100
VXLAN Overlay
(VNI 30001)
Bridge Domain
92. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 92
Layer-2 Multi-Tenancy – Bridge Domains
Host1
MAC: AA:AA:AA:AA:AA:AA
IP: 192.168.1.11
VLAN 100
VXLAN VNI 30001
Host3
MAC: CC:CC:CC:CC:CC:CC
IP: 192.168.1.33
VLAN 100
VXLAN VNI 30001
Leaf
VV
VXLAN Overlay
(VNI 30001)
VLAN 100 VLAN 100
Bridge Domain
The Bridge Domain is the Layer-2 Segment from Host to Host
In VXLAN, the Bridge Domain consists of three Components
1) The Ethernet Segment (VLAN), between Host and
Switch
2) The Hardware Resources (Bridge Domain) within the
Switch
3) The VXLAN Segment (VNI) between Switch and Switch
93. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 93
VLAN-to-VNI mapping
Host1
MAC: AA:AA:AA:AA:AA:AA
IP: 192.168.1.11
VLAN 100
VXLAN VNI 30001
Host3
MAC: CC:CC:CC:CC:CC:CC
IP: 192.168.1.33
VLAN 100
VXLAN VNI 30001
Leaf
VV
VLAN 100 VLAN 100
VXLAN Overlay
(VNI 30001)
Host2
MAC: BB:BB:BB:BB:BB:BB
IP: 192.168.1.22
VLAN 100
VXLAN VNI 30001
94. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 94
CLI Modes - VLAN based (per-Switch)
Leaf#1
vlan 100
vn-segment 30001
Leaf#2
vlan 100
vn-segment 30001
• VLAN to VNI configuration on a per-switch
basis
• VLAN becomes “Switch Local Identifier”
• VNI becomes “Network Global Identifier”
95. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 95
Per-Switch VLAN-to-VNI mapping
Host1
MAC: AA:AA:AA:AA:AA:AA
IP: 192.168.1.11
VLAN 100
VXLAN VNI 30001
Host3
MAC: CC:CC:CC:CC:CC:CC
IP: 192.168.1.33
VLAN 200
VXLAN VNI 30001
Leaf
VV
VLAN 100 VLAN 200
VXLAN Overlay
(VNI 30001)
Host2
MAC: BB:BB:BB:BB:BB:BB
IP: 192.168.1.22
VLAN 100
VXLAN VNI 30001
96. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 96
CLI Modes - VLAN based (per-Switch)
Leaf#1
vlan 100
vn-segment 30001
Leaf#2
vlan 200
vn-segment 30001
• VLAN to VNI configuration on a per-switch
basis
• VLAN becomes “Switch Local Identifier”
• VNI becomes “Network Global Identifier”
• 4k VLAN limitation has been removed
97. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 97
Per-Port VLAN-to-VNI mapping
Host1
MAC: AA:AA:AA:AA:AA:AA
IP: 192.168.1.11
VLAN 100
VXLAN VNI 30001
Host3
MAC: CC:CC:CC:CC:CC:CC
IP: 192.168.1.33
VLAN 300
VXLAN VNI 30001
Leaf
VV
VLAN 100 VLAN 300
VXLAN Overlay
(VNI 30001)
Host2
MAC: BB:BB:BB:BB:BB:BB
IP: 192.168.1.22
VLAN 200
VXLAN VNI 30001
VLAN 200
98. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 98
CLI Modes - VLAN based (per-Port)
Leaf#1
vlan 2500
vn-segment 30001
interface Ethernet 1/8
switchport mode trunk
switchport vlan mapping enable
switchport vlan mapping 100 2500
interface Ethernet 1/9
switchport mode trunk
switchport vlan mapping enable
switchport vlan mapping 200 2500
99. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 99
CLI Modes - Bridge-Domain based (per-Port)
Leaf#1
bridge-domain 100
member vni 30001
encapsulation profile vni VLAN100-30001
dot1q 100 vni 30001
encapsulation profile vni VLAN200-30001
dot1q 200 vni 30001
interface Ethernet 1/8
no switchport
service instance 1 vni
encapsulation profile VLAN100-30001 default
interface Ethernet 1/9
no switchport
service instance 1 vni
encapsulation profile VLAN200-30001 default
100. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 100
Layer-3 Multi-Tenancy
Spine
RR RR
V
V
V
V
V
V
SVI 200
SVI 100
VRF-A (VNI 50001)
VRF-B (VNI 50002)
SVI 100, Gateway IP: 192.168.1.1 (VRF-A)
SVI 200, Gateway IP: 10.10.10.1 (VRF-B)
SVI 300, Gateway IP: 172.16.1.1 (VRF-B)
Host1
IP: 192.168.1.11 (VRF-A)
VLAN 100
Host3
IP: 172.16.1.33 (VRF-B)
VLAN 300
Host2
IP: 10.10.10.22 (VRF-B)
VLAN 200
SVI 300
route
route
101. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 101
Layer-3 Multi-Tenancy – VRF-VNI or L3VNI
Host1
IP: 192.168.1.11 (VRF-A)
VLAN 100
Host3
IP: 172.16.1.33 (VRF-B)
VLAN 300
Leaf
VV
SVI 100
V
Host2
IP: 10.10.10.22 (VRF-B)
VLAN 200
SVI 200 SVI 300
VRF-A
(VNI 50001)
VRF-B
(VNI 50002)
Routing
Domain
VRF-B
Routing
Domain
VRF-A
102. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 102
Layer-3 Multi-Tenancy – VRF-VNI or L3VNI
Host1
IP: 192.168.1.11 (VRF-A)
VLAN 100
Host3
IP: 172.16.1.33 (VRF-B)
VLAN 300
Leaf
VV
SVI 100
V
Host2
IP: 10.10.10.22 (VRF-B)
VLAN 200
SVI 200 SVI 300
VRF-A
(VNI 50001)
VRF-B
(VNI 50002)
Routing
Domain
VRF-B
Routing
Domain
VRF-A
The Routing Domain is the VRF owning multiple Subnets across multiple
Switches
In VXLAN EVPN, the Routing Domain consists of three Components
1) The Routing Domains (VRF), local to the
Switch
2) The Routing Domain (L3VNI) between the Switches
3) Multi-Protocol BGP with EVPN Address-Family
103. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 103
Layer-3 Multi-Tenancy – VRF-Lite
Leaf
VV
SVI 300SVI 200SVI 100
Subnet1
IP: 192.168.1.0/24 (VRF-A)
VLAN 100
Subnet2
IP: 10.10.10.0/24 (VRF-B)
VLAN 200
Subnet3
IP: 172.16.1.0/24 (VRF-B)
VLAN 300
Host4
IP: 10.44.44.0/24 (VRF-A)
VLAN 400
SVI 400
VLAN 1002
VLAN 1001
Ethernet
104. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 104
Leaf
VV
SVI 300SVI 200SVI 100 SVI 400
VLAN 1002
VLAN 1001
Ethernet
Subnet1
IP: 192.168.1.0/24 (VRF-A)
VLAN 100
Subnet2
IP: 10.10.10.0/24 (VRF-B)
VLAN 200
Subnet3
IP: 172.16.1.0/24 (VRF-B)
VLAN 300
Host4
IP: 10.44.44.0/24 (VRF-A)
VLAN 400
Layer-3 Multi-Tenancy – VRF-Lite
vrf context VRF-A
interface eth1/10.1001
encapsulation dot1q 1001
vrf member VRF-A
ip address 10.1.1.1/24
ip router ospf 100 area 0.0.0.0
router ospf 100
vrf VRF-A
vrf context VRF-B
interface eth1/10.1002
encapsulation dot1q 1002
vrf member VRF-B
ip address 10.2.2.1/24
ip router ospf 100 area 0.0.0.0
router ospf 100
vrf VRF-B
vrf context VRF-B
interface eth1/10.1002
encapsulation dot1q 1002
vrf member VRF-B
ip address 10.2.2.2/24
ip router ospf 100 area 0.0.0.0
router ospf 100
vrf VRF-B
vrf context VRF-A
interface eth1/10.1001
encapsulation dot1q 1001
vrf member VRF-A
ip address 10.1.1.2/24
ip router ospf 100 area 0.0.0.0
router ospf 100
vrf VRF-A
105. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 105
Layer-3 Multi-Tenancy – MPLS L3VPN
Leaf
VV
SVI 300SVI 200SVI 100 SVI 400
VPN Label “Red”
VPN Label “Blue”
MPLS
Subnet1
IP: 192.168.1.0/24 (VRF-A)
VLAN 100
Subnet2
IP: 10.10.10.0/24 (VRF-B)
VLAN 200
Subnet3
IP: 172.16.1.0/24 (VRF-B)
VLAN 300
Host4
IP: 10.44.44.0/24 (VRF-A)
VLAN 400
106. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 106
Layer-3 Multi-Tenancy – MPLS L3VPN
Leaf
VV
SVI 300SVI 200SVI 100 SVI 400
VPN Label “Red”
VPN Label “Blue”
MPLS
Subnet1
IP: 192.168.1.0/24 (VRF-A)
VLAN 100
Subnet2
IP: 10.10.10.0/24 (VRF-B)
VLAN 200
Subnet3
IP: 172.16.1.0/24 (VRF-B)
VLAN 300
Host4
IP: 10.44.44.0/24 (VRF-A)
VLAN 400
vrf context VRF-A
rd 1.1.1.1:100
address-family ipv4 unicast
route-target import 100:100
route-target export 100:100
vrf context VRF-B
rd 1.1.1.1:200
address-family ipv4 unicast
route-target import 200:200
route-target export 200:200
vrf context VRF-B
rd 1.1.1.2:200
address-family ipv4 unicast
route-target import 200:200
route-target export 200:200
vrf context VRF-A
rd 1.1.1.2:100
address-family ipv4 unicast
route-target import 100:100
route-target export 100:100
router bgp 65500
address-family ipv4 unicast
neighbor 1.1.1.2 remote-as 65500
address-family vpnv4 unicast
send-community extended
vrf VRF-A
address-family ipv4 unicast
vrf VRF-B
address-family ipv4 unicast
router bgp 65500
address-family ipv4 unicast
neighbor 1.1.1.1 remote-as 65500
address-family vpnv4 unicast
send-community extended
vrf VRF-A
address-family ipv4 unicast
vrf VRF-B
address-family ipv4 unicast
107. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 107
Layer-3 Multi-Tenancy – VXLAN EVPN
Leaf
VV
SVI 300SVI 200SVI 100
Host1
MAC: AA:AA:AA:AA:AA:AA
IP: 192.168.1.11 (VRF-A)
VLAN 100
VXLAN VNI 30001
Host2
MAC: BB:BB:BB:BB:BB:BB
IP: 10.10.10.22 (VRF-B)
VLAN 200
VXLAN VNI 30002
Host3
MAC: CC:CC:CC:CC:CC:CC
IP: 172.16.1.33 (VRF-B)
VLAN 300
VXLAN VNI 30003
Host4
MAC: DD:DD:DD:DD:DD:DD
IP: 10.44.44.44 (VRF-A)
VLAN 400
VXLAN VNI 30004
SVI 400
L3VNI 50002
L3VNI 50001
VXLAN
108. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 108
Layer-3 Multi-Tenancy – VXLAN EVPN
Leaf
VV
SVI 300SVI 200SVI 100 SVI 400
L3VNI 50002
L3VNI 50001
VXLAN
Host1
MAC: AA:AA:AA:AA:AA:AA
IP: 192.168.1.11 (VRF-A)
VLAN 100
VXLAN VNI 30001
Host2
MAC: BB:BB:BB:BB:BB:BB
IP: 10.10.10.22 (VRF-B)
VLAN 200
VXLAN VNI 30002
Host3
MAC: CC:CC:CC:CC:CC:CC
IP: 172.16.1.33 (VRF-B)
VLAN 300
VXLAN VNI 30003
Host4
MAC: DD:DD:DD:DD:DD:DD
IP: 10.44.44.44 (VRF-A)
VLAN 400
VXLAN VNI 30004
vrf context VRF-A
vni 50001
rd auto
address-family ipv4 unicast
route-target both auto
route-target both auto evpn
vrf context VRF-B
vni 50002
rd auto
address-family ipv4 unicast
route-target both auto
route-target both auto evpn
vrf context VRF-B
vni 50002
rd auto
address-family ipv4 unicast
route-target both auto
route-target both auto evpn
vrf context VRF-A
vni 50001
rd auto
address-family ipv4 unicast
route-target both auto
route-target both auto evpn
router bgp 65500
address-family ipv4 unicast
neighbor 1.1.1.2 remote-as 65500
address-family l2vpn evpn
send-community extended
vrf VRF-A
address-family ipv4 unicast
advertise l2vpn evpn
vrf VRF-B
address-family ipv4 unicast
advertise l2vpn evpn
router bgp 65500
address-family ipv4 unicast
neighbor 1.1.1.1 remote-as 65500
address-family l2vpn evpn
send-community extended
vrf VRF-A
address-family ipv4 unicast
advertise l2vpn evpn
vrf VRF-B
address-family ipv4 unicast
advertise l2vpn evpn
109. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 109
Integrated Route & Bridge + Multi-Tenancy
Spine
RR RR
V
V
V
V
V
V
SVI 200
SVI 100
SVI 100
Host1
MAC: AA:AA:AA:AA:AA:AA
IP: 192.168.1.11
VLAN 100
VXLAN VNI 30001
Host3
MAC: CC:CC:CC:CC:CC:CC
IP: 192.168.1.33
VLAN 100
VXLAN VNI 30001
Host2
MAC: BB:BB:BB:BB:BB:BB
IP: 10.10.10.22
VLAN 200
VXLAN VNI 30002
bridge
L2VNI 30001
route
L3VNI 50001
VRF-A (VNI 50001)
SVI 100, Gateway IP: 192.168.1.1 (VRF-A)
SVI 200, Gateway IP: 10.10.10.1 (VRF-A)
110. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 110
Integrated Route & Bridge + Multi-Tenancy
Spine
RR RR
V
V
V
V
V
V
SVI 200
SVI 100
SVI 100
Host1
MAC: AA:AA:AA:AA:AA:AA
IP: 192.168.1.11 (VRF-A)
VLAN 100
VXLAN VNI 30001
Host3
MAC: CC:CC:CC:CC:CC:CC
IP: 192.168.1.33 (VRF-A)
VLAN 100
VXLAN VNI 30001
Host2
MAC: BB:BB:BB:BB:BB:BB
IP: 10.10.10.22 (VRF-A)
VLAN 200
VXLAN VNI 30002
VRF-A (VNI 50001)
SVI 100, Gateway IP: 192.168.1.1 (VRF-A)
SVI 200, Gateway IP: 10.10.10.1 (VRF-A)
111. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 111
Data Center Fabric Properties
Extended Namespace
Scalable Layer-2 Domains
Integrated Route and Bridge
Multi-Tenancy
112. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 112
Agenda
• Introduction to Data Center Fabrics
• VXLAN with BGP EVPN
• Overview
• Underlay
• Control & Data Plane
• Multi-Tenancy
• “Stories” and Use-Cases
• Fabric Management & Automation
114. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 114
VXLAN
applicability
evolves as the
Control Plane
evolves!
• Yesterday: VXLAN, yet another Overlay
Data-Plane only (Multicast based Flood & Learn)
• Today: VXLAN for the creation of scalable DC
Fabrics – Intra-DC
Control-Plane, active VTEP discovery, Multicast and Unicast
(Head-End Replication)
115. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 115
Story #1: Scalable Data Center Fabric
• VXLAN based Data Center Fabric
• BGP EVPN Control-Protocol (Overlay)
• OSPF for Underlay Routing (Unicast)
• PIM ASM with Anycast-RP for BUM Replication (Underlay)
• Distributed IP Anycast Gateway
116. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 116
Leaf
Leaf
Leaf
Leaf
Leaf
Border Leaf
Story #1: Scalable Data Center Fabric (1)
Spine
p2p Agg: 10.1.1.0/24
RID Agg: 10.10.10.0/24
VTEP Agg: 10.200.200.0/24
RP Agg: 10.254.254.0/24
117. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 117
interface loopback0
ip address 10.10.10.201/32
ip router ospf UNDERLAY area 0.0.0.0
router ospf UNDERLAY
router-id 10.10.10.201
interface Ethernet1/1
mtu 9192
ip address 10.1.1.2/30
ip ospf network point-to-point
ip router ospf UNDERLAY area 0.0.0.0
ip pim sparse-mode
interface Ethernet1/2
mtu 9192
ip address 10.1.1.6/30
ip ospf network point-to-point
ip router ospf UNDERLAY area 0.0.0.0
ip pim sparse-mode
interface Ethernet1/3
mtu 9192
ip address 10.1.1.10/30
ip ospf network point-to-point
ip router ospf UNDERLAY area 0.0.0.0
ip pim sparse-mode
…
Story #1: Scalable Data Center Fabric (2)
interface loopback0
ip address 10.10.10.101/32
ip router ospf UNDERLAY area 0.0.0.0
router ospf UNDERLAY
router-id 10.10.10.101
interface Ethernet1/1
mtu 9192
ip address 10.1.1.1/30
ip ospf network point-to-point
ip router ospf UNDERLAY area 0.0.0.0
ip pim sparse-mode
…
interface loopback0
ip address 10.10.10.102/32
ip router ospf UNDERLAY area 0.0.0.0
router ospf UNDERLAY
router-id 10.10.10.102
interface Ethernet1/1
mtu 9192
ip address 10.1.1.5/30
ip ospf network point-to-point
ip router ospf UNDERLAY area 0.0.0.0
ip pim sparse-mode
…
interface loopback0
ip address 10.10.10.103/32
ip router ospf UNDERLAY area 0.0.0.0
router ospf UNDERLAY
router-id 10.10.10.103
interface Ethernet1/1
mtu 9192
ip address 10.1.1.9/30
ip ospf network point-to-point
ip router ospf UNDERLAY area 0.0.0.0
ip pim sparse-mode
…
p2p Agg: 10.1.1.0/24
RID Agg: 10.10.10.0/24
VTEP Agg: 10.200.200.0/24
RP Agg: 10.254.254.0/24
118. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 118
Story #1: Scalable Data Center Fabric (3)
Rendezvous-PointRP
RP RP
interface loopback0
ip address 10.10.10.202/32
ip router ospf UNDERLAY area 0.0.0.0
interface loopback254
ip address 10.254.254.1/32
ip router ospf UNDERLAY area 0.0.0.0
ip pim sparse-mode
ip pim anycast-rp 10.254.254.1 10.254.254.202
ip pim anycast-rp 10.254.254.1 10.254.254.203
ip pim rp-address 10.254.254.1
interface loopback0
ip address 10.10.10.203/32
ip router ospf UNDERLAY area 0.0.0.0
interface loopback254
ip address 10.254.254.1/32
ip router ospf UNDERLAY area 0.0.0.0
ip pim sparse-mode
ip pim anycast-rp 10.254.254.1 10.254.254.202
ip pim anycast-rp 10.254.254.1 10.254.254.203
ip pim rp-address 10.254.254.1
interface loopback0
ip address 10.10.10.101/32
ip router ospf UNDERLAY area 0.0.0.0
ip pim rp-address 10.254.254.1
interface loopback0
ip address 10.10.10.102/32
ip router ospf UNDERLAY area 0.0.0.0
ip pim rp-address 10.254.254.1
interface loopback0
ip address 10.10.10.103/32
ip router ospf UNDERLAY area 0.0.0.0
ip pim rp-address 10.254.254.1p2p Agg: 10.1.1.0/24
RID Agg: 10.10.10.0/24
VTEP Agg: 10.200.200.0/24
RP Agg: 10.254.254.0/24
119. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 119
Story #1: Scalable Data Center Fabric (4)
p2p Agg: 10.1.1.0/24
RID Agg: 10.10.10.0/24
VTEP Agg: 10.200.200.0/24
RP Agg: 10.254.254.0/24
VTEP
VTEP
VTEP
VTEP
VTEP
VTEP
interface loopback1
ip address 10.200.200.101/32
ip router ospf UNDERLAY area 0.0.0.0
interface nve1
source-interface loopback1
host-reachability protocol bgp
interface loopback1
ip address 10.200.200.102/32
ip router ospf UNDERLAY area 0.0.0.0
interface nve1
source-interface loopback1
host-reachability protocol bgp
interface loopback1
ip address 10.200.200.103/32
ip router ospf UNDERLAY area 0.0.0.0
interface nve1
source-interface loopback1
host-reachability protocol bgp
120. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 120
Story #1: Scalable Data Center Fabric (5)
p2p Agg: 10.1.1.0/24
RID Agg: 10.10.10.0/24
VTEP Agg: 10.200.200.0/24
RP Agg: 10.254.254.0/24
VTEP
VTEP
VTEP
VTEP
VTEP
VTEP
router bgp 65500
router-id 10.10.10.202
neighbor 10.10.10.0/24 remote-as 65500
update-source loopback0
address-family l2vpn evpn
send-community both
route-reflector-client
router bgp 65500
router-id 10.10.10.203
neighbor 10.10.10.0/24 remote-as 65500
update-source loopback0
address-family l2vpn evpn
send-community both
route-reflector-client
BGP Route-ReflectorRR
RR RR
router bgp 65500
router-id 10.10.10.101
neighbor 10.10.10.202 remote-as 65500
update-source loopback0
address-family l2vpn evpn
send-community both
neighbor 10.10.10.203 remote-as 65500
update-source loopback0
address-family l2vpn evpn
send-community both
router bgp 65500
router-id 10.10.10.102
neighbor 10.10.10.202 remote-as 65500
update-source loopback0
address-family l2vpn evpn
send-community both
neighbor 10.10.10.203 remote-as 65500
update-source loopback0
address-family l2vpn evpn
send-community both
router bgp 65500
router-id 10.10.10.103
neighbor 10.10.10.202 remote-as 65500
update-source loopback0
address-family l2vpn evpn
send-community both
neighbor 10.10.10.203 remote-as 65500
update-source loopback0
address-family l2vpn evpn
send-community both
121. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 121
Story #1: Scalable Data Center Fabric (6)
VTEP
VTEP
VTEP
VTEP
VTEP
VTEP
vlan 100
vn-segment 30001
name Blue
vlan 200
vn-segment 30002
name Green
evpn
vni 30001
rd auto
route-target both auto
vni 30002
rd auto
route-target both auto
interface nve1
source-interface loopback1
host-reachability protocol bgp
member vni 30001
mcast-group 239.239.239.1
member vni 30002
mcast-group 239.239.239.2p2p Agg: 10.1.1.0/24
RID Agg: 10.10.10.0/24
VTEP Agg: 10.200.200.0/24
RP Agg: 10.254.254.0/24
122. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 122
Story #1: Scalable Data Center Fabric (7)
VTEP
VTEP
VTEP
VTEP
VTEP
VTEP
vrf context VRF-A
vni 50001
rd auto
address-family ipv4 unicast
route-target both auto
route-target both auto evpn
address-family ipv6 unicast
route-target both auto
route-target both auto evpn
interface nve1
source-interface loopback1
host-reachability protocol bgp
member vni 50001 associate-vrf
router bgp 65500
vrf VRF-A
address-family ipv4 unicast
advertise l2vpn evpn
redistribute direct route-map TAG
p2p Agg: 10.1.1.0/24
RID Agg: 10.10.10.0/24
VTEP Agg: 10.200.200.0/24
RP Agg: 10.254.254.0/24
interface Vlan100
mtu 9192
vrf member VRF-A
ip address 192.168.1.1/24 tag 21921
fabric forwarding mode anycast-gateway
interface Vlan200
mtu 9192
vrf member VRF-A
ip address 10.10.10.1/24 tag 21921
fabric forwarding mode anycast-gateway
route-map TAG permit 10
match tag 21921
123. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 123
Story #1: Scalable Data Center Fabric (8)
VTEP
VTEP
VTEP
VTEP
VTEP
VTEP
p2p Agg: 10.1.1.0/24
RID Agg: 10.10.10.0/24
VTEP Agg: 10.200.200.0/24
RP Agg: 10.254.254.0/24
interface Ethernet 2/1.10
vrf member VRF-A
ip address 172.16.0.1/30
encapsulation dot1q 5
interface Ethernet 2/1.20
vrf member VRF-B
ip address 172.16.0.1/30
encapsulation dot1q 6
router bgp 65500
vrf VRF-A
address-family ipv4 unicast
advertise l2vpn evpn
aggregate-address 10.10.10.0/24 summary-only
aggregate-address 192.168.1.0/24 summary-only
neighbor 172.16.0.1 remote-as 65599
update-source Ethernet2/1.10
address-family ipv4 unicast
…
WAN
interface Ethernet 1/15.21
vrf member VRF-A
ip address 172.16.0.2/30
encapsulation dot1q 5
interface Ethernet 1/15.22
vrf member VRF-B
ip address 172.16.0.2/30
encapsulation dot1q 6
router bgp 65599
vrf VRF-A
address-family ipv4 unicast
neighbor 172.16.0.1 remote-as 65500
update-source Ethernet1/15.21
address-family ipv4 unicast
…
124. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 124
Story #2: Scalable Data Center Fabric
• VXLAN based Data Center Fabric
• BGP EVPN Control-Protocol (Overlay)
• eBGP for Underlay Routing (Unicast)
• eBGP Multi-AS Design
• Ingress Replication for BUM (Underlay)
• Distributed IP Anycast Gateway
125. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 125
AS65501
AS65502
AS65503
AS65503
AS65504
AS65555
Story #2: Scalable Data Center Fabric (1)
Spine AS65500
p2p Agg: 10.1.1.0/24
RID Agg: 10.10.10.0/24
VTEP Agg: 10.200.200.0/24
RP Agg: 10.254.254.0/24
126. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 126
interface loopback0
ip address 10.10.10.201/32 tag 12345
interface Ethernet1/1
mtu 9192
ip address 10.1.1.2/30
interface Ethernet1/2
mtu 9192
ip address 10.1.1.6/30
interface Ethernet1/3
mtu 9192
ip address 10.1.1.10/3
router bgp 65500
router-id 10.10.10.201
address-family ipv4 unicast
redistribute direct route-map UL-TAG
neighbor 10.10.10.1 remote-as 65501
address-family ipv4 unicast
neighbor 10.10.10.5 remote-as 65502
address-family ipv4 unicast
neighbor 10.10.10.9 remote-as 65503
address-family ipv4 unicast
…
Story #2: Scalable Data Center Fabric (2)
p2p Agg: 10.1.1.0/24
RID Agg: 10.10.10.0/24
VTEP Agg: 10.200.200.0/24
RP Agg: 10.254.254.0/24
interface loopback0
ip address 10.10.10.101/32 tag 12345
interface Ethernet1/1
mtu 9192
ip address 10.1.1.1/30
router bgp 65501
router-id 10.10.10.101
address-family ipv4 unicast
redistribute direct route-map UL-TAG
template peer SPINE-UNDERLAY
remote-as 65500
address-family ipv4 unicast
neighbor 10.10.10.2
inherit peer SPINE-UNDERLAY
neighbor 10.10.10.L1-S2
inherit peer SPINE-UNDERLAY
neighbor 10.10.10.L1-S3
inherit peer SPINE-UNDERLAY
neighbor 10.10.10.L1-S4
inherit peer SPINE-UNDERLAY
…
interface loopback0
ip address 10.10.10.102/32 tag 12345
interface Ethernet1/1
mtu 9192
ip address 10.1.1.5/30
router bgp 65502
router-id 10.10.10.102
address-family ipv4 unicast
redistribute direct route-map UL-TAG
template peer SPINE-UNDERLAY
remote-as 65500
address-family ipv4 unicast
neighbor 10.10.10.6
inherit peer SPINE-UNDERLAY
neighbor 10.10.10.L1-S2
inherit peer SPINE-UNDERLAY
neighbor 10.10.10.L1-S3
inherit peer SPINE-UNDERLAY
neighbor 10.10.10.L1-S4
inherit peer SPINE-UNDERLAY
…
interface loopback0
ip address 10.10.10.103/32 tag 12345
interface Ethernet1/1
mtu 9192
ip address 10.1.1.9/30
router bgp 65503
router-id 10.10.10.103
address-family ipv4 unicast
redistribute direct route-map UL-TAG
template peer SPINE-UNDERLAY
remote-as 65500
address-family ipv4 unicast
neighbor 10.10.10.10
inherit peer SPINE-UNDERLAY
neighbor 10.10.10.L1-S2
inherit peer SPINE-UNDERLAY
neighbor 10.10.10.L1-S3
inherit peer SPINE-UNDERLAY
neighbor 10.10.10.L1-S4
inherit peer SPINE-UNDERLAY
…
route-map TAG-UL permit 10
match tag 12345
127. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 127
Story #2: Scalable Data Center Fabric (3)
p2p Agg: 10.1.1.0/24
RID Agg: 10.10.10.0/24
VTEP Agg: 10.200.200.0/24
RP Agg: 10.254.254.0/24
VTEP
VTEP
VTEP
VTEP
VTEP
VTEP
interface loopback1
ip address 10.200.200.101/32 tag 12345
interface nve1
source-interface loopback1
host-reachability protocol bgp
interface loopback1
ip address 10.200.200.102/32 tag 12345
interface nve1
source-interface loopback1
host-reachability protocol bgp
interface loopback1
ip address 10.200.200.103/32 tag 12345
interface nve1
source-interface loopback1
host-reachability protocol bgp
128. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 128
Story #2: Scalable Data Center Fabric (4)
p2p Agg: 10.1.1.0/24
RID Agg: 10.10.10.0/24
VTEP Agg: 10.200.200.0/24
RP Agg: 10.254.254.0/24
VTEP
VTEP
VTEP
VTEP
VTEP
VTEP
router bgp 65500
router-id 10.10.10.203
address-family l2vpn evpn
nexthop route-map NHUNCH
retain route-target all
neighbor 10.10.10.101 remote-as 65501
update-source loopback0
address-family l2vpn evpn
send-community both
disable-connected-check
route-map NHUNCH out
neighbor 10.10.10.102 remote-as 65502
update-source loopback0
address-family l2vpn evpn
send-community both
disable-connected-check
route-map NHUNCH out
neighbor 10.10.10.103 remote-as 65503
update-source loopback0
address-family l2vpn evpn
send-community both
disable-connected-check
route-map NHUNCH out
router bgp 65500
router-id 10.10.10.202
address-family l2vpn evpn
nexthop route-map NHUNCH
retain route-target all
neighbor 10.10.10.101 remote-as 65501
update-source loopback0
address-family l2vpn evpn
send-community both
disable-connected-check
route-map NHUNCH out
neighbor 10.10.10.102 remote-as 65502
update-source loopback0
address-family l2vpn evpn
send-community both
disable-connected-check
route-map NHUNCH out
neighbor 10.10.10.103 remote-as 65503
update-source loopback0
address-family l2vpn evpn
send-community both
disable-connected-check
route-map NHUNCH out
route-map NHUNCH permit 10
set ip next-hop unchanged
129. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 129
Story #2: Scalable Data Center Fabric (5)
p2p Agg: 10.1.1.0/24
RID Agg: 10.10.10.0/24
VTEP Agg: 10.200.200.0/24
RP Agg: 10.254.254.0/24
VTEP
VTEP
VTEP
VTEP
VTEP
VTEP
router bgp 65501
router-id 10.10.10.101
neighbor 10.10.10.202 remote-as 65500
update-source loopback0
address-family l2vpn evpn
send-community both
disable-connected-check
neighbor 10.10.10.203 remote-as 65500
update-source loopback0
address-family l2vpn evpn
send-community both
disable-connected-check
router bgp 65502
router-id 10.10.10.102
neighbor 10.10.10.202 remote-as 65500
update-source loopback0
address-family l2vpn evpn
send-community both
disable-connected-check
neighbor 10.10.10.203 remote-as 65500
update-source loopback0
address-family l2vpn evpn
send-community both
disable-connected-check
router bgp 65503
router-id 10.10.10.103
neighbor 10.10.10.202 remote-as 65500
update-source loopback0
address-family l2vpn evpn
send-community both
disable-connected-check
neighbor 10.10.10.203 remote-as 65500
update-source loopback0
address-family l2vpn evpn
send-community both
disable-connected-check
130. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 130
Story #2: Scalable Data Center Fabric (6)
VTEP
VTEP
VTEP
VTEP
VTEP
VTEP
vlan 100
vn-segment 30001
name Blue
vlan 200
vn-segment 30002
name Green
evpn
vni 30001
rd auto
route-target both 65500:30001
vni 30002
rd auto
route-target both 65500:30002
interface nve1
source-interface loopback1
host-reachability protocol bgp
member vni 30001
ingress-replication protocol bgp
member vni 30002
ingress-replication protocol bgpp2p Agg: 10.1.1.0/24
RID Agg: 10.10.10.0/24
VTEP Agg: 10.200.200.0/24
RP Agg: 10.254.254.0/24
131. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 131
Story #2: Scalable Data Center Fabric (7)
VTEP
VTEP
VTEP
VTEP
VTEP
VTEP
vrf context VRF-A
vni 50001
rd auto
address-family ipv4 unicast
route-target both 65500:50001
route-target both 65500:50001 evpn
address-family ipv6 unicast
route-target both 65500:50001
route-target both 65500:50001 evpn
interface nve1
source-interface loopback1
host-reachability protocol bgp
member vni 50001 associate-vrf
router bgp 655xx
vrf VRF-A
address-family ipv4 unicast
advertise l2vpn evpn
redistribute direct route-map TAG
p2p Agg: 10.1.1.0/24
RID Agg: 10.10.10.0/24
VTEP Agg: 10.200.200.0/24
RP Agg: 10.254.254.0/24
interface Vlan100
mtu 9192
vrf member VRF-A
ip address 192.168.1.1/24 tag 21921
fabric forwarding mode anycast-gateway
interface Vlan200
mtu 9192
vrf member VRF-A
ip address 10.10.10.1/24 tag 21921
fabric forwarding mode anycast-gateway
route-map TAG permit 10
match tag 21921
132. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 132
Story #2: Scalable Data Center Fabric (8)
VTEP
VTEP
VTEP
VTEP
VTEP
VTEP
p2p Agg: 10.1.1.0/24
RID Agg: 10.10.10.0/24
VTEP Agg: 10.200.200.0/24
RP Agg: 10.254.254.0/24
interface Ethernet 2/1.10
vrf member VRF-A
ip address 172.16.0.1/30
encapsulation dot1q 5
interface Ethernet 2/1.20
vrf member VRF-B
ip address 172.16.0.1/30
encapsulation dot1q 6
router bgp 65555
vrf VRF-A
address-family ipv4 unicast
advertise l2vpn evpn
aggregate-address 10.10.10.0/24 summary-only
aggregate-address 192.168.1.0/24 summary-only
neighbor 172.16.0.1 remote-as 65599
update-source Ethernet2/1.10
address-family ipv4 unicast
…
WAN
interface Ethernet 1/15.21
vrf member VRF-A
ip address 172.16.0.2/30
encapsulation dot1q 5
interface Ethernet 1/15.22
vrf member VRF-B
ip address 172.16.0.2/30
encapsulation dot1q 6
router bgp 65599
vrf VRF-A
address-family ipv4 unicast
neighbor 172.16.0.1 remote-as 65555
update-source Ethernet1/15.21
address-family ipv4 unicast
…
133. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 133
VXLAN
applicability
evolves as the
Control Plane
evolves!
• Yesterday: VXLAN, yet another Overlay
Data-Plane only (Multicast based Flood & Learn)
• Today: VXLAN for the creation of scalable DC
Fabrics – Intra-DC
Control-Plane, active VTEP discovery, Multicast and Unicast
(Head-End Replication)
• Future: VXLAN for DCI – Inter-DC
DCI Enhancements (ARP caching/suppress, Multi-Homing,
Failure Domain isolation, Loop Protection etc.)
134. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 134
What is the Elephant in the Room?
135. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 135
Note sure if it is a Elephant
VXLAN for Interconnecting Networks
136. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 136
Story #3: Inter-Fabric Connectivity
• Option 1: End-to-End Fabric Stretch
• Option 2: Fabric-DCI-Fabric (2-box)
• Option 3: Fabric-DCI-Fabric L3-DCI (1-box)
• Option 4: Fabric-DCI-Fabric L2-DCI (1-box)
137. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 137
V
V
V
V
V
V
V
V
• Multiple BGP-EVPN Control-Plane Domains
• End-to-End reachability for VTEP
• End-to-End reachability for BUM Replication
Multicast / Ingress Replication
• End-to-End Data-Plane encapsulation
Inter-Fabric Connectivity (Option 1)
EVPN Control-Plane Domain 1
EVPN Control-Plane Domain 2
VXLAN Encapsulation
138. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 138
V
V
V
V
V
V
V
V
• Multiple BGP-EVPN Control-Plane Domains
• Normalization via Ethernet (MPLS, VRF-lite & IEEE
802.1Q Trunk) at the Border
• Separate Data-Plane (DP) encapsulation per
Domain
Multicast / Ingress Replication
Inter-Fabric Connectivity (Option 2)
EVPN Control-Plane Domain 1
EVPN Control-Plane Domain 2
DCI
VXLAN Encapsulation
DCI Encapsulation
139. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 139
V
V
V
V
V
V
V
V
• Multiple BGP-EVPN Control-Plane Domains
• Integrated Hand-Off with Data-Plane separation
Option 3 – L3 DCI
L3-LISP, MPLS, EVPN
Option 4 – L2 DCI
OTV, L2-LISP, EVPN
Separate Data-Plane (DP) encapsulation per Domain
Multicast / Ingress Replication
Inter-Fabric Connectivity (Option 3 / Option 4)
VXLAN Encapsulation
EVPN Control-Plane Domain 1
EVPN Control-Plane Domain 2 DCI Encapsulation
140. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 140
Inter-Fabric Connectivity
Option 1 Option 2 Option 3/4
Underlay Control Plane Unified Underlay Domain Separated Underlay Domains Separated Underlay Domains
Overlay Control Plane Separated Overlay Control-Plane Domains
Overlay Data Plane Single Data-Plane Separated Data-Planes Separated Data-Planes
BUM Replication in DCI
Unified Underlay Domain (All
Multicast or All Ingress Replication)
Dependency on DCI Choice (Unicast/Multicast)
ARP Flood Suppression
(DCI)
yes yes yes
Unknown Unicast Flood
Suppression (DCI)
no yes yes
Broadcast Suppression/Limit
(DCI)
no yes yes
Layer-2 Loop Prevention Loop mitigation (Edge Protection) VPC at Border Loop mitigation (At DCI)
142. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 142
How to achieve Data Center Automation
• Simplify
Do not start with the most difficult task (low hanging Fruits)
• Standardize
Find common Denominators and create Templates
• Automate repetitive Tasks
Use Templates for Simple Tasks and use Automation (e.g. create VLAN, SVI, VRF)
• Abstract
Take a step back and look at the WHOLE
Cisco ACI
143. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 143
Network Infrastructure
IP Fabric
Underlay
Management
- Network Element-
Management
-Topology Overview
-Configuration
Deployment
Overlay
Management
- Overlay Services
(Layer 2/Layer 3)
- Service Chaining
Hybrid Overlay
- integration of
Physical and Virtual
VTEPs
Inter-Domain and
Multi-Fabric
- Seamless LISP
and MPLS
integration
- Optimizing Inter-
Domain integration
-Cross DC Mobility
API
NX-APIPuppet
Chef
Ansible
VMM Openstack
Workload Mobility, Service Agility
Multi-tenancy
Simplified Provisioning & Management
Anatomy of Data Center Automation
144. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 144
Fabric Management & Operations
Element
management:
Hardware
Management,
Health Status,
and Inventory
Day-0:
Configuration
(POAP)
Underlay
Management
Day- 1:
Configuration and
Configuration
Management
Automated
Configuration
Compute
Integration
Day-2:
Visibility,
Configuration
increments,
compare
changes.
Troubleshooting
145. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 145
Simplifying Management & Fabric Visibility
• Device Auto-Configuration
• Cabling Plan Consistency Check
• Automated Network Provisioning
• Common point of fabric access
• Tenant, Virtual Fabric & Host Visibility
146. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 146
Device Auto-Configuration: Day 0
• Underlay Configuration:
Physical interface IP configuration
Loopback interface IP configuration
Multicast Configuration for the Underlay (BUM)
Routing protocol for the underlay configuration
vPC domain
BGP EVPN + RR configuration
VTEP configuration
147. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 147
Device Auto-Configuration: Day 0.5
• Tenant Configuration including:
VPC configuration for downstream connectivity
Interface configuration
Host Ports and Port-Channels
148. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 148
Device Auto-Configuration: Day 1
• Tenant Configuration including:
VLAN configuration
VRF configuration
VNI configuration
SVI (BDI) configuration
BGP VRF (L3 Tenant) + EVPN (L2 Tenant)
Distributed IP Anycast Gateway configuration
149. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 149
Device Auto-Configuration (POAP)
Day 0, Day 0.5 and Day 1
1. Easy way to unbox, rack the device, and not enter any base CLI configuration. Just
rack, power, and plug into the management network.
2. Provides a standard and consistent configuration across of the data center network
devices.
3. Provides a standard and consistent images to deploy to all of the data center
devices.
150. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 150Cisco ConfidentialCisco Confidential© 2016 Cisco and/or its affiliates. All rights reserved. 150
Q & A
151. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 151
Recommended Reading
Using TRILL, FabricPath, and VXLAN:
Designing Massively Scalable Data Centers
(MSDC) with Overlays
• Sanjay K. Hooda
• Shyam Kapadia
• Padmanabhan Krishnan
ISBN-10: 1-58714-393-3
ISBN-13: 978-1-58714-393-9
152. © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 152
Recommended Viewing
Cisco Programmable Fabric Using VXLAN
with BGP EVPN LiveLessons
• David Jansen
• Lukas Krattiger
ISBN-10: 0-13-427229-3
ISBN-13: 978-0-13-427229-0