Die monatlichen Anlässe in Zusammenarbeit mit dem Swiss IPv6 Council behandeln verschiedene technische Themenbereiche von IPv6. Ist Dual-Stack ein guter Weg? Was sind Erfahrungen mit reinen IPv6-Netzwerken? Viele Unternehmen kämpfen mit diesen Fragen. Genau aus diesem Grund hat Cisco in San Jose einen reinen IPv6-Campus eingerichtet und sucht Antworten auf solche Fragen. In seinem Referat präsentierte Khalid Jawaid die Bemühungen zur Einführung von IPv6 in einem einzelnen Campusgebäude und die Hürden, die dies mit sich bringt. Im Weiteren sprach er über die Herausforderungen, die sich aus der Erstellung des Business Case im Jahr 2010 und der Bereitstellung von mehr als 400 Remote-Standorten und den daraus gezogenen Lehren ergaben. Die Inputs von Khalid Jawaid haben bei vielen Teilnehmern neue Perspektiven und Ideen für die Einführung von IPv6 geweckt. Gerne stellen wir Ihnen die Slides von Manuel Schweizer zur Verfügung:

1. Khalid Jawaid CCIE 6765
Solutions Engineer, Global Infrastructure Services, Cisco IT
30th Oct 2017
Deploying IPv6 only in SJC23
Cisco IT – Building an IPv6 Only
Network

2. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
• Ben Irving (Sponsor
Director)
• Travis Norling (Manager
ETE)
• Hitesh Panchal
• Charles Radke
Acknowledgements
Great Team Behind This
• Norman Fong
• Tsung Chan
• John Banner
• Many More!

3. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Agenda list
1
2
3
4
Cisco IT Overview
IPv6 Only in Building 23
and Issues
IPv6 Only DC Plans
and Issues
Q/A – Interactive
Discussion

4. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Cisco IT Overview
• 50,000+ Devices
• 300+ locations in 92 countries
• 500+ buildings
• 200,000 Sq Ft of DC space
• 1000+ labs worldwide
• 150,000+ Users
• ~ 5 Million IP Addresses (All Inclusive)
• ~ 6800 Applications

5. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Cisco IT Overview
• 11 iPoPs advertising Cisco IPv4/IPv6 space
• EIGRP for IPv6/IPv4 + BGP
• Dual Stacked Everywhere (ExceptExtranet and CVO)
• Dual Stack DC Gateways (not server VLANs)
• Management over IPv4 (Except IPv6 Service Monitoring
and SJC23)
• CNR for DHCP Services

6. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
SYDNEY
4000::/34
AU::/39
SHANGHAI
BANGALORE
4000::/34
IN::/39
SINGAPORE
4000::/34
SG::/39
HONG KONG
4000::/34
CN::/39
HK::/39
SJ
::/32
RTP
RTP::/35
Richardson
RCDN::/40
Amsterdam
4000::/36
Cisco Global Internet Presence IPv6 Advertisements
(ARIN 2001:420::/32)
TOKYO
4000::/34
JP::/39
UK
4000::/36
UK::/38

7. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Our IPv6 Timeline
2010 – 2016 – Dual Stack
2016 – 2018 – Dual Stack + IPv6 Only
SJC23 – IPv6 Only RTP IPv6 Only DC POD
2018/19 – 20??
IPv6 Only Mandate (New
Apps)
Training /
Development

8. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Confidential© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
An IPv6 Only
Experience
Goals
Product Gaps
Operational Gaps
User Experience
Knowledge/
Awareness

9. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Confidential© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
SJC23 – IPv6 Only
Access
• Single Campus Building
• Wired and Wireless
• Android and iOS
• NAT64/DNS64
• Management + Data
• UC / Collaboration
Target

10. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Physical Topology – IPv6 Only @ SJC23
VSS
6504E 6504E
Alpha Backbone (Dual
Stack)
4507E 4507E 3850 3850
5508
3702 3802
DNS64
RHEL7
BIND9
NAT64
ASR1K
HA
IPv6 Only

11. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
NAT64 Topology – IPv6 Only @ SJC23
Corp Backbone
(Dual Stack)
DNS64
RHEL7
BIND9
NAT64 HA Control
NAT64 Outside NAT64 Outside
NAT64 Inside NAT64 Inside
NAT64 HA Control
NAT64 HA Data NAT64 HA Data
L3
L2
L2 Adjacency for NAT64 HA

12. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
AP 3702/3802
Products Used SJC23
ASR1K
6504E
WLC 5508
4507R+E3850

13. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
• HSRPv2 for IPv6 (First Hop Routing)
• First Hop Security
• IPv6 Snooping (DHCPv6 Guard, Destination Guard, DHCPv6 Binding)
• ND Inspection
• RA Guard
• uRPF
• DHCPv6 Stateful (Default and Preferred)
• SLAAC (Special case)
• EIGRP for IPv6
• NAT64/DNS64
IPv6 Features Deployed

14. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
• Average 300 Users, peak 500
• 3 Months (start to finish)
• Approx. 7 – 8 engineers
• Average Traffic 250 Mbps (v6 Only Links)
• Average 32K NAT64 Xlate Entries
Statistics

15. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Charts (NAT64 Xlates)
Not available via SNMP, gather with a script

16. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Issues and Challenges

17. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Problem Statement: No fall back to NAT64 upon receiving a
SERVFAIL or FORMERR.
Symptoms: No Connectivity to website
Diagnosis: NAT64 does not create a synth AAAA back to client if it
gets a SERVFAIL
Workaround: Create a master zone on Cisco DNS64 for destination
and get manually synth AAAA (Problems when the destination fails
over)
• LTF: Webex upgrade of GSS
Issue – NAT64 fallback fails with SERVFAIL

18. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Problem Statement: AnyConnect client keeps reconnecting on MAC
Symptoms: No Connectivity – Client Reconnecting
Diagnosis: AnyConnect client software issue with NAT64 headend
causing fragmentation. Client dropping TCP Fragments due to implicit
filtering breaking TLS connection causing reconnecting loops. Also
impacts IPSec/DTLS Tunnels
Workaround: No Workaround
• LTF: Fixed in AnyConnect Client ver 4.4MR3+
Issue – AnyConnect client fails on MAC

19. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Problem Statement: Web based Spark Clients not working. Client
apps working across all platforms for all services
Symptoms: No Connectivity/Calling/Services
Diagnosis: Web Client connectivity Infrastructure is not IPv6 enabled
Workaround: No Workaround
• LTF: IPv6 Enable Web client infrastructure
Issue – Spark Web Clients not IPv6 Ready

20. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Problem Statement: Can’t poll IOS-XE for Xlate data using SNMP
Symptoms: No SNMP data
Diagnosis: Not supported
Workaround: Use a script to collect Xlate output via SSH/CLI
LTF: CSCvc13935 bug filed as Enhancement Request
Issue – NAT64 – No SNMP MIB for Xlates

21. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Problem Statement: Jabber clients failed to register with CUCM
Symptoms: No Registration
Diagnosis: IPv6 support is not available for Jabber clients below
CUCM Ver 12.0
Workaround: No Workaround
• LTF: Upgrade to CUCM 12.0 – After upgrade, all features / services
working
Issue – Jabber/Phones Fail to Register

22. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Problem Statement: Can’t poll IOS-XE for Xlate data using SNMP
Symptoms: No SNMP data
Diagnosis: Not supported
Workaround: Use a script to collect Xlate output via SSH/CLI
LTF: CSCvc13935 bug filed as Enhancement Request
Issue – NAT64 – No SNMP MIB for Xlates

23. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
• Not all apps/ drivers in standard Cisco Desktop image ipv6 ready.
Needed latest updates
• IPv4 Literals – Can’t do DNS64 and therefore no NAT64
• 802.1X – Need platform support across 4500 (In Development) – use
SGT/SGACLs as workaround
Misc Issues

24. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
• AAAA and IPv6 connectivity statistics of top websites according to
Alexa - http://www.employees.org/~dwing/aaaa-stats/
• NAT64Check
• https://nat64check.go6lab.si/
• Google’s DNS64 Service
• https://developers.google.com/speed/public-dns/docs/dns64
Web Tools with interesting stats

25. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Confidential© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
IPv6 Only DC
(PoC Stages)
• Single Pod (ACI)
• Data plane only
• NAT64/DNS64
• Stateless and Stateful NAT

26. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Products Used DC Pod
ASR1K
Nexus 9K
C9396PX
Nexus 7K

27. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
High Level Topology
NAT64
ASR1K

28. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
• WAAS – Does not support IPv6 yet
• Kubernetes – IPv6 Not Supported / In Dev
• PXE Boot – Not supported over IPv6
• Storage – IPv6 only not tested – IPv4 must be served as long as it
exists or storage pools will be fragmented (cost and operational
impact)
• More as we further develop the design / get into deployment.
Gaps and Issues

29. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
1. How do you measure User Experience?
2. Where should IPv6 go first? DC, non-
DC?
3. How do you handle privacy extensions?
Questions?

30. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
• Measure User Experience = Metric for success
• Some websites required internal zone creation (We did that for high
impact sites that failed)
• InfoSec and related tooling is critical. Ensure that the necessary
compliance is still there and working (Privacy Extensions for eg.)
• It does work – failure scenarios will mostly be specific
• Finally, there is a price to pay. For some time, IPv6 development will
trail behind latest tech/features which may be IPv4 only.
Key Takeaways

32. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
IPv6 Address Plan (Top Level)
 Global 2001:420::/32
 Americas 2001:0420::/34
 EMEA and Asia Pacific 2001:0420:4000::/34
 Global Spare1 2001:0420:8000::/34
 Global Spare2 2001:0420:C000::/34
 Global Infrastructure 2001:0420:C000::/42
 Global Mobility 2001:0420:C040::/42

33. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Address Overview
Breaking down the /32
/34 Global Level
(50% spares)
/35 - /36
per Region
/37 - /39
per Sub-Region
/40 per Campus
(256 Buildings)
/48 per Building/Branch
(16 PINs per Building/Branch)
/52 per PIN *
(4096 Subnets / PIN)
FIXED TEMPLATE VARIABLE
* PIN = Place In the Network - A framework to classify functional areas of the network
eg, Lab, Desktop, DC, DMZ etc

34. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
/52/48 /64
Subnets / PIN
(4096)
2001:0420:028C:1000::/52 - Desktop PIN
2001:0420:028C:1300::/64 – Desktop VLAN 300
2001:0420:028C:1301::/64 – Desktop VLAN 301
2001:0420:028C:2000::/52 - Lab PIN
2001:0420:028C:2001::/64 – Lab Subnet 1
2001:0420:028C:2002::/64 – Lab Subnet 2
Address Planning
Template Addressing
41
PIN
(16)
0 = Infra
1 = Desktop / Wireless
2 = Lab
3 = Guest
4 = DMZ
D = Building DC
... etc
(13th Nibble) Functional Identifier
Building/Branch
Regional Identifier

35. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Charts

36. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
• How are we handling legacy v4 embedded in apps. Do we use
464XLAT? If yes, how do we plan on retiring it?
• Ans – 2 situations :
• 1) Embedded IPv4 literal will fail in ipv6 only
• 2) Host resolving to v4 will use NAT64 and leave a foot print
• Did we include BMS, HVAC, etc? Where is the NAT64 gateway?
• Ans: Only users and Network Infra. NAT64 is at site/network core of
campus
Questions