High Performance Kea DHCP Deployments at Large Scale
•Descargar como PPT, PDF•
2 recomendaciones•422 vistas
This document discusses Kea, an open source DHCP server from ISC. It provides high-level summaries of Kea's design, features, and deployments. Kea was designed for high performance, modularity, and scalability. Notable deployments include a large ISP serving over 2 million users and Facebook using Kea in their datacenters for bare metal provisioning. Kea provides advantages like failure resilience, rapid configuration updates, and metrics collection.
Recomendados
Recomendados
Más contenido relacionado
La actualidad más candente
La actualidad más candente (20)
Destacado
Destacado (20)
Similar a High Performance Kea DHCP Deployments at Large Scale
Similar a High Performance Kea DHCP Deployments at Large Scale (20)
Último
Último (20)
High Performance Kea DHCP Deployments at Large Scale
- 1. © 2015 ISC Kea Deployments High Performance DHCP PLNOG15, Kraków, 2015-09-29 Tomek Mrugalski <tomasz(at)isc(dot)org>
- 2. © 2015 ISC Who is Tomek? M.Sc., Ph.D from Gdansk University of Technology Primary author of Dibbler Portable DHCPv6 implementation (srv, cli, relay) Supports Win 2k-Win8, Linux, BSD, ~Solaris 7 years at Intel (Network Quality Labs, chipsets) 4 years at ISC Lead Developer of Kea (former BIND10 DHCP) Occasional contributor to ISC-DHCP Active IETF participant since 2009 DHC Working Group co-chair Author of 5 RFCs, 15+ drafts 2
- 3. © 2015 ISC Why re-write DHCP? • Existing code is old enough to buy alcohol • Hardware changed (many cores) • Networks changed • DHCP landscape changed • New software development techniques • Lacking performance • Monolithic • Documentation is lacking • Complex code, difficult to extend 3
- 4. © 2015 ISC Kea Open Source BIND10 component, now standalone C++ IPv6 first High Performance On-line configuration Switchable backends Multi-core support* • Flexible Logging • Statistics • Hooks • Modular • Portable (Linux, Free/Net/OpenBSD, Mac OS) • Well documented User's Guide, Developer's guide, Contributors Guide 4
- 5. © 2015 ISC Design Assumptions Long-lived project, design for 2035 Modular Scalable Networks with millions of devices – Many thousands of leases/sec – IPv4 and IPv6 on equal footing Maintainable Unit-tests Portable – Linux, BSD, Mac OS (Windows, Solaris possible) 5
- 6. © 2015 ISC Kea :: Overview DHCPv4 Server DHCPv4 Server DHCPv6 Server DHCPv6 Server DNS Updates DNS Updates perfdhcp •general purpose DHCP library •IPv4/IPv6 packet parsing/assembly •IPv4/IPv6 options parsing/assembly •interface detection (Linux, BSD/Mac OS) •socket management •general purpose DHCP library •IPv4/IPv6 packet parsing/assembly •IPv4/IPv6 options parsing/assembly •interface detection (Linux, BSD/Mac OS) •socket management libdhcp++ JSON config JSON config 6
- 7. © 2015 ISC Kea :: current Status Basic - v4, v6 parity - Address assignment, renewal, release, expiry - On-line configuration - Switchable backends: MySQL, PostgreSQL, memfile - Options (std) Advanced - Hooks (external libraries, a'la apache modules) - Options (custom, vendor, nested, namespaces) - Host reservation (addresses, prefixes) - MAC in DHCPv6 - Statistics (JSON) - Control Channel (JSON over Unix socket) - Flexible (per component) logging 7
- 8. © 2015 ISC Kea :: Roadmap 0.92 Aug 2015 Statistics Flexible Logging Control Channel 1.1 Spring 2016 Client classification Remote mgmt 1.0 Dec 2015 Lease expiration Decline PXE boot Public benchmarks 1.2 Late 2016 ?Radius auth ?Hooks config ?Lease query ?Server- initiated reconfig Contributors! DHCPv4 over v6, Secure DHCP fixed flexiblefixed flexible 8
- 9. © 2015 ISC Switchable Lease Database MySQL, PostgreSQL – Standard tools to read/update Memfile – Custom developed in C++ – Flat file storage (CSV) – Memory-only or memory+disk-write – Very high performance in mem-only mode – High performance in memory+disk mode Abstract LeaseMgr – C++ class, add your favourite storage DHCPv4/6 server MySQL PgSQL Memfile C++ Abstract LeaseMgr Scalable2Mdevices 9
- 10. © 2015 ISC DHCPpacketprocessingflow Hooks :: Extending the Server Kea Process Custom Library 1 Custom Library 2 F1() F2() G1() G2() Receive packet Select subnet Select lease Send packet 10
- 11. © 2015 ISC Hooks API • Many hook points defined (more being added) • buffer4_receive, buffer6_receive • pkt4_receive, pkt6_receive • subnet4_select, subnet6_select • lease4_select, lease6_select • lease{4,6}_{renew, release, decline} • pkt4_send, pkt6_send • buffer4_send, buffer6_send • Replaces conditional configuration processing in ISC-DHCP • Very well documented (developer’s guide, examples) http://git.kea.isc.org/~tester/kea/doxygen/ => Hooks Framework 11
- 12. © 2015 ISC Performance Results Preliminary data. Your mileage may vary. This is single threaded, single process for now. host: iMac 27" Mid 2011, 3.4 GHz i7, 16 GB 1333 MHz DDR3, OS X 10.10.1, VMWare Fusion Pro 7.1.0 Guest: Fedora 20 64 bits, 2 GB, 2 cores, gcc/g++ 4.8.3, glibc 2.16, MariaDB 5.5.41 (MySQL fork), PostgreSQL 9.3.6. 12
- 13. © 2015 ISC Deployments Large ISP 13
- 14. © 2015 ISC Large ISP Deployment Large, country wide ISP in western Europe needs DHCP solution to support his hotspots with 2M users Timeline: • Approached ISC in Dec. 2014 • Requested several features • Roadmap updated, beta drop in Feb. 2015 • On-site developer training • Signed support contract • Kea 0.9.1 with all features in Mar 2015 • Field trials in May 2015 • Country wide deployment – Aug 2015 14
- 15. © 2015 ISC Wireless Access Gateway 1 memfile open secure Wireless Access Gateway 2 open secure Kea1 Pool Open WAG-1 Pool Open WAG-2 Kea2 Pool Secure WAG-1 Pool Secure WAG-2 memfile DRBD RedHat Cluster Suite Kea1 Pool Open WAG-1 Pool Open WAG-2 Kea2 Pool Secure WAG-1 Pool Secure WAG-2 IP1 IP2 IP1 IP2 DRBD RedHat Cluster Suite memfile memfile Node A Active Node B Stand-by Large ISP Architecture 15
- 16. © 2015 ISC Large ISP :: Conclusions • Fast response (new feature in ~2 months) • Proven to work for 2M users, will work for more • High performance (7500 new leases/sec) • Support contract • 1 request so far, solved promptly Status: Serving country-wide WiFi network in western Europe, ~2 million users 16
- 17. © 2015 ISC Kea @ Facebook Datacenter
- 18. © 2015 ISC Datacenter Deployment Overview: Large DC with inventory database, multiple DCs Redundancy problems, wants to use inventory DB for bare metal provisioning, PXE boot. Had complex git/rsync config generation with periodic restarts, took hours to propagate changes 18
- 19. © 2015 ISC Datacenter :: Overview server server server server … Rack switch 19
- 20. © 2015 ISC server server server …rack switch server server server … rack switch DHCP relay DHCP relay DHCP Server 1 active stand by DHCP Server 2 HW load balancers cost $$$ Both servers down => LB down => Problem 1 :: Failure domain 20
- 21. © 2015 ISC Problem 2 :: Bootstrapping DHCP Server serve r … switc h serve r serve r serve r … switc h serve r serve r serve r… switc h serve r serve r serve r … switc h serve r serve r serve r … switc h serve r serve r serve r … switc h serve r serve r serve r … switc h serve r serve r serve r … switc h serve r serve r serve r … switc h serve r serve r serve r … switc h serve r serve r serve r … switc h serve r serve r serve r … switc h serve r serve r serve r … switc h serve r serve r serve r … switc h serve r serve r serve r … switc h serve r serve r serve r … switc h serve r serve r serve r … switc h serve r serve r serve r … switc h serve r serve r serve r … switc h serve r serve r serve r … switc h serve r serve r serve r … switc h serve r serve r serve r… switc h serve r serve r serve r … switc h serve r serve r serve r … switc h serve r serve r serve r … switc h serve r serve r serve r … switc h serve r serve r serve r … switc h serve r serve r serve r … switc h serve r serve r serve r … switc h serve r serve r serve r … switc h serve r serve r serve r … switc h serve r serve r serve r … switc h serve r serve r serve r … switc h serve r serve r serve r … switc h serve r serve r serve r … switc h serve r serve r serve r … switc h serve r serve r serve r … switc h serve r serve r serve r … switc h serve r serve r serve r … switc h serve r serve r serve r … switc h serve r serve r serve r … switc h serve r serve r serve r … switc h serve r serve r serve r … switc h serve r serve r serve r … switc h serve r serve r serve r … switc h serve r serve r serve r … switc h serve r serve r serve r … switc h serve r serve r serve r … switc h serve r serve r remote cluster 21
- 22. © 2015 ISC Problem 3 :: Configuration Pipeline DHCP server Inventory system Periodic job Git repo delivery system Chef Infrastructu re Nanny script /etc/dhcpd/ … /etc/init.d/ restart • People changing hw => HW addr changing • Inventory update took up to 4h to propagate • Lack of instrumentation • Latencies • Errors? Exceptions? 22
- 23. © 2015 ISC Kea Deployment Kea (v4, v6) • buffer4_receive • pkt4_receive • subnet4_select • lease4_select • pkt4_send • buffer4_send Hooklibrary • Facebook specific hook lib extracts addresses from DB • Effectively “stateless” DHCP • DB changes are applicable immediately (no config sync delay, no restart, no reconfiguration downtimes) 23
- 24. © 2015 ISC DHCP Anycasting Announcing single host route over BGP, the same on all servers exaBGP DHCP server exaBGP DHCP server Region 1 Region 2 exaBGP DHCP server exaBGP DHCP server infrastructure routers BGP BGP BGP BGP 24
- 25. © 2015 ISC Facebook :: Timeline Timeline •Month 1: proof of concept developed, test deployment in 1 cluster •Month 2-3: productize, integrate with provisioning infrastructure •Month 4: mass deployment, serving all clients with Kea! 25
- 26. © 2015 ISC Facebook :: Benefits Benefits •Anycast: packets sent to the nearest server • => No more HW load balancing • => Any DHCP server instance can handle any request (in the same or other cluster, it doesn’t matter) •HW replacement is easier, changes picked up instantly •JSON configuration, trivial to manage/regenerate •Metrics! 26 https://www.isc.org/blogs/how-facebook-is-using-kea- in-the-datacenter/ https://www.usenix.org/conference/srecon15europe/p rogram/presentation/failla
- 27. © 2015 ISC Get Involved Open source Community asset Core repository is public Bug datatase public Mailing lists, jabber public Test, report bugs Contribute Support contract Submit patches (read Contributor’s Guide first) Sponsorship Development contracts Review design documents Looking for contributions: • Additional back ends • Apps via Hooks API • Basic GUI •… github.com/isc-projects/kea http://kea.isc.org 27
- 28. © 2015 ISC Questions? http://kea.isc.org, https://www.isc.org/kea/ github.com/isc-projects/kea 28
- 29. © 2015 ISC Thank you