FinagleCon_2015_Pinterest
•Descargar como PPTX, PDF•
2 recomendaciones•1,072 vistas
How we use Finagle at Pinterest. Presented at FinagleCon hosted by Twitter on 08/13/2015.
Recomendados
Más contenido relacionado
La actualidad más candente
La actualidad más candente (8)
Destacado
Destacado (20)
Similar a FinagleCon_2015_Pinterest
Similar a FinagleCon_2015_Pinterest (20)
Último
Último (20)
FinagleCon_2015_Pinterest
- 1. Finagle at Pinterest 1 Yongsheng Wu Email: yongsheng@pinterest.com Linkedin: www.linkedin.com/in/yongshengwu Pinterest: www.pinterest.com/yswu Twitter: @yswu 08-13-2015
- 2. 2 Introduction of Pinterest SOA Service Proxy Dynamic Service Discovery Python Finagle Thrift Client Genesis
- 3. Pinterest is solving the discovery problem. 3
- 4. 4
- 5. 5
- 6. 6 50+ Billion Pins categorized by people into more than 1 Billion Boards
- 7. 7 Introduction of Pinterest SOA Service Proxy Dynamic Service Discovery Python Finagle Thrift Client Genesis
- 10. 10
- 11. Why Finagle? 11 ๏Reactive Manifesto ✓Responsive ✓Resilient ✓Elastic ✓Message Driven ๏Efficiency ๏Future Programming Model ๏Visibility ๏Community
- 12. Best Practices for Service Migration 12 ๏Avoid Changes of Functionalities ๏Test-Driven Development ‣ Unit Test ‣ Integration Test ๏Dark Traffic ‣ Comparison ‣ Capacity ๏Light Traffic
- 13. Pin Creation 13
- 14. 14 Introduction of Pinterest SOA Service Proxy Dynamic Service Discovery Python Finagle Thrift Client Genesis
- 15. Service Proxy 15
- 16. Service Proxy 16
- 17. Service Proxy Request Logging 17 ๏Request logging automatically handled by proxy ‣ Sampled Access Logging ‣ Sampled Slow Request Logging ‣ Sampled Failure Request Logging ๏Log format
- 18. Service Proxy Request Logging 18 ๏Request logging automatically handled by proxy ‣ Sampled Access Logging ‣ Sampled Slow Request Logging ‣ Sampled Failure Request Logging ๏Log format
- 19. Service Proxy Request Logging 19 ๏Request logging automatically handled by proxy ‣ Sampled Access Logging ‣ Sampled Slow Request Logging ‣ Sampled Failure Request Logging ๏Log format
- 20. Service Proxy Request Logging 20 ๏Request logging automatically handled by proxy ‣ Sampled Access Logging ‣ Sampled Slow Request Logging ‣ Sampled Failure Request Logging ๏Log format
- 21. Service Proxy Request Logging 21 ๏Request logging automatically handled by proxy ‣ Sampled Access Logging ‣ Sampled Slow Request Logging ‣ Sampled Failure Request Logging ๏Log format
- 26. Service Proxy Rate Limiting 26 ๏Guava RateLimiter (Token Bucket based) and ServerSet ๏No coordination among servers in the cluster ๏Rate limiting rules ๏Max QPS to service ๏Max QPS to a particular API endpoint ๏Max QPS from a specific client to service ๏Max QPS from a specific client to a particular API endpoint
- 27. 27
- 28. 28 Introduction of Pinterest SOA Service Proxy Dynamic Service Discovery Python Finagle Thrift Client Genesis
- 32. 32 Introduction of Pinterest SOA Service Proxy Dynamic Service Discovery Python Finagle Thrift Client Genesis
- 35. 35 Introduction of Pinterest SOA Service Proxy Dynamic Service Discovery Python Finagle Thrift Client Genesis
- 36. Genesis 36 ๏Interactive Q&A Shell ๏Finagle Server and Client Generation ‣ Existing thrift IDL ๏Python Thrift Client Generation ๏Statsboard Generation ‣ System Metrics (CPU, Memory, Network, Disk) ‣ Finagle Stats (requests, failures, GC) ‣ Per API Endpoint Stats ‣ QPS ‣ Failure Count ‣ Latency
- 37. Genesis 37
- 38. Genesis 38
- 39. Genesis 39
- 40. 40 Introduction of Pinterest SOA Service Proxy Dynamic Service Discovery Python Finagle Thrift Client Genesis
- 41. Future Challenges 41 ๏Unified Data Model, Partial Retrieval ๏One-step Service Generation ๏Distributed Tracing ๏Auto-scaling ๏Cell-based Architecture ๏MUX, HTTP/2 ๏… ….
Notas del editor
- About me: Joined 3 years ago, early engineer on Infra team at Pinterest Storage systems on Twitter’s core data sets: users, tweets, social graph CRM, Chatter, Platform as a Service at Salesforce
- Pinterest if you want to be inspired, you want to discover What should I have for lunch? What art pieces I may love? Emphasize “Picked for you”
- Explore what may interest you. Explore interests, related interests, discover things. Follow interests
- Discovery at scale
- Why SOA???
- 3 founders (Ben, Evan, and Paul), 1 engineer (Yash)
- December, 2014 Currently 600+ employees, 200+ engineers We are scaling our organization really rapidly
- We want to foster long term ownership. Accountability. SLA
- Responsive: timeout setting possible at many levels Resilient: retries, connection pooling, bad host detection/eviction/retry, queue policies Elastic: dynamic service discovery Message Driven: Async RPC, thrift, protobuf, http, memcache, redis, etc. Efficiency: not Python (slow, not static type checked, single process, single request handling), JVM based platform, tens of thousands QPS per host Future: better than callbacks, way better than java future back in the days Community: very well supported, plus we know the committers :-)
- For both dark and light stages, a mechanism can easily dial up/down the traffic to the new service is a key component of the system. dark traffic most of time will cause additional load, so capacity needs to be watched closely in some cases; while light traffic and traffic to original service are typically exclusive to each other, no such a concern in most cases.
- Today we have ~100 services on prod, majority of them are finagle based, but we also have services in goLang, c++, and python. There are challenges, we want to build the same framework features for all platforms, but it takes time and effort.
- Use Java Dynamic Proxy for code injection
- sampled access logging, slow request logging, and failed request logging. make it much easier for the service owners to debug and monitor their services.
- Dimensions: client, server host, api endpoint, exception type
- massive productivity gain: several hours down to a couple of minutes good investment for team if creating a lot of services, also a building block for one-step service generation Shell: JLine2, autocompletion of file path; FB Swift for thrift IDL parsing, Python ptsd package for thrift IDL parsing
- screenshots of statsboard generated, finagle server/client generation, thrift client generation
- screenshots of statsboard generated, finagle server/client generation, thrift client generation
- Recap