SlideShare una empresa de Scribd logo
1 de 32
cache
   @TimYang
Agenda


• Feed
•      Cache
•
•   ,   status feed

•
Feed

•                 feed

    • Push(       )

    • Pull(   )

    •
Push
•     feed

    • Inbox:
    • Outbox:
•                   inbox( )

•               Inbox( )
Push
•
•


•
Pull


•                 outbox( )

•                       Inbox(   )

• Online computation
Pull


•
•
•
•


•
Cache
memory is the new disk,
and disk is the new tape.
for "real-time" web applications,
and systems that require massive scalability
     - Jim Gray
Cache
cache
          Weibo cache arch

  Inbox       hot cache




Outbox       Vector cache    Archive cache




 Social       Following        Followers     users
 Graph



Content       Hot cache          Total
Inbox
•       “         ”

• cache      id list

•                   (hot cache)

•                   get set

• redis list/set
Outbox

•
•     id list(e.g. 100)

•   id LRU
Social Graph cache
• Following ids
• Followers
 •
 •
 •            delete all
Content cache
•           cache

•
    •
•
    • Open API      xml, json...
Cache


•
•
Update Workflow


                  Update status




Content cache    Hot Inbox Vector   Outbox vector



Content cache
 replication
•   outbox hot vector

•   followers list

•   inbox
feed
                Home timeline Workflow


                     home_timeline


                      aggregator



                                         Content hot
Inbox cache         Outbox Vector
                                           cache


Inbox archive        Inbox archive      Content cache
feed

•   inbox cache

•
•         ,   following

•   id list         feed
cache

•
• hot keys
•
• mutex
•                     Content cache

• multi get n feed(n = items/ , e.g. 50)
• cache = n * (feed +                    ,
    e.g. 2k)
•                1,000   /
• 1,000    800Mbps

•1        8Gbps
•       1G       300~400Mbps

•
    •        local cache

    •
    •
hot keys
• content cache of
• create local cache
  1. get user_yaochen_local
  2. get user_yaochen
       1. set user_yaochen_local:value
  3.               delete all
cache
•                      key
      memcache

    •
    • LRU cache,
    • Memory storage
mutex
•       content item cache

    •           db

    •                        db

• key_mutex                   key

• mutex
•
    • S2
    • S2
    •
S2

• Web 2.0 Social
• What’s S2
 • Social Platform &
 • Social Applications
•   www.s2forum.org
•
• Sina App Engine
•
•
•
Q&A


• email: iso1600@gmail.com
•              @TimYang

Más contenido relacionado

La actualidad más candente

Introduction to Storm
Introduction to Storm Introduction to Storm
Introduction to Storm
Chandler Huang
 
Analyzing Historical Data of Applications on YARN for Fun and Profit
Analyzing Historical Data of Applications on YARN for Fun and ProfitAnalyzing Historical Data of Applications on YARN for Fun and Profit
Analyzing Historical Data of Applications on YARN for Fun and Profit
DataWorks Summit
 
關於SQL Injection的那些奇技淫巧
關於SQL Injection的那些奇技淫巧關於SQL Injection的那些奇技淫巧
關於SQL Injection的那些奇技淫巧
Orange Tsai
 

La actualidad más candente (20)

Introduction to Storm
Introduction to Storm Introduction to Storm
Introduction to Storm
 
Tips on High Performance Server Programming
Tips on High Performance Server ProgrammingTips on High Performance Server Programming
Tips on High Performance Server Programming
 
Introduction to memcached
Introduction to memcachedIntroduction to memcached
Introduction to memcached
 
The Web of data and web data commons
The Web of data and web data commonsThe Web of data and web data commons
The Web of data and web data commons
 
A simple introduction to redis
A simple introduction to redisA simple introduction to redis
A simple introduction to redis
 
Caching solutions with Redis
Caching solutions   with RedisCaching solutions   with Redis
Caching solutions with Redis
 
Pegasus: Designing a Distributed Key Value System (Arch summit beijing-2016)
Pegasus: Designing a Distributed Key Value System (Arch summit beijing-2016)Pegasus: Designing a Distributed Key Value System (Arch summit beijing-2016)
Pegasus: Designing a Distributed Key Value System (Arch summit beijing-2016)
 
Service mesh(istio) monitoring
Service mesh(istio) monitoringService mesh(istio) monitoring
Service mesh(istio) monitoring
 
우아한테크세미나-우아한멀티모듈
우아한테크세미나-우아한멀티모듈우아한테크세미나-우아한멀티모듈
우아한테크세미나-우아한멀티모듈
 
Analyzing Historical Data of Applications on YARN for Fun and Profit
Analyzing Historical Data of Applications on YARN for Fun and ProfitAnalyzing Historical Data of Applications on YARN for Fun and Profit
Analyzing Historical Data of Applications on YARN for Fun and Profit
 
關於SQL Injection的那些奇技淫巧
關於SQL Injection的那些奇技淫巧關於SQL Injection的那些奇技淫巧
關於SQL Injection的那些奇技淫巧
 
Redis overview for Software Architecture Forum
Redis overview for Software Architecture ForumRedis overview for Software Architecture Forum
Redis overview for Software Architecture Forum
 
(Re)Indexing Large Repositories in Alfresco
(Re)Indexing Large Repositories in Alfresco(Re)Indexing Large Repositories in Alfresco
(Re)Indexing Large Repositories in Alfresco
 
Redis Reliability, Performance & Innovation
Redis Reliability, Performance & InnovationRedis Reliability, Performance & Innovation
Redis Reliability, Performance & Innovation
 
Pegasus KV Storage, Let the Users focus on their work (2018/07)
Pegasus KV Storage, Let the Users focus on their work (2018/07)Pegasus KV Storage, Let the Users focus on their work (2018/07)
Pegasus KV Storage, Let the Users focus on their work (2018/07)
 
Redis Streams plus Spark Structured Streaming
Redis Streams plus Spark Structured StreamingRedis Streams plus Spark Structured Streaming
Redis Streams plus Spark Structured Streaming
 
Cassandra Introduction & Features
Cassandra Introduction & FeaturesCassandra Introduction & Features
Cassandra Introduction & Features
 
Best Practices for Running PostgreSQL on AWS - DAT314 - re:Invent 2017
Best Practices for Running PostgreSQL on AWS - DAT314 - re:Invent 2017Best Practices for Running PostgreSQL on AWS - DAT314 - re:Invent 2017
Best Practices for Running PostgreSQL on AWS - DAT314 - re:Invent 2017
 
실전 프론트엔드 개발 - 취직 그리고 이직
실전 프론트엔드 개발 - 취직 그리고 이직실전 프론트엔드 개발 - 취직 그리고 이직
실전 프론트엔드 개발 - 취직 그리고 이직
 
Facebook architecture
Facebook architectureFacebook architecture
Facebook architecture
 

Destacado

人人网技术经理张铁安 Feed系统结构浅析
人人网技术经理张铁安 Feed系统结构浅析人人网技术经理张铁安 Feed系统结构浅析
人人网技术经理张铁安 Feed系统结构浅析
isnull
 
设计模式分享
设计模式分享设计模式分享
设计模式分享
michaelsh
 
Erlang分布式系统的的领域语言
Erlang分布式系统的的领域语言Erlang分布式系统的的领域语言
Erlang分布式系统的的领域语言
Feng Yu
 
Depoimento de Amaury Jr ne PF
Depoimento de Amaury Jr ne PFDepoimento de Amaury Jr ne PF
Depoimento de Amaury Jr ne PF
Luis Nassif
 
20140717 about openstreetmap
20140717 about openstreetmap20140717 about openstreetmap
20140717 about openstreetmap
Taichi Furuhashi
 
Lesson 7 1 Part 2 1
Lesson 7  1 Part 2 1Lesson 7  1 Part 2 1
Lesson 7 1 Part 2 1
mradcliffe
 
Fleury no inferno
Fleury no infernoFleury no inferno
Fleury no inferno
Luis Nassif
 
About OSM japan at the Association of Japanese Geographers
About OSM japan at the Association of Japanese GeographersAbout OSM japan at the Association of Japanese Geographers
About OSM japan at the Association of Japanese Geographers
Taichi Furuhashi
 

Destacado (20)

新浪微博Feed服务架构
新浪微博Feed服务架构新浪微博Feed服务架构
新浪微博Feed服务架构
 
Build scalable microblog qcon beijing 2010
Build scalable microblog qcon beijing 2010Build scalable microblog qcon beijing 2010
Build scalable microblog qcon beijing 2010
 
分布式Key Value Store漫谈
分布式Key Value Store漫谈分布式Key Value Store漫谈
分布式Key Value Store漫谈
 
新浪微博分布式缓存与队列-2013版
新浪微博分布式缓存与队列-2013版新浪微博分布式缓存与队列-2013版
新浪微博分布式缓存与队列-2013版
 
Feed服务架构-新浪微博新员工培训议题
Feed服务架构-新浪微博新员工培训议题Feed服务架构-新浪微博新员工培训议题
Feed服务架构-新浪微博新员工培训议题
 
高性能队列Fqueue的设计和使用实践
高性能队列Fqueue的设计和使用实践高性能队列Fqueue的设计和使用实践
高性能队列Fqueue的设计和使用实践
 
人人网技术经理张铁安 Feed系统结构浅析
人人网技术经理张铁安 Feed系统结构浅析人人网技术经理张铁安 Feed系统结构浅析
人人网技术经理张铁安 Feed系统结构浅析
 
设计模式分享
设计模式分享设计模式分享
设计模式分享
 
Java常见问题排查
Java常见问题排查Java常见问题排查
Java常见问题排查
 
Erlang分布式系统的的领域语言
Erlang分布式系统的的领域语言Erlang分布式系统的的领域语言
Erlang分布式系统的的领域语言
 
13 tips for a highly engaging feed
13 tips for a highly engaging feed13 tips for a highly engaging feed
13 tips for a highly engaging feed
 
Depoimento de Amaury Jr ne PF
Depoimento de Amaury Jr ne PFDepoimento de Amaury Jr ne PF
Depoimento de Amaury Jr ne PF
 
20140717 about openstreetmap
20140717 about openstreetmap20140717 about openstreetmap
20140717 about openstreetmap
 
Lesson 7 1 Part 2 1
Lesson 7  1 Part 2 1Lesson 7  1 Part 2 1
Lesson 7 1 Part 2 1
 
Fleury no inferno
Fleury no infernoFleury no inferno
Fleury no inferno
 
About OSM japan at the Association of Japanese Geographers
About OSM japan at the Association of Japanese GeographersAbout OSM japan at the Association of Japanese Geographers
About OSM japan at the Association of Japanese Geographers
 
Miguli Basic
Miguli BasicMiguli Basic
Miguli Basic
 
Space hackathon5
Space hackathon5Space hackathon5
Space hackathon5
 
Collaborative 3D Modelling and Printing: What you See is not Directly What Yo...
Collaborative 3D Modelling and Printing: What you See is not Directly What Yo...Collaborative 3D Modelling and Printing: What you See is not Directly What Yo...
Collaborative 3D Modelling and Printing: What you See is not Directly What Yo...
 
Alfredorodriguez
AlfredorodriguezAlfredorodriguez
Alfredorodriguez
 

Similar a 微博cache设计谈

Timyang新浪微博设计谈
Timyang新浪微博设计谈Timyang新浪微博设计谈
Timyang新浪微博设计谈
Cevin Cheung
 
Tup2 新浪杨卫华
Tup2 新浪杨卫华Tup2 新浪杨卫华
Tup2 新浪杨卫华
yongboy
 
SDEC2011 NoSQL concepts and models
SDEC2011 NoSQL concepts and modelsSDEC2011 NoSQL concepts and models
SDEC2011 NoSQL concepts and models
Korea Sdec
 
BuilHigh Performance Weibo Platform-Qcon2011
BuilHigh Performance Weibo Platform-Qcon2011BuilHigh Performance Weibo Platform-Qcon2011
BuilHigh Performance Weibo Platform-Qcon2011
Yiwei Ma
 
A Case Study of NoSQL Adoption: What Drove Wordnik Non-Relational?
A Case Study of NoSQL Adoption: What Drove Wordnik Non-Relational?A Case Study of NoSQL Adoption: What Drove Wordnik Non-Relational?
A Case Study of NoSQL Adoption: What Drove Wordnik Non-Relational?
DATAVERSITY
 
Introducing Amplify
Introducing AmplifyIntroducing Amplify
Introducing Amplify
appendTo
 

Similar a 微博cache设计谈 (20)

Timyang新浪微博设计谈
Timyang新浪微博设计谈Timyang新浪微博设计谈
Timyang新浪微博设计谈
 
Tup2 新浪杨卫华
Tup2 新浪杨卫华Tup2 新浪杨卫华
Tup2 新浪杨卫华
 
杨卫华:微博cache设计浅谈
杨卫华:微博cache设计浅谈杨卫华:微博cache设计浅谈
杨卫华:微博cache设计浅谈
 
Kiosk / PHP
Kiosk / PHP Kiosk / PHP
Kiosk / PHP
 
Cache on Delivery
Cache on DeliveryCache on Delivery
Cache on Delivery
 
SDEC2011 NoSQL concepts and models
SDEC2011 NoSQL concepts and modelsSDEC2011 NoSQL concepts and models
SDEC2011 NoSQL concepts and models
 
BuilHigh Performance Weibo Platform-Qcon2011
BuilHigh Performance Weibo Platform-Qcon2011BuilHigh Performance Weibo Platform-Qcon2011
BuilHigh Performance Weibo Platform-Qcon2011
 
04 standard class library c#
04 standard class library c#04 standard class library c#
04 standard class library c#
 
ログ収集プラットフォーム開発におけるElasticsearchの運用
ログ収集プラットフォーム開発におけるElasticsearchの運用ログ収集プラットフォーム開発におけるElasticsearchの運用
ログ収集プラットフォーム開発におけるElasticsearchの運用
 
Amazon Aurora로 안전하게 migration 하기
Amazon Aurora로 안전하게 migration 하기Amazon Aurora로 안전하게 migration 하기
Amazon Aurora로 안전하게 migration 하기
 
A Case Study of NoSQL Adoption: What Drove Wordnik Non-Relational?
A Case Study of NoSQL Adoption: What Drove Wordnik Non-Relational?A Case Study of NoSQL Adoption: What Drove Wordnik Non-Relational?
A Case Study of NoSQL Adoption: What Drove Wordnik Non-Relational?
 
Introducing Amplify
Introducing AmplifyIntroducing Amplify
Introducing Amplify
 
Html5: Something wicked this way comes (Hack in Paris)
Html5: Something wicked this way comes (Hack in Paris)Html5: Something wicked this way comes (Hack in Paris)
Html5: Something wicked this way comes (Hack in Paris)
 
Nuxeo World Session: CMIS - What's Next?
Nuxeo World Session: CMIS - What's Next?Nuxeo World Session: CMIS - What's Next?
Nuxeo World Session: CMIS - What's Next?
 
OSDC 2013 | Introduction into Chef by Andy Hawkins
OSDC 2013 | Introduction into Chef by Andy HawkinsOSDC 2013 | Introduction into Chef by Andy Hawkins
OSDC 2013 | Introduction into Chef by Andy Hawkins
 
OSMC 2016 - Monitor your infrastructure with Elastic Beats by Monica Sarbu
OSMC 2016 - Monitor your infrastructure with Elastic Beats by Monica SarbuOSMC 2016 - Monitor your infrastructure with Elastic Beats by Monica Sarbu
OSMC 2016 - Monitor your infrastructure with Elastic Beats by Monica Sarbu
 
OSMC 2016 | Monitor your Infrastructure with Elastic Beats by Monica Sarbu
OSMC 2016 | Monitor your Infrastructure with Elastic Beats by Monica SarbuOSMC 2016 | Monitor your Infrastructure with Elastic Beats by Monica Sarbu
OSMC 2016 | Monitor your Infrastructure with Elastic Beats by Monica Sarbu
 
ActiveRecordのテーブル名の話
ActiveRecordのテーブル名の話ActiveRecordのテーブル名の話
ActiveRecordのテーブル名の話
 
初心者向けGAE/Java説明資料
初心者向けGAE/Java説明資料初心者向けGAE/Java説明資料
初心者向けGAE/Java説明資料
 
iPhone dev intro
iPhone dev introiPhone dev intro
iPhone dev intro
 

Último

Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
panagenda
 
Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native Applications
WSO2
 

Último (20)

TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
 
Corporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptxCorporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptx
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...
 
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century education
 
MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024
 
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?
 
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
 
Navi Mumbai Call Girls 🥰 8617370543 Service Offer VIP Hot Model
Navi Mumbai Call Girls 🥰 8617370543 Service Offer VIP Hot ModelNavi Mumbai Call Girls 🥰 8617370543 Service Offer VIP Hot Model
Navi Mumbai Call Girls 🥰 8617370543 Service Offer VIP Hot Model
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
 
Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)
 
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodPolkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
 
Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native Applications
 
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
 
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
 
Artificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : UncertaintyArtificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : Uncertainty
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
ICT role in 21st century education and its challenges
ICT role in 21st century education and its challengesICT role in 21st century education and its challenges
ICT role in 21st century education and its challenges
 
Real Time Object Detection Using Open CV
Real Time Object Detection Using Open CVReal Time Object Detection Using Open CV
Real Time Object Detection Using Open CV
 

微博cache设计谈

Notas del editor