Recommended
Recommended
More Related Content
Viewers also liked
Recently uploaded
Recently uploaded (20)
Cassandra at Formspring - How it Solved Scalability Issues
- 1. Cassandra @Formspring Tuesday, August 30, 2011
- 2. Cassandra @Formspring Yet another Knight, Dragon and Princess Story. Tuesday, August 30, 2011
- 3. Formspring helps people find out more about each other by sharing interesting & personal responses Tuesday, August 30, 2011
- 4. 25M users Tuesday, August 30, 2011
- 5. 3.5B responses Tuesday, August 30, 2011
- 6. Tuesday, August 30, 2011
- 7. Tuesday, August 30, 2011
- 8. Tuesday, August 30, 2011
- 9. Tuesday, August 30, 2011
- 10. Tuesday, August 30, 2011
- 11. Tuesday, August 30, 2011
- 12. Tuesday, August 30, 2011
- 13. Tuesday, August 30, 2011
- 14. Tuesday, August 30, 2011
- 15. Tuesday, August 30, 2011
- 16. Tuesday, August 30, 2011
- 17. Tuesday, August 30, 2011
- 18. Tuesday, August 30, 2011
- 19. Tuesday, August 30, 2011
- 20. Tuesday, August 30, 2011
- 21. Tuesday, August 30, 2011
- 22. Follow[ing|ers] Tuesday, August 30, 2011
- 23. Tuesday, August 30, 2011
- 24. Tuesday, August 30, 2011
- 25. Tuesday, August 30, 2011
- 26. Tuesday, August 30, 2011
- 27. Our situation before Cassandra Tuesday, August 30, 2011
- 28. MySQL + Memcache Tuesday, August 30, 2011
- 29. MySQL + Memcache • Flat MySQL table Tuesday, August 30, 2011
- 30. MySQL + Memcache • Flat MySQL table • Indexes (Followers vs Following) Tuesday, August 30, 2011
- 31. MySQL + Memcache • Flat MySQL table • Indexes (Followers vs Following) • Memcache Tuesday, August 30, 2011
- 32. Memcache • Stored as JSON list Tuesday, August 30, 2011
- 33. Memcache • Stored as JSON list • Can get out of sync Tuesday, August 30, 2011
- 34. MySQL • Hundreds of millions of rows Tuesday, August 30, 2011
- 35. MySQL • Hundreds of millions of rows • m2.4xl EC2 instance Tuesday, August 30, 2011
- 36. It all started with a Feature. Tuesday, August 30, 2011
- 37. QOTD Tuesday, August 30, 2011
- 38. QOTD Tuesday, August 30, 2011
- 39. QOTD • Feb: 35K Followers Tuesday, August 30, 2011
- 40. QOTD • Feb: 35K Followers • Mid-Feb: 100K Tuesday, August 30, 2011
- 41. QOTD • Feb: 35K Followers • Mid-Feb: 100K • Growing about 30K a week. Tuesday, August 30, 2011
- 42. QOTD • Feb: 35K Followers • Mid-Feb: 100K • Growing about 30K a week. • 200K: closer to Memcache 1MB limit. Tuesday, August 30, 2011
- 43. QOTD • Feb: 35K Followers • Mid-Feb: 100K • Growing about 30K a week. • 200K: closer to Memcache 1MB limit. • (with gz compaction . . .) Tuesday, August 30, 2011
- 44. Then we added more features . . . Tuesday, August 30, 2011
- 45. Tuesday, August 30, 2011
- 46. Tuesday, August 30, 2011
- 47. Temporary solution Tuesday, August 30, 2011
- 48. Redis • Key : JSON blob of IDs Tuesday, August 30, 2011
- 49. Redis • Key : JSON blob of IDs • ~2-3s / insert Tuesday, August 30, 2011
- 50. Frontend Ask Mem MySQL Cache Tuesday, August 30, 2011
- 51. Frontend Ask Outbox Mem QW MySQL Cache Tuesday, August 30, 2011
- 52. Frontend Ask SELECT account_id FROM follow Outbox WHERE following = 1234 Mem QW MySQL Cache Tuesday, August 30, 2011
- 53. Frontend Ask Outbox OH SHIT Mem QW MySQL Cache Tuesday, August 30, 2011
- 54. Frontend Ask Outbox OH SHIT Mem QW MySQL Cache Tuesday, August 30, 2011
- 55. Frontend Ask Outbox OH SHIT Mem QW MySQL Cache Gazillion messages Tuesday, August 30, 2011
- 56. Frontend Ask Outbox OH SHIT Mem QW MySQL Cache Inbox Inbox Inbox Gazillion messages QWInbox Inbox QW QWInbox Inbox QW QWInbox Inbox QW QWInbox Inbox QW QWInbox Inbox QW QWInbox Inbox QW QWInbox Inbox QW QW QW QWs Tuesday, August 30, 2011
- 57. Frontend Ask Outbox OH SHIT Mem QW MySQL Cache Inbox Inbox Inbox Gazillion messages QWInbox Inbox QW QWInbox Inbox QW QWInbox Inbox QW QWInbox Inbox QW QWInbox Inbox QW QWInbox Inbox QW QWInbox Inbox QW QW QW Inbox QWs Object Tuesday, August 30, 2011
- 58. Frontend Ask Outbox *HANGING* OH SHIT Mem QW MySQL Cache Inbox Inbox Inbox Gazillion messages QWInbox Inbox QW QWInbox Inbox QW QWInbox Inbox QW QWInbox Inbox QW QWInbox Inbox QW QWInbox Inbox QW QWInbox Inbox QW QW QW Inbox QWs Object Tuesday, August 30, 2011
- 59. Frontend Ask Outbox *OOM* OH SHIT Mem QW MySQL Cache Inbox Inbox Inbox Gazillion messages QWInbox Inbox QW QWInbox Inbox QW QWInbox Inbox QW QWInbox Inbox QW QWInbox Inbox QW QWInbox Inbox QW QWInbox Inbox QW QW QW Inbox QWs Object Tuesday, August 30, 2011
- 60. Frontend Ask Outbox Outbox Outbox Outbox OH SHIT Mem Outbox QW Outbox QW MySQL Outbox QW QW QW QW QW Cache Inbox Inbox Inbox Gazillion messages QWInbox Inbox QW QWInbox Inbox QW QWInbox Inbox QW QWInbox Inbox QW QWInbox Inbox QW QWInbox Inbox QW QWInbox Inbox QW QW QW Inbox QWs Object Tuesday, August 30, 2011
- 61. Frontend Ask Outbox Outbox Outbox Outbox OH SHIT Mem Outbox QW Outbox QW MySQL QW OOM QW QW QW Cache Inbox Inbox Inbox Gazillion messages QWInbox Inbox QW QWInbox Inbox QW QWInbox Inbox QW QWInbox Inbox QW QWInbox Inbox QW QWInbox Inbox QW QWInbox Inbox QW QW QW Inbox QWs Object Tuesday, August 30, 2011
- 62. We need a solution Tuesday, August 30, 2011
- 63. Fast Tuesday, August 30, 2011
- 64. Tuesday, August 30, 2011
- 65. Step 1: Research Tuesday, August 30, 2011
- 66. Tuesday, August 30, 2011
- 67. Tuesday, August 30, 2011
- 68. Tuesday, August 30, 2011
- 69. Tuesday, August 30, 2011
- 70. Tuesday, August 30, 2011
- 71. Our first shot with Cassandra Tuesday, August 30, 2011
- 72. All Responses Tuesday, August 30, 2011
- 73. Tuesday, August 30, 2011
- 74. Explain our stack here Tuesday, August 30, 2011
- 75. Index of responses to questions on the site. Tuesday, August 30, 2011
- 76. Index of responses to questions on the site. Tuesday, August 30, 2011
- 77. Explain our stack here Tuesday, August 30, 2011
- 78. Tuesday, August 30, 2011
- 79. Ask Followers Tuesday, August 30, 2011
- 80. How to scale ? Tuesday, August 30, 2011
- 81. Build a Social Graph Tuesday, August 30, 2011
- 82. Social Graph Tuesday, August 30, 2011
- 83. Build a Social Graph • Python Tuesday, August 30, 2011
- 84. Build a Social Graph • Python • Cassandra (Obviously..) Tuesday, August 30, 2011
- 85. Build a Social Graph • Python • Cassandra (Obviously..) • PyCassa Tuesday, August 30, 2011
- 86. Build a Social Graph • Python • Cassandra (Obviously..) • PyCassa • Thrift Tuesday, August 30, 2011
- 87. 4 types of actions • Following Tuesday, August 30, 2011
- 88. 4 types of actions • Following • Followers Tuesday, August 30, 2011
- 89. 4 types of actions • Following • Followers • Blocks Tuesday, August 30, 2011
- 90. 4 types of actions • Following • Followers • Blocks • Blockers Tuesday, August 30, 2011
- 91. Thrift Tuesday, August 30, 2011
- 92. Tuesday, August 30, 2011
- 93. Tuesday, August 30, 2011
- 94. Cassandra Tuesday, August 30, 2011
- 95. Cassandra Tuesday, August 30, 2011
- 96. Cassandra Tuesday, August 30, 2011
- 97. Cassandra Tuesday, August 30, 2011
- 98. Cassandra Tuesday, August 30, 2011
- 99. Cassandra Tuesday, August 30, 2011
- 100. Python + Pycassa Tuesday, August 30, 2011
- 101. Python + Pycassa Column Families Tuesday, August 30, 2011
- 102. Python + Pycassa Get list of Follow[ers/ing] Tuesday, August 30, 2011
- 103. Python + Pycassa More Queries Tuesday, August 30, 2011
- 104. Python + Pycassa Follow user Tuesday, August 30, 2011
- 105. Switch to Production Tuesday, August 30, 2011
- 106. Simultaneous writes to MySQL and Cassandra Tuesday, August 30, 2011
- 107. Switch to Production Cassandra MySQL Read x Write x x Tuesday, August 30, 2011
- 108. Idempotence is your friend. Tuesday, August 30, 2011
- 109. Migration from MySQL to Cassandra Tuesday, August 30, 2011
- 110. Load batches of user info into Cassandra. Tuesday, August 30, 2011
- 111. It’s OK if it fails. Tuesday, August 30, 2011
- 112. Remember idempotence is your friend. Tuesday, August 30, 2011
- 113. Adjust your capacity Tuesday, August 30, 2011
- 114. From 4 instances to a 12 instances Tuesday, August 30, 2011
- 115. New Ask Followers Infrastructure Tuesday, August 30, 2011
- 116. Frontend Ask Cassandra Cluster Tuesday, August 30, 2011
- 117. Frontend Ask Outbox Cassandra QW Cluster Tuesday, August 30, 2011
- 118. Frontend Ask Outbox Cassandra QW Cluster Iterative KW Tuesday, August 30, 2011
- 119. Frontend Ask Outbox Cassandra QW Cluster 0, N) ser_id, ers(u get_follow Iterative KW Tuesday, August 30, 2011
- 120. Frontend Ask Outbox Cassandra QW Cluster ’, ..., ‘ 1999’] ‘1234 , ‘1238 [ Iterative KW Tuesday, August 30, 2011
- 121. Frontend Ask Outbox Cassandra QW Cluster ’, ..., ‘ 1999’] ‘1234 , ‘1238 [ Iterative KW N Messages Tuesday, August 30, 2011
- 122. Frontend Ask Outbox Cassandra QW Cluster ’, ..., ‘ 1999’] ‘1234 , ‘1238 [ Iterative KW N Messages Inbox QW Tuesday, August 30, 2011
- 123. Frontend Ask Outbox Cassandra QW Cluster ’, ..., ‘ 1999’] ‘1234 , ‘1238 [ Iterative KW Inbox N Inbox QW Object Tuesday, August 30, 2011
- 124. Frontend Ask Outbox Cassandra QW Cluster +N) ser_i d, N, N u wers( get _follo Iterative KW Inbox N Inbox QW Object Tuesday, August 30, 2011
- 125. Count Following 14K/min hits at peak time Tuesday, August 30, 2011
- 126. Count Followers 40K/min hits at peak time Tuesday, August 30, 2011
- 127. Reduce our #instances 12x 12x Inbox Inbox Inbox Inbox Inbox Inbox Inbox Inbox QWs QWsInbox Inbox QWs QWsInbox Inbox QWs QWsInbox Inbox QWs QWsInbox Inbox QWs QWsInbox Inbox QWs QWsInbox Inbox QWs QWsInbox Inbox QWs QWsInbox Inbox QWs QWsInbox Inbox QWs QWsInbox Inbox QWs QWsInbox Inbox QWs Inbox Outbox QWs QWs QWs QWs QWs QWs QWs QWs QWs Tuesday, August 30, 2011
- 128. Reduce our #instances 12x 4x Inbox Inbox Inbox Inbox QWs QWsInbox Inbox QWs QWsInbox Inbox QWs QWsInbox Inbox QWs QWsInbox Inbox QWs QWsInbox Inbox Inbox Inbox QWs QWsInbox Inbox Inbox Inbox QWs QWs Outbox QWs QWs QWs QWs QWs QWs QWs Tuesday, August 30, 2011
- 129. Reduce our #instances 12x 4x Inbox Inbox Inbox QWsInbox QWsInbox QWsInbox QWsInbox QWsInbox Inbox QWsInbox QWsInbox Inbox Inbox QWsInbox Inbox Outbox QWs Inbox QWs Inbox QWsInbox QWs QWs QWsInbox Inbox QWs QWs QWs+ QWs QWs QWs Iterator QW Tuesday, August 30, 2011
- 130. Counts for free with 0.8.1 Tuesday, August 30, 2011
- 131. Counting . . . Tuesday, August 30, 2011
- 132. Counting is hard Let’s go Shopping. Tuesday, August 30, 2011
- 133. Counting • Fast Tuesday, August 30, 2011
- 134. Counting • Fast • Not-accurate Tuesday, August 30, 2011
- 135. Counting • Fast • Not-accurate • Faking “Read-repair” Tuesday, August 30, 2011
- 136. Counting Tuesday, August 30, 2011
- 137. Ghetto Count Tuesday, August 30, 2011
- 138. Cassandra allowed us: • To create new features. Tuesday, August 30, 2011
- 139. Cassandra allowed us: • To create new features. • To scale. Tuesday, August 30, 2011
- 140. Cassandra allowed us: • To create new features. • To scale. • To save money. Tuesday, August 30, 2011
- 141. Cassandra allowed us: • To create new features. • To scale. • To save money. • To sleep. (Almost there) Tuesday, August 30, 2011
- 142. Cassandra allowed us: • To create new features. • To scale. • To save money. • To sleep. (Almost there) • Lots of fun :) Tuesday, August 30, 2011
- 143. Come work with us ! Tuesday, August 30, 2011