Se ha denunciado esta presentación.
Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.

Redis in Practice

33.091 visualizaciones

Publicado el

Noah Davis & Luke Melia of Weplay share a series of examples of Redis in the real world. In doing so, they cover a survey of Redis' features, approach, history and philosophy. Most examples are drawn from the Weplay team's experience using Redis to power features on Weplay.com, a social site for youth sports.

Publicado en: Tecnología

Redis in Practice

  1. 1. Redis in PracticeNoSQL NYC • December 2nd, 2010 Noah Davis & Luke Melia
  2. 2. About Noah & LukeWeplay.comRubyistsRunning Redis in production since mid-2009@noahd1 and @lukemelia on Twitter
  3. 3. Pronouncing Redis
  4. 4. Tonight’s gameplanFly-by intro to RedisRedis in Practice View counts Q&A throughout, please. Global locks We love being Presence interrupted. Social activity feeds Friend suggestions Caching
  5. 5. Salvatore Sanfilippo Original author of Redis. @antirez on Twitter. Lives in Italy.Recently hired by VMWare. Great project leader.
  6. 6. Describing RedisRemote dictionary server“advanced, fast, persistent key- value database”“Data structures server”“memcached on steroids”
  7. 7. In Salvatore’s words “I see Redis definitely more as a flexible toolthan as a solution specialized to solve a specific problem: his mixed soul of cache, store, and messaging server shows this very well.” - Salvatore Sanfilippo
  8. 8. QualitiesWritten in CFew dependenciesFastSingle-threadedLots of clients availableInitial release was March 2009
  9. 9. FeaturesKey-value store where a value is one of: scalar/string, list, set, sorted sets, hashPersistence: in-memory, snapshots, append-only log, VMReplication: master-slave, configureablePub-SubExpiry“Transaction”-yIn development: Redis Cluster
  10. 10. What Redis ain’t (...yet)Big dataSeamless scalingAd-hoc query toolThe only data store you’ll ever need
  11. 11. Who’s using Redis?VMWare GroovesharkGithub SuperfeedrCraigslist RavelryEngineYard mediaFAILThe Guardian WeplayForrst More...PostRank
  12. 12. Our infrastructureApp server App server Utility server MySQL master Redis master MySQL slave Redis slave
  13. 13. Redis in Practice #1View Counts
  14. 14. View countspotential for massive amounts of simple writes/readsvaluable data, but not mission criticallots of row contention in SQL
  15. 15. Redis incrementing $ redis-cli redis> incr "medium:301:views" (integer) 1 redis> incr "medium:301:views" (integer) 2 redis> incrby "medium:301:views" 5 (integer) 7 redis> decr "medium:301:views" (integer) 6
  16. 16. Redis in Practice #2Distributed Locks
  17. 17. Subscribe to a Weplay Calendar Subscribe to “ics”
  18. 18. Subscription challengesGenerally static content for long periods of time, but withshort sessions of frequent updatesExpensive to compute on the fly
  19. 19. Without Redis/Locking CANCEL EVENT BACKGROUND Pubisher QUEUE Queue: Publish ICS Publish Contention/Redundancy ADD EVENT BACKGROUND Pubisher QUEUE Queue: Publish ICS Publish
  20. 20. Redis Locking: SetNX redis = Redis.new redis.setnx "locking.key", Time.now + 2.hours => true redis.setnx "locking.key", Time.now + 2.hours => false redis.del "locking.key" => true¨ redis.setnx "locking.key", Time.now + 2.hours => true
  21. 21. Redis: Obtained LockADD EVENT REDIS SETNX "group_34_publish_ics" Returns: 1 BACKGROUND PUBLISHER REDIS QUEUE 5 minutes later: PUBLISH.... Queue Job: Publish.publish_ics("34") DEL "group_34_publish_ics"
  22. 22. Redis: Locked Out CANCEL EVENT REDIS SETNX "group_34_publish_ics" Returns: 0 ADD EVENT REDIS SETNX "group_34_publish_ics" Returns: 0
  23. 23. Redis in Practice #3 Presence
  24. 24. “Who’s online?”Weplay members told usthey wanted to be ableto see which of theirfriends were online...
  25. 25. About sets0 to N elements Adding a value to a set does not require youUnordered to check if the valueNo repeated members exists in the set first
  26. 26. Working with Redis sets 1/3# SADD key, member# Adds the specified member to the set stored at keyredis = Redis.newredis.sadd my_set, foo # => trueredis.sadd my_set, bar # => trueredis.sadd my_set, bar # => falseredis.smembers my_set # => ["foo", "bar"]
  27. 27. Working with Redis sets 2/3# SUNION key1 key2 ... keyN# Returns the members of a set resulting from the union of all# the sets stored at the specified keys.# SUNIONSTORE <i>dstkey key1 key2 ... keyN</i></b># Works like SUNION but instead of being returned the resulting# set is stored as dstkey.redis = Redis.newredis.sadd set_a, fooredis.sadd set_a, barredis.sadd set_b, barredis.sadd set_b, bazredis.sunion set_a, set_b # => ["foo", "baz", "bar"]redis.sunionstore set_ab, set_a, set_bredis.smembers set_ab # => ["foo", "baz", "bar"]
  28. 28. Working with Redis sets 3/3# SINTER key1 key2 ... keyN# Returns the members that are present in all# sets stored at the specified keys.redis = Redis.newredis.sadd set_a, fooredis.sadd set_a, barredis.sadd set_b, barredis.sadd set_b, bazredis.sinter set_a, set_b # => ["bar"]
  29. 29. Approach 1/2
  30. 30. Approach 2/2
  31. 31. Implementation 1/2# Defining the keysdef current_key  key(Time.now.strftime("%M"))enddef keys_in_last_5_minutes  now = Time.now  times = (0..5).collect {|n| now - n.minutes }  times.collect{ |t| key(t.strftime("%M")) }enddef key(minute)  "online_users_minute_#{minute}"end
  32. 32. Implementation 2/2# Tracking an Active User, and calculating who’s onlinedef track_user_id(id)  key = current_key  redis.sadd(key, id)enddef online_user_ids  redis.sunion(*keys_in_last_5_minutes)enddef online_friend_ids(interested_user_id)  redis.sunionstore("online_users", *keys_in_last_5_minutes)  redis.sinter("online_users", "user:#{interested_user_id}:friend_ids")end
  33. 33. Redis in Practice #4Social Activity Feeds
  34. 34. Social Activity Feeds
  35. 35. Via SQL, Approach 1/2 Doesn’t scale to more complex social graphs e.g. friends who are SELECT activities.* FROM activities JOIN friendships f1 ON f1.from_id = ‘hidden’ from your feed activities.actor_id JOIN friendships f2 ON f2.to_id = activities.actor_id May still require multiple WHERE f1.to_id = ? OR f2.from_id = ? ORDER BY activities.id DESC queries to grab LIMIT 15 unindexed data
  36. 36. Via SQL, Approach 2/2 user_id 11 activity_id 96 22 96 11 97 22 97 33 97 11 98 22 98 Friend: 11 11 99 Activity: 100 11 100 Actor 1 Friend: 22 INSERTS 22 100 Teammate: 33 33 100
  37. 37. Large SQL tableGrew quicklyDifficult to maintainDifficult to prune
  38. 38. Redis Lists 1/2 redis> lpush "teams" "yankees" (integer) 1 redis> lpush "teams" "redsox" (integer) 2 redis> llen "teams" (integer) 2 redis> lrange "teams" 0 1 1. "redsox" 2. "yankees" redis> lrange "teams" 0 -1 1. "redsox" 2. "yankees" LTRIM, LLEN, LRANGE
  39. 39. Redis Lists 2/2 redis> ltrim "teams" 0 0 OK redis> lrange "teams" 0 -1 1. "redsox" LTRIM
  40. 40. via Redis, 1/2 LPUSH “user:11:feed”, “100” LTRIM “user:11:feed”, 0, 50 LPUSH user:11:feed, 100 Key Value Friend: 11 LTRIM user:11:feed, 0, 50Activity: 100 user:11:feed [100,99,97,96] LPUSH user:22:feed, 100 Actor 1 Friend: 22 LTRIM user:22:feed, 0, 50 user:22:feed [100,99,98] LPUSH user:33:feed, 100 Teammate: 33 LTRIM user:33:feed, 0, 100 user:33:feed [100,99]
  41. 41. via Redis, 2/2 Key Value Friend: 11 user:11:feed [100,99,97,96] Friend: 22 user:22:feed [100,99,98] Activity: 100 Actor 1 user:33:feed [100,99] Teammate: 33 Key Value New York new_york:feed [100,67] Boston boston:feed [100,99]
  42. 42. Rendering the Feed
  43. 43. Redis in Practice #5Friend Suggestions
  44. 44. Friend Suggestionssuggest new connections based onexisting connectionsexpanding the social graph andmirroring real world connections is key
  45. 45. The Concept PaulMe George John Ringo
  46. 46. The Yoko Factor PaulMe George John Yoko Ringo
  47. 47. The Approach 1/2 Sarah 2 Frank Donny Me 2 Ted Erika
  48. 48. The Approach 2/2 Sarah 2 Frank Donny Me Ted 3 Erika Sue
  49. 49. ZSETS in Redis“Sorted Sets”Each member of the set has a scoreOrdered by the score at all times
  50. 50. Friend Suggestions in Rediszincrby “user:1:suggestions” 1 “donny”zincrby “user:1:suggestions” 1 “donny” Sarah Frank Donny Mezincrby “user:1:suggestions” 1 “erika” Ted Erikazincrby “user:1:suggestions” 1 “erika” [ Donny 2 , Erika 2 ]
  51. 51. Friend Suggestions in Rediszincrby “user:1:suggestions” 1 “erika” Sarah Frank Donny Me Tedzrevrange “user:1:suggestions” 0 1 Erika Sue [ Erika 3 , Donny 2 ]
  52. 52. Redis in Practice #6 Caching
  53. 53. Suitability for cachingExcellent match for managed denormalization ex. friendships, teams, teammatesExcellent match where you would benefit from persistenceand/or replicationHistorically, not a good match for a “generational cache,” inwhich you want to optimize memory use by evicting least-recently used (LRU) keys
  54. 54. As an LRU cacheTTL-support since inception, but with unintuitive behavior Writing to volatile key replaced it and cleared the TTLRedis 2.2 changes this behavior and adds key features: ability to write to, and update expiry of volatile keys maxmemory [bytes], maxmemory-policy [policy] policies: volatile-lru, volatile-ttl, volatile-random, allkeys-lru, allkeys-random
  55. 55. Easy to adaptNamespaced Rack::Session, Rack::Cache, I18n and cacheRedis cache stores for Ruby web frameworks implementation is under 1,000 LOC
  56. 56. Contentious benchmarks
  57. 57. Anyquestions? Thanks! Follow us on Twitter: @noahd1 and @lukemelia Tell your friends in youth sports about Weplay.com

×