Introduction to Redis, include example use-case used in Mindtalk

1. Introduction to
Rizky Abdilah, Mindtalk, @rizkyabdilah

2. Background
● First release in March 2009
● Created n maintained by Salvatore Sanfilippo
● Mostly like Memcache, but with superset
feature
● In-memory database, key-value store
● Open Source, BSD license
● Sponsored by VMWare

3. What to Expect
● Introduction to Redis:
– Data Structure Server, Key Feature
– Scale it up, Persistence, Who's using Redis
● Redis and Memcache:
– Difference and Similarity
● Tips with Redis
● Redis on Mindtalk

4. Data Structure Server
● String => “This is String”
● List => [“dog”, “fox”, “dog”, “cat”]
● Hash Table, Dictionary => {“name”: “Rizky”}
● Set => set([“dog”, “fox”, “cat”])
● Sorted Set:
– set([(“dog”, 1), (“fox”, 4), (“cat”, 6)])

5. Key Features and Cool Stuff
● All data is in memory (fast)
● Support for persistence (rdb, aof)
● Time To Live key (set expired)
● Atomic Operation (INCR)
● Lua Scripting (v2.6)
● Pub/Sub
● Awesome Community (Friendly)

6. Persistence
● RDB, saved periodically or immediately,
snashotting:
– save 900 1 - (15 min, 1 key changed)
– LZF compression
● AOF (Append Only File, work like journal)
– appendonly yes
– Data safety, sacrificing speed
● Background and Foreground save

7. Scale it Up
● Master-slave, replication for HA
● Single-threaded, multiple instance in one
machine
● Cluster? (to be released)

8. Who's Using Redis
● Mindtalk
● Twitter
● Instagram
● Github (Resque)
● YouPorn
● Etc (http://redis.io/topics/whos-using-redis)

9. BO (Basic Operation): “String”
● SET => SET foo “bar”
● GET => GET foo
● INCR => INCR user_id
– Increment integer value by 1
● INCRBY => INCRBY user_id 2
– Increment integer value by given amount
● SETEX => SETEX foo 5 “bar”
– Set the value and the expired time

10. BO List: [“list”]
● [L|R]PUSH => LPUSH jobs:android “bla”
– Prepend or append element in a list
● [L|R]POP => LPOP jobs:android
– Remove and get first element in a list
● LLEN => LLEN jobs:android
– Return length of a list
● LRANGE => LRANGE jobs:android 0 -1
– Return all elements in a list
● LINDEX => LINDEX jobs:android 0
– Get element in a list by its index

11. BO Hash Table: {“key”: “val”}
● HSET => HSET user:$id name “Rizky”
– Set value in hash table
● HGET => HGET user:$id name
– Return value inside hash table
● HGETALL => HGETALL user:$id
– Return all data in hash table

12. BO Set: set([“1”])
● SADD => SADD uniq_ip_visitor “192.168.1.1”
– Add element in Set
● SREM => SREM uniq_ip_visitor “192.168.1.1”
– Remove element in Set
● SMEMBERS => SMEMBERS uniq_ip_visitor
– Return all elements in Set

13. BO Sorted Set: set([(“cat”, 56)])
● ZADD => ZADD post_view 1 $post_id
– Add element in sorted set
● ZINCRBY => ZINCRBY post_view 1 $post_id
– increment
● ZRANGE => ZRANGE post_view 0 -1
– Return all data with ascending sort
● ZREVRANGE => ZREVRANGE post_view 0 -1
– Return all data with descending sort

14. Advanced Operation
● Intersect, Diff, Union Set (join?)
● Intersect, Diff, Union Sorted Set

15. Advanced Operation [2]
● Blocking List: B[L|R]POP
● Multi SET/GET
● Bit Operation: SETBIT
● Bitwise Operation: BITOP
● Append String: APPEND
● Scription: EVAL
● Etc (http://redis.io/commands)

16. Redis and Memcache: Similarity
● RAM only
● TTL Support
● Communicate via TCP

17. Redis and Memcache: “Difference”
● Single vs Multithreaded
● Redis Persistence concern
● Redis native support for list, hash, set, etc
● Redis native support for Pub/Sub
● Memcache support for UDP while Redis
concern with durability and data safety
(TCP only)

18. Tips with Redis
● Use hash table (HSET) to compress memory
usage, instead of using plain String (SET) -
http://mndt.lk/Vr7L - 70MB (SET) vs 16MB
(HSET)
● Hash and list ziplist max-size, around 1024
– Redis will encode the value inside hash table if the
biggest entry didn't exceed the threshold (config)
– hash-max-ziplist-entries and list-max-
ziplist-entries

19. Redis Way
● “But the Redis Way is that the user must
understand how things work so that he is able
to pick the best compromise, and to
understand how the system will behave
exactly.” - Salvatore Sanfilippo
● So, Redis or Memcache?

20. Redis on Mindtalk
● Cache
● REST API Rate Limiting
● Message Queue (digaku.rpush)
● Most Viewed Article (last 2 hour)

21. REST API Rate Limiting
● Goal: To limit API Call from Client

22. Message Queue
● Implemented using BLPOP and RPUSH
command

23. Most Viewed Article (last 2 Hour)

24. Most Viewed Article (last 2 Hour)
● Batch processing log-file, result saved in
Redis using Sorted Set
● ZADD article_view:$hour $count $post_id
● ZREVRANGE article_view:$hour 0 9

25. Pub/Sub: digaku.notifier

26. Pub/Sub: digaku.notifier
● Long-pooling in client and server
● Implemented using Redis PUBLISH and
SUBSCRIBE command