Redis is an open source, in-memory key-value data store that can be used as a cache or database. It supports various data structures like strings, hashes, lists, sets and sorted sets. Redis has features for transactions, pub/sub messaging and scripting. Data can be sharded across multiple Redis instances for large datasets or high throughput needs. Common use cases include caching, sessions, pub/sub, rate limiting and autocompletion.
2. Who am I?
Maarten Balliauw
Antwerp, Belgium
Developer Advocate, JetBrains
Founder, MyGet
Focus on web & cloud
http://blog.maartenballiauw.be
@maartenballiauw
5. Redis
“Redis is an open source, BSD licensed,
networked, single-threaded, in-memory key-
value cache and store. It is often referred to as a data
structure server since keys can contain strings, hashes, lists, sets,
sorted sets, bitmaps and hyperloglogs.”
REmote DIctionary Server
6. Redis
Key-value cache and store (value can be a couple of things)
In-memory (no persistence, but you can)
Single-threaded (atomic operations & transactions)
Networked (it’s a server and it does master/slave)
Some other stuff (scripting, pub/sub, Sentinel, snapshot, …)
Things to remember
7. Persistence?
Dump memory to disk (faster restore after restart)
AOF (append-only files)
RDB (Redis DB snapshots)
With or without: all your data must fit in memory
9. The Little Redis Book
http://openmymind.net/redis.pdf
By Karl Seguin
10. Data types
Type Example Key Example value
String cache:/home <html><head><title>Home page</title>
Hash categories:1 Field
name
description
numproducts
Member
Ships
Pirate ships for sale
50000
Set categories:1:products 20 11 56 89 32 4
List products:20:comments [0] -> “That skull flag is awesome!”
[1] -> “Parrot bit my finger, not happy!”
[2] -> “Parrot keeps swearing at me.”
Sorted set products:bestsellers Field
Eye Patch
Parrot
Score
84632
82120
+ Bitmap, Hyperloglog
12. Data types
Type Example Key Example value
String cache:/home
user:nextid
<html><head><title>Home page</title>
2
Hash user:1 Field
name
handle
Member
Maarten
@maartenballiauw
Set user:1:followers 10 42 99 85
List user:1:timeline [0] -> { ... }
[1] -> { ... }
[2] -> { ... }
Sorted set trending:no:tags Field
#ndcoslo
#redis
Score
84632
82120
+ Bitmap, Hyperloglog
13. Keys
Good practice: use a pattern for keys
E.g. users:1:maarten
Get by id:
KEYS users:1:*
GET ......
Get by name:
KEYS users:*:maarten
GET ......
O(N) operation – use with caution!
16. Transactions
MULTI, EXEC, DISCARD, WATCH
No rollbacks (just discard queue)
Failures are because of you, not Redis
Optimistic locking with WATCH
Only execute transaction queue if watched keys did not change
20. Scripting
Run Lua scripts on Redis
http://www.redis.io/commands/eval
EVAL ‘return “Hello, World!”’ 0
Scripts are cached (SCRIPT FLUSH)
Scripts can be used as functions
Although we must pass the body all the time (or use SCRIPT LOAD +
EVALSHA)
Helper functions available!
Base, table, string, math, debug, struct, cjson, cmsgpack, redis.sha1hex
22. What if I need more memory?
Sharding
On client (consistent hashing)
Using a proxy (Twemproxy - https://github.com/twitter/twemproxy)
Cluster
On server (Redis Cluster) tip: always connect to >= 2 nodes
var Redis = require('ioredis');
var cluster = new Redis.Cluster([
{ port: 6380, host: '127.0.0.1' },
{ port: 6381, host: '127.0.0.1' }
]);
cluster.set('foo', 'bar');
cluster.get('foo', function (err, res) { // ... });
24. When can I use Redis?
Output cache
Session state
General-purpose cache (e.g. for database)
“Cache with benefits”
Pub/sub
Generally: when use case maps and data fits in RAM
25. When should I avoid Redis?
More data than can fit in RAM
Can use multiple, but even then: SUM(RAM) == max. data size
Data and query model fits well in a relational model
Materialize certain queries in Redis if needed
Avoid Redis for large objects...
...with lots of concurrent read/write operations
26. Counting stuff
How would you count “likes” if you were Facebook?
SQL
UPDATE posts SET likes = likes + 1 WHERE postid = 1234
Locking! Will slow down the application...
Redis
INCR post:12345
(Every once in a while, check keys to persist)
27. Counting stuff (+ popularity)
And how would you get the 5 most popular posts?
Use a sorted set
Elements are scored (= # of likes)
We can use ZREVRANGE to get the top X posts
ZREVRANGE posts:likes 0 5 withscores
28. Get the latest 5 product reviews
No need to go to the database!
Use a Redis List, truncated at 5 items
Need more? Query the database
29. Rate limiting
API should only allows 5 requests per 60 seconds
Redis List
Record 5 latest requests
If we’re at 5, check the leftmost item versus current time
1 2 3 4 5
12:20:25 12:20:22 12:20:21 12:20:18 12:19:50
30. Autocompletion
How to provide autocompletion over our million-
records product catalog?
SQL “LIKE”
Lucene / ElasticSearch
Redis Sorted Set
31. Intersecting collections
“Which friends do we have in common?”
Algorithm could be:
Find friends that we have in common
Find friends they have in common
Score the # of connections
32. Sort
We have a series of bugs
Priority, Status, Title, ...
A user follows a few of these bugs
How to store?
How to query? (give me my top 5 bugs by priority)
hset bugs:1 priority 5
hset bugs:2 priority 2
hset bugs:3 priority 1
SORT bugsifollow BY bugs:*->priority
34. Conclusion
Redis is not just a cache
Data types
Transactions
Pub/sub
Scripting
Sharding/partitioning
Patterns
1 thing to remember:
http://openmymind.net/redis.pdf
Start server
Show config file and some of the options in there
Connect client, mention on localhost it autoconnects as I’m using default ports and such
Run some commands:
get name
set name “Maarten”
get name
getrange name 2 4
expire name 5
get name
Explain simple commands, yet powerful (e.g. substring)
Check config with “info” (also cache misses/hits etc.)
Demo some of the data types. Here’s a bunch to choose from:
String:
get cache:/home
set cache:/home "Home page"
get cache:/home
getrange cache:/home 2 5
expire cache:/home 5
get cache:/home
incr site:visits
incrby site:visits 6
Hash:
hmset categories:1 name Books numproducts 4
hgetall categories:1
hincrby categories:1 numproducts 1
hget categories:1 name
hset categories:1 numproducts 500
hgetall categories:1
Set:
sadd categories:1:products 20 11 56 89 32 4
sadd categories:1:products 99
scard categories:1:products
sadd categories:2:products 20 11
sinter categories:1:products categories:2:products
sdiff categories:1:products categories:2:products
List:
rpush products:20:comments "Awesome product!"
rpush products:20:comments "This sucks."
rpush products:20:comments "Yarr!"
lindex products:20:comments 1
llen products:20:comments
brpop products:20:comments queue 0 #blocking at the end! pub/sub?
Sorted Set:
zadd products:bestsellers 100 "Eye Patch" 10 "Parrot" 1 "Ship"
zrevrange products:bestsellers 0 1
zrevrank products:bestsellers "Ship"
zscore products:bestsellers "Ship"
Ask which tech (C# or Node)
Connect to localhost, perhaps flushall firstSET where “Sweden”SET conference “Unknown”SET attendees 0
Connect a second clientMULTIINCRBY attendees 100SET conference “CloudBurst”EXEC
Worked fine! Now let’s do this on client 1:WATCH attendeesMULTIINCRBY attendees 100And on client 2:SET attendees 99And on client 1:EXEC
.NET:
var transaction = redis.GetDatabase().CreateTransaction();
transaction.AddCondition(Condition.KeyExists("foo"));
transaction.StringSetAsync("foo", "bar");
bool committed = transaction.Execute();
Use two clients
Client 1: SUBSCRIBE news
Client 2: PUBLISH news “This just in”
Let’s do that in .NET:
ConnectionMultiplexer redis = ConnectionMultiplexer.Connect("localhost:6379");
ISubscriber subscriber = redis.GetSubscriber();
subscriber.Subscribe("news", (channel, message) =>
{
Console.WriteLine("News: {0}", (string)message);
});
ISubscriber publisher = redis.GetSubscriber();
publisher.Publish("news", "This just in!");
Console.ReadLine();
Send a message from one of the clients, too
Keyspace notifications:CONFIG SET notify-keyspace-events KEAPSUBSCRIBE '__key*__:*'