2. Agenda
● What is Redis
● Advantages
● Competitor
● Persistence
● Eviction Policies
● Optimization
● Master Failure case
3. Redis
Redis is an open source in memory data structure store, used
as a cache and database, it is developed by an Italian
developer Salvatore Sanfilippo.
Redis 4.0.2 is the latest stable version.
● NoSql key/value store
● Written in C
● LRU eviction
● On-disk persistence
● Lua scripting
● Multiple Data Structures
4. DataTypes
● Strings: Redis string is a sequence of bytes
● Lists: Redis Lists are simply lists of strings, sorted by insertion order
● Sets: Redis Sets are an unordered collection of strings
● Sorted Sets: Redis Sorted Sets are, similarly to Redis Sets, non
repeating collections of Strings. The difference is that every member
of a Sorted Set is associated with score
● Hashes: Redis Hashes are maps between string fields and string
values
5. Advantages
● Flexible
● No schema
● Cache and data persistence (MongoDb + memcached)
● Rich data types
● Very Fast
110000 sets per second
81000 gets per second
6. Memcached vs Redis
● Memcached only supports String -> String mapping
● Memcached has No replication
● Redis has better community
● Memcached limits keys name to 250 bytes, whereas
redis allows as large as 512MB
● Memcached has no persistence storage option
7. Redis Persistence
● Snapshotting : takes the data as it exists at one moment
in time and writes it to disk
● AppenOnlyFile (AOF) : disk is synced to write
commands, as soon as there is any change in data it
gets synced to disk, this reduces the overall throughput.
One of the primary reasons why you’d want to store in-
memory data on disk is so that you have it later, or so
that you can back it up to a remote location in the case of
failure
8. Snapshotting
● Any Redis client can initiate a snapshot by calling the BGSAVE
command. Redis will fork, and the child process will write the
snapshot to disk while the parent process continues to respond to
commands.
● A Redis client can also initiate a snapshot by calling the SAVE
command, which causes Redis to stop responding to any/all
commands until the snapshot completes. This command isn’t
commonly used, only when we don’t have enough memory for a
BGSAVE.
● If Redis is configured with save lines, such as save 60 10000 , Redis
will automatically trigger a BGSAVE operation if 10,000 writes have
occurred within 60 seconds since the last successful save has
started.
9. Append Only File (AOF)
● Append-only log files keep a record of data changes that occur by
writing each change to the end of the file.
● Over time, a growing AOF could cause your disk to run out of
space, but more commonly, upon restart, Redis will be executing
every command in the AOF in order. When handling large AOFs,
Redis can take a very long time to start up
● BGREWRITEAOF can be used, which will rewrite the AOF to be as
short as possible by removing redundant commands.
BGREWRITEAOF works similarly to the snapshotting BGSAVE:
performing a fork and subsequently rewriting the append-only log in
the child
10. ●
What to name the snapshot on disk, how often to perform an automatic snapshot,
whether to compress the snapshot, and whether to keep accepting writes on failure.
●
The second group of options configure the AOF subsystem, telling Redis whether to use
it, how often to sync writes to disk, whether to sync during AOF compaction, and how
often to compact the AOF.
11. Redis Eviction Policies
●
noeviction: return errors when the memory limit was reached and the client is trying to
execute commands that could result in more memory to be used (most write commands, but
DEL and a few more exceptions).
●
allkeys-lru: evict keys by trying to remove the less recently used (LRU) keys first, in order to
make space for the new data added.
● volatile-lru: evict keys by trying to remove the less recently used (LRU) keys first, but only
among keys that have an expire set, in order to make space for the new data added.
●
allkeys-random: evict keys randomly in order to make space for the new data added.
● volatile-random: evict keys randomly in order to make space for the new data added, but
only evict keys with an expire set.
●
volatile-ttl: evict keys with an expire set, and try to evict keys with a shorter time to live (TTL)
first, in order to make space for the new data added.
12. New LFU mode(4.0)
Using LFU Redis will try to track the frequency of access
of items, so that the ones used rarely are evicted while
the one used often have an higher chance of remaining
in memory.
● volatile-lfu Evict using approximated LFU among the keys with an
expire set.
● allkeys-lfu Evict any key using LFU.
13. Optimization
A typical doubly linked list, we have structures called nodes, which
represent each value in the list
Each of these nodes has pointers to the previous and next nodes in the
list, as well as a pointer to the string in the node.
Each string stored as three parts:
● An integer representing the length
● An integer representing the number of remaining free bytes
● String itself followed by a null character
14. Optimization
● Each of these three strings that are each three characters long will actually
take up space for three pointers, two integers (the length and remaining
bytes in the value), plus the string and an extra byte.
● On the other hand, the ziplist representation will store a sequence of
length, length, string elements. The first length is the size of the previous
entry (for easy scanning in both directions), the second length is the size of
the current entry, and the string is the stored data itself
● By not storing additional pointers and metadata, the ziplist can cut down
overhead from 21 bytes each to roughly 2 bytes