SlideShare una empresa de Scribd logo
1 de 12
Descargar para leer sin conexión
An Introduction to Top Redis
Use Cases
by Dr. Josiah Carlson
@dr_josiah
Agenda
● Who am I?
● What is Redis?
● Top use cases
● Questions
Who am I?
● A guy who does a lot of stuff with Redis and Python
○ Book:
■ Redis in Action http://manning.com/carlson
■ Read for free http://bit.ly/ria-free
○ Libraries: https://github.com/josiahcarlson
■ rom, RPQueue, lua_call, ...
○ Mailing list: https://groups.google.com/forum/#!forum/redis-db
○ Blog: dr-josiah.com
○ Twitter: @dr_josiah
What is Redis?
● In-memory non-relational key -> data structure server
○ integers, floats, strings, bitmaps, hyperloglogs, lists, sets, hashes, sorted sets(,
geoindex)
○ Plus pubsub, key change notifications, ...
● Optional on-disk persistence
● Optional master/slave replication
○ Optional sentinel for high-availability/failover
● Optional separate clustering mode as of April 2015
● Server-side Lua scripts (like stored procedures, only easier)
● Third party client-sharding via clients or Twemproxy/Nutcracker
Use-case 1: Distributed locking
● 10-100x faster than Zookeeper by default
○ can be tuned for the same reliability
● Multiple locks at the same time
○ job input/output coordination
■ MySQL DDL/truncate operations - locking outside can save your bacon
○ locks that “never” time out
○ locks that time out, or require refreshing
○ counting semaphores for multiple lock holders
Use-case 2: Analytics
● Stop counting after the fact
○ Handle 100-200k events/second without pipelining/batching
○ Handle 1M+ events/second with pipelining/batching
■ Send one batched call at the end of the request
● Use a Lua script for hierarchical counters
● As simple as a counter, or with a little work min/max/avg/stddev
● Minor modifications get you rate limiting
○ Login, API, bandwidth utilization, …
○ With locking (and counting semaphores), can limit outgoing API usage
○ Pick your semantic: counters, sliding window, leaky bucket, or something else
Use-case 3: Web cookies
● No more:
○ encrypting/signing your cookies badly (still need to make them https-only)
○ running into 4k cookie limits
○ slow mobile requests due to overweight cookies
● Easily structured cookie data storage
○ Shopping carts
○ Last X pages visited per user
■ Real-time analytics/trend analysis over recent pages, where to go, what to buy, etc. (people
who viewed this item bought item X some Y% of the time)
○ A listing of recent logins per user, with cookie deletion
Use-case 4: Queues/message passing
● Libraries in most languages
○ Use an off-the-shelf library, or write one that fits your required semantics (0/1 or 1+)
● Prioritization, delayed, cron-scheduled, every X seconds, etc.
● Blocking queues with listeners listening on multiple queues
● Broadcasting with pubsub
● If you like Redis, but want a queue
○ Salvatore has written Disque, a 1+ queue system
Use-case 5: Anything you’re using Memcached for
● Memcached does strings and ints
● Redis does
○ integers, floats, strings, bitmaps, hyperloglogs, lists, sets, hashes, sorted sets(,
geoindex)
○ pubsub (generally, and keyspace changes)
○ optimistic locking and basic transactions with WATCH/MULTI/EXEC
○ Lua scripting
■ … a literally uncountable number of things that Memcached can’t do
...and many more
● DB row caching
● Web page caching
● Autocomplete
● Dynamically updated toplists for
games
● Generalized voting/ranking (Reddit,
Hacker News, etc.)
● Unique visitor counts on hourly,
daily, weekly, and/or monthly basis
● Custom search engines/ad targeting
● Rate-limited geo notifications
● As a general data store (various
libraries, some with pseudo-relational
layer)
Questions?
Thank you!
@dr_josiah

Más contenido relacionado

La actualidad más candente

A Linked Data based index of library institutions
A Linked Data based index of library institutionsA Linked Data based index of library institutions
A Linked Data based index of library institutions
Felix Ostrowski
 

La actualidad más candente (6)

A Linked Data based index of library institutions
A Linked Data based index of library institutionsA Linked Data based index of library institutions
A Linked Data based index of library institutions
 
Data corruption
Data corruptionData corruption
Data corruption
 
ACS CINF Luncheon talk (Boston 2018)
ACS CINF Luncheon talk (Boston 2018)ACS CINF Luncheon talk (Boston 2018)
ACS CINF Luncheon talk (Boston 2018)
 
Publishing the British National Bibliography as Linked Open Data / Corine Del...
Publishing the British National Bibliography as Linked Open Data / Corine Del...Publishing the British National Bibliography as Linked Open Data / Corine Del...
Publishing the British National Bibliography as Linked Open Data / Corine Del...
 
Crawling the Web for Structured Documents
Crawling the Web for Structured DocumentsCrawling the Web for Structured Documents
Crawling the Web for Structured Documents
 
Code4Lib Keynote 2011
Code4Lib Keynote 2011Code4Lib Keynote 2011
Code4Lib Keynote 2011
 

Destacado

Riak seattle-meetup-august
Riak seattle-meetup-augustRiak seattle-meetup-august
Riak seattle-meetup-august
pharkmillups
 
Riak Use Cases : Dissecting The Solutions To Hard Problems
Riak Use Cases : Dissecting The Solutions To Hard ProblemsRiak Use Cases : Dissecting The Solutions To Hard Problems
Riak Use Cases : Dissecting The Solutions To Hard Problems
Andy Gross
 
Developing polyglot persistence applications (SpringOne India 2012)
Developing polyglot persistence applications (SpringOne India 2012)Developing polyglot persistence applications (SpringOne India 2012)
Developing polyglot persistence applications (SpringOne India 2012)
Chris Richardson
 

Destacado (20)

Redis and its many use cases
Redis and its many use casesRedis and its many use cases
Redis and its many use cases
 
Redis use cases
Redis use casesRedis use cases
Redis use cases
 
Introduction to Redis Data Structures: Sorted Sets
Introduction to Redis Data Structures: Sorted SetsIntroduction to Redis Data Structures: Sorted Sets
Introduction to Redis Data Structures: Sorted Sets
 
See who is using MemSQL
See who is using MemSQLSee who is using MemSQL
See who is using MemSQL
 
Redis Use Patterns (DevconTLV June 2014)
Redis Use Patterns (DevconTLV June 2014)Redis Use Patterns (DevconTLV June 2014)
Redis Use Patterns (DevconTLV June 2014)
 
Riak seattle-meetup-august
Riak seattle-meetup-augustRiak seattle-meetup-august
Riak seattle-meetup-august
 
Riak add presentation
Riak add presentationRiak add presentation
Riak add presentation
 
Basho and Riak at GOTO Stockholm: "Don't Use My Database."
Basho and Riak at GOTO Stockholm:  "Don't Use My Database."Basho and Riak at GOTO Stockholm:  "Don't Use My Database."
Basho and Riak at GOTO Stockholm: "Don't Use My Database."
 
Redis : Play buzz uses Redis
Redis : Play buzz uses RedisRedis : Play buzz uses Redis
Redis : Play buzz uses Redis
 
Scaling with Riak at Showyou
Scaling with Riak at ShowyouScaling with Riak at Showyou
Scaling with Riak at Showyou
 
Modelling Data as Graphs (Neo4j)
Modelling Data as Graphs (Neo4j)Modelling Data as Graphs (Neo4j)
Modelling Data as Graphs (Neo4j)
 
Riak Use Cases : Dissecting The Solutions To Hard Problems
Riak Use Cases : Dissecting The Solutions To Hard ProblemsRiak Use Cases : Dissecting The Solutions To Hard Problems
Riak Use Cases : Dissecting The Solutions To Hard Problems
 
An intro to Neo4j and some use cases (JFokus 2011)
An intro to Neo4j and some use cases (JFokus 2011)An intro to Neo4j and some use cases (JFokus 2011)
An intro to Neo4j and some use cases (JFokus 2011)
 
Recommendations with Neo4j (FOSDEM 2015)
Recommendations with Neo4j (FOSDEM 2015)Recommendations with Neo4j (FOSDEM 2015)
Recommendations with Neo4j (FOSDEM 2015)
 
Advanced Neo4j Use Cases with the GraphAware Framework
Advanced Neo4j Use Cases with the GraphAware FrameworkAdvanced Neo4j Use Cases with the GraphAware Framework
Advanced Neo4j Use Cases with the GraphAware Framework
 
Modelling Data in Neo4j (plus a few tips)
Modelling Data in Neo4j (plus a few tips)Modelling Data in Neo4j (plus a few tips)
Modelling Data in Neo4j (plus a few tips)
 
Product catalog using MongoDB
Product catalog using MongoDBProduct catalog using MongoDB
Product catalog using MongoDB
 
Best Buy Web 2.0
Best Buy Web 2.0Best Buy Web 2.0
Best Buy Web 2.0
 
The BestBuy.com Cloud Architecture
The BestBuy.com Cloud ArchitectureThe BestBuy.com Cloud Architecture
The BestBuy.com Cloud Architecture
 
Developing polyglot persistence applications (SpringOne India 2012)
Developing polyglot persistence applications (SpringOne India 2012)Developing polyglot persistence applications (SpringOne India 2012)
Developing polyglot persistence applications (SpringOne India 2012)
 

Similar a Introduction to some top Redis use cases

Mp26 : Tachyon, sloppiness is bliss
Mp26 : Tachyon, sloppiness is blissMp26 : Tachyon, sloppiness is bliss
Mp26 : Tachyon, sloppiness is bliss
Montreal Python
 
You Can Run a Website: CMS for the Library Layman
You Can Run a Website: CMS for the Library LaymanYou Can Run a Website: CMS for the Library Layman
You Can Run a Website: CMS for the Library Layman
Veronica R
 
2010 AIRI Petabyte Challenge - View From The Trenches
2010 AIRI Petabyte Challenge - View From The Trenches2010 AIRI Petabyte Challenge - View From The Trenches
2010 AIRI Petabyte Challenge - View From The Trenches
George Ang
 
Open Chemistry, JupyterLab and data: Reproducible quantum chemistry
Open Chemistry, JupyterLab and data: Reproducible quantum chemistryOpen Chemistry, JupyterLab and data: Reproducible quantum chemistry
Open Chemistry, JupyterLab and data: Reproducible quantum chemistry
Marcus Hanwell
 
Overview of no sql
Overview of no sqlOverview of no sql
Overview of no sql
Sean Murphy
 

Similar a Introduction to some top Redis use cases (20)

Mp26 : Tachyon, sloppiness is bliss
Mp26 : Tachyon, sloppiness is blissMp26 : Tachyon, sloppiness is bliss
Mp26 : Tachyon, sloppiness is bliss
 
You Can Run a Website: CMS for the Library Layman
You Can Run a Website: CMS for the Library LaymanYou Can Run a Website: CMS for the Library Layman
You Can Run a Website: CMS for the Library Layman
 
Big data at scrapinghub
Big data at scrapinghubBig data at scrapinghub
Big data at scrapinghub
 
Better Python Coding with Prefect Blocks
Better Python Coding with Prefect BlocksBetter Python Coding with Prefect Blocks
Better Python Coding with Prefect Blocks
 
Session 10 handling bigger data
Session 10 handling bigger dataSession 10 handling bigger data
Session 10 handling bigger data
 
Session 10 handling bigger data
Session 10 handling bigger dataSession 10 handling bigger data
Session 10 handling bigger data
 
2010 AIRI Petabyte Challenge - View From The Trenches
2010 AIRI Petabyte Challenge - View From The Trenches2010 AIRI Petabyte Challenge - View From The Trenches
2010 AIRI Petabyte Challenge - View From The Trenches
 
PostgreSQL and Sphinx pgcon 2013
PostgreSQL and Sphinx   pgcon 2013PostgreSQL and Sphinx   pgcon 2013
PostgreSQL and Sphinx pgcon 2013
 
21st Athens Big Data Meetup - 1st Talk - Fast and simple data exploration wit...
21st Athens Big Data Meetup - 1st Talk - Fast and simple data exploration wit...21st Athens Big Data Meetup - 1st Talk - Fast and simple data exploration wit...
21st Athens Big Data Meetup - 1st Talk - Fast and simple data exploration wit...
 
Caveats
CaveatsCaveats
Caveats
 
Alexander Sibiryakov- Frontera
Alexander Sibiryakov- FronteraAlexander Sibiryakov- Frontera
Alexander Sibiryakov- Frontera
 
The Internet in Database: A Cassandra Use Case
The Internet in Database: A Cassandra Use CaseThe Internet in Database: A Cassandra Use Case
The Internet in Database: A Cassandra Use Case
 
Introduction to Big Data Technologies & Applications
Introduction to Big Data Technologies & ApplicationsIntroduction to Big Data Technologies & Applications
Introduction to Big Data Technologies & Applications
 
AWS big-data-demystified #1.1 | Big Data Architecture Lessons Learned | English
AWS big-data-demystified #1.1  | Big Data Architecture Lessons Learned | EnglishAWS big-data-demystified #1.1  | Big Data Architecture Lessons Learned | English
AWS big-data-demystified #1.1 | Big Data Architecture Lessons Learned | English
 
Open Chemistry, JupyterLab and data: Reproducible quantum chemistry
Open Chemistry, JupyterLab and data: Reproducible quantum chemistryOpen Chemistry, JupyterLab and data: Reproducible quantum chemistry
Open Chemistry, JupyterLab and data: Reproducible quantum chemistry
 
Red Hat Storage Server Administration Deep Dive
Red Hat Storage Server Administration Deep DiveRed Hat Storage Server Administration Deep Dive
Red Hat Storage Server Administration Deep Dive
 
Frontera: open source, large scale web crawling framework
Frontera: open source, large scale web crawling frameworkFrontera: open source, large scale web crawling framework
Frontera: open source, large scale web crawling framework
 
NYT Web Archive
NYT Web ArchiveNYT Web Archive
NYT Web Archive
 
WebCamp 2016: Python. Вячеслав Каковский: Real-time мессенджер на Python. Осо...
WebCamp 2016: Python. Вячеслав Каковский: Real-time мессенджер на Python. Осо...WebCamp 2016: Python. Вячеслав Каковский: Real-time мессенджер на Python. Осо...
WebCamp 2016: Python. Вячеслав Каковский: Real-time мессенджер на Python. Осо...
 
Overview of no sql
Overview of no sqlOverview of no sql
Overview of no sql
 

Último

AKTU Computer Networks notes --- Unit 3.pdf
AKTU Computer Networks notes ---  Unit 3.pdfAKTU Computer Networks notes ---  Unit 3.pdf
AKTU Computer Networks notes --- Unit 3.pdf
ankushspencer015
 

Último (20)

Introduction to Multiple Access Protocol.pptx
Introduction to Multiple Access Protocol.pptxIntroduction to Multiple Access Protocol.pptx
Introduction to Multiple Access Protocol.pptx
 
UNIT-III FMM. DIMENSIONAL ANALYSIS
UNIT-III FMM.        DIMENSIONAL ANALYSISUNIT-III FMM.        DIMENSIONAL ANALYSIS
UNIT-III FMM. DIMENSIONAL ANALYSIS
 
DJARUM4D - SLOT GACOR ONLINE | SLOT DEMO ONLINE
DJARUM4D - SLOT GACOR ONLINE | SLOT DEMO ONLINEDJARUM4D - SLOT GACOR ONLINE | SLOT DEMO ONLINE
DJARUM4D - SLOT GACOR ONLINE | SLOT DEMO ONLINE
 
Introduction and different types of Ethernet.pptx
Introduction and different types of Ethernet.pptxIntroduction and different types of Ethernet.pptx
Introduction and different types of Ethernet.pptx
 
Booking open Available Pune Call Girls Koregaon Park 6297143586 Call Hot Ind...
Booking open Available Pune Call Girls Koregaon Park  6297143586 Call Hot Ind...Booking open Available Pune Call Girls Koregaon Park  6297143586 Call Hot Ind...
Booking open Available Pune Call Girls Koregaon Park 6297143586 Call Hot Ind...
 
(INDIRA) Call Girl Aurangabad Call Now 8617697112 Aurangabad Escorts 24x7
(INDIRA) Call Girl Aurangabad Call Now 8617697112 Aurangabad Escorts 24x7(INDIRA) Call Girl Aurangabad Call Now 8617697112 Aurangabad Escorts 24x7
(INDIRA) Call Girl Aurangabad Call Now 8617697112 Aurangabad Escorts 24x7
 
University management System project report..pdf
University management System project report..pdfUniversity management System project report..pdf
University management System project report..pdf
 
MANUFACTURING PROCESS-II UNIT-2 LATHE MACHINE
MANUFACTURING PROCESS-II UNIT-2 LATHE MACHINEMANUFACTURING PROCESS-II UNIT-2 LATHE MACHINE
MANUFACTURING PROCESS-II UNIT-2 LATHE MACHINE
 
Coefficient of Thermal Expansion and their Importance.pptx
Coefficient of Thermal Expansion and their Importance.pptxCoefficient of Thermal Expansion and their Importance.pptx
Coefficient of Thermal Expansion and their Importance.pptx
 
KubeKraft presentation @CloudNativeHooghly
KubeKraft presentation @CloudNativeHooghlyKubeKraft presentation @CloudNativeHooghly
KubeKraft presentation @CloudNativeHooghly
 
(ANVI) Koregaon Park Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...
(ANVI) Koregaon Park Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...(ANVI) Koregaon Park Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...
(ANVI) Koregaon Park Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...
 
Online banking management system project.pdf
Online banking management system project.pdfOnline banking management system project.pdf
Online banking management system project.pdf
 
The Most Attractive Pune Call Girls Budhwar Peth 8250192130 Will You Miss Thi...
The Most Attractive Pune Call Girls Budhwar Peth 8250192130 Will You Miss Thi...The Most Attractive Pune Call Girls Budhwar Peth 8250192130 Will You Miss Thi...
The Most Attractive Pune Call Girls Budhwar Peth 8250192130 Will You Miss Thi...
 
AKTU Computer Networks notes --- Unit 3.pdf
AKTU Computer Networks notes ---  Unit 3.pdfAKTU Computer Networks notes ---  Unit 3.pdf
AKTU Computer Networks notes --- Unit 3.pdf
 
The Most Attractive Pune Call Girls Manchar 8250192130 Will You Miss This Cha...
The Most Attractive Pune Call Girls Manchar 8250192130 Will You Miss This Cha...The Most Attractive Pune Call Girls Manchar 8250192130 Will You Miss This Cha...
The Most Attractive Pune Call Girls Manchar 8250192130 Will You Miss This Cha...
 
College Call Girls Nashik Nehal 7001305949 Independent Escort Service Nashik
College Call Girls Nashik Nehal 7001305949 Independent Escort Service NashikCollege Call Girls Nashik Nehal 7001305949 Independent Escort Service Nashik
College Call Girls Nashik Nehal 7001305949 Independent Escort Service Nashik
 
Introduction to IEEE STANDARDS and its different types.pptx
Introduction to IEEE STANDARDS and its different types.pptxIntroduction to IEEE STANDARDS and its different types.pptx
Introduction to IEEE STANDARDS and its different types.pptx
 
(ANJALI) Dange Chowk Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...
(ANJALI) Dange Chowk Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...(ANJALI) Dange Chowk Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...
(ANJALI) Dange Chowk Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...
 
Top Rated Pune Call Girls Budhwar Peth ⟟ 6297143586 ⟟ Call Me For Genuine Se...
Top Rated  Pune Call Girls Budhwar Peth ⟟ 6297143586 ⟟ Call Me For Genuine Se...Top Rated  Pune Call Girls Budhwar Peth ⟟ 6297143586 ⟟ Call Me For Genuine Se...
Top Rated Pune Call Girls Budhwar Peth ⟟ 6297143586 ⟟ Call Me For Genuine Se...
 
High Profile Call Girls Nagpur Meera Call 7001035870 Meet With Nagpur Escorts
High Profile Call Girls Nagpur Meera Call 7001035870 Meet With Nagpur EscortsHigh Profile Call Girls Nagpur Meera Call 7001035870 Meet With Nagpur Escorts
High Profile Call Girls Nagpur Meera Call 7001035870 Meet With Nagpur Escorts
 

Introduction to some top Redis use cases

  • 1. An Introduction to Top Redis Use Cases by Dr. Josiah Carlson @dr_josiah
  • 2. Agenda ● Who am I? ● What is Redis? ● Top use cases ● Questions
  • 3. Who am I? ● A guy who does a lot of stuff with Redis and Python ○ Book: ■ Redis in Action http://manning.com/carlson ■ Read for free http://bit.ly/ria-free ○ Libraries: https://github.com/josiahcarlson ■ rom, RPQueue, lua_call, ... ○ Mailing list: https://groups.google.com/forum/#!forum/redis-db ○ Blog: dr-josiah.com ○ Twitter: @dr_josiah
  • 4. What is Redis? ● In-memory non-relational key -> data structure server ○ integers, floats, strings, bitmaps, hyperloglogs, lists, sets, hashes, sorted sets(, geoindex) ○ Plus pubsub, key change notifications, ... ● Optional on-disk persistence ● Optional master/slave replication ○ Optional sentinel for high-availability/failover ● Optional separate clustering mode as of April 2015 ● Server-side Lua scripts (like stored procedures, only easier) ● Third party client-sharding via clients or Twemproxy/Nutcracker
  • 5. Use-case 1: Distributed locking ● 10-100x faster than Zookeeper by default ○ can be tuned for the same reliability ● Multiple locks at the same time ○ job input/output coordination ■ MySQL DDL/truncate operations - locking outside can save your bacon ○ locks that “never” time out ○ locks that time out, or require refreshing ○ counting semaphores for multiple lock holders
  • 6. Use-case 2: Analytics ● Stop counting after the fact ○ Handle 100-200k events/second without pipelining/batching ○ Handle 1M+ events/second with pipelining/batching ■ Send one batched call at the end of the request ● Use a Lua script for hierarchical counters ● As simple as a counter, or with a little work min/max/avg/stddev ● Minor modifications get you rate limiting ○ Login, API, bandwidth utilization, … ○ With locking (and counting semaphores), can limit outgoing API usage ○ Pick your semantic: counters, sliding window, leaky bucket, or something else
  • 7. Use-case 3: Web cookies ● No more: ○ encrypting/signing your cookies badly (still need to make them https-only) ○ running into 4k cookie limits ○ slow mobile requests due to overweight cookies ● Easily structured cookie data storage ○ Shopping carts ○ Last X pages visited per user ■ Real-time analytics/trend analysis over recent pages, where to go, what to buy, etc. (people who viewed this item bought item X some Y% of the time) ○ A listing of recent logins per user, with cookie deletion
  • 8. Use-case 4: Queues/message passing ● Libraries in most languages ○ Use an off-the-shelf library, or write one that fits your required semantics (0/1 or 1+) ● Prioritization, delayed, cron-scheduled, every X seconds, etc. ● Blocking queues with listeners listening on multiple queues ● Broadcasting with pubsub ● If you like Redis, but want a queue ○ Salvatore has written Disque, a 1+ queue system
  • 9. Use-case 5: Anything you’re using Memcached for ● Memcached does strings and ints ● Redis does ○ integers, floats, strings, bitmaps, hyperloglogs, lists, sets, hashes, sorted sets(, geoindex) ○ pubsub (generally, and keyspace changes) ○ optimistic locking and basic transactions with WATCH/MULTI/EXEC ○ Lua scripting ■ … a literally uncountable number of things that Memcached can’t do
  • 10. ...and many more ● DB row caching ● Web page caching ● Autocomplete ● Dynamically updated toplists for games ● Generalized voting/ranking (Reddit, Hacker News, etc.) ● Unique visitor counts on hourly, daily, weekly, and/or monthly basis ● Custom search engines/ad targeting ● Rate-limited geo notifications ● As a general data store (various libraries, some with pseudo-relational layer)