SlideShare a Scribd company logo
1 of 67
Indexing with
   MongoDB
        Luke Ehresman
CopperEgg - www.copperegg.com
What is an index?
What is an index?

• Pointers to documents
What is an index?

• Pointers to documents
• Efficiently organized for quick scanning
What is an index?

• Pointers to documents
• Efficiently organized for quick scanning
• Maintained in a tree structure
How do we find x = 6 without an index?




{x:0} {x:1} {x:2} {x:3} {x:4} {x:5} {x:6} {x:7} {x:8} {x:9}
How do we find x = 6 without an index?




{x:0} {x:1} {x:2} {x:3} {x:4} {x:5} {x:6} {x:7} {x:8} {x:9}
How do we find x = 6 without an index?




{x:0} {x:1} {x:2} {x:3} {x:4} {x:5} {x:6} {x:7} {x:8} {x:9}
How do we find x = 6 without an index?




{x:0} {x:1} {x:2} {x:3} {x:4} {x:5} {x:6} {x:7} {x:8} {x:9}
How do we find x = 6 without an index?




{x:0} {x:1} {x:2} {x:3} {x:4} {x:5} {x:6} {x:7} {x:8} {x:9}
How do we find x = 6 without an index?




{x:0} {x:1} {x:2} {x:3} {x:4} {x:5} {x:6} {x:7} {x:8} {x:9}
How do we find x = 6 without an index?




{x:0} {x:1} {x:2} {x:3} {x:4} {x:5} {x:6} {x:7} {x:8} {x:9}
How do we find x = 6 without an index?




{x:0} {x:1} {x:2} {x:3} {x:4} {x:5} {x:6} {x:7} {x:8} {x:9}
How do we find x = 6 without an index?




{x:0} {x:1} {x:2} {x:3} {x:4} {x:5} {x:6} {x:7} {x:8} {x:9}




                 6 documents scanned
How do we find x = 6 with an index?
How do we find x = 6 with an index?

                        {x:4}
                <                 >



        {x:2}                             {x:6}
    <           >                     <           >


{x:1}           {x:3}           {x:5}             {x:7}
How do we find x = 6 with an index?

                        {x:4}
                <                 >



        {x:2}                             {x:6}
    <           >                     <           >


{x:1}           {x:3}           {x:5}             {x:7}
How do we find x = 6 with an index?

                        {x:4}
                <                 >



        {x:2}                             {x:6}
    <           >                     <           >


{x:1}           {x:3}           {x:5}             {x:7}


    Only 2 documents scanned
In reality, it’s even better than that...
In reality, it’s even better than that...


                                    {x:0, name:‘George’, ....}
                                    {x:1, name:‘Jane’, ....}
                                    {x:2, name:‘Judy’, ....}
                                    {x:3, name:‘Elroy’, ....}
                                    {x:4, name:‘Astro’, ....}
                                    {x:5, name:‘Rosie’, ....}
                                    {x:6, name:‘Fred’, ....}
                                    {x:7, name:‘Wilma’, ....}
                                    {x:8, name:‘Barney’, ....}
                                    {x:9, name:‘Betty’, ....}
In reality, it’s even better than that...


                        {x:4}                             {x:0, name:‘George’, ....}
                                                          {x:1, name:‘Jane’, ....}
                <                 >                       {x:2, name:‘Judy’, ....}
                                                          {x:3, name:‘Elroy’, ....}
                                                          {x:4, name:‘Astro’, ....}
                                                          {x:5, name:‘Rosie’, ....}
        {x:2}                             {x:6}           {x:6, name:‘Fred’, ....}
                                                          {x:7, name:‘Wilma’, ....}
    <           >                     <           >       {x:8, name:‘Barney’, ....}
                                                          {x:9, name:‘Betty’, ....}

{x:1}           {x:3}           {x:5}             {x:7}
In reality, it’s still even better than that...
In reality, it’s still even better than that...

•MongoDB stores data and indexes in extents
In reality, it’s still even better than that...

•MongoDB stores data and indexes in extents
•Only loads extents into memory as needed
In reality, it’s still even better than that...

•MongoDB stores data and indexes in extents
•Only loads extents into memory as needed
•The full index does not need to be in memory
In reality, it’s still even better than that...

•MongoDB stores data and indexes in extents
•Only loads extents into memory as needed
•The full index does not need to be in memory
                            {x:4}
                    <                 >



            {x:2}                             {x:6}
        <           >                     <           >


    {x:1}           {x:3}           {x:5}             {x:7}
When to index?
When to index?

• Frequently queried fields
When to index?

• Frequently queried fields
• Low response time
When to index?

• Frequently queried fields
• Low response time
• Sorting
When to index?

• Frequently queried fields
• Low response time
• Sorting
• Avoid full collection scans
Things to know...
Things to know...
• Indexes maintain order
Things to know...
• Indexes maintain order
• Indexes will slow writes
Things to know...
• Indexes maintain order
• Indexes will slow writes
• Indexes take up space
Things to know...
• Indexes maintain order
• Indexes will slow writes
• Indexes take up space
• Index maintenance will cause writes
Things to know...
• Indexes maintain order
• Indexes will slow writes
• Indexes take up space
• Index maintenance will cause writes
• “unique” parameter forces uniqueness
How to create an index
How to create an index
    db.people.ensureIndex({...})
How to create an index
        db.people.ensureIndex({...})

{_id:1} is already created for each collection
How to create an index
            db.people.ensureIndex({...})

   {_id:1} is already created for each collection

      You can call ensureIndex multiple times
(it only creates an index if it doesn’t already exist)
How to create an index
            db.people.ensureIndex({...})

   {_id:1} is already created for each collection

      You can call ensureIndex multiple times
(it only creates an index if it doesn’t already exist)

     Blocks your mongod process unless you
             specify background:true.
Indexes on missing fields
Indexes on missing fields

null is assumed if a field is missing
Indexes on missing fields

         null is assumed if a field is missing

db.people.insert({name:‘Fred’, children:[‘Pebbles’]});
db.people.insert({name:‘Pebbles’})

db.people.ensureIndex({children:1})

db.people.find({children:null});
(returns Pebbles and uses the index)
Indexes on missing fields

         null is assumed if a field is missing

db.people.insert({name:‘Fred’, children:[‘Pebbles’]});
db.people.insert({name:‘Pebbles’})

db.people.ensureIndex({children:1})

db.people.find({children:null});
(returns Pebbles and uses the index)

         Optional sparse indexes exclude
           empty fields for efficiency
Compound indexes
Compound indexes

{
    name: “George”,
    age:45,
    gender:”M”,
    ...
}
Compound indexes

{
    name: “George”,        {gender:1, age:-1}
    age:45,
    gender:”M”,
    ...
}
Compound indexes

{
    name: “George”,          {gender:1, age:-1}
    age:45,
    gender:”M”,
    ...
}                         gender:”M”
                            age:45
                                              {name:‘George’, ....}
                            age:12
                                              {name:‘Jane’, ....}
                            age:3
                                              {name:‘Judy’, ....}
                          gender:”F”
                                              {name:‘Elroy’, ....}
                            age:44
                                              {name:‘Astro’, ....}
                            age:16
                                              {name:‘Rosie’, ....}
                            age:10
Indexes on arrays
Indexes on arrays
{
    name: “George”,
    age:45,
    gender:”M”,
    children: [‘Judy’, ‘Elroy’]
    ...
}
Indexes on arrays
{
    name: “George”,
    age:45,
                                  {children:1}
    gender:”M”,
    children: [‘Judy’, ‘Elroy’]
    ...
}
Indexes on arrays
{
    name: “George”,
    age:45,
                                  {children:1}
    gender:”M”,
    children: [‘Judy’, ‘Elroy’]
    ...
}
                                  Actually creates a new entry
                                     in the index for each
                                     element of the array.
Indexes on arrays
 {
     name: “George”,
     age:45,
                                   {children:1}
     gender:”M”,
     children: [‘Judy’, ‘Elroy’]
     ...
 }
                                   Actually creates a new entry
                                      in the index for each
                                      element of the array.
children:‘Judy’                         {name:‘George’, ....}
children:‘Elroy’                        {name:‘Fred’, ...}
children:‘Bam-Bam’                      {name:‘Barney’ ...}
Covered indexes
Covered indexes
Queries that can be resolved with only the index
(does not need to fetch the original document)
Covered indexes
Queries that can be resolved with only the index
(does not need to fetch the original document)


{
    name: “George”,
    age:45,
    gender:”M”,
    children: [‘Judy’, ‘Elroy’]
    ...
}
Covered indexes
Queries that can be resolved with only the index
(does not need to fetch the original document)


{
    name: “George”,
    age:45,                       db.people.ensureIndex({name:1, age:1});
    gender:”M”,                   db.people.find({name:‘George’}, {_id:0, age:1});
    children: [‘Judy’, ‘Elroy’]
    ...
}
How does it know which index to use?
How does it know which index to use?

db.people.ensureIndex({name:1});
db.people.ensureIndex({age:1});

db.people.find({age:{$gte:21}});
How does it know which index to use?

db.people.ensureIndex({name:1});
db.people.ensureIndex({age:1});

db.people.find({age:{$gte:21}});

 scan


 index on name


 index on age
How does it know which index to use?

db.people.ensureIndex({name:1});
db.people.ensureIndex({age:1});

db.people.find({age:{$gte:21}});

 scan

                            terminated
 index on name


 index on age         remember
Query performance analysis


db.people.find({name:‘Fred’}).explain()

{
    “cursor” : “BasicCursor”,
    “indexBounds” : {},
    “nscanned” : 88731,
    “nscannedObjects” : 88731,
    “n” : 1,
    “millis” : 108
}
Query performance analysis


db.people.ensureIndex({name:1})
db.people.find({name:‘Fred’}).explain()

{
    “cursor” : “BtreeCursor name_1”,
    ...
}
Query performance analysis


db.people.ensureIndex({name:1})
db.people.find({name:‘Fred’}).explain()

{
    “cursor” : “BtreeCursor name_1”,
    ...
}



                     Demo
Credits
• Much of this information was gleaned from
  these two presentations:
• http://speakerdeck.com/u/mongodb/p/
  indexing-and-query-optimizer-kevin-
  hanson-10gen
• http://www.slideshare.net/mongodb/
  indexing-with-mongodb

More Related Content

What's hot

Php code for online quiz
Php code for online quizPhp code for online quiz
Php code for online quiz
hnyb1002
 
Improving RDF Search Performance with Lucene and SIREN
Improving RDF Search Performance with Lucene and SIRENImproving RDF Search Performance with Lucene and SIREN
Improving RDF Search Performance with Lucene and SIREN
Mike Hugo
 
Gareth hayes. non alphanumeric javascript-php and shared fuzzing
Gareth hayes. non alphanumeric javascript-php and shared fuzzingGareth hayes. non alphanumeric javascript-php and shared fuzzing
Gareth hayes. non alphanumeric javascript-php and shared fuzzing
Yury Chemerkin
 

What's hot (17)

Coding Horrors
Coding HorrorsCoding Horrors
Coding Horrors
 
Php code for online quiz
Php code for online quizPhp code for online quiz
Php code for online quiz
 
Improving RDF Search Performance with Lucene and SIREN
Improving RDF Search Performance with Lucene and SIRENImproving RDF Search Performance with Lucene and SIREN
Improving RDF Search Performance with Lucene and SIREN
 
Gareth hayes. non alphanumeric javascript-php and shared fuzzing
Gareth hayes. non alphanumeric javascript-php and shared fuzzingGareth hayes. non alphanumeric javascript-php and shared fuzzing
Gareth hayes. non alphanumeric javascript-php and shared fuzzing
 
How to Avoid Common Data Visualization Pitfalls and Being Led Astray By Your ...
How to Avoid Common Data Visualization Pitfalls and Being Led Astray By Your ...How to Avoid Common Data Visualization Pitfalls and Being Led Astray By Your ...
How to Avoid Common Data Visualization Pitfalls and Being Led Astray By Your ...
 
Real World CouchDB
Real World CouchDBReal World CouchDB
Real World CouchDB
 
MongoDB .local Munich 2019: Tips and Tricks++ for Querying and Indexing MongoDB
MongoDB .local Munich 2019: Tips and Tricks++ for Querying and Indexing MongoDBMongoDB .local Munich 2019: Tips and Tricks++ for Querying and Indexing MongoDB
MongoDB .local Munich 2019: Tips and Tricks++ for Querying and Indexing MongoDB
 
Airline reservation project using JAVA in NetBeans IDE
Airline reservation project using JAVA in NetBeans IDEAirline reservation project using JAVA in NetBeans IDE
Airline reservation project using JAVA in NetBeans IDE
 
Building a Scalable Inbox System with MongoDB and Java
Building a Scalable Inbox System with MongoDB and JavaBuilding a Scalable Inbox System with MongoDB and Java
Building a Scalable Inbox System with MongoDB and Java
 
MongoDB .local London 2019: Tips and Tricks++ for Querying and Indexing MongoDB
MongoDB .local London 2019: Tips and Tricks++ for Querying and Indexing MongoDBMongoDB .local London 2019: Tips and Tricks++ for Querying and Indexing MongoDB
MongoDB .local London 2019: Tips and Tricks++ for Querying and Indexing MongoDB
 
MongoDB With Style
MongoDB With StyleMongoDB With Style
MongoDB With Style
 
Strongly Typed Languages and Flexible Schemas
Strongly Typed Languages and Flexible SchemasStrongly Typed Languages and Flexible Schemas
Strongly Typed Languages and Flexible Schemas
 
Webinar: Strongly Typed Languages and Flexible Schemas
Webinar: Strongly Typed Languages and Flexible SchemasWebinar: Strongly Typed Languages and Flexible Schemas
Webinar: Strongly Typed Languages and Flexible Schemas
 
Schema Design with MongoDB
Schema Design with MongoDBSchema Design with MongoDB
Schema Design with MongoDB
 
ETL for Pros: Getting Data Into MongoDB
ETL for Pros: Getting Data Into MongoDBETL for Pros: Getting Data Into MongoDB
ETL for Pros: Getting Data Into MongoDB
 
MongoDB Schema Design: Four Real-World Examples
MongoDB Schema Design: Four Real-World ExamplesMongoDB Schema Design: Four Real-World Examples
MongoDB Schema Design: Four Real-World Examples
 
Dev Jumpstart: Schema Design Best Practices
Dev Jumpstart: Schema Design Best PracticesDev Jumpstart: Schema Design Best Practices
Dev Jumpstart: Schema Design Best Practices
 

Viewers also liked

Performance Tuning on the Fly at CMP.LY
Performance Tuning on the Fly at CMP.LYPerformance Tuning on the Fly at CMP.LY
Performance Tuning on the Fly at CMP.LY
MongoDB
 
Indexing and Query Optimizer (Aaron Staple)
Indexing and Query Optimizer (Aaron Staple)Indexing and Query Optimizer (Aaron Staple)
Indexing and Query Optimizer (Aaron Staple)
MongoSF
 
MongoDB Indexing Constraints and Creative Schemas
MongoDB Indexing Constraints and Creative SchemasMongoDB Indexing Constraints and Creative Schemas
MongoDB Indexing Constraints and Creative Schemas
MongoDB
 
NoSQL i dlaczego go nie potrzebujesz? [OlCamp]
NoSQL i dlaczego go nie potrzebujesz? [OlCamp]NoSQL i dlaczego go nie potrzebujesz? [OlCamp]
NoSQL i dlaczego go nie potrzebujesz? [OlCamp]
Filip Tepper
 
User Data Management with MongoDB
User Data Management with MongoDB User Data Management with MongoDB
User Data Management with MongoDB
MongoDB
 
MongoDB Performance Tuning
MongoDB Performance TuningMongoDB Performance Tuning
MongoDB Performance Tuning
MongoDB
 
MongoDB Analytics: Learn Aggregation by Example - Exploratory Analytics and V...
MongoDB Analytics: Learn Aggregation by Example - Exploratory Analytics and V...MongoDB Analytics: Learn Aggregation by Example - Exploratory Analytics and V...
MongoDB Analytics: Learn Aggregation by Example - Exploratory Analytics and V...
MongoDB
 

Viewers also liked (19)

Fast querying indexing for performance (4)
Fast querying   indexing for performance (4)Fast querying   indexing for performance (4)
Fast querying indexing for performance (4)
 
Indexing with MongoDB
Indexing with MongoDBIndexing with MongoDB
Indexing with MongoDB
 
Performance Tuning on the Fly at CMP.LY
Performance Tuning on the Fly at CMP.LYPerformance Tuning on the Fly at CMP.LY
Performance Tuning on the Fly at CMP.LY
 
MongoDB and Indexes - MUG Denver - 20160329
MongoDB and Indexes - MUG Denver - 20160329MongoDB and Indexes - MUG Denver - 20160329
MongoDB and Indexes - MUG Denver - 20160329
 
Indexing and Performance Tuning
Indexing and Performance TuningIndexing and Performance Tuning
Indexing and Performance Tuning
 
Indexing and Query Optimizer (Aaron Staple)
Indexing and Query Optimizer (Aaron Staple)Indexing and Query Optimizer (Aaron Staple)
Indexing and Query Optimizer (Aaron Staple)
 
MongoDB Days UK: Indexing and Performance Tuning
MongoDB Days UK: Indexing and Performance TuningMongoDB Days UK: Indexing and Performance Tuning
MongoDB Days UK: Indexing and Performance Tuning
 
MongoDB Indexing Constraints and Creative Schemas
MongoDB Indexing Constraints and Creative SchemasMongoDB Indexing Constraints and Creative Schemas
MongoDB Indexing Constraints and Creative Schemas
 
NoSQL i dlaczego go nie potrzebujesz? [OlCamp]
NoSQL i dlaczego go nie potrzebujesz? [OlCamp]NoSQL i dlaczego go nie potrzebujesz? [OlCamp]
NoSQL i dlaczego go nie potrzebujesz? [OlCamp]
 
Geo-Indexing w/MongoDB
Geo-Indexing w/MongoDBGeo-Indexing w/MongoDB
Geo-Indexing w/MongoDB
 
User Data Management with MongoDB
User Data Management with MongoDB User Data Management with MongoDB
User Data Management with MongoDB
 
Webinar: Index Tuning and Evaluation
Webinar: Index Tuning and EvaluationWebinar: Index Tuning and Evaluation
Webinar: Index Tuning and Evaluation
 
Phplx mongodb
Phplx mongodbPhplx mongodb
Phplx mongodb
 
Indexing and Query Optimizer (Richard Kreuter)
Indexing and Query Optimizer (Richard Kreuter)Indexing and Query Optimizer (Richard Kreuter)
Indexing and Query Optimizer (Richard Kreuter)
 
Indexing
IndexingIndexing
Indexing
 
MongoDB Performance Tuning
MongoDB Performance TuningMongoDB Performance Tuning
MongoDB Performance Tuning
 
MongoDB Analytics: Learn Aggregation by Example - Exploratory Analytics and V...
MongoDB Analytics: Learn Aggregation by Example - Exploratory Analytics and V...MongoDB Analytics: Learn Aggregation by Example - Exploratory Analytics and V...
MongoDB Analytics: Learn Aggregation by Example - Exploratory Analytics and V...
 
Webinar: Working with Graph Data in MongoDB
Webinar: Working with Graph Data in MongoDBWebinar: Working with Graph Data in MongoDB
Webinar: Working with Graph Data in MongoDB
 
Optimizing MongoDB: Lessons Learned at Localytics
Optimizing MongoDB: Lessons Learned at LocalyticsOptimizing MongoDB: Lessons Learned at Localytics
Optimizing MongoDB: Lessons Learned at Localytics
 

Similar to Indexing with MongoDB

Similar to Indexing with MongoDB (6)

Data Representation - Day 2
Data Representation - Day 2Data Representation - Day 2
Data Representation - Day 2
 
Processing & Dataviz
Processing & DatavizProcessing & Dataviz
Processing & Dataviz
 
JSON-stat & JS: the JSON-stat Javascript Toolkit
JSON-stat & JS: the JSON-stat Javascript ToolkitJSON-stat & JS: the JSON-stat Javascript Toolkit
JSON-stat & JS: the JSON-stat Javascript Toolkit
 
Haskell in the Real World
Haskell in the Real WorldHaskell in the Real World
Haskell in the Real World
 
Trees
TreesTrees
Trees
 
Thinking Functionally In Ruby
Thinking Functionally In RubyThinking Functionally In Ruby
Thinking Functionally In Ruby
 

Recently uploaded

Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Victor Rentea
 

Recently uploaded (20)

Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a Fresher
 
Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...
Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...
Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...
 
Artificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : UncertaintyArtificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : Uncertainty
 
ICT role in 21st century education and its challenges
ICT role in 21st century education and its challengesICT role in 21st century education and its challenges
ICT role in 21st century education and its challenges
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...
 
AWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of Terraform
 
MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024
 
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
 
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
 
Exploring Multimodal Embeddings with Milvus
Exploring Multimodal Embeddings with MilvusExploring Multimodal Embeddings with Milvus
Exploring Multimodal Embeddings with Milvus
 
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century education
 
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
 
Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...
Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...
Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...
 
AXA XL - Insurer Innovation Award Americas 2024
AXA XL - Insurer Innovation Award Americas 2024AXA XL - Insurer Innovation Award Americas 2024
AXA XL - Insurer Innovation Award Americas 2024
 
[BuildWithAI] Introduction to Gemini.pdf
[BuildWithAI] Introduction to Gemini.pdf[BuildWithAI] Introduction to Gemini.pdf
[BuildWithAI] Introduction to Gemini.pdf
 
Corporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptxCorporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptx
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
 

Indexing with MongoDB

  • 1. Indexing with MongoDB Luke Ehresman CopperEgg - www.copperegg.com
  • 2. What is an index?
  • 3. What is an index? • Pointers to documents
  • 4. What is an index? • Pointers to documents • Efficiently organized for quick scanning
  • 5. What is an index? • Pointers to documents • Efficiently organized for quick scanning • Maintained in a tree structure
  • 6. How do we find x = 6 without an index? {x:0} {x:1} {x:2} {x:3} {x:4} {x:5} {x:6} {x:7} {x:8} {x:9}
  • 7. How do we find x = 6 without an index? {x:0} {x:1} {x:2} {x:3} {x:4} {x:5} {x:6} {x:7} {x:8} {x:9}
  • 8. How do we find x = 6 without an index? {x:0} {x:1} {x:2} {x:3} {x:4} {x:5} {x:6} {x:7} {x:8} {x:9}
  • 9. How do we find x = 6 without an index? {x:0} {x:1} {x:2} {x:3} {x:4} {x:5} {x:6} {x:7} {x:8} {x:9}
  • 10. How do we find x = 6 without an index? {x:0} {x:1} {x:2} {x:3} {x:4} {x:5} {x:6} {x:7} {x:8} {x:9}
  • 11. How do we find x = 6 without an index? {x:0} {x:1} {x:2} {x:3} {x:4} {x:5} {x:6} {x:7} {x:8} {x:9}
  • 12. How do we find x = 6 without an index? {x:0} {x:1} {x:2} {x:3} {x:4} {x:5} {x:6} {x:7} {x:8} {x:9}
  • 13. How do we find x = 6 without an index? {x:0} {x:1} {x:2} {x:3} {x:4} {x:5} {x:6} {x:7} {x:8} {x:9}
  • 14. How do we find x = 6 without an index? {x:0} {x:1} {x:2} {x:3} {x:4} {x:5} {x:6} {x:7} {x:8} {x:9} 6 documents scanned
  • 15. How do we find x = 6 with an index?
  • 16. How do we find x = 6 with an index? {x:4} < > {x:2} {x:6} < > < > {x:1} {x:3} {x:5} {x:7}
  • 17. How do we find x = 6 with an index? {x:4} < > {x:2} {x:6} < > < > {x:1} {x:3} {x:5} {x:7}
  • 18. How do we find x = 6 with an index? {x:4} < > {x:2} {x:6} < > < > {x:1} {x:3} {x:5} {x:7} Only 2 documents scanned
  • 19. In reality, it’s even better than that...
  • 20. In reality, it’s even better than that... {x:0, name:‘George’, ....} {x:1, name:‘Jane’, ....} {x:2, name:‘Judy’, ....} {x:3, name:‘Elroy’, ....} {x:4, name:‘Astro’, ....} {x:5, name:‘Rosie’, ....} {x:6, name:‘Fred’, ....} {x:7, name:‘Wilma’, ....} {x:8, name:‘Barney’, ....} {x:9, name:‘Betty’, ....}
  • 21. In reality, it’s even better than that... {x:4} {x:0, name:‘George’, ....} {x:1, name:‘Jane’, ....} < > {x:2, name:‘Judy’, ....} {x:3, name:‘Elroy’, ....} {x:4, name:‘Astro’, ....} {x:5, name:‘Rosie’, ....} {x:2} {x:6} {x:6, name:‘Fred’, ....} {x:7, name:‘Wilma’, ....} < > < > {x:8, name:‘Barney’, ....} {x:9, name:‘Betty’, ....} {x:1} {x:3} {x:5} {x:7}
  • 22. In reality, it’s still even better than that...
  • 23. In reality, it’s still even better than that... •MongoDB stores data and indexes in extents
  • 24. In reality, it’s still even better than that... •MongoDB stores data and indexes in extents •Only loads extents into memory as needed
  • 25. In reality, it’s still even better than that... •MongoDB stores data and indexes in extents •Only loads extents into memory as needed •The full index does not need to be in memory
  • 26. In reality, it’s still even better than that... •MongoDB stores data and indexes in extents •Only loads extents into memory as needed •The full index does not need to be in memory {x:4} < > {x:2} {x:6} < > < > {x:1} {x:3} {x:5} {x:7}
  • 28. When to index? • Frequently queried fields
  • 29. When to index? • Frequently queried fields • Low response time
  • 30. When to index? • Frequently queried fields • Low response time • Sorting
  • 31. When to index? • Frequently queried fields • Low response time • Sorting • Avoid full collection scans
  • 33. Things to know... • Indexes maintain order
  • 34. Things to know... • Indexes maintain order • Indexes will slow writes
  • 35. Things to know... • Indexes maintain order • Indexes will slow writes • Indexes take up space
  • 36. Things to know... • Indexes maintain order • Indexes will slow writes • Indexes take up space • Index maintenance will cause writes
  • 37. Things to know... • Indexes maintain order • Indexes will slow writes • Indexes take up space • Index maintenance will cause writes • “unique” parameter forces uniqueness
  • 38. How to create an index
  • 39. How to create an index db.people.ensureIndex({...})
  • 40. How to create an index db.people.ensureIndex({...}) {_id:1} is already created for each collection
  • 41. How to create an index db.people.ensureIndex({...}) {_id:1} is already created for each collection You can call ensureIndex multiple times (it only creates an index if it doesn’t already exist)
  • 42. How to create an index db.people.ensureIndex({...}) {_id:1} is already created for each collection You can call ensureIndex multiple times (it only creates an index if it doesn’t already exist) Blocks your mongod process unless you specify background:true.
  • 44. Indexes on missing fields null is assumed if a field is missing
  • 45. Indexes on missing fields null is assumed if a field is missing db.people.insert({name:‘Fred’, children:[‘Pebbles’]}); db.people.insert({name:‘Pebbles’}) db.people.ensureIndex({children:1}) db.people.find({children:null}); (returns Pebbles and uses the index)
  • 46. Indexes on missing fields null is assumed if a field is missing db.people.insert({name:‘Fred’, children:[‘Pebbles’]}); db.people.insert({name:‘Pebbles’}) db.people.ensureIndex({children:1}) db.people.find({children:null}); (returns Pebbles and uses the index) Optional sparse indexes exclude empty fields for efficiency
  • 48. Compound indexes { name: “George”, age:45, gender:”M”, ... }
  • 49. Compound indexes { name: “George”, {gender:1, age:-1} age:45, gender:”M”, ... }
  • 50. Compound indexes { name: “George”, {gender:1, age:-1} age:45, gender:”M”, ... } gender:”M” age:45 {name:‘George’, ....} age:12 {name:‘Jane’, ....} age:3 {name:‘Judy’, ....} gender:”F” {name:‘Elroy’, ....} age:44 {name:‘Astro’, ....} age:16 {name:‘Rosie’, ....} age:10
  • 52. Indexes on arrays { name: “George”, age:45, gender:”M”, children: [‘Judy’, ‘Elroy’] ... }
  • 53. Indexes on arrays { name: “George”, age:45, {children:1} gender:”M”, children: [‘Judy’, ‘Elroy’] ... }
  • 54. Indexes on arrays { name: “George”, age:45, {children:1} gender:”M”, children: [‘Judy’, ‘Elroy’] ... } Actually creates a new entry in the index for each element of the array.
  • 55. Indexes on arrays { name: “George”, age:45, {children:1} gender:”M”, children: [‘Judy’, ‘Elroy’] ... } Actually creates a new entry in the index for each element of the array. children:‘Judy’ {name:‘George’, ....} children:‘Elroy’ {name:‘Fred’, ...} children:‘Bam-Bam’ {name:‘Barney’ ...}
  • 57. Covered indexes Queries that can be resolved with only the index (does not need to fetch the original document)
  • 58. Covered indexes Queries that can be resolved with only the index (does not need to fetch the original document) { name: “George”, age:45, gender:”M”, children: [‘Judy’, ‘Elroy’] ... }
  • 59. Covered indexes Queries that can be resolved with only the index (does not need to fetch the original document) { name: “George”, age:45, db.people.ensureIndex({name:1, age:1}); gender:”M”, db.people.find({name:‘George’}, {_id:0, age:1}); children: [‘Judy’, ‘Elroy’] ... }
  • 60. How does it know which index to use?
  • 61. How does it know which index to use? db.people.ensureIndex({name:1}); db.people.ensureIndex({age:1}); db.people.find({age:{$gte:21}});
  • 62. How does it know which index to use? db.people.ensureIndex({name:1}); db.people.ensureIndex({age:1}); db.people.find({age:{$gte:21}}); scan index on name index on age
  • 63. How does it know which index to use? db.people.ensureIndex({name:1}); db.people.ensureIndex({age:1}); db.people.find({age:{$gte:21}}); scan terminated index on name index on age remember
  • 64. Query performance analysis db.people.find({name:‘Fred’}).explain() { “cursor” : “BasicCursor”, “indexBounds” : {}, “nscanned” : 88731, “nscannedObjects” : 88731, “n” : 1, “millis” : 108 }
  • 67. Credits • Much of this information was gleaned from these two presentations: • http://speakerdeck.com/u/mongodb/p/ indexing-and-query-optimizer-kevin- hanson-10gen • http://www.slideshare.net/mongodb/ indexing-with-mongodb

Editor's Notes

  1. \n
  2. \n
  3. \n
  4. \n
  5. \n
  6. \n
  7. \n
  8. \n
  9. \n
  10. \n
  11. \n
  12. \n
  13. \n
  14. \n
  15. \n
  16. \n
  17. \n
  18. \n
  19. \n
  20. \n
  21. \n
  22. \n
  23. \n
  24. \n
  25. \n
  26. \n
  27. \n
  28. \n
  29. \n
  30. \n
  31. \n
  32. \n
  33. \n
  34. \n
  35. \n
  36. \n
  37. \n
  38. \n
  39. \n
  40. \n
  41. \n
  42. \n
  43. \n
  44. \n
  45. \n
  46. \n
  47. \n
  48. \n
  49. \n
  50. \n
  51. \n
  52. \n
  53. \n
  54. \n