Come and hear more about our new full-text search operator for MongoDB Atlas. This is a significant enhancement to MongoDB search features and is the easiest and most powerful full-text search solution for databases on MongoDB Atlas. This talk is important for anyone who has implemented search or is considering a search feature in their MongoDB application. You will see a demo of $searchBeta, learn about how it works, discover specific features to help you deliver relevant search results, and learn how you can start using full-text search in your application today.

1. #MDBlocal
Atlas Search Deep Dive
Will Chow
Senior Solutions Architect, MongoDB
TORONTO

2. #MDBLocal
{
“name”: “Will Chow”,
“MongoDB”: {
“positions”: [“Senior Solutions Architect”],
“since” : new Date (“2018-10”),
},
“Pre-MongoDB”: {
“positions”: [“Systems Engineer”, “Architect”, “Developer”],
“companies”: [ “Cloudera”, “IBM”, “DataMirror”]
},
“likes”: [“running”, “renovating” ]
}
About Me

3. #MDBLocal
AGENDA
Why Full
Text
Search?
Atlas
Search
Features
Queries +
How They
Work
Indexes and
Analyzers
Architecture

4. Why Full-Text Search?
Search is a requirement
for most applications

5. #MDBLocal
Why didn’t
[thing I want]
show up first?

6. #MDBLocal
Relevance is Complicated… Developer: “This result doesn’t
match the most terms.”
Marketing: “Why isn’t the promo on
top?”
Product: “If I type in an exact
product name, just skip the search.”
UX: “Users need categories
instead.”
CEO: “Change the top result right
NOW!”
Customer: “Je ne trouve pas ce que
je cherche …”

7. #MDBLocal
RelevanceisCritical

8. #MDBLocal
Searching with Text Search - $text
db.movies.find
({$text: {$search: "chikago"}},
{score: {$meta: "textScore"},
_id : 0,
title: 1,
fullplot: 1
}).sort
({score:{$meta:"textScore"}}).pretty()
[] (empty set)

9. #MDBLocal
Sync

10. #MDBLocal
Atlas Search Features
Native
Powered by
Performant
Text Indexes
Powerful
Query
Operators
Configurable
Language
Analyzers
Built-in
Highlighting
Flexible
Scoring

11. #MDBLocal
Atlas Search Features
Native
Powered by
Score is the Measure of Relevance Flexible
Scoring

12. #MDBLocal
So, build the best search
for your application
Atlas Search
Full Text Search Indexes
$searchBeta

13. #MDBLocal
How do I use it?
Create a cluster
on MongoDB
Atlas 4.2
(M30+)
Create a
database and
collection
Create a full
text index on
that collection
in Atlas
Query via
$searchBeta
aggregation
pipeline

14. #MDBLocal
Full Text Search Index
Creation
Dynamic Mapping
Static Mapping

15. #MDBLocal
Sample_mflix movies
document model

16. #MDBLocal
Searching with Full Text Search - $searchBeta
db.movies.aggregate([
{$searchBeta: { term: {
path: "title",
query: "chikago",
fuzzy: { "maxEdits": 1}},
highlight: { path: "fullplot"}}},
{$project: {
_id : 0,
title: 1,
score: { $meta: 'searchScore' },
highlights: { $meta: "searchHighlights"}}},
{$limit : 1}]).pretty()
{"title" : "Chicago",
"score" : 4.210028171539307,
highlights" : [{
path" : "fullplot",
"texts" : [
{"value" : "Murderess Velma Kelly finds herself on
death row in 1920s ",
"type" : "text"},
{"value" : "Chicago",
"type" : "hit"}],
"score" : 0.863726019859314}]}

17. #MDBLocal
db.movies.aggregate([
{ $match": {
"title": "The Godfather"
}},
{ $sort: { title: 1 } }
])
$searchBeta
db.movies.aggregate([
{ $searchBeta": {
"search": { query: "The Godfather",
path: "title",
score: { boost: {value: 3}}}}
])
Sort occurs after filter Sort occurs during filter
Query-time Scoring

18. DEMO
A Movie Search Engine using
Atlas Search
Will Chow

19. Architecture
How FTS works

20. #MDBLocal
MongoDB +
• Pre-existing functionality
• Highlights, Fuzzy-matching, Query-time scoring and
more
• Analyzers
• Language support
• Western languages: English, French, etc.
• Eastern Languages: CJK, Hindi, Thai, etc.
• Inverted index structure = fast searches

21. #MDBLocal
Inverted Index
{ _id: 1,
S: “The quick brown fox jumped over the lazy dog” }
{ _id: 2,
S: “Quick brown foxes leap over lazy dogs in summer” }
TERM DOC
The 1
Quick 2
brown 1, 2
fox 1
foxes 2
jumped 1
leap 2
TERM DOC
the 1
quick 1, 2
brown 1, 2
fox 1, 2
in 2
jump 1, 2
dog 1, 2
STEMMING,SYNONYMS

22. #MDBLocal
Real-world Inverted Index: A Concordance

23. #MDBLocal
Inverted Indices and Analyzers
“_id”: 3,
“title” : “Planes, Trains & Automobiles”
{
“planes” : [3, …],
“trains” : [3, …],
“automobiles” : [3, … ]
},
lucene.simple lucene.english
{
“plane” : [3, …],
“train” : [3, …],
“automobile” : [3, …],
}
lucene.keyword
{
“Planes, Trains &
Automobiles” : [3, …]
},

24. #MDBLocal
Querying and Analyzers
search: { query: “planes trains”,
path: “title” }
term(“plane”) OR
term(“train”)
term: { query: “planws”,
path: { value: “title”,
multi: ”simple”},
fuzzy: {maxEdits: 1 } }
Fuzzy(“planws”,
maxEdits: 1)
term: { query: “Planes(.*)”,
path: { value: “title”,
multi: ”keyword”},
regex: true} }
Regex(“Planes(.*)”)
lucene.keyword lucene.simple lucene.english

25. #MDBLocal
MongoDB Atlas FTS components
mongod mongos mongot (NEW!)
● $searchBeta aggregation
pipeline stage
● Talks mongodb wire
protocol to mongot
● Shard aware
implementation
● scatter-gather queries
● Based on Apache Lucene 8
● Integrated into MongoDB
Atlas
● Separate java process
from mongod
● collocated with mongod

26. #MDBLocal
FTS Indexing: Initial Sync
MongoDB Atlas
Query
Index Definition
Automation Agent
mongotmongod
Collection Scan
Complete!

27. #MDBLocal
FTS Indexing: Steady State
Documents
mongotmongod
changestream
MongoDB Atlas
(per node)
mongot watches
the changestream
continuously and
updates the
search index

28. #MDBLocal
Query Lifecycle
Wire Protocol
(over internet to
MongoDB Atlas) aggregate([ {$searchBeta: {
search: {
path: “name”,
query: “star wars”
}
}}]
mongod
Wire protocol
(localhost)
Lucene booleanQuery:
(should
(term(“name”, “star”),
term(“name”, wars”))
search: {
path: “name”,
query: “star wars”
}
[ { _id: “123”,score:
1.23,highlights: […] },
{…}]
{ “name” :
{ “star” : [123,124],
“wars” : [123,125,…]
}
}
Lookup([{_id: “123”],
{…}])
mongot
db.col.aggregate([
{$searchBeta: {
search: {
path: “name”,
query: “star wars”
}
}}])
app
[ { _id: “123”, title:
“Star Wars”}, {…}]
MongoDB Atlas Host

29. #MDBLocal
Sharding
• Merge
• Sort by score
mongos
request
result
per shard
per shard
• Scatter/gather
(each shard)
mongot,
mongod
mongot,
mongod
Primary Secondaries
Query

30. What’s next?
Feature Roadmap

31. #MDBLocal
• Expanded data type support
• Nums, dates, geo
• Synonyms
• Improved operators/syntax
• Architecture/performance
improvements
2019 Roadmap

32. #MDBLocal
Feedback
Visit cloud.mongodb.com
Please provide feedback using
the link on the FTS page.
We are listening!

33. DEMO
http://bit.ly/AtlasSearch_Movies

34. Key Takeaways
Ø Apache Lucene 8
Ø Uses MongoDB Query Language
Ø Wide variety of query operators – fuzzy, wildcard
Ø Flexible Scoring and Highlights
Ø Configurable Indexes
Ø Saves you time!!!

35. THANK YOU

36. #MDBlocal
Every session you rate enters you into a
drawing for a gift card!
https://www.surveymonkey.com/r/C8TFLRC
MongoDB Atlas Full-Text
Search Deep Dive