Se ha denunciado esta presentación.
Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.
Concurrency Control in MongoDB 3.0
Kaloian Manassiev
Senior Engineer at MongoDB
kaloianm@mongodb.com
@kaloianm
2
Audience
• Operations engineers
• Application developers
• Third-party storage engine developers
• Anybody who is curious
3
What is concurrency control?
4
What is concurrency control?
• Locking
• Data consistency
$100 + $100 = $200 (not $100 )
• MVCC
5
Collection Collection
6
Collection Collection
Storage Engine API
7
What you will learn
• Top level concurrency control
–Instance, database and collection
• How top level cooperates with t...
8
What you will NOT learn
• WiredTiger internals
–Separate session on this
9
Talk outline
• MongoDB concurrency control
• Multiple-granularity locking
• WiredTiger vs MMAP V1
• Questions
10
MongoDB 2.0 (and before)
THE
TOP LOCK
R W
R OK NO
W NO NO
11
MongoDB 2.2
Lock per database
db1.coll.insert({a:1}) db4.coll.find({c:5})
12
MongoDB 2.2
THE
TOP LOCK
+ intents
13
TOP
db1.coll.insert({a:1}) db4.coll.find({c:5})
IX IS
X S
14
TOP
db.coll.insert({a:1}) db.coll.find({c:5})
IX IS
X S
15
Why intents?
• Get rid of the global lock?
• Use read (shared) lock?
16
TOP
db1.coll.insert({a:1}) db4.coll.find({c:5})
S S
X S
17
TOP
db1.coll.insert({a:1}) db.fsyncLock({lock:1})
S S
X
18
TOP
db1.coll.insert({a:1}) db.fsyncLock({lock:1})
IX S
X
19
TOP
db1.coll.insert({a:1}) db.fsyncLock({lock:1})
IX S
XS S S S
20
TOP
db.fsyncLock({lock:1})
S
S S S S
db1.coll.find({a:1})
IS
S
21
Intents
• “Intention” to access one or more
children of an item
–Compatible with other intents
–Need to acquire lock fu...
22
Locks
• “Lock” an item for particular access
–Item being locked
–All items below it
23
Intents compatibility
IS IX S X
IS OK OK OK NO
IX OK OK NO NO
S OK NO OK NO
X NO NO NO NO
24
MongoDB 3.0
TOP
25
Multiple-granularity locking
TOP
26
Lock Manager
• Ensures the locking protocol is
obeyed
• Very low overhead
• Testable
27
Lock Manager
• Improved fairness
• Support for locking policies
• Lock statistics gathering
28
TOP
db.coll1.update({a:1}) db.coll1.update({a:4})
IX IX
IX IX
IX IX
X X
29
Storage Engine API
TOP
30
Storage Engine API
TOP
Storage Engine API
31
TOP
db.coll1.update({a:1}) db.coll1.update({a:4})
IX IX
IX IX
IX IX
WiredTiger MVCC
32
TOP
db.coll1.update({a:1}) db.coll1.drop()
IX IX
IX X
IX
WiredTiger MVCC
33
How does MMAP V1 work?
• Supports collection concurrency
• Takes lock on the collection
–Instead of intent
34
TOP
db.coll1.update({a:1}) db.coll2.update({a:4})
IX IX
IX IX
X X
MMAP V1
35
TOP
db.coll1.update({a:1}) db.coll1.find({a:4})
IX IS
IX IS
X S
MMAP V1
36
WiredTiger
• Intents: Global, Database, Collection
• Document-level concurrency left to the
storage engine
37
MMAP V1
• Intents: Global, Database
• Locks: Collection
38
Conclusion
• Storage engines have direct control
over concurrency
–Completely decoupled from top-level
locking
–Enabled...
39
Conclusion
• Retrofitted MMAP V1 to use multi-
granularity locks
–Support for collection-level locking
–Got rid of the ...
QUESTIONS?
EXTRA SLIDES
42
43
44
45
46
47
48
49
Próxima SlideShare
Cargando en…5
×

Concurrency Control in MongoDB 3.0

8.571 visualizaciones

Publicado el

This talk will describe the changes which went into MongoDB 3.0 in order to allow storage engines to achieve their maximum concurrency potential. In MongoDB 3.0, concurrency control has been separated into two levels: top-level, which protects the database catalog, and storage engine-level, which allows each individual storage engine implementation to manage its own concurrency. We will start from the top and introduce the concept of multi-granularity locking and how it protects the database catalog. We will then explain how the MongoDB lock manager works and how it allows storage engines to manage their own concurrency control without imposing any additional overhead.

  • Inicia sesión para ver los comentarios

Concurrency Control in MongoDB 3.0

  1. 1. Concurrency Control in MongoDB 3.0 Kaloian Manassiev Senior Engineer at MongoDB kaloianm@mongodb.com @kaloianm
  2. 2. 2 Audience • Operations engineers • Application developers • Third-party storage engine developers • Anybody who is curious
  3. 3. 3 What is concurrency control?
  4. 4. 4 What is concurrency control? • Locking • Data consistency $100 + $100 = $200 (not $100 ) • MVCC
  5. 5. 5 Collection Collection
  6. 6. 6 Collection Collection Storage Engine API
  7. 7. 7 What you will learn • Top level concurrency control –Instance, database and collection • How top level cooperates with the storage engine
  8. 8. 8 What you will NOT learn • WiredTiger internals –Separate session on this
  9. 9. 9 Talk outline • MongoDB concurrency control • Multiple-granularity locking • WiredTiger vs MMAP V1 • Questions
  10. 10. 10 MongoDB 2.0 (and before) THE TOP LOCK R W R OK NO W NO NO
  11. 11. 11 MongoDB 2.2 Lock per database db1.coll.insert({a:1}) db4.coll.find({c:5})
  12. 12. 12 MongoDB 2.2 THE TOP LOCK + intents
  13. 13. 13 TOP db1.coll.insert({a:1}) db4.coll.find({c:5}) IX IS X S
  14. 14. 14 TOP db.coll.insert({a:1}) db.coll.find({c:5}) IX IS X S
  15. 15. 15 Why intents? • Get rid of the global lock? • Use read (shared) lock?
  16. 16. 16 TOP db1.coll.insert({a:1}) db4.coll.find({c:5}) S S X S
  17. 17. 17 TOP db1.coll.insert({a:1}) db.fsyncLock({lock:1}) S S X
  18. 18. 18 TOP db1.coll.insert({a:1}) db.fsyncLock({lock:1}) IX S X
  19. 19. 19 TOP db1.coll.insert({a:1}) db.fsyncLock({lock:1}) IX S XS S S S
  20. 20. 20 TOP db.fsyncLock({lock:1}) S S S S S db1.coll.find({a:1}) IS S
  21. 21. 21 Intents • “Intention” to access one or more children of an item –Compatible with other intents –Need to acquire lock further down –No overhead
  22. 22. 22 Locks • “Lock” an item for particular access –Item being locked –All items below it
  23. 23. 23 Intents compatibility IS IX S X IS OK OK OK NO IX OK OK NO NO S OK NO OK NO X NO NO NO NO
  24. 24. 24 MongoDB 3.0 TOP
  25. 25. 25 Multiple-granularity locking TOP
  26. 26. 26 Lock Manager • Ensures the locking protocol is obeyed • Very low overhead • Testable
  27. 27. 27 Lock Manager • Improved fairness • Support for locking policies • Lock statistics gathering
  28. 28. 28 TOP db.coll1.update({a:1}) db.coll1.update({a:4}) IX IX IX IX IX IX X X
  29. 29. 29 Storage Engine API TOP
  30. 30. 30 Storage Engine API TOP Storage Engine API
  31. 31. 31 TOP db.coll1.update({a:1}) db.coll1.update({a:4}) IX IX IX IX IX IX WiredTiger MVCC
  32. 32. 32 TOP db.coll1.update({a:1}) db.coll1.drop() IX IX IX X IX WiredTiger MVCC
  33. 33. 33 How does MMAP V1 work? • Supports collection concurrency • Takes lock on the collection –Instead of intent
  34. 34. 34 TOP db.coll1.update({a:1}) db.coll2.update({a:4}) IX IX IX IX X X MMAP V1
  35. 35. 35 TOP db.coll1.update({a:1}) db.coll1.find({a:4}) IX IS IX IS X S MMAP V1
  36. 36. 36 WiredTiger • Intents: Global, Database, Collection • Document-level concurrency left to the storage engine
  37. 37. 37 MMAP V1 • Intents: Global, Database • Locks: Collection
  38. 38. 38 Conclusion • Storage engines have direct control over concurrency –Completely decoupled from top-level locking –Enabled document-level control
  39. 39. 39 Conclusion • Retrofitted MMAP V1 to use multi- granularity locks –Support for collection-level locking –Got rid of the global lock usage for journaling
  40. 40. QUESTIONS?
  41. 41. EXTRA SLIDES
  42. 42. 42
  43. 43. 43
  44. 44. 44
  45. 45. 45
  46. 46. 46
  47. 47. 47
  48. 48. 48
  49. 49. 49

×