SCALING MONGODB IN THE CLOUD        Simon Maynard - Bugsnag CTO               @snmaynard
WHERE HAVE I USED MONGODB?
HEYZAP• Largest   mobile gaming social network• MongoDB     the main datastore   • Also   MySQL & Redis• High   number of ...
BUGSNAG                       bugsnag.com• Exception   tracking service for mobile and web• MongoDB      only persistent d...
WHAT ARE THE PROS & CONS OF        MONGODB?
MONGODB PROS & CONS    Pros•   Schemaless•   Fire & Forget•   Scalable writes / reads•   Fast!
MONGODB PROS & CONS    Pros                          Cons•   Schemaless                •   Schemaless•   Fire & Forget    ...
WHEN SHOULD YOU THINK ABOUT         SCALING?
• From    the start!• Monitor• Anticipate• React   early
WHAT ARE THE KEY RESOURCES?
RAM•   Heavily reliant on available RAM•   “Working set” should fit in RAM•   Indexes & documents
RAM      in RAM      not in RAM
I/O•   When data is not in RAM, MongoDB hits the disk•   Ensure this happens infrequently•   When it does, it should be fa...
HOW TO KEEP I/O FAST•   Fast filesystem - 10gen recommends xfs•   Use RAID - e.g. RAID 10 (stripe of mirrors)•   Increase fi...
HOW CAN YOU ARCHITECT MONGODB          TO SCALE?
VERTICAL SCALING•   Buy more resources on single machine    •   RAM    •   I/O
HORIZONTAL SCALING  •   Buy more machines      •   Replica sets      •   Sharding
REPLICA SETS•   Scales reads well•   One primary, many secondaries•   Read from all members•   Write to primary only•   In...
SHARDING•   Many primaries, many secondaries•   Scales writes and reads•   Harder to set up well
WHAT OTHER TECHNIQUES TO SCALE?
STANDARD RULES•   Standard DB scaling rules apply to MongoDB    •   Use skip() and limit()    •   Return subsets of fields ...
SCHEMA DESIGN•   De-normalize                   {                       "_id" : ObjectId("505bd6a6c6b6b99254000003"),     ...
SCHEMA DESIGN•   Indexes should be minimized in size and number        {                                                  ...
SCHEMA DESIGN•   Minimize key lengths on small documents•   Can reduce storage requirements and performance increase{    "...
SCHEMA DESIGN•   Minimize key lengths on small documents•   Can reduce storage requirements and performance increase{     ...
PROFILER•   MongoDB has a built in profiler•   Use the profiler all the time•   db.setProfilingLevel(1, 100)•   ‘show profile’...
PROFILER OUTPUT"ts" : ISODate("2012-09-24T23:24:28.908Z"),                                                     "nscanned" ...
PROFILER OUTPUT"ts" : ISODate("2012-09-24T23:24:28.908Z"),                                                     "nscanned" ...
WHAT SHOULD I MONITOR?
MONITORS•   Chart the index size•   Chart the number of current ops•   Monitor index misses•   Monitor replication lag•   ...
HOW CAN I MONITOR MONGODB?
db.currentOp(){    "opid" : 783608,    "active" : true,    "secs_running" : 149,    "op" : "query",    "ns" : "bugsnag.acc...
db.serverStatus()"locks" : {                                    ! "misses" : 0,! "bugsnag" : {                            ...
db.stats(){!   "db" : "bugsnag",!   "collections" : 14,!   "objects" : 68081951,!   "avgObjSize" : 10147.85350585104,!   "...
MONGOTOP         ns               total   read   write   bugsnag.events          80ms   12ms   68ms   bugsnag.projects    ...
MONGOSTAT        insert query update delete getmore command flushes faults locked db idx miss %localhost 147   210    51   ...
MONGO MONITORING SERVICE•   MMS is 10gen hosted Mongo monitoring•   Available as web app (https://mms.10gen.com)•   Androi...
KIBANA & LOGSTASH•   Logstash is open-source log parser - http://logstash.net/•   Kibana is an alternative UI for Logstash...
•   Questions?•   Check out www.bugsnag.com•   Follow me on twitter @snmaynard
Próxima SlideShare
Cargando en…5
×

Mongo scaling

4.843 visualizaciones

Publicado el

Publicado en: Tecnología, Empresariales
  • good
       Responder 
    ¿Estás seguro?    No
    Tu mensaje aparecerá aquí
  • On slide 21 ('Standard rules'), using skip() on large datasets is usually NOT a standard good practice. As much as feasible, you want to use range queries instead, which is less easy for paging and forces you to consider paging in the data model, typically by storing extra information based on expected queries in the collection in order to prevent having to skip.

    Cool recommendation about logstash + kibana: these are not yet known enough. Note that there is an alternative node.js implementation of logstash ( https://github.com/bpaquet/node-logstash ) which may be more convenient to deploy than the original.
       Responder 
    ¿Estás seguro?    No
    Tu mensaje aparecerá aquí

Mongo scaling

  1. 1. SCALING MONGODB IN THE CLOUD Simon Maynard - Bugsnag CTO @snmaynard
  2. 2. WHERE HAVE I USED MONGODB?
  3. 3. HEYZAP• Largest mobile gaming social network• MongoDB the main datastore • Also MySQL & Redis• High number of reads, fewer writes
  4. 4. BUGSNAG bugsnag.com• Exception tracking service for mobile and web• MongoDB only persistent datastore • Redis caching• Lots of writes, fewer reads
  5. 5. WHAT ARE THE PROS & CONS OF MONGODB?
  6. 6. MONGODB PROS & CONS Pros• Schemaless• Fire & Forget• Scalable writes / reads• Fast!
  7. 7. MONGODB PROS & CONS Pros Cons• Schemaless • Schemaless• Fire & Forget • Fire & Forget• Scalable writes / reads • No joins• Fast! • No transactions • Database level locking
  8. 8. WHEN SHOULD YOU THINK ABOUT SCALING?
  9. 9. • From the start!• Monitor• Anticipate• React early
  10. 10. WHAT ARE THE KEY RESOURCES?
  11. 11. RAM• Heavily reliant on available RAM• “Working set” should fit in RAM• Indexes & documents
  12. 12. RAM in RAM not in RAM
  13. 13. I/O• When data is not in RAM, MongoDB hits the disk• Ensure this happens infrequently• When it does, it should be fast• EBS throughput sucks
  14. 14. HOW TO KEEP I/O FAST• Fast filesystem - 10gen recommends xfs• Use RAID - e.g. RAID 10 (stripe of mirrors)• Increase file descriptor limits• Turn off atime and diratime• Tweak read-ahead settings• http://www.mongodb.org/display/DOCS/Production+Notes
  15. 15. HOW CAN YOU ARCHITECT MONGODB TO SCALE?
  16. 16. VERTICAL SCALING• Buy more resources on single machine • RAM • I/O
  17. 17. HORIZONTAL SCALING • Buy more machines • Replica sets • Sharding
  18. 18. REPLICA SETS• Scales reads well• One primary, many secondaries• Read from all members• Write to primary only• Inconsistent reads from secondaries
  19. 19. SHARDING• Many primaries, many secondaries• Scales writes and reads• Harder to set up well
  20. 20. WHAT OTHER TECHNIQUES TO SCALE?
  21. 21. STANDARD RULES• Standard DB scaling rules apply to MongoDB • Use skip() and limit() • Return subsets of fields • Index all your queries • Run explain() on new/slow queries
  22. 22. SCHEMA DESIGN• De-normalize { "_id" : ObjectId("505bd6a6c6b6b99254000003"), "author" : "Simon Maynard", "post" : "Hey everyone!", "comments" : [ { "author" : "anonymous", "text" : "Hey!", },{ "author" : "James Smith", "text" : "Hey Simon!", } }
  23. 23. SCHEMA DESIGN• Indexes should be minimized in size and number { { "name" : "Angry Birds", "name" : "Angry Birds", "android" : true, "platform" : 3 "iphone" : true } }
  24. 24. SCHEMA DESIGN• Minimize key lengths on small documents• Can reduce storage requirements and performance increase{ "_id":"AHAHSPGPGSAVKLPAPHSVGKSALR", "game_id":"8122", "user_id":"1854", "session_start":"51067007", "session_end":"51067085"} 92 bytes
  25. 25. SCHEMA DESIGN• Minimize key lengths on small documents• Can reduce storage requirements and performance increase{ { "_id":"AHAHSPGPGSAVKLPAPHSVGKSALR", "_id":"AHAHSPGPGSAVKLPAPHSVGKSALR", "game_id":"8122", "g":"8122", "user_id":"1854", "u":"1854", "session_start":"51067007", "s":"51067007", "session_end":"51067085" "e":"51067085"} } 92 bytes 58 bytes About 1/3 memory saved!
  26. 26. PROFILER• MongoDB has a built in profiler• Use the profiler all the time• db.setProfilingLevel(1, 100)• ‘show profile’ shows recent profiles• Stored in db.system.profile
  27. 27. PROFILER OUTPUT"ts" : ISODate("2012-09-24T23:24:28.908Z"), "nscanned" : 1,"op" : "query", "scanAndOrder" : true,"ns" : "bugsnag.errors", "numYield" : 0,"query" : { "lockStats" : { "query" : { "timeLockedMicros" : { }, "errorHash":"2ff33b4f86543972577cdee34f60e4b2", "timeAcquiringMicros" : { "project_id" :"4ff24b7e2511bb1a70000004" "r" : NumberLong(2), }, "w" : NumberLong(3) "orderby" : { } "_id" : 1 }, } "nreturned" : 1,}, "responseLength" : 5240,"ntoreturn" : 1, "millis" : 0,"ntoskip" : 0,
  28. 28. PROFILER OUTPUT"ts" : ISODate("2012-09-24T23:24:28.908Z"), "nscanned" : 1,"op" : "query", "scanAndOrder" : true,"ns" : "bugsnag.errors", "numYield" : 0,"query" : { "lockStats" : { "query" : { "timeLockedMicros" : { }, "errorHash":"2ff33b4f86543972577cdee34f60e4b2", "timeAcquiringMicros" : { "project_id" :"4ff24b7e2511bb1a70000004" "r" : NumberLong(2), }, "w" : NumberLong(3) "orderby" : { } "_id" : 1 }, } "nreturned" : 1,}, "responseLength" : 5240,"ntoreturn" : 1, "millis" : 0,"ntoskip" : 0,
  29. 29. WHAT SHOULD I MONITOR?
  30. 30. MONITORS• Chart the index size• Chart the number of current ops• Monitor index misses• Monitor replication lag• Monitor I/O performance (iostat)• Monitor disk space
  31. 31. HOW CAN I MONITOR MONGODB?
  32. 32. db.currentOp(){ "opid" : 783608, "active" : true, "secs_running" : 149, "op" : "query", "ns" : "bugsnag.accounts", "query" : { "_id" : ObjectId("505bd6a6c6b6b99254000003"), }, "waitingForLock" : false, "numYields" : 349,}
  33. 33. db.serverStatus()"locks" : { ! "misses" : 0,! "bugsnag" : { ! "resets" : 0, "timeLockedMicros" : { ! "missRatio" : 0 ! "r" : NumberLong(1639187950), } ! "w" : NumberLong(1313312267) }, }, "opcounters" : { "timeAcquiringMicros" : { ! "insert" : 13674147, "r" : NumberLong(1041368094), ! "query" : 5261723, "w" : NumberLong(630905947) ! "update" : 2576757, } ! "delete" : 22324, }, ! "getmore" : 4459,}, ! "command" : 4382007"indexCounters" : { },! "btree" : { "accesses" : 610645909, ! "hits" : 610645909,
  34. 34. db.stats(){! "db" : "bugsnag",! "collections" : 14,! "objects" : 68081951,! "avgObjSize" : 10147.85350585104,! "dataSize" : 690885147618,! "storageSize" : 1290028235245,! "numExtents" : 67,! "indexes" : 28,! "indexSize" : 21240430449,! "fileSize" : 1925185536051,! "nsSizeMB" : 16,! "ok" : 1}
  35. 35. MONGOTOP ns total read write bugsnag.events 80ms 12ms 68ms bugsnag.projects 2ms 2ms 0ms bugsnag.users 1ms 1ms 0msbugsnag.system.indexes 4ms 4ms 0ms
  36. 36. MONGOSTAT insert query update delete getmore command flushes faults locked db idx miss %localhost 147 210 51 13 4 215 0 0 14% 0
  37. 37. MONGO MONITORING SERVICE• MMS is 10gen hosted Mongo monitoring• Available as web app (https://mms.10gen.com)• Android client also available from Google Play
  38. 38. KIBANA & LOGSTASH• Logstash is open-source log parser - http://logstash.net/• Kibana is an alternative UI for Logstash - http://kibana.org/• Cool trend analysis for mongo logs
  39. 39. • Questions?• Check out www.bugsnag.com• Follow me on twitter @snmaynard

×