My slides from MongoAustin 2011

1. Stash the Hash!
MongoDB is the MashupDB
Mongo Austin February 15, 2011
WYNNNETHERLAND

2. whoami

3. +

5. Two years ago...

7. x
No SQL?
x

8. MongoDB
x

9. Let's just skip to here already!
x

10. One year ago

11. CTO of Amazon
NoSQL Smackdown!

12. Today

13. is it web scale?

14. x

15. How do we get to here?
x
x

16. Enlightenment

17. Almost
^
Two years with

18. Our journey

19. TweetCongress.org

20. “We the Tweeple of the United
States, in order to form a more
perfect government, establish
communication, and promote
transparency do hereby tweet the
Congress of the United States of
America.”

21. and aggregator
^
The idea: A Twitter directory for the US Congress

23. When does NOSQL make sense?
★ Your data is stored and retrieved mainly by primary key, without
complex joins.
★ You have a non-trivial amount of data, and the thought of managing
lots of RDBMS shards and replication failure scenarios gives you the fear.
http://www.metabrew.com/article/anti-rdbms-a-list-of-distributed-key-value-stores/

24. Key value stores
Project Voldemort
me ver y cool
Ringo
Scalaris
So space
Kai
ojects in this
Dynomite
MemcacheDB
pr
ThruDB
CouchDB
Cassandra
HBase
Hypertable
Redis
Tokyo Cabinet/Tyrant
Riak
Neo4J

25. Tokyo Cabinet

27. CouchDB
Apache CouchDB is a distributed, fault-tolerant and
schema-free document-oriented database accessible via
a RESTful HTTP/JSON API.
http://couchdb.apache.org/

28. SQL CouchDB
Prede ned, explicit schema Dynamic, implicit schema
Collection of named documents with varying
Uniform tables of data
structure
Normalized. Objects spread across tables. Denormalized. Docs usually self contained. Data
Duplication reduced. often duplicated.
Must know schema to read/write a complete
Must know only document name
object
Dynamic queries of static schemas Static queries of dynamic schemas
http://damienkatz.net/files/What is CouchDB.pdf
http://damienkatz.net/files/What is CouchDB.pdf

29. SQL CouchDB
Prede ned, explicit schema Dynamic, implicit schema
Collection of named documents with varying
Uniform tables of data
structure
Normalized. Objects spread across tables.
Duplication reduced.
TheDenormalized. Docs usually selfdetails
devil'soften duplicated.contained. Data
in the
Must know schema to read/write a complete
Must know only document name
object
Dynamic queries of static schemas Static queries of dynamic schemas
http://damienkatz.net/files/What is CouchDB.pdf
http://damienkatz.net/files/What is CouchDB.pdf

30. why I

31. Mongo speaks JSON

32. Document Storage (BSON) B is for Binary
{ author: 'joe',
created: Date('03-28-2009'),
title: 'Yet another blog post',
text: 'Here is the text...',
tags: [ 'example', 'joe' ],
comments: [ { author: 'jim', comment: 'I disagree' },
{ author: 'nancy', comment: 'Good post' }
]
}
Sure wish JSON did this...
http://www.mongodb.org/display/DOCS/BSON

33. That looks like JSON

34. versatility

36. Runs like Hayes.
Hits like Mays.

37. Introducing
★ Built For Speedcan be very fast
★ Dynamic Queries and Indexes
★ Replication and Failover
★ Sharding
★ Map / Reduce
★ Geospatial stuffs

38. MongoDB is great for
★ Websites
Your own queryable API mirror
★ Caching
★ High volume, low value
★ High scalability stash the hash
★ Storage of program objects and JSON

39. Not as great for
★ Highly transactional
★ Financial stuffs
★ Problems requiring SQL

40. Installation
★ mkdir -p /data/db
★ download pre-built for OSX and unzip to /usr/local/
★ cp -R /usr/local/pathtomongo/bin /usr/local/bin
Ruby driver for MongoDB
★ gem install mongo
Native C
★ gem install mongo_ext extensions ( go turbo! )
★ gem install mongo_mapper

41. brew install mongodb

42. Contents of mongo/bin what's in the box?
★ mongod - The MongoDB server
★ mongo - the JavaScript interactive shell
★ mongoexport - export data as JSON or csv
★ mongoimport - As advertised
was?
★ mongodump - Like mysqldump why? What did you think it
★ mongorestore - Restore from mongodump les
★ mongos - Auto-sharding module (getting better with every build)

43. Some new terms

44. When I say think
database database
Well that one isn't new...

45. Databases in MongoDB
★ Made up of multiple collections
★ Are created on-the- y when rst referenced

46. When I say think
collection table

47. Collections in MongoDB
★ Schema-less but typed!
★ For grouping documents into smaller query sets (speed)
★ Indexable by one or more key
★ Are created on-the- y when rst referenced
★ Capped collections: Fixed size, older records dropped after limit
reached

48. When I say think
document record or row

49. The Ruby ecosystem

50. The Ruby driver

51. Querying
db.collection.find({'first_name': 'John'}) # finds all Johns
db.collection.find({'first_name': /^wynn/i}) # regex
db.collection.find_first({'_id':1}) # finds first with _id of 1
db.collection.find({'age': {'$gte': 21}}) # finds possible drinkers
db.collection.find({'author.first_name':'John'}) # subdocument
db.collection.find({$where:'this.age >= 6 && this.age <= 18'})

52. More
Querying
$in, $nin, $all, $ne, $gt, $gte, $lt, $lte, $size, $where
:fields (like :select in active record)
:limit, :offset for pagination
:sort ascending or descending [['foo', 1], ['bar', -1]]
count and group (uses map/reduce)
db.collection.mapReduce(mapfunction,reducefunction[,options]);

53. Updating
$inc, $set, $unset, $push, $pushAll, $addToSet,
$pop, $pull, $pullAll, $rename, $bit

55. akfast w it h
B re
Nune maker
John

56. MongoMapper from @jnunemaker
★ Mongo is not MySQL
★ DSL for modeling domain should also teach you Mongo
★ Now in version 0.8.6
I voted for "Nunemapper"

57. Features
★ Typecasting
★ Callbacks (ActiveSupport Callbacks)
★ Validations
★ Connection and database can differ per document
★ Create and Update with single or multiple
★ Delete and Destroy and _all counterparts
Be careful. Ordering can be tricky.
★ Find: id, ids, :all, : rst, :last
★ Associations

58. Example
class User
include MongoMapper::Document
key :name, String, :required => true, :length => 5..100
key :email, String, :required => true, :index => true
key :age, Integer, :numeric => true
key :active, Boolean, :default => true
one :address
many :articles
end
Included as module, not subclassed
class Address
include MongoMapper::Document
key :street, String
key :city, String
key :state, String, :length => 2
key :zip, Integer, :numeric => true, :length => 5
end

59. Queries
User.where(:age.gt => 27).sort(:age).all
User.where(:age.gt => 27).sort(:age.desc).all
User.where(:age.gt => 27).sort(:age).limit(1).all
User.where(:age.gt => 27).sort(:age).skip(1).limit(1).all

60. Scopes
class User
include MongoMapper::Document
# plain old vanilla scopes with fancy queries
scope :johns, where(:name => 'John')
# plain old vanilla scopes with hashes
scope :bills, :name => 'Bill'
# dynamic scopes with parameters
scope :by_name, lambda { |name| where(:name => name) }
scope :by_ages, lambda { |low, high| where(:age.gte => low, :age.lte => high) }
# Yep, even plain old methods work as long as they return a query
def self.by_tag(tag)
where(:tags => tag)
end
# You can even make a method that returns a scope
def self.twenties; by_ages(20, 29) end
key :name, String
key :tags, Array
end

61. Scopes
# simple scopes
pp User.johns.first
pp User.bills.first
# scope with arg
pp User.by_name('Frank').first
# scope with two args
pp User.by_ages(20, 29).all
# chaining class methods on scopes
pp User.by_ages(20, 40).by_tag('ruby').all
# scope made using method that returns scope
pp User.twenties.all

62. Mongoid

63. Mongomatic

64. Mongomatic
require 'mongomatic'
class User < Mongomatic::Base
def validate
self.errors.add "name", "can't be empty" if self["name"].blank?
self.errors.add "email", "can't be empty" if self["email"].blank?
end
end
# set the db for all models:
Mongomatic.db = Mongo::Connection.new.db("mongomatic_test")
# or you can set it for a specific model:
User.db = Mongo::Connection.new.db("mongomatic_test_user")

65. Mongomatic
User.empty?
=> true
u = User.new(:name => "Ben")
=> #<user:0x00000100d0cbf8 @doc="{"name"=">"Ben"}, @removed=false>
u.valid?
=> false
u["email"] = "me@somewhere.com"
u.valid?
=> true
u.insert
=> BSON::ObjectId('4c32834f0218236321000001')
User.empty?
=> false

66. MongoDB is open source

67. Mongo runs in the cloud

68. #mongodb

69. How can you help?
★ We need an awesome admin GUI
★ Port some plugins (might get easier with ActiveModel support
coming soon)
★ Build something cool

70. Lessons learned in production the fine print
★ The laws of computing are still in effect Upserts FTW!
★ Indexes are important no matter what the salesman told ya about
performance
★ Data modeling. Deep or Wide? The answer is yes!
★ MongoDB and MongoMapper are in active development
Very responsive yet very volatile changes!

71. 10gen

72. Get the backstory on

73. Questions?
wynn@hp.com
@pengwynn