MongoDB & Mongomapper 4 real

4 real

railswaycon 2010, berlin.

jan krutisch <jan.krutisch@mindmatters.de>
http://jan.krutisch.de/

Montag, 31. Mai 2010

http://www.cashbits.de/


mongodb you say?


document database


NoS
QL
inclu
ded!

document database


no ﬁxed schema
no migrations


rich data structure


id title descr pos_lat pos_lng


{
"_id" : ObjectId("4c00245062610475a005afcd"),
"address" : "Bernstorffstr. 174n22767 HamburgnDE",
"description" : null,
"position" : {
"lat" : 53.5600912,
"lng" : 9.9596977
},
"tags" : [
"hausarzt",
"naturheilverfahren",
"akupunktur",
"allgemeinmedizin"
],
"title" : "Dr. med. Lilo Eisenbarth",
"loxicon_id" : 808261
}


✗

BSON

‣ Binary serialized JSON
‣ http://bsonspec.org/
‣ Goals: Lightweight, Traversable, Efﬁcient
‣ Format for Datastorage and Wire


rich queries


Queries
‣ expressed BSON query documents
‣ very ﬂexible
‣ relatively simple query expressions
‣ pretty close to SQL conceptually
‣ examples will follow
‣ on top: map/reduce for aggregation


Scaling

‣ Master > Slave replication
‣ Replica Pairs (with Arbiter)
‣ Replica Sets (Target: 1.6)
‣ Autosharding (Target: 1.6)


A few words on durability

‣ MongoDB only fsyncs every <n> seconds
‣ There‘s a desaster waiting to happen!
‣ When in production, replicate!
‣ This is not as bad as it sounds.


Installation/Hosting


OS X:
$ brew install mongodb


Ubuntu/Debian:
theres an apt for that


excu
se
the p
Ubuntu/Debian: un

theres an apt for that


http://www.mongodb.org/display/DOCS/Downloads

http://mongohq.com/

http://mongomachine.com/

basic usage


$ mongo


> use test
switched to db test

db.quotes.save({
text: "You can observe a lot just by watching.",
from: "Yogi Berra", created_at: new Date()
});

db.quotes.save({
text: "Silence is one of the hardest arguments to refute.",
from: "Josh Billings", created_at: new Date()
});


let‘s query


db.quotes.find();
// returns all records in collection.

db.quotes.find({from: "Yogi Berra"});
{
"_id" : ObjectId("4c0022551496fc2051e93695"),
"text" : "You can observe a lot just by watching.",
"from" : "Yogi Berra",
"created_at" : "Fri May 28 2010 22:06:45 GMT+0200 (CEST)"
}


$lt <
$gt >
$lte <=
$gte >=
$ne !=


db.quotes.find({from: {"$ne": "Yogi Berra"}});
{
"_id" : ObjectId("4c0022551496fc2051e93696"),
"text" : "Silence is one of the hardest arguments to refute.",
"from" : "Josh Billings",
"created_at" : "Fri May 28 2010 22:06:45 GMT+0200 (CEST)"
}


$in IN (2,3,4)
$nin NOT IN
$all [2,3] ~ [1,2,3]


db.quotes.find({from:{ "$in":["Yogi Berra","Josh Billings"]}});
{ "_id" : ObjectId("4c0022551496fc2051e93695"), "text" : "You can..."...}
{ "_id" : ObjectId("4c0022551496fc2051e93696"), "text" : "Silence..."...}

db.arrays.save({list: [1,2,3]});

db.arrays.find({list:{ "$in":[3,4]}});
{ "_id" : ObjectId("4c0025461496fc2051e93697"), "list" : [ 1, 2, 3 ] }

db.arrays.find({list:{ "$all":[3,5]}});


$mod yah, RLY
$size okay
$exists NOT NULL
$type huh?


db.arrays.find({list:{ "$mod":[4,0]}});

db.arrays.find({list:{ "$size":3}});

db.arrays.find({list: {"$exists": true}});
[...]

db.arrays.find({list: {"$type": 1}});
[...]


...and...


db.quotes.find({from: /^Yog/});
[...]

db.quotes.find({from: /^Yog/});
[...]

db.quotes.find("this.from == 'Yogi Berra'");
[...]

db.quotes.find({"$where": "this.from == 'Yogi Berra'"});
[...]


sort()


db.quotes.find().sort({from:1})
{"from" : "Josh Billings" ... }
{"from" : "Yogi Berra" ...}

db.quotes.find().sort({from:-1})
{"from" : "Yogi Berra" ...}
{"from" : "Josh Billings" ... }


limit()


skip() // == OFFSET


count()


db.quotes.find().count();
2


Indices


Indexing

‣ Same concept as SQL-Indices
‣ You want them. (Same concept as with...)
‣ Sort order, unique, compound, geospatial


db.quotes.ensureIndex({from: 1});

db.quotes.ensureIndex({from: -1});

db.quotes.ensureIndex({text: 1}, {unique: true});

db.quotes.ensureIndex({from: 1, text: 1});

db.quotes.dropIndexes();

db.quotes.dropIndex({from: 1, text: 1});

db.quotes.reIndex();


map/reduce,
we can haz it, too


function() {
this.tags.forEach(function(z) {
emit(z, {count: 1});
});
}

function(key, values) {
var total = 0;
values.forEach(function(v) { total += v });
return {count: total}
}


(it‘s not fast...)


one more thing


GridFS


GridFS file storage

‣ Binary fields in BSON limited to 4MB
‣ GridFS API fixes that, files stored as chunks
‣ Use the language drivers


I‘m in u‘r rubies,
querying teh MongoDB!


ruby integration
‣ mongo gem
‣ bson/ bson_ext gem

‣ mongo_mapper
‣ mongoid


Basic driver usage


require 'rubygems'
require 'mongo'

db = Mongo::Connection.new.db("test")
doc = {
:text => "You can observe a lot just by watching.",
:from => "Yogi Berra",
:created_at => Time.now
}
db['quotes'].insert(doc)
db['quotes'].find.each do |row|
puts row.inspect
end

{
"_id"=>$oid4bffe2896261046e79000001,
"from"=>"Yogi Berra",
"created_at"=>Fri May 28 15:34:33 UTC 2010,
"text"=>"You can observe a lot just by watching."
}


require 'rubygems'
require 'mongo'


100.times do |i|
db['numbers'].insert({"i" => i})
end

db['numbers'].find("i" => {"$lt" => 2}).each do |row|
puts row.inspect
end

# {"_id"=>$oid4bffe4396261046f25000001, "i"=>0}
# {"_id"=>$oid4bffe4396261046f25000002, "i"=>1}


db['text_entries'].drop_index("tags_1")
db['text_entries'].create_index("tags")
db['text_entries'].index_information


GridFS usage



grid = Mongo::Grid.new(db)

id = grid.put("You can put Strings in here",
:filename => 'test.txt')

file = grid.get(id)
puts file.filename
puts file.read

grid.delete(id)

grid.put(
File.open("/Users/jankrutisch/Dropbox/Photos/IMGP8989.jpg")
)


fs = Mongo::GridFileSystem.new(db)

fs.open("test.txt", "w") do |f|
f.write "You can put stuff in here"
end

fs.open("test.txt", "r") do |f|
puts f.read
end

fs.delete("test.txt")


mongo_mapper
‣ By John Nunemaker (@jnunemaker)
‣ works
‣ a few quirks
‣ almost completely undocumented
‣ Some stuff is still missing


# config/initializers/mongo_mapper.rb
File.open(File.join(Rails.root, 'config/mongodb.yml'), 'r') do |f|
@settings = YAML.load(f)[Rails.env]
end
MongoMapper.connection = Mongo::Connection.from_uri(@settings["connection"]) if @settings["connection"]

MongoMapper.database = @settings["database"]


class Loop
include MongoMapper::Document
key :name
key :public, Boolean
key :message_id
key :plays_and_downloads, Integer

belongs_to :user
timestamps!
end


@loops = Loop.all(
! :user_id => {"$exists" => true},
! :order => 'created_at DESC',
! :limit => 10
)


„created_at DESC“ ?!?


mongoid


mongoid
‣ By Durran Jordan (Hashrocket)
‣ Two major versions:
‣ 1.x (currently 1.9) for Rails 2.3 compatibility
‣ 2.x (currently 2.x beta) for Rails 3 compatibility
‣ Good documentation
‣ API is better (?)


File.open(File.join(RAILS_ROOT, 'config/mongodb.yml'), 'r') do |f|
@settings = YAML.load(f)[RAILS_ENV]
end

Mongoid::Config.instance.from_hash(@settings)


class Loop
include Mongoid::Document
include Mongoid::Timestamps

field :name
field :public, :type => Boolean
field :message_id
field :plays_and_downloads, :type => Integer

belongs_to_related :user
end


Criteria API

‣ A bit like Arel
‣ chainable method calls
‣ Named scopes


Embedded Documents


class Person
field :first_name
field :last_name
embeds_one :address
embeds_many :phones
end

class Address
field :street
field :city
field :state
field :post_code
embedded_in :person, :inverse_of => :address
end


acts_as_attachment


class Loop
include Mongoid::Timestamps

include Mongoid::Grid
field :name
field :public, :type => Boolean
field :message_id
field :plays_and_downloads, :type => Integer

attachment :nan
end


using MongoDB 4 real


Installation is easy


(if you‘re using Ubuntu)


setting up replication


$ mongod --master
or
master = true # mongodb.conf

$ mongod --slave --source
slaveserver.example.com

slave = true
source = slaveserver.example.com


OpLog size!


„security“


memory usage?


limits?


stability?


bonus level


you‘ve made it this far!


MongoDB explain()ed


db.text_entries.find({tags: "restaurant"}).limit(10).explain();
{
! "cursor" : "BtreeCursor tags_1",
! "indexBounds" : [
! ! [
! ! ! {
! ! ! ! "tags" : "restaurant"
! ! ! },
! ! ! {
! ! ! ! "tags" : "restaurant"
! ! ! }
! ! ]
! ],
! "nscanned" : 210,
! "nscannedObjects" : 210,
! "n" : 10,
! "millis" : 0,
! [...]
}


My personal impression


Extremely easy to grasp


Normalisation suXXorZ


Migrations hurt


Seems to be very fast


Some issues for small
projects


thanks for listening.


Moi
‣ jan.krutisch@mindmatters.de
‣ http://jan.krutisch.de/
‣ http://github.com/halfbyte/
‣ http://twitter.com/halfbyte
‣ http://www.mindmatters.de/


Pointers

‣ http://www.mongodb.org/
‣ http://www.mongoid.org/
‣ http://github.com/jnunemaker/mongo_mapper


MongoDB & Mongomapper 4 real

Recomendados

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente (20)

Similar a MongoDB & Mongomapper 4 real

Similar a MongoDB & Mongomapper 4 real (7)

Más de jan_mindmatters

Más de jan_mindmatters (13)

Último

Último (20)

MongoDB & Mongomapper 4 real