Btree Nosql Oak

Apache
CouchDB
Monday, November 2, 2009

how couchdb treats the disks
append only btree

Replication
JSON
Map Reduce
HTTP

intro:
offline apps
simple development
easy to deploy

Stuart Langridge - Canonical

! !


15 million desktops by the end of the year.
solve the data island problem


raindrop - next gen messaging platform
name: your own personal piece of the cloud


Couchapps
-- easiest way to deploy an Ajax application that needs to store data.
-- capable of scaling up with CouchDB. (etags etc)

“Ground Computing”
@jhuggins

http://www.ﬂickr.com/photos/mcpig/872293700/

app runs near the user. lower latency, lower cost.

Oﬄine by Default

http://www.ﬂickr.com/photos/shane-h/280084650

reliable
telephone poles / like akamai but for dynamic database backed applications.
- same application on the client and server or the cloud

http://apod.nasa.gov/apod/ap050930.html

“Of the Web”

http://jacobian.org/writing/of-the-web/

Let me tell you something: Django may be built for the Web, but CouchDB is built of the Web. I've never seen software that so
completely embraces the philosophies behind HTTP. ... this is what the software of the future looks like. Jacob Kaplan-Moss --
October, 2007

perfect spot - discovered not invented

When you can move data via replication, it

changes the physics of the web

so data is useful independent of location.

Lockless
Storage

html5 story

js

ruby

transactions?

-- getting CouchDB right involves going all the way to core lockless storage engine.

performance, map reduce, and deployment

B-Tree


data structure
sorted by keys
minimize disk seeks


logical btree


indexes - efficient
key lookups
range scans

Append

Only

robust / reliable
never touch bytes that are on disk
can’t be inconsistent


append only - in action
update J to a new value (rewrite j’s leaf node)


rewrite the path along the tree to J
up the parent path until we write the root.


all pointers point backwards into the ﬁle


update cost?

only has O(log n) cost where n is the # of keys


another update - add 5.

rewrite path to Z node to add 5


time follows the blue arrow.
top half is actually an accumulation of updates like the bottom.
note: batch efficiency -- save block updates


2 day test run
couchdb over long haul
4 writers, 16 readers, random access, small keyspace - lots of 409s etc.

Incremental

Reduce

takes advantage of the write pattern we’ve seen
setup main storage doc count example


rereduce = false
count # of keys in the block


rereduce = true
sum # of docs in child nodes
ﬂexible range querying


intermediate reductions stored during a view calculation
updating J doesn’t change any counts, adding the 5 key does


we are rewriting the parent nodes anyway, so it makes sense to run the reduce calculation

Concurrent
Access

readers see a consistent view
we can support many readers
reads don’t block writes
writes don’t block reads


concurrent readers
list all docs, then a write comes in, you don’t want to be surprised by an extra doc
readers can be as long lived as they want without blocking writers or seeing inconsistencies

Local Web
Platform

user interviews

http://www.longnow.org/projects/clock/orrery/

append-only
no ﬁxup phase
reliable and concurrent
sequential write pattern (few writer seeks)
hot backups
compaction

Btree Nosql Oak

Recomendados

Recomendados

Más contenido relacionado

Similar a Btree Nosql Oak

Similar a Btree Nosql Oak (20)

Más de Chris Anderson

Más de Chris Anderson (6)

Último

Último (20)

Btree Nosql Oak