Couche Base par Tugdual Grall

Win some cool stuﬀ, send a mail to

winning@couchbase.com

with NORMANDYJUG in the subject.

Wednesday, March 20, 13

Normandy
JUG
-‐
March
19th
2013

Introduc)on
to
NoSQL
with
Couchbase

Tugdual
“Tug”
Grall
Technical
Evangelist

email:
tug@couchbase.com twi0er:
@tgrall

About
me

• Tugdual
“Tug”
Grall • Web

-‐ Couchbase -‐

@tgrall
-‐ Technical
Evangelist -‐

hQp://blog.grallandco.com
-‐ eXo -‐

tgrall
-‐ CTO • NantesJUG
co-‐founder

-‐ Oracle • Pet
Project
:

-‐ Developer/Product
Manager • hQp://www.resultri.com

-‐ Mainly
Java/SOA
-‐ Developer
in
consul)ng
ﬁrms


INTRO
TO
NOSQL


RDBMS
ARE
NOT
ENOUGH?


Growth
is
the
New
Reality

• Instagram gained nearly 1 million users overnight when then
expanded to Android


Draw
Something
-‐
Social
Game

35 million monthly active users in 1 month !!!


By
contrast....
The Simpson’s : Tapped Out
Daily Active Users (Millions)


How
do
you
take
this
growth?
Applica@on
Scales
Out
Just
add
more
commodity
web
servers

System
Cost
Applica)on
Performance

Web/App
Server
Tier

Users

RDBMS
Scales
Up
Get
a
bigger,
more
complex
server

System
Cost
Applica)on
Performance

Won’t
scale

beyond
this

point
Rela@onal
Database
Users

RDBMS
is
good
for
many
thing,
but
hard
to
scale

Scaling
out
RDBMS

• Run
Many
SQL
Servers
Web/App
Server
Tier

• Data
could
be
shared
-‐ Done
by
the
applica)on
code
Memcached
Tier

• Caching
for
faster
response
)me

MySQL
Tier


NoSQL
Technology
Scales
Out
Applica@on
Scales
Out
Just
add
more
commodity
web
servers

System
Cost
Applica)on
Performance

Web/App
Server
Tier

Users

NoSQL
Database
Scales
Out
Cost
and
performance
mirrors
app
@er

System
Cost
Applica)on
Performance

NoSQL
Distributed
Data
Store

Users

Scaling
out
ﬂa;ens
the
cost
and
performance
curves

A
New
Technology?

Bigtable Dynamo Cassandra Voldemort
November
2006 October
2007 August
2008 February
2009

Building
new
database
to
answer
the
following
requirements
• No
schema
required
before
inser)ng
data
• No
schema
change
required
to
change
data
format
• Auto-‐sharding
without
applica)on
par)cipa)on
• Distributed
queries
• Integrated
main
memory
caching
• Data
synchroniza)on
(
mul)-‐datacenter)

Very
few
organiza@ons
want
to
(fewer
can)
build
and
maintain
database
soQware
technology.
But
every
organiza@on
building
interac@ve
web
applica@ons
needs
this
technology.


What
Is
Biggest
Data
Management
Problem
Driving

Use
of
NoSQL
in
Coming
Year?

49%

35%
29%

16% 12% 11%

Lack
of
ﬂexibility/ Inability
to
scale
Performance
Cost All
of
these Other
rigid
schemas out
data challenges

Source:
Couchbase
Survey,
December
2011,
n
=
1351.


NO
SQL
TAXONOMIES


NoSQL
Catalog
Key-‐Value Data
Structure Document Column Graph
(memory
only)
Cache

Memcached Redis
(memory/disk)
Database

Membase Couchbase Cassandra Neo4j

MongoDB


NoSQL
Catalog
Key-‐Value Data
Structure Document Column Graph
(memory
only)
Cache

Memcached Redis

Coherence
(memory/disk)
Database

Membase Couchbase Cassandra Neo4j

MongoDB HBase InﬁniteGraph


Hadoop
?


OperaIonal
vs.
AnalyIc
Databases

Real-‐Cme,
AnalyCc
InteracCve
Databases Databases

NoSQL

Fast
access
Get
insights
from

to
data data

Couchbase Cassandra Cloudera
MongoDB Hbase Hortonworks
Mapr

COUCHBASE


Couchbase
Server
Core
Principles

Easy
Consistent
High

Scalability PE
RF O R M A N C E Performance
Grow
cluster
without
Consistent
sub-‐millisecond

applica^on
changes,
without
read
and
write
response
^mes

down^me
with
a
single
click with
consistent
high
throughput

Always
On
JSON
JSON JSO
Flexible
Data

24x365 Model
JSON N
JSON

No
down^me
for
so`ware
JSON
document
model
with
no

upgrades,
hardware
ﬁxed
schema.
maintenance,
etc.


Couchbase
2.0
New
Features
JSON support Indexing and
Querying

Incremental Map Cross data center
Reduce replication


Couchbase
Handles
Real
World
Scale


8092
Couchbase
Server
2.0
Architecture
11211 11210
Query
API Memcapable

1.0 Memcapable

2.0 Sub^tle

Moxi
Query
Engine

REST
management
API/Web
UI

vBucket
state
and
replica)on
manager
Memcached

Global
singleton
supervisor

Rebalance
orchestrator
Conﬁgura)on
manager

Node
health
monitor
Process
monitor
Couchbase
EP
Engine

Heartbeat
Data
Manager Cluster
Manager
storage
interface

New
Persistence
Layer hdp on
each
node one
per
cluster

Erlang/OTP

HTTP Erlang
port
mapper Distributed
Erlang
8091 4369 21100
-‐
21199


Couchbase
Server
2.0
Architecture
8092 11211 11210
Query
API Memcapable

1.0 Memcapable

2.0

Moxi
Query
Engine

REST
management
API/Web
UI

vBucket
state
and
replica)on
manager
Object-‐level
Cache

Global
singleton
supervisor
RAM
Cache,

Rebalance
orchestrator
Conﬁgura)on
manager

Node
health
monitor
Server/Cluster

Process
monitor
Indexing
&

Heartbeat
Couchbase
EP
Engine
Management
&

Persistence
storage
interface
Communica@on
Management (Erlang)
New
(C
Persistence
Disk

&
V8)
Persistence
Layer hdp on
each
node one
per
cluster

Erlang/OTP

The Unreasonable Effectiveness of C by Damien Katz
HTTP Erlang
port
mapper Distributed
Erlang
8091 4369 21100
-‐
21199


Open
Source
Project
Apache
2.0

hQps://github.com/couchbase/ Gerrit: hQp://review.couchbase.org/
hQps://github.com/couchbaselabs/


SETTING
UP
TO
DEVELOP


Install
Couchbase
Server
2.0

Ubuntu

RedHat

Mac
OS
X

Windows

or
build
from
sources


Oﬃcial
SDKs

Ruby

libcouchbase

Clojure

Python

www.couchbase.com/develop
Go


Client
SDKs

c)on
on n e
make
c y
ol og
Couchbase

eceiv
e
top
r
Couchbase
Client Topology
App
Server Update


COUCHBASE
OPERATIONS


Write
OperaIon
Doc
1
App
Server

3 3
Managed
Cache 2
To
other
node Replica^on

Doc
1
Queue

Disk
Queue
Disk

Couchbase
Server
Node


Basic
OperaIons
APP
SERVER
1 APP
SERVER
2

COUCHBASE
Client
Library COUCHBASE
Client
Library
CLUSTER
MAP CLUSTER
MAP

• Docs
distributed
evenly
across
servers

READ/WRITE/UPDATE
• Each
server
stores
both
ac@ve
and
replica

SERVER
1 SERVER
2 SERVER
3 docs
Only
one
doc
ac)ve
at
a
)me
ACTIVE ACTIVE ACTIVE
• Client
library
provides
app
with
simple

Doc
5 Doc Doc
4 Doc Doc
1 Doc
interface
to
database
Doc
2 Doc Doc
7 Doc Doc
2 Doc • Cluster
map
provides
map

to
which
server
doc
is
on
Doc
9 Doc Doc
8 Doc Doc
6 Doc App
never
needs
to
know

• App
reads,
writes,
updates
docs
REPLICA REPLICA REPLICA
• Mul@ple
app
servers
can
access
same

Doc
4 Doc Doc
6 Doc Doc
7 Doc
document
at
same
@me
Doc
1 Doc Doc
3 Doc Doc
9 Doc

Doc
8 Doc Doc
2 Doc Doc
5 Doc

COUCHBASE
SERVER

CLUSTER


Store
&
Retrieve
OperaIons
• get
(key)
–
Retrieve
a
document
• set
(key,
value)
–
Store
a
document,
overwrites
if
exists
• add
(key,
value)
–
Store
a
document,
error/excepôn
if
exists
• replace
(key,
value)
–
Store
a
document,
error/excepôn
if
doesn’t
exist
• cas
(key,
value,
cas)
–
Compare
and
swap,
mutate
document
only
if
it
hasn’t
changed

while
execu^ng
this
operaôn


Atomic
Counter
OperaIons
These
opera^ons
are
always
executed
in
order
atomically.

• set
(key,
value)
–
Use
set
to
ini^alize
the
counter
• cb.set(“my_counter”,
1)
• incr
(key)
–
Increase
an
atomic
counter
value,
default
by
1
• cb.incr(“my_counter”)
#
now
it’s
2
• decr
(key)
–
Decrease
an
atomic
counter
value,
default
by
1
• cb.decr(“my_counter”)
#
now
it’s
1


Mental
Adjustments

• In
SQL
we
tend
to
want
to
avoid
hilng
the
database
as
much
as

possible
– Even
with
caching
and
indexing
tricks,
and
massive

improvements
over
the
years,
SQL
s^ll
gets
bogged
down
by

complex
joins
and
huge
indexes,
so
we
avoid
making

database
calls

• In
Couchbase,
get’s
and
set’s
are
so
fast
they
are
trivial,
not

bo0lenecks,
this
is
hard
for
many
people
to
accept
at
ﬁrst;

Mul^ple
get
statements
are
commonplace,
don’t
avoid
it!


JSON
Document
Structure
meta
{
Meta
Informa@on
“id”:
“u::jasdeep@couchbase.com”,
Including
Key “rev”:
“1-‐0002bce0000000000”,
“ﬂags”:
0,
“expira@on”:
0,
All
Keys
Unique
and

“type”:
“json”
Kept
in
RAM }

document
{
“uid”:
123456,
Document
Value “ﬁrstname”:
“jasdeep”,
“lastname”:
“Jaitla”,
Most
Recent
In
Ram
“age”:
22,
“favorite_colors”:
[“blue”,
“black”],
And
Persisted
To
Disk
“email”:
“jasdeep@couchbase.com”
}


DEMONSTRATION


Add
Nodes
to
Cluster
APP
SERVER
1 APP
SERVER
2

COUCHBASE
Client
Library COUCHBASE
Client
Library

CLUSTER
MAP CLUSTER
MAP

READ/WRITE/UPDATE READ/WRITE/UPDATE

SERVER
1 SERVER
2 SERVER
3 SERVER
4 SERVER
5
ACTIVE ACTIVE ACTIVE ACTIVE ACTIVE • Two
servers
added
Doc
5 Doc Doc
4 Doc Doc
1 Doc
One-‐click
opera@on

• Docs
automa@cally

Doc
2 Doc Doc
7 Doc Doc
2 Doc
rebalanced
across
cluster
Even
distribu)on
of
docs
Doc
9 Doc Doc
8 Doc Doc
6 Doc Minimum
doc
movement

• Cluster
map
updated
REPLICA REPLICA REPLICA REPLICA REPLICA
• App
database

Doc
4 Doc Doc
6 Doc Doc
7 Doc
calls
now
distributed

over
larger
number
of

Doc
1 Doc Doc
3 Doc Doc
9 Doc servers
Doc
8 Doc Doc
2 Doc Doc
5 Doc

COUCHBASE
SERVER

CLUSTER

User
Conﬁgured
Replica
Count
=
1


Fail
Over
Node
APP
SERVER
1 APP
SERVER
2

COUCHBASE
Client
Library COUCHBASE
Client
Library

CLUSTER
MAP CLUSTER
MAP

SERVER
1 SERVER
2 SERVER
3 SERVER
4 SERVER
5
• App
servers
accessing
docs
ACTIVE ACTIVE ACTIVE ACTIVE ACTIVE

Doc
5 Doc Doc
4 Doc Doc
1 Doc Doc
9 Doc Doc
6 Doc • Requests
to
Server
3
fail

Doc
2 Doc Doc
7 Doc Doc
2 Doc Doc
8 Doc Doc
• Cluster
detects
server
failed
Promotes
replicas
of
docs
to
ac)ve
Updates
cluster
map
Doc
1 Doc
3
• Requests
for
docs
now
go
to

appropriate
server
REPLICA REPLICA REPLICA REPLICA REPLICA

Doc
4 Doc Doc
6 Doc Doc
7 Doc Doc
5 Doc Doc
8 Doc
• Typically
rebalance

would
follow
Doc
1 Doc Doc
3 Doc Doc
9 Doc Doc
2 Doc

COUCHBASE
SERVER

CLUSTER

User
Conﬁgured
Replica
Count
=
1


Indexing
and
Querying
APP
SERVER
1 APP
SERVER
2

COUCHBASE
Client
Library COUCHBASE
Client
Library

CLUSTER
MAP CLUSTER
MAP

Query

SERVER
1 SERVER
2 SERVER
3
ACTIVE
• Indexing
work
is
distributed
amongst

ACTIVE ACTIVE
nodes
Doc
5 Doc Doc
5 Doc Doc
5 Doc
• Large
data
set
possible
Doc
2 Doc Doc
2 Doc Doc
2 Doc
• Parallelize
the
eﬀort
Doc
9 Doc Doc
9 Doc Doc
9 Doc • Each
node
has
index
for
data
stored
on
it

REPLICA REPLICA REPLICA • Queries
combine
the
results
from

required
nodes
Doc
4 Doc Doc
4 Doc Doc
4 Doc

Doc
1 Doc Doc
1 Doc Doc
1 Doc

Doc
8 Doc Doc
8 Doc Doc
8 Doc

COUCHBASE
SERVER

CLUSTER


Cross
Data
Center
ReplicaIon
(XDCR)
Couchbase
Cluster Couchbase
Cluster
West
Coast
Data
Center East
Coast
Data
Center
SERVER
1 SERVER
2 SERVER
3 SERVER
1 SERVER
2 SERVER
3
RAM
CACHE RAM
CACHE RAM
CACHE RAM
CACHE RAM
CACHE RAM
CACHE
Doc
1 Doc Doc
1 Doc Doc
1 Doc Doc
1 Doc Doc
1 Doc Doc
1 Doc

Doc
2 Doc Doc
2 Doc Doc
2 Doc Doc
2 Doc Doc
2 Doc Doc
2 Doc

Doc
6 Doc Doc
6 Doc Doc
6 Doc Doc
6 Doc Doc
6 Doc Doc
6 Doc

DISK DISK DISK DISK DISK DISK
Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc




Map
FuncIon


ElasIc
Search
Adaptor
• Elastic Search is good for ad-hoc queries and faceted browsing
• Our adapter is aware of changing Couchbase topology
• Indexed by Elastic Search after stored to disk in Couchbase

Elas@cSearch


I’m
Excited
to
See
What
You
Build,
Q
&
A

Couchbase
Docs Contact
me
on
Twider
www.couchbase.com/docs/index-‐full.html @tgrall

Couchbase
Forums Contact
me
by
Email
www.couchbase.com/forums tug@couchbase.com

IRC Learn
More
About
Design
Paderns
#couchbase CouchbaseModels.com
#libcouchbase
Senng
up
for
Ruby
on
Rails
CouchbaseOnRails.com


Q&A


Couche Base par Tugdual Grall

Recomendados

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente (6)

Similar a Couche Base par Tugdual Grall

Similar a Couche Base par Tugdual Grall (20)

Más de Normandy JUG

Más de Normandy JUG (20)

Último

Último (20)

Couche Base par Tugdual Grall