Introduction to couchbase server 20

1. Introduc+on
to
Couchbase
Server
2.0
1

2. Couchbase
Server
NoSQL
Document
Database
2.0
2

3. Couchbase
Server
Easy
Consistent
High
Scalability
PE
RF O R M A N C E
Performance
Grow
cluster
without
Consistent
sub-­‐millisecond
applica+on
changes,
without
read
and
write
response
+mes
down+me
with
a
single
click
with
consistent
high
throughput
Always
On
Flexible
Data
24x365
Model
JSON
JSON JSO
N
JSON
JSON
No
down+me
for
soGware
JSON
document
model
with
upgrades,
hardware
no
fixed
schema.
maintenance,
etc.
3

4. Flexible
Data
Model
{
“ID”:
1,
“FIRST”:
“DipN”,
“LAST”:
“Borkar”,
“ZIP”:
“94040”,
“CITY”:
“MV”,
“STATE”:
“CA”
}
JSON
JSON
JSON
JSON
• No
need
to
worry
about
the
database
when
changing
your
applica+on
• Records
can
have
different
structures,
there
is
no
fixed
schema
• Allows
painless
data
model
changes
for
rapid
applica+on
development
4

5. Couchbase
Server
Features
Built-­‐in
clustering
–
All
nodes
equal
Data
replica+on
with
auto-­‐failover
Zero-­‐-­‐down+me
maintenance
Clone
to
grow
and
scale
horizontally
Built-­‐in
managed
cached
Monitoring
and
administra+on
APIs
and
GUI
SDK
for
a
variety
of
languages
5

6. New
in
2.0
JSON
support
Indexing
and
Querying
JSON
JSON JSO
JSON N
JSON
Incremental
Map
Reduce
Cross
data
center
replicaNon
6

7. Couchbase
Server
2.0
Architecture
8092
11211
11210
Query
API
Memcapable
1.0
Memcapable
2.0
Moxi
Query
Engine
REST
management
API/Web
UI
vBucket
state
and
replica+on
manager
Memcached
Global
singleton
supervisor
Rebalance
orchestrator
Configura+on
manager
Node
health
monitor
Process
monitor
Heartbeat
Couchbase
EP
Engine
Data
Manager
Cluster
Manager
storage
interface
New
Persistence
Layer
hp
on
each
node
one
per
cluster
Erlang/OTP
HTTP
Erlang
port
mapper
Distributed
Erlang
8091
4369
21100
-­‐
21199
7

8. Couchbase
Server
2.0
Architecture
8092
11211
11210
Query
API
Memcapable
1.0
Memcapable
2.0
Moxi
Query
Engine
REST
management
API/Web
UI
vBucket
state
and
replica+on
manager
Object-­‐level
Cache
Global
singleton
supervisor
Rebalance
orchestrator
Configura+on
manager
Node
health
monitor
Process
monitor
Heartbeat
Couchbase
EP
Engine
storage
interface
New
Persistence
Layer
hp
on
each
node
one
per
cluster
Erlang/OTP
HTTP
Erlang
port
mapper
Distributed
Erlang
8091
4369
21100
-­‐
21199
8

9. Couchbase
deployment
Web
Applica+on
Couchbase
Client
Library
Data
Flow
Cluster
Management
9

10. COUCHBASE
OPERATIONS
10

11. Single
node
-­‐
Couchbase
Write
Opera+on
2
Doc
1
App
Server
3
2
3
Managed
Cache
To
other
node
Replica+on
Queue
Doc
1
Disk
Queue
Disk
Couchbase
Server
Node
11

12. Single
node
-­‐
Couchbase
Update
Opera+on
2
Doc
1’
App
Server
3
2
3
Managed
Cache
To
other
node
Replica+on
Doc
1
Doc
1’
Queue
Disk
Queue
Disk
Doc
1
Couchbase
Server
Node
12

13. Single
node
-­‐
Couchbase
Read
Opera+on
2
Doc
1
GET
App
Server
3
2
3
Managed
Cache
To
other
node
Replica+on
Queue
Doc
1
Disk
Queue
Disk
Doc
1
Couchbase
Server
Node
13

14. Single
node
-­‐
Couchbase
Cache
Evic+on
2
Doc
6
2
3
4
5
App
Server
3
2
3
Managed
Cache
To
other
node
Replica+on
Queue
Doc
1
Disk
Queue
Disk
Doc
1
Doc
6
Doc
5
Doc
4
Doc
3
Doc
2
Couchbase
Server
Node
14

15. Single
node
–
Couchbase
Cache
Miss
2
Doc
1
GET
App
Server
3
2
3
Managed
Cache
To
other
node
Replica+on
Queue
Doc
1
Doc
5
4
4
Doc
Doc
Doc
3
2
Doc
Disk
Queue
Disk
Doc
1
Doc
6
Doc
5
Doc
4
Doc
3
Doc
2
Couchbase
Server
Node
15

16. Cluster
wide
-­‐
Basic
Opera+on
APP
SERVER
1
APP
SERVER
2
COUCHBASE
Client
Library
COUCHBASE
Client
Library
CLUSTER
MAP
CLUSTER
MAP
READ/WRITE/UPDATE
SERVER
1
SERVER
2
SERVER
3
• Docs
distributed
evenly
across
ACTIVE
ACTIVE
ACTIVE
servers
Doc
5
Doc
Doc
4
Doc
Doc
1
Doc
• Each
server
stores
both
acNve
and
replica
docs
Doc
2
Doc
Doc
7
Doc
Doc
2
Doc
Only
one
server
ac+ve
at
a
+me
• Client
library
provides
app
with
Doc
9
Doc
Doc
8
Doc
Doc
6
Doc
simple
interface
to
database
REPLICA
REPLICA
REPLICA
• Cluster
map
provides
map
to
which
server
doc
is
on
Doc
4
Doc
Doc
6
Doc
Doc
7
Doc
App
never
needs
to
know
Doc
1
Doc
Doc
3
Doc
Doc
9
Doc
• App
reads,
writes,
updates
docs
Doc
8
Doc
Doc
2
Doc
Doc
5
Doc
• MulNple
app
servers
can
access
same
document
at
same
Nme
COUCHBASE
SERVER
CLUSTER
User
Configured
Replica
Count
=
1
16

17. Cluster
wide
-­‐
Add
Nodes
to
Cluster
APP
SERVER
1
APP
SERVER
2
COUCHBASE
Client
Library
COUCHBASE
Client
Library
CLUSTER
MAP
CLUSTER
MAP
READ/WRITE/UPDATE
READ/WRITE/UPDATE
SERVER
1
SERVER
2
SERVER
3
SERVER
4
SERVER
5
• Two
servers
added
ACTIVE
ACTIVE
ACTIVE
ACTIVE
ACTIVE
One-­‐click
operaNon
Doc
5
Doc
Doc
4
Doc
Doc
1
Doc
• Docs
automaNcally
rebalanced
across
Doc
2
Doc
Doc
7
Doc
Doc
2
Doc
cluster
Even
distribu+on
of
docs
Minimum
doc
movement
Doc
9
Doc
Doc
8
Doc
Doc
6
Doc
• Cluster
map
updated
REPLICA
REPLICA
REPLICA
REPLICA
REPLICA
• App
database
Doc
4
Doc
Doc
6
Doc
Doc
7
Doc
calls
now
distributed
over
larger
number
of
Doc
1
Doc
Doc
3
Doc
Doc
9
Doc
servers
Doc
8
Doc
Doc
2
Doc
Doc
5
Doc
COUCHBASE
SERVER
CLUSTER
User
Configured
Replica
Count
=
1
17

18. Cluster
wide
-­‐
Fail
Over
Node
APP
SERVER
1
APP
SERVER
2
COUCHBASE
Client
Library
COUCHBASE
Client
Library
CLUSTER
MAP
CLUSTER
MAP
SERVER
1
SERVER
2
SERVER
3
SERVER
4
SERVER
5
• App
servers
accessing
docs
ACTIVE
ACTIVE
ACTIVE
ACTIVE
ACTIVE
• Requests
to
Server
3
fail
Doc
5
Doc
Doc
4
Doc
Doc
1
Doc
Doc
9
Doc
Doc
6
Doc
• Cluster
detects
server
failed
Promotes
replicas
of
docs
to
Doc
2
Doc
Doc
7
Doc
Doc
2
Doc
Doc
8
Doc
Doc
ac+ve
Updates
cluster
map
Doc
1
Doc
3
• Requests
for
docs
now
go
to
REPLICA
REPLICA
REPLICA
REPLICA
REPLICA
appropriate
server
Doc
4
Doc
Doc
6
Doc
Doc
7
Doc
Doc
5
Doc
Doc
8
Doc
• Typically
rebalance
would
follow
Doc
1
Doc
Doc
3
Doc
Doc
9
Doc
Doc
2
Doc
COUCHBASE
SERVER
CLUSTER
User
Configured
Replica
Count
=
1
18

19. DEMO
TIME
19

20. Indexing
and
Querying
–
The
basics
• Define
materialized
views
on
JSON
documents
and
then
query
across
the
data
set
• Using
views
you
can
define
• Primary
indexes
• Simple
secondary
indexes
(most
common
use
case)
• Complex
secondary,
ter+ary
and
composite
indexes
• Aggrega+ons
(reduc+on)
• Indexes
are
eventually
indexed
• Queries
are
eventually
consistent
with
respect
to
documents
• Built
using
Map/Reduce
technology
• Map
and
Reduce
func+ons
are
wrihen
in
Javascript
20

21. Eventually
indexed
Views
–
Data
flow
2
Doc
1
App
Server
Couchbase
Server
Node
3
2
3
Managed
Cache
To
other
node
Replica+on
Queue
Doc
1
Disk
Queue
Disk
Doc
1
View
engine
21

22. Cluster
wide
-­‐
Indexing
and
Querying
APP
SERVER
1
APP
SERVER
2
COUCHBASE
Client
Library
COUCHBASE
Client
Library
CLUSTER
MAP
CLUSTER
MAP
Query
SERVER
1
SERVER
2
SERVER
3
• Indexing
work
is
distributed
ACTIVE
ACTIVE
ACTIVE
amongst
nodes
Doc
5
Doc
Doc
5
Doc
Doc
5
Doc
• Large
data
set
possible
Doc
2
Doc
Doc
2
Doc
Doc
2
Doc
• Parallelize
the
effort
Doc
9
Doc
• Each
node
has
index
for
data
stored
Doc
9
Doc
Doc
9
Doc
on
it
REPLICA
REPLICA
REPLICA
• Queries
combine
the
results
from
Doc
4
Doc
required
nodes
Doc
4
Doc
Doc
4
Doc
Doc
1
Doc
Doc
1
Doc
Doc
1
Doc
Doc
8
Doc
Doc
8
Doc
Doc
8
Doc
COUCHBASE
SERVER
CLUSTER
User
Configured
Replica
Count
=
1
22

23. Cross
Data
Center
Replica+on
–
The
basics
• Replicate
your
Couchbase
data
across
clusters
• Clusters
may
be
spread
across
geos
• Configured
on
a
per-­‐bucket
basis
• Supports
unidirec+onal
and
bidirec+onal
opera+on
• Applica+on
can
read
and
write
from
both
clusters
(ac+ve
–
ac+ve
replica+on)
• Replica+on
throughput
scales
out
linearly
• Different
from
intra-­‐cluster
replica+on
23

24. Cross
data
center
replica+on
–
Data
flow
2
Doc
1
App
Server
Couchbase
Server
Node
3
2
3
Managed
Cache
To
other
node
Replica+on
Queue
Doc
1
Disk
Queue
Disk
Doc
1
XDCR
Queue
To
other
cluster
24

25. Cluster
wide
-­‐
XDCR
SERVER
1
SERVER
2
SERVER
3
ACTIVE
ACTIVE
ACTIVE
COUCHBASE
SERVER
CLUSTER
Doc
Doc
Doc
NY
DATA
CENTER
Doc
2
Doc
Doc
Doc
9
Doc
Doc
RAM
RAM
RAM
Doc
Doc
Doc
Doc
Doc
Doc
Doc
Doc
Doc
DISK
DISK
DISK
SERVER
1
SERVER
2
SERVER
3
ACTIVE
ACTIVE
ACTIVE
Doc
Doc
Doc
Doc
2
Doc
Doc
Doc
9
Doc
Doc
RAM
RAM
RAM
COUCHBASE
SERVER
CLUSTER
Doc
Doc
Doc
Doc
Doc
Doc
Doc
Doc
Doc
SF
DATA
CENTER
DISK
DISK
DISK
25

26. DEMO
TIME
26

27. Demo:
The
next
big
social
game
3
Objects
(documents)
within
game:
• Players
• Monsters
• Items
Gameplay:
• Players
fight
monsters
• Monsters
drop
items
• Players
own
items
27

28. Player
Document
{
"jsonType":
"player",
"uuid":
"35767d02-­‐a958-­‐4b83-­‐8179-­‐616816692de1",
"name":
"Keith4540",
"hitpoints":
75,
Player
ID
"experience":
663,
"level":
4,
"loggedIn":
false
}
28

29. Item
Document
{
Item
ID
"jsonType":
"item",
"name":
"Katana_e5890c94-­‐11c6-­‐65746ce6c560",
"uuid":
"e5890c94-­‐11c6-­‐4856-­‐a7a6-­‐65746ce6c560",
"ownerId":
"Dale9887"
}
Player
ID
29

30. Monster
Document
{
Monster
ID
"jsonType":
"monster",
"name":
"Bauchan9932",
"uuid":
"d10dfc1b-­‐0412-­‐4140-­‐b4ec-­‐affdbf2aa5ec",
"hitpoints":
370,
"experienceWhenKilled":
52,
"itemProbability":
0.5050581341872865
}
30

31. GAME
ON!
31

32. Full
Text
Search
Integra+on
• Elas+c
Search
is
good
for
ad-­‐hoc
queries
and
faceted
browsing
• Couchbase
adapter
uses
XDCR
to
push
muta+ons
to
ESDocs
are
indexed
by
Elas+c
Search
• Couchbase
ES
Adapter
is
cluster-­‐aware
ElasNcSearch
Unidirec+onal
Cross
Data
Center
Replica+on
32

33. Full
Text
Search
Application Server
Couchbase SDK ES Queries over HTTP
Do
ta TS c
Da ry Qu Re
ue er fs
MRQ y
Couchbase Server Cluster ElasticSearch Server Cluster
MR MR MR MR
Views Views Views Views
XDCR-based
Cross Data Center Replication CB-ES Transport
33

34. Couchbase
SDKs
Java
SDK
User
Code
.Net
SDK
CouchbaseClient
cb
=
new
CouchbaseClient(listURIs,
Java
client
API
"aBucket",
"letmein");
cb.set("hello",
0,
"world");
cb.get("hello");
Couchbase
Java
Library
PHP
SDK
(spymemcached)
Ruby
SDK
Couchbase
Server
…and
many
more
hhp://www.couchbase.com/develop
34

35. QUESTIONS?
35
3

36. THANK
YOU
CHIYOUNG@COUCBASE.COM
@CHIYOUNGSEO
36
3