Explains how to setup sharded cluster of replica sets. Sharding and replication deployment workshop

1. MongoDB
Replication and Sharding
Workshop
by Harun Yardımcı
@h_yardimci

2. What we will do today?
- Following Replicated Shard

3. What is MongoDB?
● Document-oriented database
● in MongoDB you store JSON-like documents
with dynamic schemas
● Bridge the gap between key-value stores
and relational databases

4. What is MongoDB?

5. Why MongoDB?
● Document-oriented
● High performance
● High availability
● Easy scalability
● Rich query language + MapReduce

6. Use Cases
Well Suited
● Archiving and event logging
● Document and Content Management Systems
● ECommerce Often in combination with an RDBMS for the final
order processing and accounting
● Gaming Small read/writes are a good fit for MongoDB
● Mobile Specifically, the server-side infrastructure of mobile
systems. Geospatial.
● Operational data store of a web site MongoDB is very good at
real-time inserts, updates, and queries. Specific web use
case examples:
● content management
● comment storage, management, voting
● user registration, profile, session data
● Real-time stats/analytics

7. What is a Document?
{'id':1, 'category_name' : 'Computer'}

8. Mongo Data Model
Key:
'category_name'
Value:
'Computer'
Field: key-value pair
'category_name':'Computer'
Document: set of fields
{'id':1, 'category_name' : 'Computer'}

9. Mongo Data Model
Collection: set of documents
(say categories)
{'id':1, 'category_name' : 'Computer'},
{'id':2, 'category_name' : 'Mobile'},
...
Database: set of collections
categories
products
members

10. Simple Usage and Introduction

11. First Run
$ mkdir -p /data/db/ Create a data path
and give permissions
$ chown -R mongod:mongod /data/db/
Start mongod deamon
$ mongod [--dbpath /data/db/] &
$ mongo Connect to mongod
> show dbs
> use admin
> show collections

12. Intro
CRUD Operations
> use testdb
switched to db testdb
> j = { name : "deneme" };
{"name" : "deneme"}
> t = { x : 3 };
{ "x" : 3 }
> db.col.save(j);
> db.col.insert(j); /* see the error message */
> db.col.save(t);
> for (var x = 1; x <= 20; x++) db.col.save({x:x, j:x*x})
>

13. Intro
> db.col.find();
{ "_id" : ObjectId("4c2209f9f3924d31102bd84a"), "name" : "mongo" }
{ "_id" : ObjectId("4c2209fef3924d31102bd84b"), "x" : 3 }
{ "_id" : ObjectId("4f971ab51c3fde3bc55f286f"), "x" : 1, "j" : 1 }
{ "_id" : ObjectId("4f971ab51c3fde3bc55f2870"), "x" : 2, "j" : 4 }
{ "_id" : ObjectId("4f971ab51c3fde3bc55f2871"), "x" : 3, "j" : 9 }
…
has more
> db.col.remove({"x":"3"});
>

14. The Big Picture!

15. Replication
Two Types of Replications
● Master / Slave Replication
● ReplicaSet

16. Master-Slave Rep.
$ mkdir -p /data/db/ms Different data paths
$ mkdir -p /data/db/sl for each instances
$ bin/mongod --master [--port <port>] [--dbpath /data/masterdb/]
$ bin/mongod --slave [--port <port>] --source <masterhostname>[:<port>]
[--dbpath /data/slavedb/]
…
> db.printReplicationInfo() - on master
> db.printSlaveReplicationInfo() - on slave
> use admin
> db.runCommand({resync: 1})

17. ReplicaSet - Voting
Consensus Vote
For a node to be elected primary, it must receive a
majority of votes by the following formula
(floor(5/2)+1)
Arbiters - {_id: 2, host: 'localhost:27019', arbiterOnly:
true}
Reachable Node – heartbeat
Each replica set is limited to 12 total nodes and 7 voting nodes.

18. ReplicaSet
Starting The Nodes myRSet
localhost localhost localhost
27017 27018 27019
$ mkdir -p /data/r0 /data/r0 /data/r1 /data/r2
$ mkdir -p /data/r1
$ mkdir -p /data/r2
$ mongod --replSet myRSet --port 27017 --dbpath /data/r0
$ mongod --replSet myRSet --port 27018 --dbpath /data/r1
$ mongod --replSet myRSet --port 27019 --dbpath /data/r2

19. ReplicaSet
Initiating The Set
> config = {_id: 'myRSet', members: [
{_id: 0, host: 'localhost:27017'},
{_id: 1, host: 'localhost:27018'},
{_id: 2, host: 'localhost:27019'}]
}
> rs.initiate(config);
{
"info" : "Config now saved locally. Should come online in about a
minute.",
"ok" : 1
}
> rs.status();

20. Sharding
Sharding Components
● Shard Servers
● Config Servers
● mongos Router

21. Sharding

22. Sharding
$ mkdir -p /data/db/s1 /data/db/s2 /data/db/config
$ mongod --shardsvr --port 27001 --dbpath /data/db/s1 &
$ mongod --shardsvr --port 27002 --dbpath /data/db/s2 &
$ mongod --configsvr --port 27003 --dbpath /data/db/config &
$ mongos --port 27017 --configdb localhost:27003 &

23. Sharding
Add Shards to Config
Connect to mongos to add shards
> use admin
> db.runCommand( {addShard : "localhost:27001"} );
{"ok" : 1 , "added" : "localhost:27001"}
> db.runCommand( {addShard : "localhost:27002"} );
{"ok" : 1 , "added" : "localhost:27002"}

24. Sharding
Enable Sharding
> db.runCommand( { enablesharding: "test_database"} );
{ "ok" : 1 }
> db.runCommand( { shardcollection :
"test_database.myCollection", key : {"_id" :1} })
{ "collectionsharded" : "test_database.myCollection",
"ok" : 1 }

25. ReplicaSet + Sharding
host1$ mongod --shardsvr --replSet rs_a
host2$ mongod --shardsvr --replSet rs_a Same replica set name
host3$ mongod --shardsvr --replSet rs_a
> cfg = {
_id : "rs_a",
members : [
{_id : 0, host : "host1:27018", priority : 1},
{_id : 1, host : "host2:27018", priority : 1},
{_id : 2, host : "host3:27018", priority : 0}
]
}
> rs.initiate(cfg)

26. ReplicaSet + Sharding
host1$ mongod --configsvr
host2$ mongod --configsvr
host3$ mongod --configsvr
$ mongos --configdb host1:27019,host2:27019,host3:27019
$ mongo
> db.adminCommand( { addShard :
"rs_a/host1:27018,host2:27018,host3:27018" } )
> db.adminCommand( { addShard :
"rs_b/host4:27018,host5:27018,host6:27018" } )
> db.adminCommand( { addShard :
"rs_c/host7:27018,host8:27018,host9:27018" } )

27. What is Next?
Please
Try it Yourself
and
Share Your Experiences
Thanks