Más contenido relacionado La actualidad más candente (20) Similar a MongoDB: Intro & Application for Big Data (20) Más de Takahiro Inoue (20) MongoDB: Intro & Application for Big Data9. {
"_id" : ObjectId("4dcd3ebc9278000000005158"),
"timestamp" : ISODate("2011-05-13T14:22:46.777Z"),
"binary" : BinData(0,""),
"string" : "abc",
"number" : 3,
"subobj" : {"subA": 1, "subB": { "subsubC": 2 }},
"array" : [1, 2, 3],
"dbref" : [_id1, _id2, _id3]
}
10. {
"_id" : ObjectId("4dcd3ebc9278000000005158"),
"nickname" : "doryokujin"
},{
"_id" : ObjectId("4dcd3ebc9278000000005159"),
"firstname" : "Takahiro",
"lastname" : "Inoue",
"mail" : "mr.stoicman@gmail.com",
"twitter" : "@doryokujin"
},...
11. {
"_id" : ObjectId("4dcd3ebc9278000000005158"),
"timestamp" : ISODate("2011-05-13T14:22:46.777Z"),
"binary" : BinData(0,""),
"string" : "abc",
"number" : 3,
"subobj" : {"subA": 1, "subB": 2 },
"array" : [1, 2, 3],
padding
}
12. {
"_id" : ObjectId("4dcd3ebc9278000000005158"),
"timestamp" : ISODate("2011-05-13T14:22:46.777Z"),
"binary" : BinData(0,""),
"string" : "def",
"number" : 4,
"subobj" : {"subA": 1, "subB": 2 },
"array" : [1, 2, 3, 4, 5, 6],
"newkey" : "In-place"
}
16. {
"_id" : ObjectId("4dcd3ebc9278000000005158"),
"timestamp" : ISODate("2011-05-13T14:22:46.777Z"),
"binary" : BinData(0,""),
"string" : "abc",
"number" : 3,
"subobj" : {"subA": 1, "subB": 2 },
"array" : [1, 2, 3],
}
26. Cluster
Shard Servers (Data)
config Servers
(Shard Configration) shard1 shard2 shard3
[ a, f ) [ k, n) [ o, t ) Chunk
[ f, k ) [ n, o ) [ t, } )
mongos Servers (Routers)
27. Shard
1
( mongos
primary primary
)
Shard
cinfig Shard
mongos
35. Every Server:
Large HDD(500GB )
Large Memory(16GB )
Slave Delay in Master Data on
Preparation for AmazonS3
User Error
Master Data on S3
Non Sharding,
Replica Set
38. From Text Logs: (Large)
From mySQL: (Small)
From Other NoSQL: (Middle)
Temporary Raw data storage
is HDFS, not MongoDB
We only need result data
to discover new features
41. Fluent
Structured logging
Pluggable architecture
Reliable forwarding
e Event Collector Service
42. Fluent
Structured logging
Pluggable architecture
Reliable forwarding
e Event Collector Service
Sadayuki Furuhashi
Treasure Data, Inc.
@frsyuki
43. “2011-04-01 host1 myapp: cmessage size=12MB user=me”
2011-04-01 myapp.message {
“on_host”: ”host1”,
2011-04-01 myapp.message {
”combined”: true, “on_host”: ”host1”,
”combined”: true,
“size”: 12000000, “size”: 12000000,
“user”: “me”
“user”: “me” }
}
48. aggregate aggregate aggregate aggregate
log log log log
key1 key2 key3 shuffle
aggregate aggregate aggregate
aggregate