GraphDB data structure introduction
•
29 recomendaciones•3,780 vistas
Introduce graphDB structure with Neo4j.
Recomendados
Más contenido relacionado
Similar a GraphDB data structure introduction
Similar a GraphDB data structure introduction (20)
Último
Último (20)
GraphDB data structure introduction
- 1. Introduction data structure for GraphDB mixi.inc Shunya Kimura @kimuras
- 2. agenda • Motivation • Introduction • Structure for Node • Create Node • Appendix
- 3. Motivation
- 4. • Just interesting in GraphDB data structure • Why fast • Mysterious
- 5. Introduction
- 6. Neo4j • OSS implement of GraphDB • High availability • Scales to billions of nods and relationships • High speed querying through traversals • For more Information about Neo4j • http://www.slideshare.net/skimura/
- 7. Introduction for Graph Node Name: Kimura Job: Eng Relationship
- 8. Introduction for Graph Node Name: Kimura Job: Eng Relationship
- 9. Introduction for Graph Node Data Structure Create Node Relationship
- 11. A node data structure Node Manager
- 12. A node data structure Node Manager Node Cache
- 13. A node data structure Node Manager Node Cache Relation Cache
- 14. A node data structure Node Manager Node Cache Relation Cache Cache Manager
- 15. A node data structure Node Manager Node Cache Relation Cache Cache Manager Property Index Manager
- 16. A node data structure Node Manager Node Cache Relation Cache Cache Manager Property Index Transaction Manager Manager
- 17. A node data structure Node Manager Node Cache Relation Cache Cache Manager Property Index Transaction Manager Persistence Manager Manager
- 18. A node data structure Node Manager Node Cache Relation Cache Cache Manager Property Index Transaction Manager Persistence Manager Manager id id id id relationships relationships relationships relationships Node Node Node Node
- 19. Node Manager • NodeCache: Caching node id and node object • RelationCache: Caching rel id and Relation object • Cache Manager: Management cache threads • Property Index Manager: Management all nodes Properties • Persistence Manager: Management store
- 20. Relationship structure id relationships Node
- 21. Relationship structure id relationships Node
- 22. Relationship structure Array structure for relationships id relationships Node
- 23. Relationship structure relationships In Blocks Out Blocks Node
- 24. Relationship structure Node Node relationships In Blocks Node Node Out Blocks Node Node Node Node Node
- 25. Relationship structure In Direction Node Node relationships In Blocks Node Node Out Blocks Node Node Node Node Node
- 26. Relationship structure In Direction Node Node relationships In Blocks Node Node Out Blocks Node Node Node Out Direction Node Node
- 27. Relationship structure Node id relationships
- 28. Relationship structure Node id relationships relationships
- 29. Relationship structure Node id relationships relationships In blocks ID:1 ID:2 ID:3 ID:4 ID:5 ID:6 ID:7 ID:8 ID:9
- 30. Relationship structure Node id relationships relationships In blocks ID:1 ID:2 ID:3 ID:4 ID:5 ID:6 ID:7 ID:8 ID:9 Out blocks ID:10 ID:11 ID:12 ID:13 ID:14 ID:15 ID:16 ID:17 ID:18
- 31. Relationship structure Node id Last In block relationships relationships In blocks ID:1 ID:2 ID:3 ID:4 ID:5 ID:6 ID:7 ID:8 ID:9 Out blocks ID:10 ID:11 ID:12 ID:13 ID:14 ID:15 ID:16 ID:17 ID:18
- 32. Relationship structure Node id Last In block relationships Last Out block relationships In blocks ID:1 ID:2 ID:3 ID:4 ID:5 ID:6 ID:7 ID:8 ID:9 Out blocks ID:10 ID:11 ID:12 ID:13 ID:14 ID:15 ID:16 ID:17 ID:18
- 33. Create Nodes
- 34. Create Node • Understanding data structure with creating nodes • Create Node A • Create Node B • Create Relationship A to B
- 35. Create Node GraphDatabaseService graphDb = new EmbeddedGraphDatabase( DB_PATH ); Node NodeA = graphDb.createNode(); Node NodeB = graphDb.createNode(); Relationship relationship = NodeA.createRelationshipTo( NodeB, RelTypes.KNOWS );
- 36. Create Node GraphDatabaseService graphDb = new EmbeddedGraphDatabase( DB_PATH ); Node NodeA = graphDb.createNode(); Node NodeB = graphDb.createNode(); Relationship relationship = NodeA.createRelationshipTo( NodeB, RelTypes.KNOWS ); NodeA
- 37. Create Node GraphDatabaseService graphDb = new EmbeddedGraphDatabase( DB_PATH ); Node NodeA = graphDb.createNode(); Node NodeB = graphDb.createNode(); Relationship relationship = NodeA.createRelationshipTo( NodeB, RelTypes.KNOWS ); NodeA NodeB
- 38. Create Node GraphDatabaseService graphDb = new EmbeddedGraphDatabase( DB_PATH ); Node NodeA = graphDb.createNode(); Node NodeB = graphDb.createNode(); Relationship relationship = NodeA.createRelationshipTo( NodeB, RelTypes.KNOWS ); NodeA NodeB
- 40. Introduction Create Node NodeA
- 41. Introduction Create Node Node Manager Node Cache Persistence Manager Relation Cache NodeA
- 42. Introduction Create Node Create New ID id Node Manager Node Cache Persistence Manager Relation Cache NodeA
- 43. Introduction Create Node Create New ID id Create Node Node Manager id Node Cache relationships Persistence Manager NodeA Relation Cache NodeA
- 44. Introduction Create Node Create New ID id Create Node Node Manager id register Node cache Node Cache relationships Persistence Manager NodeA Relation Cache NodeA
- 45. Introduction Create Node Create New ID id Create Node Node Manager id register Node cache Node Cache relationships Persistence Manager register Persistence Manager NodeA Relation Cache NodeA
- 46. Cache • You can choice some cache type • Strong Reference Cache • Soft LRU Cache (default) • Weak LRU Cache • No Cache
- 47. What is LRU Cache • discards the least recently used items first. http://en.wikipedia.org/wiki/Cache_algorithms
- 48. What is LRU Cache • discards the least recently used items first. http://en.wikipedia.org/wiki/Cache_algorithms On memory KEY VALUE 1 VAL_1 2 VAL_2 3 VAL_3 4 VAL_4 5 VAL_5
- 49. What is LRU Cache • discards the least recently used items first. http://en.wikipedia.org/wiki/Cache_algorithms On memory KEY VALUE 1 VAL_1 2 VAL_2 Priority 3 VAL_3 4 VAL_4 5 VAL_5
- 50. What is LRU Cache • discards the least recently used items first. http://en.wikipedia.org/wiki/Cache_algorithms On memory KEY VALUE GET KEY:5 1 VAL_1 2 VAL_2 Priority 3 VAL_3 4 VAL_4 5 VAL_5
- 51. What is LRU Cache • discards the least recently used items first. http://en.wikipedia.org/wiki/Cache_algorithms On memory KEY VALUE GET KEY:5 5 VAL_5 1 VAL_1 Priority 2 VAL_2 3 VAL_3 4 VAL_4
- 52. What is LRU Cache • discards the least recently used items first. http://en.wikipedia.org/wiki/Cache_algorithms On memory KEY VALUE 5 VAL_5 1 VAL_1 Priority 2 VAL_2 3 VAL_3 4 VAL_4
- 53. What is LRU Cache • discards the least recently used items first. http://en.wikipedia.org/wiki/Cache_algorithms On memory KEY VALUE PUT KEY:6 5 VAL_5 1 VAL_1 Priority 2 VAL_2 3 VAL_3 4 VAL_4
- 54. What is LRU Cache • discards the least recently used items first. http://en.wikipedia.org/wiki/Cache_algorithms On memory KEY VALUE PUT KEY:6 6 VAL_6 5 VAL_5 Priority 1 VAL_1 2 VAL_2 3 VAL_3
- 55. Soft LRU Cache
- 56. Soft LRU Cache Soft LRU Cache Cache (ConcurrentHashMap) Queue (SoftReferenceQueue)
- 57. Soft LRU Cache ConcurrentHashMap KEY VALUE 1 ref_1 Soft LRU Cache 2 ref_2 3 ref_3 Cache (ConcurrentHashMap) Queue (SoftReferenceQueue)
- 58. Soft LRU Cache ConcurrentHashMap KEY VALUE 1 ref_1 Soft LRU Cache 2 ref_2 3 ref_3 Cache (ConcurrentHashMap) SoftRefrenceQueue Queue (SoftReferenceQueue) KEY VALUE(※SoftRefrence) 1 ref_1 2 ref_2 3 ref_3 ※Soft reference objects, which are cleared at the discretion of the garbage collector in response to memory demand. http://docs.oracle.com/javase/7/docs/api/java/lang/ref/SoftReference.html
- 59. Soft LRU Cache ConcurrentHashMap KEY VALUE 1 ref_1 Soft LRU Cache 2 ref_2 3 ref_3 Cache (ConcurrentHashMap) SoftRefrenceQueue Queue (SoftReferenceQueue) KEY VALUE(※SoftRefrence) 1 ref_1 2 ref_2 3 ref_3 ※Soft reference objects, which are cleared at the discretion of the garbage collector in response to memory demand. http://docs.oracle.com/javase/7/docs/api/java/lang/ref/SoftReference.html
- 60. Comparing Cache type Cache Type Queue Type Reference Type Weak LRU Cache WeakReferenceQueue WeakReference Soft LRU Cache SoftRefrenceQueue SoftReference Strong Reference None None Cache Remaining Reference on memory Strong > Soft > Weak
- 61. Introduction Create Node Create New ID id Create Node Node Manager id register Node cache Node Cache relationships Persistence Manager register Persistence Manager NodeA Relation Cache NodeA
- 62. Introduction Create Node Create New ID id Create Node Node Manager id register Node cache Node Cache relationships Persistence Manager register Persistence Manager NodeA Relation Cache NodeA NodeB
- 63. Introduction Create Node NodeA NodeB
- 64. Introduction Create Node NodeA NodeB
- 65. Introduction Create Node id Node Manager relationships Node Cache NodeA Persistence Manager id Relation Cache relationships NodeB NodeA NodeB
- 66. Introduction Create Node id Node Manager relationships Node Cache NodeA Register OUT direction to NodeB Persistence Manager id Relation Cache relationships NodeB NodeA NodeB
- 67. Introduction Create Node id Node Manager relationships Node Cache NodeA Register OUT direction to NodeB Persistence Manager id Relation Cache relationships Register IN direction from NodeA NodeB NodeA NodeB
- 68. Introduction Create Node id Node Manager relationships Node Cache NodeA Persistence Manager id Relation Cache relationships NodeB NodeA NodeB
- 69. Introduction Create Node Create New Relation ID rel id id Node Manager relationships Node Cache NodeA Persistence Manager id Relation Cache relationships NodeB NodeA NodeB
- 70. Introduction Create Node Create New Relation ID create Relationship rel id id Node Manager relationships Node Cache NodeA Relation Persistence Manager id Relation Cache relationships NodeB NodeA NodeB
- 71. Introduction Create Node Create New Relation ID create Relationship rel id id Node Manager relationships Node Cache NodeA Relation Persistence Manager id Relation Cache relationships NodeB NodeA NodeB
- 72. Appendix
- 73. Implemented Graph Algorithms • Centrality • Betweenness Centrality • Closeness Centrality • Path • A* • Dijkstart • Shortest Path
- 74. announcement
- 75. (NL ) (DSIRNLP) (Tokyo NLP) (Japan.R) ( ) (TokyoWebmining) (MongoDB Japan) http://www.ipsj.or.jp/10jigyo/taikai/74kai/event_2-6.html
- 76. Thanks!
Notas del editor
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n