RDBMS to Graph
•
6 recomendaciones•1,032 vistas
Relational databases were conceived to digitize paper forms and automate well-structured business processes, and still have their uses. But RDBMS cannot model or store data and its relationships without complexity, which means performance degrades with the increasing number and levels of data relationships and data size. Additionally, new types of data and data relationships require schema redesign that increases time to market. A native graph database like Neo4j naturally stores, manages, analyzes, and uses data within the context of connections meaning Neo4j provides faster query performance and vastly improved flexibility in handling complex hierarchies than SQL.
Recomendados
Más contenido relacionado
La actualidad más candente
La actualidad más candente (20)
Similar a RDBMS to Graph
Similar a RDBMS to Graph (20)
Más de Neo4j
Más de Neo4j (20)
Último
Último (20)
RDBMS to Graph
- 4. HR-tools Supply Payments Logistics CRM Support TRADITIONAL DATA STRUCTURE RDBMS RDBMS RDBMSRDBMS RDBMS RDBMS
- 5. SHIFT TOWARDS SYSTEMS OF ENGAGEMENT
- 6. Users Engaging With DevicesUsers Engaging With Users Devices Engaging With Devices SYSTEMS OF ENGAGEMENT
- 7. SHIFT TOWARDS SYSTEMS OF ENGAGEMENT You are here! Data volume
- 8. SYSTEMS OF RECORD Relational Database Model Structured Pre-computed Based on rigid rules SYSTEMS OF ENGAGEMENT NoSQL Database Model Highly Flexible Real-Time Queries Highly Contextual
- 10. SYSTEMS OF ENGAGEMENT This is data modeled as a graph!
- 13. Intuitiveness
- 15. Speed “We found Neo4j to be literally thousands of times faster than our prior MySQL solution, with queries that require 10-100 times less code. Today, Neo4j provides eBay with functionality that was previously impossible.” - Volker Pacher, Senior Developer “Minutes to milliseconds” performance Queries up to 1000x faster than RDBMS or other NoSQL
- 16. Intuitivness Speed, because Native Graph Database Agility
- 18. Employee ID Name PictureRef Building Office Departme nt Title Degree1 Uni1 Major1 4951870 John Doe s3://acme- pics/ 4951870.p ng 1200 124A Eng Software Engineer II MS Harvard Computer Science 9765207 Jane Smith s3://acme- pics/ 9765207.p ng 1300 187D BizOps Sr Operations Associate BS Stanford Physics 4150915 Shyam Bhatt s3://acme- pics/ 4150915.p ng 45 432C Sales Enterprise Sales Assoc MBA Penn Accountin g 7566243 Kathryn Bates s3://acme- pics/ 7566243.p ng 44 334B Eng Staff Software Engineer PhD UCB Computer Science
- 22. Think of a Relational DB Query EmpID Name PictureRef 4951870 John Doe s3://acme-pics/ 4951870.png 9765207 Jane Smith s3://acme-pics/ 9765207.png 4150915 Shyam Bhatt s3://acme-pics/ 4150915.png 7566243 Kathryn Bates s3://acme-pics/ 7566243.png EmpID Manager ID StartDate EndDate 4951870 9765207 20170101 null 9765207 7566243 20150130 null 4150915 8795882 20141215 20150312 7566243 8509238 20120605 20140124 EmpID Building Office 4951870 1200 124A 9765207 1300 187D 4150915 45 432C
- 23. Think of a Relational DB Query EmpID Name PictureRef 4951870 John Doe s3://acme-pics/ 4951870.png 9765207 Jane Smith s3://acme-pics/ 9765207.png 4150915 Shyam Bhatt s3://acme-pics/ 4150915.png 7566243 Kathryn Bates s3://acme-pics/ 7566243.png EmpID Manager ID StartDate EndDate 4951870 9765207 20170101 null 9765207 7566243 20150130 null 4150915 8795882 20141215 20150312 7566243 8509238 20120605 20140124 EmpID Building Office 4951870 1200 124A 9765207 1300 187D 4150915 45 432C
- 24. Think of a Relational DB Query EmpID Name PictureRef 4951870 John Doe s3://acme-pics/ 4951870.png 9765207 Jane Smith s3://acme-pics/ 9765207.png 4150915 Shyam Bhatt s3://acme-pics/ 4150915.png 7566243 Kathryn Bates s3://acme-pics/ 7566243.png EmpID Manager ID StartDate EndDate 4951870 9765207 20170101 null 9765207 7566243 20150130 null 4150915 8795882 20141215 20150312 7566243 8509238 20120605 20140124 EmpID Building Office 4951870 1200 124A 9765207 1300 187D 4150915 45 432C 16+ Index Lookups Expensive!
- 25. (Partial) Graph View 1 Index Lookup (find :Employee nodes) Then Index-Free Adjacency :Employee {id:4951870} :Employee {id:9765207} :Office {id: 1200124a} :Building {id: 1200} [:IS_MANAGED_BY] [:HAS_OFFICE] [:LOCATED_IN]
- 27. A Naturally Adaptive Model A Query Language Designed for Connectedness + =Agility
- 28. Cypher Typical Complex SQL Join The Same Query using Cypher MATCH (boss)-[:MANAGES*0..3]->(sub), (sub)-[:MANAGES*1..3]->(report) WHERE boss.name = “John Doe” RETURN sub.name AS Subordinate, count(report) AS Total Project Impact Less time writing queries Less time debugging queries Code that’s easier to read
- 29. ABOUT ME • Developed web apps for 5 years including e-commerce, business workflow, more. • Worked at Google for 8 years on Google Apps, Cloud Platform • Technologies: Python, Java, BigQuery, Oracle, MySQL, OAuth ryan@neo4j.com @ryguyrg
- 30. NEO4j USE CASES Real Time Recommendations Master Data Management Fraud Detection Identity & Access Management Graph Based Search Network & IT-Operations
- 31. NEO4j USE CASES Real Time Recommendations Master Data Management Fraud Detection Identity & Access Management Graph Based Search Network & IT-Operations GRAPH THINKING: Real Time Recommendations VIEWED VIEWED BOUGHT VIEWED BOUGHT BOUGHT BOUGHT BOUGHT
- 32. “As the current market leader in graph databases, and with enterprise features for scalability and availability, Neo4j is the right choice to meet our demands.” Marcos Wada Software Developer, Walmart NEO4j USE CASES Real Time Recommendations Master Data Management Fraud Detection Identity & Access Management Graph Based Search Network & IT-Operations
- 33. NEO4j USE CASES Real Time Recommendations Master Data Management Fraud Detection Identity & Access Management Graph Based Search Network & IT-Operations GRAPH THINKING: Master Data Management MANAGES MANAGES LEADS REGION M ANAG ES MANAGES REGION LEADS LEADS COLLABORATES
- 34. Neo4j is the heart of Cisco HMP: used for governance and single source of truth and a one-stop shop for all of Cisco’s hierarchies. NEO4j USE CASES Real Time Recommendations Master Data Management Fraud Detection Identity & Access Management Graph Based Search Network & IT-Operations
- 35. NEO4j USE CASES Real Time Recommendations Master Data Management Fraud Detection Identity & Access Management Graph Based Search Network & IT-Operations GRAPH THINKING: Master Data Management Solu%on Support Case Support Case Knowledge Base Ar%cle Message Knowledge Base Ar%cle Knowledge Base Ar%cle Neo4j is the heart of Cisco’s Helpdesk Solution too.
- 36. NEO4j USE CASES Real Time Recommendations Master Data Management Fraud Detection Identity & Access Management Graph Based Search Network & IT-Operations GRAPH THINKING: Fraud Detection O PENED_ACCO UNT HAS IS_ISSUED HAS LIVES LIVES IS_ISSUED OPENED_ACCOUNT
- 37. “Graph databases offer new methods of uncovering fraud rings and other sophisticated scams with a high-level of accuracy, and are capable of stopping advanced fraud scenarios in real-time.” Gorka Sadowski Cyber Security Expert NEO4j USE CASES Real Time Recommendations Master Data Management Fraud Detection Identity & Access Management Graph Based Search Network & IT-Operations
- 38. GRAPH THINKING: Graph Based Search NEO4j USE CASES Real Time Recommendations Master Data Management Fraud Detection Identity & Access Management Graph Based Search Network & IT-Operations PUBLISH INCLUDE INCLUDE CREATE CAPTURE IN IN SOURCE USES USES IN IN USES SOURCE SOURCE
- 39. Uses Neo4j to manage the digital assets inside of its next generation in-flight entertainment system. NEO4j USE CASES Real Time Recommendations Master Data Management Fraud Detection Identity & Access Management Graph Based Search Network & IT-Operations
- 40. NEO4j USE CASES Real Time Recommendations Master Data Management Fraud Detection Identity & Access Management Graph Based Search Network & IT-Operations BROWSES CONNECTS BRIDGES ROUTES POWERS ROUTES POWERS POWERS HOSTS QUERIES GRAPH THINKING: Network & IT-Operations
- 41. Uses Neo4j for network topology analysis for big telco service providers NEO4j USE CASES Real Time Recommendations Master Data Management Fraud Detection Identity & Access Management Graph Based Search Network & IT-Operations
- 42. GRAPH THINKING: Identity And Access Management NEO4j USE CASES Real Time Recommendations Master Data Management Fraud Detection Identity & Access Management Graph Based Search Network & IT-Operations TRUSTS TRUSTS ID ID AUTHENTICATES AUTHENTICATES O W NS OWNS CAN_READ
- 43. UBS was the recipient of the 2014 Graphie Award for “Best Identify And Access Management App” NEO4j USE CASES Real Time Recommendations Master Data Management Fraud Detection Identity & Access Management Graph Based Search Network & IT-Operations
- 44. Neo4j Adoption by Selected Verticals SOFTWARE FINANCIAL SERVICES RETAIL MEDIA & BROADCASTING SOCIAL NETWORKS TELECOM HEALTHCARE
- 45. AGENDA • Use Cases • SQL Pains • Building a Neo4j Application • Moving from RDBMS -> Graph Models • Walk through an Example • Creating Data in Graphs • Querying Data
- 47. SQL Day in the Life of a RDBMS Developer
- 55. SELECT p.name, c.country, c.leader, p.hair, u.name, u.pres, u.state FROM people p LEFT JOIN country c ON c.ID=p.country LEFT JOIN uni u ON p.uni=u.id WHERE u.state=‘CT’
- 61. • Complex to model and store relationships • Performance degrades with increases in data • Queries get long and complex • Maintenance is painful SQL Pains
- 62. • Easy to model and store relationships • Performance of relationship traversal remains constant with growth in data size • Queries are shortened and more readable • Adding additional properties and relationships can be done on the fly - no migrations Graph Gains
- 64. What does this Graph look like?
- 66. Property Graph Model CREATE (:Person { name:“Dan”} ) - [:LOVES]-> (:Person { name:“Ann”} ) LOVES LABEL PROPERTY NODE NODE LABEL PROPERTY
- 68. MATCH (p:Person)-[:WENT_TO]->(u:Uni), (p)-[:LIVES_IN]->(c:Country), (u)-[:LED_BY]->(l:Leader), (u)-[:LOCATED_IN]->(s:State) WHERE s.abbr = ‘CT’ RETURN p.name, c.country, c.leader, p.hair, u.name, l.name, s.abbr
- 71. How do you use Neo4j? CREATE MODEL + LOAD DATA QUERY DATA
- 72. How do you use Neo4j?
- 74. How do you use Neo4j?
- 77. Java Stored Procedures and Functions
- 81. Architectural Options Data Storage and Business Rules Execu5on Data Mining and Aggrega5on Applica'on Graph Database Cluster Neo4j Neo4j Neo4j Ad Hoc Analysis Bulk Analy'c Infrastructure Hadoop, EDW … Data Scien'st End User Databases Rela5onal NoSQL Hadoop
- 82. RDBMS to Graph Options MIGRATE ALL DATA MIGRATE SUBSET DUPLICATE SUBSET Non-Graph Queries Graph Queries Graph Queries Non-Graph Queries All Queries Rela3onal Database Graph Database Application Application Application Non Graph Data All Data
- 83. FROM RDBMS TO GRAPHS
- 86. Northwind
- 87. Northwind - the canonical RDBMS Example
- 91. Starting with the ER Diagram
- 92. Locate the Foreign Keys
- 93. Drop the Foreign Keys
- 94. Find the JOIN Tables
- 95. (Simple) JOIN Tables Become Relationships
- 96. Attributed JOIN Tables -> Relationships with Properties
- 97. Querying a Subset Today
- 98. As a Graph
- 100. using openCypher
- 101. Property Graph Model CREATE (:Employee{ firstName:“Steven”} ) -[:REPORTS_TO]-> (:Employee{ firstName:“Andrew”} ) REPORTS_TO Steven Andrew LABEL PROPERTY NODE NODE LABEL PROPERTY
- 102. Who do people report to? MATCH (e:Employee)<-[:REPORTS_TO]-(sub:Employee) RETURN *
- 103. Who do people report to?
- 104. Who do people report to? MATCH (e:Employee)<-[:REPORTS_TO]-(sub:Employee) RETURN e.employeeID AS managerID, e.firstName AS managerName, sub.employeeID AS employeeID, sub.firstName AS employeeName;
- 105. Who do people report to?
- 106. Who does Robert report to? MATCH p=(e:Employee)<-[:REPORTS_TO]-(sub:Employee) WHERE sub.firstName = ‘Robert’ RETURN p
- 107. Who does Robert report to?
- 108. What is Robert’s reporting chain? MATCH p=(e:Employee)<-[:REPORTS_TO*]-(sub:Employee) WHERE sub.firstName = ‘Robert’ RETURN p
- 109. What is Robert’s reporting chain?
- 110. Who’s the Big Boss? MATCH (e:Employee) WHERE NOT (e)-[:REPORTS_TO]->() RETURN e.firstName as bigBoss
- 111. Who’s the Big Boss?
- 112. Product Cross-Selling MATCH (choc:Product {productName: 'Chocolade'}) <-[:INCLUDES]-(:Order)<-[:SOLD]-(employee), (employee)-[:SOLD]->(o2)-[:INCLUDES]->(other:Product) RETURN employee.firstName, other.productName, COUNT(DISTINCT o2) as count ORDER BY count DESC LIMIT 5;
- 114. (ASIDE ON GRAPH COMPUTE)
- 115. Shortest Path Between Airports MATCH p = shortestPath( (a:Airport {code:”SFO”})-[*0..2]-> (b:Airport {code: “MSO”})) RETURN p
- 116. (END ASIDE ON GRAPH COMPUTE)
- 117. POWERING AN APP
- 118. Simple App
- 119. Simple App
- 120. Simple Python Code
- 121. Simple Python Code
- 122. Simple Python Code
- 123. Simple Python Code
- 124. LOADING OUR DATA
- 125. CSV
- 126. CSV files for Northwind
- 127. CSV files for Northwind
- 128. 3 Steps to Creating the Graph IMPORT NODES CREATE INDEXES IMPORT RELATIONSHIPS
- 129. Importing Nodes // Create customers USING PERIODIC COMMIT LOAD CSV WITH HEADERS FROM "https:// raw.githubusercontent.com/neo4j-contrib/developer-resources/ gh-pages/data/northwind/customers.csv" AS row CREATE (:Customer {companyName: row.CompanyName, customerID: row.CustomerID, fax: row.Fax, phone: row.Phone}); // Create products USING PERIODIC COMMIT LOAD CSV WITH HEADERS FROM "https:// raw.githubusercontent.com/neo4j-contrib/developer-resources/ gh-pages/data/northwind/products.csv" AS row CREATE (:Product {productName: row.ProductName, productID: row.ProductID, unitPrice: toFloat(row.UnitPrice)});
- 130. Importing Nodes // Create suppliers USING PERIODIC COMMIT LOAD CSV WITH HEADERS FROM "https:// raw.githubusercontent.com/neo4j-contrib/developer-resources/ gh-pages/data/northwind/suppliers.csv" AS row CREATE (:Supplier {companyName: row.CompanyName, supplierID: row.SupplierID}); // Create employees USING PERIODIC COMMIT LOAD CSV WITH HEADERS FROM "https:// raw.githubusercontent.com/neo4j-contrib/developer-resources/ gh-pages/data/northwind/employees.csv" AS row CREATE (:Employee {employeeID:row.EmployeeID, firstName: row.FirstName, lastName: row.LastName, title: row.Title});
- 131. Creating Relationships USING PERIODIC COMMIT LOAD CSV WITH HEADERS FROM "https:// raw.githubusercontent.com/neo4j-contrib/developer-resources/ gh-pages/data/northwind/orders.csv" AS row MATCH (order:Order {orderID: row.OrderID}) MATCH (customer:Customer {customerID: row.CustomerID}) MERGE (customer)-[:PURCHASED]->(order); USING PERIODIC COMMIT LOAD CSV WITH HEADERS FROM "https:// raw.githubusercontent.com/neo4j-contrib/developer-resources/ gh-pages/data/northwind/products.csv" AS row MATCH (product:Product {productID: row.ProductID}) MATCH (supplier:Supplier {supplierID: row.SupplierID}) MERGE (supplier)-[:SUPPLIES]->(product);
- 132. Creating Relationships USING PERIODIC COMMIT LOAD CSV WITH HEADERS FROM "https://raw.githubusercontent.com/neo4j- contrib/developer-resources/gh-pages/data/northwind/orders.csv" AS row MATCH (order:Order {orderID: row.OrderID}) MATCH (product:Product {productID: row.ProductID}) MERGE (order)-[pu:INCLUDES]->(product) ON CREATE SET pu.unitPrice = toFloat(row.UnitPrice), pu.quantity = toFloat(row.Quantity); USING PERIODIC COMMIT LOAD CSV WITH HEADERS FROM "https://raw.githubusercontent.com/neo4j- contrib/developer-resources/gh-pages/data/northwind/orders.csv" AS row MATCH (order:Order {orderID: row.OrderID}) MATCH (employee:Employee {employeeID: row.EmployeeID}) MERGE (employee)-[:SOLD]->(order);
- 133. High Performance LOADing neo4j-import 4.58 million things and their relationships… Loads in 100 seconds!
- 134. JDBC apoc.load.jdbc
- 139. WRAPPING UP
- 140. “We found Neo4j to be literally thousands of times faster than our prior MySQL solution, with queries that require 10 to 100 times less code. Today, Neo4j provides eBay with functionality that was previously impossible.” Volker Pacher Senior Developer
- 141. THANK YOU! Ryan Boyd @ryguyrg ryan@neo4j.com