The path to success with graph database and graph data science_ Neo4j GraphSummit 2023 Dublin.pptx
•Descargar como PPTX, PDF•
0 recomendaciones•98 vistas
What’s new, and what’s next? Product innovation moves rapidly at Neo4j – learn how graph technology can provide you with the tools to get much more from your data!
Recomendados
Recomendados
Más contenido relacionado
La actualidad más candente
La actualidad más candente (20)
Similar a The path to success with graph database and graph data science_ Neo4j GraphSummit 2023 Dublin.pptx
Similar a The path to success with graph database and graph data science_ Neo4j GraphSummit 2023 Dublin.pptx (20)
Más de Neo4j
Más de Neo4j (20)
Último
Último (20)
The path to success with graph database and graph data science_ Neo4j GraphSummit 2023 Dublin.pptx
- 1. © 2023 Neo4j, Inc. All rights reserved. © 2023 Neo4j, Inc. All rights reserved. The Path To Success With Graph Database and Data Science Michael Down 1
- 2. © 2023 Neo4j, Inc. All rights reserved. © 2022 Neo4j, Inc. All rights reserved. Neo4j Graph Data Platform 2 BUSINESS USERS DEVELOPERS DATA SCIENTISTS DATA ANALYSTS Enterprise Ready Data Science & MLOps Graph Data Science OLAP Data Science and Analytics Tools, algorithms, and Integrated ML framework AutoML Integrations Discovery & Visualization Low-code querying, data modeling and exploration tools Neo4j Bloom BI Connectors Neo4j Browser Language interfaces Application Development Tools & Frameworks Tools and APIs for rapid prototyping and development Graph Query Language Cypher and GQL as the lingua franca for graphs Transactions Analytics Graph Database Data Consolidation Contextualization OLTP Native Graph Database The core component of Neo4j platform Runs Anywhere Run by yourself or as DBaaS by Neo4j, in the cloud or on premises Data Connectors Ecosystem & Integrations Rich set of connectors to plug into existing data ecosystems Data Sources
- 3. © 2023 Neo4j, Inc. All rights reserved. Engineering Expertise >1000 person-years investment First mover advantage Maturity, Most enterprise deployments Largest graph community Growing at 80%+ annually Neo4j Graph Database Capabilities Hybrid Workloads Native Graph Architecture Powers Graph Data Science Rich Toolset Enterprise Trust Runs Anywhere 3
- 4. © 2023 Neo4j, Inc. All rights reserved. 4 Native Graph Architecture Native Graph Storage Native Graph Processing • No mismatch • Data integrity / ACID • Schema flexible • 1000x faster than relational • K-Hop now 10-1000x faster than version 4 Fabric • Federation of scaled out shards • Instant composite database Composite DB Autonomous Clustering • Elastic scale-out for high throughput • 100s of machines across clusters Data integrity and high speed also true in scaled out situations
- 5. © 2023 Neo4j, Inc. All rights reserved. Hybrid Workload Duality 5 Intelligent Applications Transactions - Security - Performance & Scalability - ACID Consistency - Intelligent Modeling - Extensive & Supported Algo Library - Scalable - Graph Visualization - Graph Transformations Graph Transactions Graph Analytics & Data Science
- 6. © 2023 Neo4j, Inc. All rights reserved. Powers Neo4j Graph Data Science Graph Data Science MACHINE LEARNING Analytics Feature Engineering Data Exploration Graph Data Science TensorFlow KNIME Python 6 Project your graph for in-memory analytics ● Unparalleled analytical processing ● .. with 60+ Algorithms for predictive analytics ● .. and pipeline to supervised AI/ML models ● Making AI smarter!
- 7. © 2023 Neo4j, Inc. All rights reserved. Developer Productivity: Rich tooling and easy onramp ops manager 7 data importer Visualize and explore your data Query editor and results visualizer Code-free data loader and modeler AuraWorkspace Unified Workspace
- 8. © 2023 Neo4j, Inc. All rights reserved. 8 Plugs into your data and development ecosystem Neo4j BI Connector Apache Spark Connector Apache Kafka Connector Data Warehouse Connector Java Python .NET JavaScript Go
- 9. © 2023 Neo4j, Inc. All rights reserved. Enterprise-Grade: Security and Trust Built In Single Sign-On Secure Development Practices Dedicated VPC Role- & Schema-Based Access Control Encryption (At-Rest, In-Transit, and Intra Cluster) SOC 2 Type 1 9
- 10. © 2023 Neo4j, Inc. All rights reserved. ● Real-time Performance at Scale ● Automatic Upgrades, Patches, Backups ● Scale on Demand, No Downtime ● High Availability ● Multi Cloud, Any Region ● Enterprise-grade Security ● Simple Capacity-Based Pricing 10 Run Anywhere: self managed, or by Neo4j ● Full administrative control ● On-premises or via cloud marketplace ● Fit where cloud isn’t appropriate (e.g. special compliance scenarios) ● Easy migration to AuraDB Self-Managed
- 11. © 2023 Neo4j, Inc. All rights reserved. Forward looking investments Developer Experience Complete multi-cloud availability AuraDB on Azure in addition to GCP, AWS Making graph ubiquitous with GQL compliance Programmatic management and monitoring with APIs for AuraDB Solidifying Neo4j as the data store of record: CDC + next-gen Kafka connector Theme: the first-choice and primary database that graph-powers any application Performance at Scale Analytic step-up performance with Parallel Cypher Queries Improved mem-to-storage ratio / Lower TCO with Freki next-gen storage Even more autonomous clustering with declarative server management Operational Trust Better monitoring and tuning with query analyzer in Neo4j Ops Manager Integrated observability with AuraDB metrics and log streaming Customer managed security in AuraDB with customer managed encryption keys and customer managed RBAC
- 12. © 2023 Neo4j, Inc. All rights reserved. © 2023 Neo4j, Inc. All rights reserved. Neo4j Graph Data Science
- 13. © 2023 Neo4j, Inc. All rights reserved. What’s important? Prioritization Who has the most connections? Who has the highest page rank? Who is an influencer? What’s unusual? Anomaly & Fraud Detection Where is a community forming? What are the group dynamics? What’s unusual about this data? What’s next? Predictions What’s the most common path? Who is in the same community? What relationship will form? 13 Graph Structure Improves Data Science Outcomes
- 14. © 2023 Neo4j, Inc. All rights reserved. And created Neo4j Graph Data Science: Eliminate Pain & Optimize Data Science Workflows with the data you already have Eliminates Pain Optimizes Data Science Flows Complex joins operations Mining Multiple Tables Tedious Manual Approximations Brute Force Comparisons Fractured Data 95% reduction in computation time 500x faster than open source libraries Improves Customer Outcomes 20-30% improvement in model performance 600% improvement in traffic $5 Million of additional fraud detected 3x better churn predictability 5x reduction in factory production lead time 14
- 15. © 2023 Neo4j, Inc. All rights reserved. 15 Data Scientists > Native Python Client > Apache Arrow integration > Unified ML pipelines We invest in four key areas Built by data scientists, for data scientists Better Predictions > 65+ Graph algorithms & embeddings > Graph native ML Pipelines > Vertex AI & SageMaker Integrations The best graph data science and ML engine Ecosystem > Apache Spark & Kafka Connectors > Native BI Connector > Data Warehouse Connector > GNN library support Seamlessly works with your data stack and pipeline Production Ready > Compatible with all major clouds > Enterprise Scale & Security > Deploy anywhere Go to production with speed, scale, and security
- 16. © 2023 Neo4j, Inc. All rights reserved. 16 With The Largest Catalog of Graph Algorithms Pathfinding & Search Centrality & Importance Community Detection Supervised Machine Learning Heuristic Link Prediction Similarity Graph Embeddings …and more Graph algorithms are a set of instructions that visit the nodes of a graph to analyze the relationships in connected data.
- 17. © 2023 Neo4j, Inc. All rights reserved. And Full Support Across the entire ML Lifecycle Feature Engineering Model Training & Tuning Model Deployment Data Collection & Preparation Exploratory Data Analysis Model Evaluation & Selection Drivers, Connectors, Fast Import/Export Graph Queries, Algorithms, and Visualization Graph Embeddings & Algorithms Predict APIs, Model/Graph Catalog Operations, Connectors Graph Native ML Pipelines Unsupervised Graph Algorithms Graph Features -> External ML Pipelines 17
- 18. © 2023 Neo4j, Inc. All rights reserved. And made it seamless for all ecosystems and pipelines Graph Data Science BI & VISUALIZATIONS INGEST STORE PROCESS Apache Kafka MACHINE LEARNING Cloud Functions Neo4j Bloom PubSub DataProc Analytics Feature Engineering Data Exploration Graph Data Science Business Applications & Existing Systems Files (unstructured, structured) TensorFlow KNIME Python Cloud Storage AWS Lambda 18 Graph Database
- 19. © 2023 Neo4j, Inc. All rights reserved. View the most well connected and influential nodes Recommendations from shared user interactions and associations Our Visualizations Make analysis easy to understand 19
- 20. © 2023 Neo4j, Inc. All rights reserved. 20 What’s in it for you: ● Improve model accuracy by 30% ● Simplify processes and remove headaches ● More projects into production without additional hiring Neo4j Graph Data Science Analytics Feature Engineering Data Exploration Graph Data Science Queries & Search Machine Learning Visualization
- 21. © 2023 Neo4j, Inc. All rights reserved. 21 Customer Case Study: Fraud Detection Correctly identify account holders committing fraud Results: ● 300% increase in fraud detection ● 10% true positive escalations (industry standard < 1%) ● Reduced false positive escalations ● 150% increase in payment flow
- 22. © 2023 Neo4j, Inc. All rights reserved. 22 How to get started… 3. Graph Native Machine Learning Learn features in your graph that you don’t even know are important yet using embeddings. Predict links, labels, and missing data with in-graph supervised ML models. Identify associations, anomalies, and trends using unsupervised machine learning. 2. Graph Algorithms 1. Knowledge Graphs Find the patterns you’re looking for in connected data
- 23. © 2023 Neo4j, Inc. All rights reserved. 23 What’s New in Graph Data Science
- 24. © 2023 Neo4j, Inc. All rights reserved. Algos & Embeddings HashGNN Embedding: Faster approach than GNNs for knowledge graphs KMeans Cluster data based on properties like graph embeddings Leiden Algorithm: Fast and scalable modularity based community detection Image courtesy of: Traag, V.A., Waltman, L. & van Eck, N.J. Image courtesy of: javatpoint.com Leiden Algorithm: K-means Clustering: 24
- 25. © 2023 Neo4j, Inc. All rights reserved. ML Pipelines Autotuning: Find optimal hyperparameters to improve model performance Multilayer Perceptrons (MLPs): Fully connected neural networks now available for Link Prediction and Node Classification 25
- 26. © 2023 Neo4j, Inc. All rights reserved. GNN Support Graph Sampling: sample a representative subgraph from a larger graph for training complex models Graph Export: use our projections in other graph ML libraries like Deep Graph Library (DGL), PyG, and Tensorflow GNN Image courtesy of Google Cloud 26
- 27. © 2023 Neo4j, Inc. All rights reserved. 27 Other Data Stores Transactions Analytics Graph Database Graph Data Science Integrated AI/Machine Learning Data Integrations & Connectors Admin Cypher Drivers & APIs Dev Tools Application Layer: Digital Twin, Recommendation, Fraud Detection, Cybersecurity, … Query Browser GraphQL Analytics & AI/Machine Learning Pipelines The Neo4j Graph Data Platform Flexible Graph Schema Performance, Reliability & Integrity Scale-Up & Scale-Out Architecture Development Tools Breadth Enterprise Ecosystem
- 28. © 2023 Neo4j, Inc. All rights reserved. © 2023 Neo4j, Inc. All rights reserved. 28 THANK YOU
Notas del editor
- At the heart -- Neo4j Graph Database Development tools for transactional use cases and applications Data Science workbench for analytics and ML workloads Built in data visualization Languages and APIs Connectors to move data in and out, streaming or bulk Optimised infrastructure environment All of this designed to run anywhere -- desktop, on-premises, hybrid or public cloud, available as a cloud service ---------------------------------------------------- At the heart of our platform, of course is our native graph database on which serves as the foundation of all the capabilities tools for discovery and visualization and code free/low code tooling (for devs and data scientists) data visualization tools helped executives understand intricacies within supply chains (inc covd vaccine distribution) Plug and play with on machine learning frameworks: Sage maker or Vertex AI an other toolkits like tensor flow, etc. Neo4j is Google’s graph database partner and GV is an investor in Neo4j Friction-free graph data science - more time experimenting, discovering value, less time wrangling data. Cypher, openCypher, GQL. Neo4j is the leader in graph query languages and maintains that leadership in driving standards for the industry. GQL is the same people as SQL - this is a big deal! Ensures value in learning is maintained, simplifies systems. Tools and connectors, kafka, JDBC, snowflake, data warehouses, etc Development tools and frameworks so you can use the platform productively, natively in your chosen environment Drivers for all languages OGMs for Spring, Java, Python, etc
- [intro slide for the next set of slides] The platform is built on some key technology innovations that make us unique in the industry. We’re going to talk about each capability in the following slides
- Native graph storage means there is no mismatch between the business domain and the data model as it is stored. What you see on the white board is what you get. Data is stored in an ACID compliant way, making us fit as a primary transactional database because we ensure there is no data is consistent and without loss The flexibility of the schema allows you to expand the data model as business requirements evolve. Native graph processing means lightning fast retrieval of data due to our ability to traverse nodes and relationships. We are continuing to improve our significant performance advantage over relational databases. For example, the recent version 5 has enhanced multiple hop queries to be even faster than what we had in version 4. Version 5 also saw significant improvements in the ability to scale out. The millisecond performance response time, and the ACID compliant data integrity are maintained as we allow you to scale out elastically across clusters using Autonomous Clustering, one of the most advanced clustering architectures of any database. You can also take advantage sharding to deal with very large data sets more efficiently, and use Fabric to treat the entire data set as if they belonged to a single graph.
- “Wave-particle duality” analogy Can be used for transactions AND analytics If you develop your modern applications on Neo4j, you will reap the benefits multiple times down the road. You have only 1 database, not 2 separate databases with the headache associated with ETL (extraction, transformation, and loading) Only 1 data model to deal with. Unlike in relational or other NoSQL when you have 1 DB for operational and another for complex queries
- Ne4j graph database is at the heart of our our product, graph data science. It allows you to run powerful, in-memory analytics with unparalleled performance and scope We also provide a pipeline to supervised AI/ML models, enriching data used for AI and making predictions [for full story, stay tuned for Graph Data Science section]
- The following three have been consolidated into the Workspace (for AuraDB): Neo4j Data Importer allows you to import your data for modeling purposes. And quickly test your data model against real data before you start developing. Neo4j Browser is a developer-focused tool to write Cypher queries and explore your Neo4j graph database. It’s a great way to learn about Neo4j and Cypher and review results in both illustrative and tabular form – all from within your browser. Neo4j Bloom is our graph data visualization tool that works out of the box with your Neo4j database. Developers, Data Scientists, and Data Analysts can explore nodes, relationships, and graph patterns with a natural type-to-search interface. Bloom also helps to boost cross-team collaboration with codeless search-to-story design so that anyone can run simple pre-canned queries without having to learn Cypher and most recently, we’ve recently integrated our Graph Data Science library with Bloom available now in AuraDS – other tools With Neo4j Desktop, develop apps with your favorite programming language like GraphQL, Node.js, Python, Go, Java, .Net, PHP and more. Connect and query remote Neo4j databases anywhere: On-prem, Self-hosted on any cloud, fully managed via Neo4j Aura and create unlimited local databases in a project based workspace And lastly, Neo4j GraphQL Library — an official library for building GraphQL APIs that bring the power of graphs to GraphQL. The low-code aspect of the library enables developers to use their GraphQL expertise to rapidly prototype building applications without the need to first learn Cypher or code Cypher into the front-end of the application. It’s open source and we’re getting awesome contributions from the developer community. Neo4j Ops Manager allows admins to manage all self-managed Neo4j databases, instances, and clusters; and monitor them from a single UI based console. Additional Notes (If time allows) Desktop * Gives you a single user development license for Neo4j Enterprise in an easy to manage single installation - no need to manage Java or neo4j installs yourself. * Organise your different Neo4j installations / versions for different projects * Provides tooling to make your development experience easier, including easier access to logs, configuration files, filesystem and plugin management * Gives you easy access to Neo4j's latest releases of Browser and Bloom as well as third party graph applications * Primarily intended for local development, but also allows you to connect to remote instances Browser * The best place to author Cypher queries and inspect query returns * Assistance writing queries provides autocompletion of labels, rel types and properties * Visualise your query returns in the most appropriate way - either as tabular results or as a graph * Inspect query plans and profile queries to understand how they can be optimised * A great place to test parameterising of your queries * Gain insights into high level database information (label and rel type counts) and review running queries. * Learn about the graph database and Cypher query language via interactive guided content
- Spend 10 seconds on this slide. Say we are always up to date on the latest compliance and security measures. Most recently we got SOC2 Type 2 certified. TLS 1.2/1.3, with encrypted bucket storage Okta, Active Directory, Google *& Others AWS PrivateLink Private Service Connect GCP - Node, Edge, and Traversal Level Granularity Cyber Academy Code Scanning & Pentesting CCPA GDPR ISO 27001 Soc 2 Type 1 And more recently Type 2
- Available as a fully managed cloud database service in addition to self managed. On the cloud of your choice, or on your premises. Completely automatic provisioning, upgrades and patches Scale on demand, with just a click Highly available Secure and reliable Simple consumption based pricing AuraDB has a free tier you can get started with now!
- THEME / Our vision. First DB means first choice DB, ie, Neo4j is among the leading databases developers and architects think of when developing any application, not just your traditional use cases. Primary DB means Neo4j is the system of record, not just a secondary db for fast queries. And by using Neo4j as the primary DB, you get dividends down the road such as the ability to execute fast complex queries, perform analytics, and complement AI/ML systems. Dev experience In order to promote Neo4j as the first choice database among developers, there are enhancements that we need to develop in order to make our vision a reality. Developers want to develop and operate from / in the cloud. To that end we are making AuraDB available in Azure in addition to GCP and AWS Neo4j engineers constitute the vast majority of contributors to GQL, the open source standard that is emerging as the equivalent of SQL for graph. This will promote adoption of GQL and graph technology, and Neo4j! Developers are interested in how Neo4j improves their productivity or capability. For example, we could pluck out "graph element types" from the GQL standard and describe how that enables composable schema -- the data equivalent of reusable software components, an evergreen DX theme. Developers love APIs. We are working on database management APIs for AuraDB. Performance at Scale In order to ensure architects and developers feel comfortable about adopting Neo4j as the primary DB, there are a few important things we are working on: CDC supports Neo4j as the primary DB, since it allows developers to extract data out of Neo4j to the target systems for other purposes.This is a feature asked by major customers like Tesla, for example. Parallel Cypher Queries solidifies our positioning as being very fast not only for transactional application data, but also for users performing analytical queries (before you even use Data Science). By using Neo4j as a primary DB, you get free analytical capabilities that you otherwise would not get if you used a relational database. We also want to keep the cost of using Neo4j in the cloud reasonable for you, and frankly for us. That is what the Freki project is for. Declarative server management is the ability to tag servers with labels, which you can use to declare certain autonomous clustering rules are applicable to those servers labeled a certain way. Even easier scale-out with Autonomous Clustering Operations And if Neo4j is going to be the primary database, we need to ensure its operations are even more solid than it already is: For self managed Neo4j, the Ops Manager is being enhanced with a query analyzer so you can better understand your queries and fine tune them With log and metrics streaming we’re providing you a better way to integrate Neo4j monitoring to your existing observability infrastructure Finally, for AuraDB, we are providing a couple enhancements that allow you to more easily self manage certain security aspects.
- Graph Structures Improve Data Science outcomes. This is because information that is traditionally trapped in rows and columns can be easily uncovered in a graph. This is especially true for use cases that focus on prioritization, anomaly and fraud detection, and predictions because of the connected nature of the use cases. So when data science, teams are looking to answer the questions of what's important, what's unusual and what's coming next, choosing a graph structure can improve their outcomes. We see these three big questions across departments and industries - whether it is financial services firms trying to identify fraud, customer service looking for the most important support article, , people teams looking to recommend training and upskilling, or retailers looking to recommend products - graph structure strengthens the analysis for all of these - and other use cases. Additional Notes: What’s Important? Prioritization Listen for words like: Best Top performing Highest converting Most challenging What’s Unusual? Anomaly and Fraud Detection Listen for words around behavior like: Unusual Anomalous Strange Odd Weird What’s next? Predictions Listen for words like: Recommend Optimize Improve Likely
- Graph Data Science Simplifies and Optimizes Data Science workflows by eliminating a lot of the tedious work as well as providing improvements in model accuracy and performance. So the graph structure lends itself to use cases where relationships matter - and Neo4j Graph Data Science further improves the outcomes and simplifies the process. Because the data is no longer stuck in rows and columns, uncovering the hidden relationships becomes much easier and the analysis is much better. To start with, let’s talk about how we simplify the work the data science teams have to do: it eliminates a massive amount of arduous complexity that a data scientist would have to go through to answer those types of questions in a traditional shape. It eliminates the complex joins. And mining multiple tables. The tedious manual approximations the brute force comparisons and managing with fractured data. In addition to making it easier, graph data science, also improves the outcomes of the analysis. We're seeing a 95% reduction in computation time. And we find that this is a 500 times faster approach than using open source libraries. Not only that, but the outcomes are better - 20 to 30 percent improvement in model performance. In specific use cases, our customers are seeing things like a 600% improvement in traffic, to their sites. Five million dollars of additional fraud detection and three times better churn predictability using graph data science. So not only does neo4j graph data science, make it incredibly easy to do this work, it also makes the outcomes of the work better. 600% improvement in traffic = Meredith corp via customer 360/entity resolution $5m fraud = Large TelCo in APAC 3x better churn predictability = Large TelCo in APAC 5x reduction in factory production lead time = global pharmaceutical company
- The only graph data science solution built for data scientists to improve predictions and ML models, at scale, with seamless integration across the data stack to get more data science projects to production.
- Neo4j Graph Data Science provides full ML Pipeline support - both supervised and unsupervised. This coverage makes it seamless for data scientists to use Graph Data Science as they would with traditional data science tools - even bringing graph features into external ML pipelines through embeddings.
- Oftentimes the work of data scientists are ignored or underestimated because the business stakeholders don’t believe the analysis over historic norms or gut feel. Graph also allows the business stakeholders to see how relationships are formed and why how the outcomes are formed. The visual cues provided by a graph help drive adoption.
- Graph Data Science is when you leverage relationships between data points for analytics and ML - not just the features from the data points themselves as is the case with traditional Data Science. Hidden relationships and patterns emerge when data is analyzed in a connected structure - allowing for new insights that help better answer some of today’s most pressing questions. Use graph analytics/algos - - centrality, similarity, etc. But also use graph features to enhance existing ML models Then go full stack for graph machine learning models. Queries: find the patterns you know exist. Machine Learning: feature engineering, and predictions. - graph algorithms, Visualization: explore, collaborate, and explain.
- As with all financial institutions - and actually most institutions - Banking Circle was hoping to improve its ability to identify fraud across transactions. Using algos like centrality, community detection, and similarity, Banking Circle was able to increase their fraud detection by 300% - for other customers of ours, they find fraud in the millions of dollars using these techniques, they were also able to improve true positive escalations, reducing false positive escalations - that means they were better able to predict what was true fraud and what wasn’t. These improved flows and checks led to a 150% increase in payment flow so customers weren’t slowed down by escalations and investigations.