The document discusses data mesh vs data fabric architectures. It defines data mesh as a decentralized data processing architecture with microservices and event-driven integration of enterprise data assets across multi-cloud environments. The key aspects of data mesh are that it is decentralized, processes data at the edge, uses immutable event logs and streams for integration, and can move all types of data reliably. The document then provides an overview of how data mesh architectures have evolved from hub-and-spoke models to more distributed designs using techniques like kappa architecture and describes some use cases for event streaming and complex event processing.
4. What is a Data Mesh?
4
Microservice
Patterns
Log-based
Integrations
Polyglot Data
Movement
Data Mesh is a data-tier architecture to integrate and
govern enterprise data assets across distributed multi-cloud
environments – two defining characteristics are:
(1) De-centralized data processing; no ETL/Hubs/Lake monoliths
(2) Event-driven; real-time where possible, batch only when necessary
Microservices-centric:
• For the administration, deployment and monitoring of the core
frameworks of data movement and governance
• “Sidecar Proxy” style pattern for Events and Data; Aligns with
Service Mesh frameworks (Kubernetes, Istio, etc)
Immutable event-logs for data integrations:
• Messaging and data store events are globally accessible via
immutable event logs
• Logs may be used to drive Streaming or Batch integrations
Distributed data movement of all types of data
• A data mesh moves data: Relational, NoSQL, JSON, Graph…
• Relational data consistency (ACID) during data movement
• Must work reliably with enterprise OLTP data sets
https://en.wikipedia.org/wiki/ACID
Data
Mesh
Event
Streaming
Immutable
Logs
Data
Replication
Polyglot
Persistence
Edge / 5G
Frameworks
Domain
Driven
Design
Service Mesh
“Sidecars”
Data
Mesh
14. Today’s Demo: Retail / Inventory Analysis
Training
Data
Customer
Data
Merchandising
Data
Orders
Data
Data Preparation
Data Science
Data
Flow
Obj
Store
Prepared
Bulk Data
Prepared
Event Data
Autonomous Data Warehouse
Real Time
Analytics
Mobile / SMS
Alerts
Data / Micro
Services
Data
Visualization
ML
Model
Data Catalog
Weather
Data
Analytics Cloud
Real-time Inventory Alerts, Data
Integration, and Predictive Stocking
Self-Service Data Preparation, Data
Integration and Data Visualization
Data Governance, Search and Access
15. Today’s Demo: Retail / Inventory Analysis
Training
Data
Customer
Data
Merchandising
Data
Orders
Data
Data Preparation
Data Science
Data
Flow
Obj
Store
Prepared
Bulk Data
Prepared
Event Data
Autonomous Data Warehouse
Real Time
Analytics
Mobile / SMS
Alerts
Data / Micro
Services
Data
Visualization
ML
Model
OCI Data Catalog
Weather
Data
Analytics Cloud