SlideShare una empresa de Scribd logo
1 de 15
Descargar para leer sin conexión
Designing Master Data Services for
Application Integration
David Loshin
Knowledge Integrity, Inc.
www.knowledge-integrity.com

© 2014 Knowledge Integrity, Inc.
www.knowledge-integrity.com
(301)754-6350

1
Consolidation: The Typical Approach to MDM


Motivating factors have largely focused on consolidation of data
into a single repository rather than how shared master data is
to be used
Customer
Service

Finance

Sales

Marketing

Financial
Data

Customer
Data

Human
Resources

H/R
Data

Sales
Data

Legal
Data

Marketing
Data

Legal

Fulfillment
Data

Centralized
Repository

© 2014 Knowledge Integrity, Inc.
www.knowledge-integrity.com
(301)754-6350

Fulfillment

?
2
Why Consolidation Should Not Drive MDM Integration

Loss of
Knowledge

Loss of
Meaning

Organizational
Misalignment

No Migration
Plan

No Process
Governance

Missing User
Input

“YADS”

Cloud Issues

Master Data
Consolidation

© 2014 Knowledge Integrity, Inc.
www.knowledge-integrity.com
(301)754-6350

Shadow IT

3
Migration Strategy for Sharing Master Data


As the master registry/hub are brought online, there must be plans
for migrating functional applications to the MDM environment to
benefit

Finance

Sales

Marketing

Service

H/R

Transactions

Compliance

H/R
Data

Transaction
Data

Compliance
Data

Master Data

Financial
Data

Sales
Data

Marketing
Data

Service
Data

© 2014 Knowledge Integrity, Inc.
www.knowledge-integrity.com
(301)754-6350

4
Consider Typical Master Data Use Cases




Presumes the existence of a master data repository, index, and
identity resolution capability
Support for operational and analytical activities, such as:







Satisfy data management and governance policies, such as:






Managed assignment of unique identifiers
Search and retrieve unified views of entity information
Managing relationships
Managing identifier cross-reference
Batch matching services
Duplicate analysis and elimination
Data standardization/cleansing

Enable Queries, Reporting, Analysis





Direct queries
JOINs against other resources
Feeding analytics applications
© 2014 Knowledge Integrity, Inc.
www.knowledge-integrity.com
(301)754-6350

5
Master Data Services Stack

Search/Retri
eve Entity

Assign
EID

Enrichment
Reconcile
Identities

Governance
Merge

Standardize/C
leanse

Bulk/Batch
Bulk CrossReference

Identity Services
Split

Bulk
Match/Resol
ution

Relationship
Management

Bulk
Update

Relationship Management

Generate EIDs

Find

ID X-Ref

Break

Relate

Data Management
Search

Retrieve

Update

Create

Deactivate

Matching/Identity Resolution
Data Access
Entity Indexes
Entity Repositories

© 2014 Knowledge Integrity, Inc.
www.knowledge-integrity.com
(301)754-6350

Entity Relationships

6
Master Entity Repositories
Entity Indexes
Entity Repositories



Instantiated repositories






Entity Relationships

Profiles
Relationships
Index

Consider typical data interactions supporting enterprise use
of master data:





Ingestion
Profiling
Probabilistic matching
Internal master data services for management of repository
© 2014 Knowledge Integrity, Inc.
www.knowledge-integrity.com
(301)754-6350

7
Data Access

Data Access



Enable access to shared master data transparently to service
layers





Direct access to physical data repository
Virtualized/federated access to conceptual master repository

Query access



SELECTs
JOINs

© 2014 Knowledge Integrity, Inc.
www.knowledge-integrity.com
(301)754-6350

8
Core Services
Governance
Merge

Identity Services
Split

Relationship Management

Generate EIDs

Find

ID X-Ref

Relate

Break

Data Management
Search

Retrieve

Update

Create

Deactivate

Matching/Identity Resolution



Matching/Identity Resolution




Data management








Algorithms for probabilistic matching
Identity search (probabilistic search, key-based search, similarity scoring)
Register/create a new master entity record
Access master entity record
Update master entity record
Deactivate a master entity record

Identity Services



Guarantee the generation of unique identifiers
Manage cross-reference of internal and external identifiers
© 2014 Knowledge Integrity, Inc.
www.knowledge-integrity.com
(301)754-6350

9
Core Services
Governance

Identity Services

Merge

Split

Relationship Management

Generate EIDs

Find

ID X-Ref

Relate

Break

Data Management
Search

Retrieve

Update

Create

Deactivate

Matching/Identity Resolution



Relationship management






Find all relationships associated with an entity
Relate two or more entities and associate nature of relationship
Break/deactivate a relationship

Governance



Merge two records when they can be determined to represent the same entity
Split a record into multiple records when it can be determined that there is a
false positive

© 2014 Knowledge Integrity, Inc.
www.knowledge-integrity.com
(301)754-6350

10
Application Services

Search/Retri
eve Entity






Bulk
Match/Resol
ution

Invokes core search, retrieve, or create
Provide unique entity identifiers
Data standardization
Alignment of otherwise-assigned identifiers for cross-referencing
Cleansing

Batch Services






Standardize/C
leanse

Bulk/Batch
Bulk CrossReference

Enrichment




Enrichment
Reconcile
Identities

Assign EID




Relationship
Management

Search/Retrieve Entity




Assign
EID

Deliver result set of master records matching a provided set of candidate identities
Apply batch updates
Batch identity resolution

Relationship Management





Find all relationships associated with an entity
Relate two or more entities and associate nature of relationship (e.g. household)
Break/deactivate a relationship
© 2014 Knowledge Integrity, Inc.
www.knowledge-integrity.com
(301)754-6350

11

Bulk
Update
Delivering a Unified View Based on Consumer Rules


Data
Source 1

Master
Data
Index

Collection &
Assembly
Data
Source 2

…
Data
Source N

Virtualized
presentation of
federated data



The master index maps the
canonical representation of a
specific entity (beneficiary,
providers, etc.) to the locations
in the original sources (or
replicas of those source)
holding data instances that
have been linked via the
identity resolution processes
The accessibility of the different
sources and the ability to apply
those business rules at the data
user’s discretion will provide a
consolidated view that best
meets the data users’
requirements at the time the
data is reused

© 2014 Knowledge Integrity, Inc.
www.knowledge-integrity.com
(301)754-6350

12
Master Data Facilitated via Federation/Virtualization
1
2
Master Data Services
3

Standardized Canonical Relational Views

8

4
Transform and Map

Master Index

Federated/Virtualized
5
Data Access

Parse and Standardize,
Normalize, Cleanse

7

© 2014 Knowledge Integrity, Inc.
www.knowledge-integrity.com
(301)754-6350

13
Summary









Review the business process to understand uses of shared
master data
Identify integration points within business processes
Characterize usage scenarios
Determine required functionality
Assess existing systemic support for required functionality
Design application integration service APIs
Design and implement application integration services

© 2014 Knowledge Integrity, Inc.
www.knowledge-integrity.com
(301)754-6350

14
Questions and Open Discussion


www.knowledge-integrity.com



If you have questions,
comments, or suggestions,
please contact me
David Loshin
301-754-6350
loshin@knowledge-integrity.com

www.dataqualitybook.com

www.mdmbook.com

© 2014 Knowledge Integrity, Inc.
www.knowledge-integrity.com
(301)754-6350

15

Más contenido relacionado

La actualidad más candente

La actualidad más candente (19)

PSCU follows PCI Compliance Guidelines for Self-Service BI through Webi - SAB...
PSCU follows PCI Compliance Guidelines for Self-Service BI through Webi - SAB...PSCU follows PCI Compliance Guidelines for Self-Service BI through Webi - SAB...
PSCU follows PCI Compliance Guidelines for Self-Service BI through Webi - SAB...
 
Office 365 security concerns, EU General Data Protection Regulation (GDPR)
Office 365 security concerns, EU General Data Protection Regulation (GDPR) Office 365 security concerns, EU General Data Protection Regulation (GDPR)
Office 365 security concerns, EU General Data Protection Regulation (GDPR)
 
K Ziai Share Point At Ut
K Ziai Share Point At UtK Ziai Share Point At Ut
K Ziai Share Point At Ut
 
Data Loss Prevention in Office 365
Data Loss Prevention in Office 365Data Loss Prevention in Office 365
Data Loss Prevention in Office 365
 
Enacting the Data Subjects Access Rights for GDPR with Data Services and Data...
Enacting the Data Subjects Access Rights for GDPR with Data Services and Data...Enacting the Data Subjects Access Rights for GDPR with Data Services and Data...
Enacting the Data Subjects Access Rights for GDPR with Data Services and Data...
 
Governance and Architecture in Data Integration
Governance and Architecture in Data IntegrationGovernance and Architecture in Data Integration
Governance and Architecture in Data Integration
 
White Paper-1-AnalytiX Mapping Manager-Governance And Architecture In Data In...
White Paper-1-AnalytiX Mapping Manager-Governance And Architecture In Data In...White Paper-1-AnalytiX Mapping Manager-Governance And Architecture In Data In...
White Paper-1-AnalytiX Mapping Manager-Governance And Architecture In Data In...
 
Azure Data Loss Prevention
Azure Data Loss PreventionAzure Data Loss Prevention
Azure Data Loss Prevention
 
Healthcare payer - Big data integration
Healthcare payer - Big data integrationHealthcare payer - Big data integration
Healthcare payer - Big data integration
 
Centrifuge Systems Overview 2 14
Centrifuge Systems Overview 2 14Centrifuge Systems Overview 2 14
Centrifuge Systems Overview 2 14
 
Securing sensitive data for the health care industry
Securing sensitive data for the health care industrySecuring sensitive data for the health care industry
Securing sensitive data for the health care industry
 
Why you should use common data service final
Why you should use common data service finalWhy you should use common data service final
Why you should use common data service final
 
Case Study - Global Collaboration Multidisciplinary Professional Services
Case Study - Global Collaboration Multidisciplinary Professional ServicesCase Study - Global Collaboration Multidisciplinary Professional Services
Case Study - Global Collaboration Multidisciplinary Professional Services
 
DRM_Evolution_2005-03-17
DRM_Evolution_2005-03-17DRM_Evolution_2005-03-17
DRM_Evolution_2005-03-17
 
Accelerate Self-service Analytics with Universal Semantic Model
Accelerate Self-service Analytics with Universal Semantic Model Accelerate Self-service Analytics with Universal Semantic Model
Accelerate Self-service Analytics with Universal Semantic Model
 
CyberSecurity in a World of Connected Devices: IoT Security
CyberSecurity in a World of Connected Devices: IoT SecurityCyberSecurity in a World of Connected Devices: IoT Security
CyberSecurity in a World of Connected Devices: IoT Security
 
Data integration alipour-hafezi
Data integration alipour-hafeziData integration alipour-hafezi
Data integration alipour-hafezi
 
Customer-Centric Data Management for Better Customer Experiences
Customer-Centric Data Management for Better Customer ExperiencesCustomer-Centric Data Management for Better Customer Experiences
Customer-Centric Data Management for Better Customer Experiences
 
Gdpr ccpa steps to near as close to compliancy as possible with low risk of f...
Gdpr ccpa steps to near as close to compliancy as possible with low risk of f...Gdpr ccpa steps to near as close to compliancy as possible with low risk of f...
Gdpr ccpa steps to near as close to compliancy as possible with low risk of f...
 

Similar a EDW Webinar: Designing Master Data Services for Application Integration

Denodo Partner Connect: A Review of the Top 5 Differentiated Use Cases for th...
Denodo Partner Connect: A Review of the Top 5 Differentiated Use Cases for th...Denodo Partner Connect: A Review of the Top 5 Differentiated Use Cases for th...
Denodo Partner Connect: A Review of the Top 5 Differentiated Use Cases for th...
Denodo
 
Understanding Your Crown Jewels: Finding, Organizing, and Profiling Sensitive...
Understanding Your Crown Jewels: Finding, Organizing, and Profiling Sensitive...Understanding Your Crown Jewels: Finding, Organizing, and Profiling Sensitive...
Understanding Your Crown Jewels: Finding, Organizing, and Profiling Sensitive...
DataWorks Summit
 
Unlock the Power of Mainframe Data for Democratized Cloud Analytics
Unlock the Power of Mainframe Data for Democratized Cloud AnalyticsUnlock the Power of Mainframe Data for Democratized Cloud Analytics
Unlock the Power of Mainframe Data for Democratized Cloud Analytics
Precisely
 
MDM Strategy & Roadmap
MDM Strategy & RoadmapMDM Strategy & Roadmap
MDM Strategy & Roadmap
victorlbrown
 

Similar a EDW Webinar: Designing Master Data Services for Application Integration (20)

Using Collaboration for Metadata, Semantics and Lineage by David Loshin
Using Collaboration for Metadata, Semantics and Lineage by David LoshinUsing Collaboration for Metadata, Semantics and Lineage by David Loshin
Using Collaboration for Metadata, Semantics and Lineage by David Loshin
 
Working With Different Kinds of Data
Working With Different Kinds of DataWorking With Different Kinds of Data
Working With Different Kinds of Data
 
Leveraging AI and ML for efficient data integration.pdf
Leveraging AI and ML for efficient data integration.pdfLeveraging AI and ML for efficient data integration.pdf
Leveraging AI and ML for efficient data integration.pdf
 
Collaborative Metadata Management with David Loshin
Collaborative Metadata Management with David LoshinCollaborative Metadata Management with David Loshin
Collaborative Metadata Management with David Loshin
 
Ensuring Reliable Data Feeds for DeFi Applications
Ensuring Reliable Data Feeds for DeFi ApplicationsEnsuring Reliable Data Feeds for DeFi Applications
Ensuring Reliable Data Feeds for DeFi Applications
 
Model Confidence for Master Data with David Loshin
Model Confidence for Master Data with David LoshinModel Confidence for Master Data with David Loshin
Model Confidence for Master Data with David Loshin
 
Data Marketplace and the Role of Data Virtualization
Data Marketplace and the Role of Data VirtualizationData Marketplace and the Role of Data Virtualization
Data Marketplace and the Role of Data Virtualization
 
Denodo Partner Connect: A Review of the Top 5 Differentiated Use Cases for th...
Denodo Partner Connect: A Review of the Top 5 Differentiated Use Cases for th...Denodo Partner Connect: A Review of the Top 5 Differentiated Use Cases for th...
Denodo Partner Connect: A Review of the Top 5 Differentiated Use Cases for th...
 
Initiate - Presentation to Salesforce CIO Council
Initiate - Presentation to Salesforce CIO CouncilInitiate - Presentation to Salesforce CIO Council
Initiate - Presentation to Salesforce CIO Council
 
Understanding Your Crown Jewels: Finding, Organizing, and Profiling Sensitive...
Understanding Your Crown Jewels: Finding, Organizing, and Profiling Sensitive...Understanding Your Crown Jewels: Finding, Organizing, and Profiling Sensitive...
Understanding Your Crown Jewels: Finding, Organizing, and Profiling Sensitive...
 
Fast Data Strategy Houston Roadshow Presentation
Fast Data Strategy Houston Roadshow PresentationFast Data Strategy Houston Roadshow Presentation
Fast Data Strategy Houston Roadshow Presentation
 
Your Data is Waiting. What are the Top 5 Trends for Data in 2022? (ASEAN)
Your Data is Waiting. What are the Top 5 Trends for Data in 2022? (ASEAN)Your Data is Waiting. What are the Top 5 Trends for Data in 2022? (ASEAN)
Your Data is Waiting. What are the Top 5 Trends for Data in 2022? (ASEAN)
 
Unlock the Power of Mainframe Data for Democratized Cloud Analytics
Unlock the Power of Mainframe Data for Democratized Cloud AnalyticsUnlock the Power of Mainframe Data for Democratized Cloud Analytics
Unlock the Power of Mainframe Data for Democratized Cloud Analytics
 
DRM Evolution 2005 03 17
DRM Evolution 2005 03 17DRM Evolution 2005 03 17
DRM Evolution 2005 03 17
 
Keynote Theatre. Keynote Day 2. 16:30 Evelyn de Souza
Keynote Theatre. Keynote Day 2. 16:30   Evelyn de Souza Keynote Theatre. Keynote Day 2. 16:30   Evelyn de Souza
Keynote Theatre. Keynote Day 2. 16:30 Evelyn de Souza
 
180926 ihan webinar 2
180926 ihan webinar 2180926 ihan webinar 2
180926 ihan webinar 2
 
MDM Strategy & Roadmap
MDM Strategy & RoadmapMDM Strategy & Roadmap
MDM Strategy & Roadmap
 
Down to Business: Taking Action Quickly with Linked Data Services
Down to Business: Taking Action Quickly with Linked Data ServicesDown to Business: Taking Action Quickly with Linked Data Services
Down to Business: Taking Action Quickly with Linked Data Services
 
Driving your data forward with FME
Driving your data forward with FMEDriving your data forward with FME
Driving your data forward with FME
 
Data Virtualization: Introduction and Business Value (UK)
Data Virtualization: Introduction and Business Value (UK)Data Virtualization: Introduction and Business Value (UK)
Data Virtualization: Introduction and Business Value (UK)
 

Más de DATAVERSITY

The Data Trifecta – Privacy, Security & Governance Race from Reactivity to Re...
The Data Trifecta – Privacy, Security & Governance Race from Reactivity to Re...The Data Trifecta – Privacy, Security & Governance Race from Reactivity to Re...
The Data Trifecta – Privacy, Security & Governance Race from Reactivity to Re...
DATAVERSITY
 
Data Strategy Best Practices
Data Strategy Best PracticesData Strategy Best Practices
Data Strategy Best Practices
DATAVERSITY
 

Más de DATAVERSITY (20)

Architecture, Products, and Total Cost of Ownership of the Leading Machine Le...
Architecture, Products, and Total Cost of Ownership of the Leading Machine Le...Architecture, Products, and Total Cost of Ownership of the Leading Machine Le...
Architecture, Products, and Total Cost of Ownership of the Leading Machine Le...
 
Data at the Speed of Business with Data Mastering and Governance
Data at the Speed of Business with Data Mastering and GovernanceData at the Speed of Business with Data Mastering and Governance
Data at the Speed of Business with Data Mastering and Governance
 
Exploring Levels of Data Literacy
Exploring Levels of Data LiteracyExploring Levels of Data Literacy
Exploring Levels of Data Literacy
 
Building a Data Strategy – Practical Steps for Aligning with Business Goals
Building a Data Strategy – Practical Steps for Aligning with Business GoalsBuilding a Data Strategy – Practical Steps for Aligning with Business Goals
Building a Data Strategy – Practical Steps for Aligning with Business Goals
 
Make Data Work for You
Make Data Work for YouMake Data Work for You
Make Data Work for You
 
Data Catalogs Are the Answer – What is the Question?
Data Catalogs Are the Answer – What is the Question?Data Catalogs Are the Answer – What is the Question?
Data Catalogs Are the Answer – What is the Question?
 
Data Catalogs Are the Answer – What Is the Question?
Data Catalogs Are the Answer – What Is the Question?Data Catalogs Are the Answer – What Is the Question?
Data Catalogs Are the Answer – What Is the Question?
 
Data Modeling Fundamentals
Data Modeling FundamentalsData Modeling Fundamentals
Data Modeling Fundamentals
 
Showing ROI for Your Analytic Project
Showing ROI for Your Analytic ProjectShowing ROI for Your Analytic Project
Showing ROI for Your Analytic Project
 
How a Semantic Layer Makes Data Mesh Work at Scale
How a Semantic Layer Makes  Data Mesh Work at ScaleHow a Semantic Layer Makes  Data Mesh Work at Scale
How a Semantic Layer Makes Data Mesh Work at Scale
 
Is Enterprise Data Literacy Possible?
Is Enterprise Data Literacy Possible?Is Enterprise Data Literacy Possible?
Is Enterprise Data Literacy Possible?
 
The Data Trifecta – Privacy, Security & Governance Race from Reactivity to Re...
The Data Trifecta – Privacy, Security & Governance Race from Reactivity to Re...The Data Trifecta – Privacy, Security & Governance Race from Reactivity to Re...
The Data Trifecta – Privacy, Security & Governance Race from Reactivity to Re...
 
Emerging Trends in Data Architecture – What’s the Next Big Thing?
Emerging Trends in Data Architecture – What’s the Next Big Thing?Emerging Trends in Data Architecture – What’s the Next Big Thing?
Emerging Trends in Data Architecture – What’s the Next Big Thing?
 
Data Governance Trends - A Look Backwards and Forwards
Data Governance Trends - A Look Backwards and ForwardsData Governance Trends - A Look Backwards and Forwards
Data Governance Trends - A Look Backwards and Forwards
 
Data Governance Trends and Best Practices To Implement Today
Data Governance Trends and Best Practices To Implement TodayData Governance Trends and Best Practices To Implement Today
Data Governance Trends and Best Practices To Implement Today
 
2023 Trends in Enterprise Analytics
2023 Trends in Enterprise Analytics2023 Trends in Enterprise Analytics
2023 Trends in Enterprise Analytics
 
Data Strategy Best Practices
Data Strategy Best PracticesData Strategy Best Practices
Data Strategy Best Practices
 
Who Should Own Data Governance – IT or Business?
Who Should Own Data Governance – IT or Business?Who Should Own Data Governance – IT or Business?
Who Should Own Data Governance – IT or Business?
 
Data Management Best Practices
Data Management Best PracticesData Management Best Practices
Data Management Best Practices
 
MLOps – Applying DevOps to Competitive Advantage
MLOps – Applying DevOps to Competitive AdvantageMLOps – Applying DevOps to Competitive Advantage
MLOps – Applying DevOps to Competitive Advantage
 

Último

Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
panagenda
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
 
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Victor Rentea
 

Último (20)

TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
 
Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...
Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...
Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...
 
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWEREMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
 
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
 
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
 
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ..."I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
ICT role in 21st century education and its challenges
ICT role in 21st century education and its challengesICT role in 21st century education and its challenges
ICT role in 21st century education and its challenges
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
AI in Action: Real World Use Cases by Anitaraj
AI in Action: Real World Use Cases by AnitarajAI in Action: Real World Use Cases by Anitaraj
AI in Action: Real World Use Cases by Anitaraj
 
AI+A11Y 11MAY2024 HYDERBAD GAAD 2024 - HelloA11Y (11 May 2024)
AI+A11Y 11MAY2024 HYDERBAD GAAD 2024 - HelloA11Y (11 May 2024)AI+A11Y 11MAY2024 HYDERBAD GAAD 2024 - HelloA11Y (11 May 2024)
AI+A11Y 11MAY2024 HYDERBAD GAAD 2024 - HelloA11Y (11 May 2024)
 
Introduction to Multilingual Retrieval Augmented Generation (RAG)
Introduction to Multilingual Retrieval Augmented Generation (RAG)Introduction to Multilingual Retrieval Augmented Generation (RAG)
Introduction to Multilingual Retrieval Augmented Generation (RAG)
 
Exploring Multimodal Embeddings with Milvus
Exploring Multimodal Embeddings with MilvusExploring Multimodal Embeddings with Milvus
Exploring Multimodal Embeddings with Milvus
 
MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024
 
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a Fresher
 
Vector Search -An Introduction in Oracle Database 23ai.pptx
Vector Search -An Introduction in Oracle Database 23ai.pptxVector Search -An Introduction in Oracle Database 23ai.pptx
Vector Search -An Introduction in Oracle Database 23ai.pptx
 
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost SavingRepurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
 
Understanding the FAA Part 107 License ..
Understanding the FAA Part 107 License ..Understanding the FAA Part 107 License ..
Understanding the FAA Part 107 License ..
 
Platformless Horizons for Digital Adaptability
Platformless Horizons for Digital AdaptabilityPlatformless Horizons for Digital Adaptability
Platformless Horizons for Digital Adaptability
 
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
 

EDW Webinar: Designing Master Data Services for Application Integration

  • 1. Designing Master Data Services for Application Integration David Loshin Knowledge Integrity, Inc. www.knowledge-integrity.com © 2014 Knowledge Integrity, Inc. www.knowledge-integrity.com (301)754-6350 1
  • 2. Consolidation: The Typical Approach to MDM  Motivating factors have largely focused on consolidation of data into a single repository rather than how shared master data is to be used Customer Service Finance Sales Marketing Financial Data Customer Data Human Resources H/R Data Sales Data Legal Data Marketing Data Legal Fulfillment Data Centralized Repository © 2014 Knowledge Integrity, Inc. www.knowledge-integrity.com (301)754-6350 Fulfillment ? 2
  • 3. Why Consolidation Should Not Drive MDM Integration Loss of Knowledge Loss of Meaning Organizational Misalignment No Migration Plan No Process Governance Missing User Input “YADS” Cloud Issues Master Data Consolidation © 2014 Knowledge Integrity, Inc. www.knowledge-integrity.com (301)754-6350 Shadow IT 3
  • 4. Migration Strategy for Sharing Master Data  As the master registry/hub are brought online, there must be plans for migrating functional applications to the MDM environment to benefit Finance Sales Marketing Service H/R Transactions Compliance H/R Data Transaction Data Compliance Data Master Data Financial Data Sales Data Marketing Data Service Data © 2014 Knowledge Integrity, Inc. www.knowledge-integrity.com (301)754-6350 4
  • 5. Consider Typical Master Data Use Cases   Presumes the existence of a master data repository, index, and identity resolution capability Support for operational and analytical activities, such as:      Satisfy data management and governance policies, such as:     Managed assignment of unique identifiers Search and retrieve unified views of entity information Managing relationships Managing identifier cross-reference Batch matching services Duplicate analysis and elimination Data standardization/cleansing Enable Queries, Reporting, Analysis    Direct queries JOINs against other resources Feeding analytics applications © 2014 Knowledge Integrity, Inc. www.knowledge-integrity.com (301)754-6350 5
  • 6. Master Data Services Stack Search/Retri eve Entity Assign EID Enrichment Reconcile Identities Governance Merge Standardize/C leanse Bulk/Batch Bulk CrossReference Identity Services Split Bulk Match/Resol ution Relationship Management Bulk Update Relationship Management Generate EIDs Find ID X-Ref Break Relate Data Management Search Retrieve Update Create Deactivate Matching/Identity Resolution Data Access Entity Indexes Entity Repositories © 2014 Knowledge Integrity, Inc. www.knowledge-integrity.com (301)754-6350 Entity Relationships 6
  • 7. Master Entity Repositories Entity Indexes Entity Repositories  Instantiated repositories     Entity Relationships Profiles Relationships Index Consider typical data interactions supporting enterprise use of master data:     Ingestion Profiling Probabilistic matching Internal master data services for management of repository © 2014 Knowledge Integrity, Inc. www.knowledge-integrity.com (301)754-6350 7
  • 8. Data Access Data Access  Enable access to shared master data transparently to service layers    Direct access to physical data repository Virtualized/federated access to conceptual master repository Query access   SELECTs JOINs © 2014 Knowledge Integrity, Inc. www.knowledge-integrity.com (301)754-6350 8
  • 9. Core Services Governance Merge Identity Services Split Relationship Management Generate EIDs Find ID X-Ref Relate Break Data Management Search Retrieve Update Create Deactivate Matching/Identity Resolution  Matching/Identity Resolution   Data management       Algorithms for probabilistic matching Identity search (probabilistic search, key-based search, similarity scoring) Register/create a new master entity record Access master entity record Update master entity record Deactivate a master entity record Identity Services   Guarantee the generation of unique identifiers Manage cross-reference of internal and external identifiers © 2014 Knowledge Integrity, Inc. www.knowledge-integrity.com (301)754-6350 9
  • 10. Core Services Governance Identity Services Merge Split Relationship Management Generate EIDs Find ID X-Ref Relate Break Data Management Search Retrieve Update Create Deactivate Matching/Identity Resolution  Relationship management     Find all relationships associated with an entity Relate two or more entities and associate nature of relationship Break/deactivate a relationship Governance   Merge two records when they can be determined to represent the same entity Split a record into multiple records when it can be determined that there is a false positive © 2014 Knowledge Integrity, Inc. www.knowledge-integrity.com (301)754-6350 10
  • 11. Application Services Search/Retri eve Entity    Bulk Match/Resol ution Invokes core search, retrieve, or create Provide unique entity identifiers Data standardization Alignment of otherwise-assigned identifiers for cross-referencing Cleansing Batch Services     Standardize/C leanse Bulk/Batch Bulk CrossReference Enrichment   Enrichment Reconcile Identities Assign EID   Relationship Management Search/Retrieve Entity   Assign EID Deliver result set of master records matching a provided set of candidate identities Apply batch updates Batch identity resolution Relationship Management    Find all relationships associated with an entity Relate two or more entities and associate nature of relationship (e.g. household) Break/deactivate a relationship © 2014 Knowledge Integrity, Inc. www.knowledge-integrity.com (301)754-6350 11 Bulk Update
  • 12. Delivering a Unified View Based on Consumer Rules  Data Source 1 Master Data Index Collection & Assembly Data Source 2 … Data Source N Virtualized presentation of federated data  The master index maps the canonical representation of a specific entity (beneficiary, providers, etc.) to the locations in the original sources (or replicas of those source) holding data instances that have been linked via the identity resolution processes The accessibility of the different sources and the ability to apply those business rules at the data user’s discretion will provide a consolidated view that best meets the data users’ requirements at the time the data is reused © 2014 Knowledge Integrity, Inc. www.knowledge-integrity.com (301)754-6350 12
  • 13. Master Data Facilitated via Federation/Virtualization 1 2 Master Data Services 3 Standardized Canonical Relational Views 8 4 Transform and Map Master Index Federated/Virtualized 5 Data Access Parse and Standardize, Normalize, Cleanse 7 © 2014 Knowledge Integrity, Inc. www.knowledge-integrity.com (301)754-6350 13
  • 14. Summary        Review the business process to understand uses of shared master data Identify integration points within business processes Characterize usage scenarios Determine required functionality Assess existing systemic support for required functionality Design application integration service APIs Design and implement application integration services © 2014 Knowledge Integrity, Inc. www.knowledge-integrity.com (301)754-6350 14
  • 15. Questions and Open Discussion  www.knowledge-integrity.com  If you have questions, comments, or suggestions, please contact me David Loshin 301-754-6350 loshin@knowledge-integrity.com www.dataqualitybook.com www.mdmbook.com © 2014 Knowledge Integrity, Inc. www.knowledge-integrity.com (301)754-6350 15