SlideShare a Scribd company logo
1 of 26
Data Quality Technical Architecture Components
Contents ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Data Quality Best-in-Class Properties ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Data Quality Architecture Concepts ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],The 1:1 relationship of architectural concepts and architectural  components is intentional
DQ Technical Architecture –  Core Functional Components Data Storage Data Replication Reference Data Data Flow Data Management Data Access Workflow Access Business Rule Access Packaged Application Management Service Definition Service Execution Service Choreography (Web) Service Management Service Registry Workflow Definition Workflow Execution Workflow Management Business Rule Definition Business Rule Automation Business Rule Management Gateway External Invocation Management Public Service Registry DQ External Interface Management Service Publication Case Management Policy Management
DQ Technical Architecture –  Core Functional Components Data Storage Data Replication Reference Data Data Flow Data Management Data Access Workflow Access Business Rule Access Packaged Application Management Service Definition Service Execution Service Choreography (Web) Service Management Service Registry Workflow Definition Workflow Execution Workflow Management Business Rule Definition Business Rule Automation Business Rule Management Gateway External Invocation Management Public Service Registry DQ External Interface Management Service Publication Case Management Policy Management Fundamental Pillars of Production Systems
DQ Technical Architecture –  Web Service Management System
DQ Technical Architecture – Workflow Management System
DQ Technical Architecture –  Business Rule Management System
DQ Technical Architecture – Data Management System
DQ Technical Architecture –  Packaged Application Management Integration of Crystal Report Software Integration of BO/DQ Software
DQ Technical Architecture –  External Invocation Management
DQ Services -  Components Member Provider Claim Subject Area: Other Service Area: Enrichment Profiling  Other De-duping Data Area: Address Name Phone SSN Other TIN Tool Area: BO/DQ Project Crystal Report Other Rules Area: Mandated Rule Business Rule Technology  Rule Industry Standard  Rule Software Area: Shell Script JAVA Wrapper Other Method Area: Set() Throw() Get() Run()
DQ Rules – Mandated Rules ,[object Object],[object Object],[object Object],[object Object],[object Object]
DQ Rules – Business Rules ,[object Object],[object Object],[object Object]
DQ Rules – Technology Rules ,[object Object],[object Object]
DQ Rules – Industry Standard Rules  ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
DQ Services –  EEP E1 Invocation (As-is) requests executes using processes for has applies invokes invokes Conceptual Member Subject Area: Enrichment Service Area: Data Area: Address Method Area: Run() Tool Area: BO/DQ Project Tool End User Workspace Driver Rules Area: Mandated Rule Software Area: JAVA Wrapper Software Area: Shell  Script
DQ Services – Address Standardization (To-be) Address standardization for member data  to match zip code with city and state. Conceptual
DQ Services –  Address Standardization (To-be) Address standardization for member data  to match zip code with city and state. executes using processes for has applies requests invokes Conceptual Member Subject Area: Profiling Service Area: Data Area: Address Method Area: Get() Tool Area: Crystal Report Tool End User Workspace Driver Rules Area: Industry Standard Rule Software Area: Other (SQL)
DQ Services –  Mixed Case (To-be) Convert all upper case name  to mixed case name  Conceptual Draft
DQ Services –  Mixed Case (To-be) Convert all upper case name  to mixed case name executes using processes for has applies requests invokes Conceptual Draft Provider Subject Area: Enrichment Service Area: Data Area: Name Method Area: Set() Tool Area: BO/DQ Project Tool End User Workspace Driver Rules Area: Business Rules Software Area: Shell/JAVA
DQ Services – Enrichment services ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],For
Data Acquisition Type – Member  ,[object Object],[object Object]
DQ Services – Profiling services ,[object Object],[object Object],[object Object]
DQ Services – De-duping services ,[object Object],[object Object],[object Object],[object Object]

More Related Content

What's hot

Business Value Through Reference and Master Data Strategies
Business Value Through Reference and Master Data StrategiesBusiness Value Through Reference and Master Data Strategies
Business Value Through Reference and Master Data StrategiesDATAVERSITY
 
Data Architecture Strategies: Data Architecture for Digital Transformation
Data Architecture Strategies: Data Architecture for Digital TransformationData Architecture Strategies: Data Architecture for Digital Transformation
Data Architecture Strategies: Data Architecture for Digital TransformationDATAVERSITY
 
Data Governance Best Practices
Data Governance Best PracticesData Governance Best Practices
Data Governance Best PracticesDATAVERSITY
 
Building a Data Governance Strategy
Building a Data Governance StrategyBuilding a Data Governance Strategy
Building a Data Governance StrategyAnalytics8
 
Data Quality & Data Governance
Data Quality & Data GovernanceData Quality & Data Governance
Data Quality & Data GovernanceTuba Yaman Him
 
Capability Model_Data Governance
Capability Model_Data GovernanceCapability Model_Data Governance
Capability Model_Data GovernanceSteve Novak
 
Data Migration Steps PowerPoint Presentation Slides
Data Migration Steps PowerPoint Presentation Slides Data Migration Steps PowerPoint Presentation Slides
Data Migration Steps PowerPoint Presentation Slides SlideTeam
 
Enterprise Data Architecture Deliverables
Enterprise Data Architecture DeliverablesEnterprise Data Architecture Deliverables
Enterprise Data Architecture DeliverablesLars E Martinsson
 
Data Catalog as the Platform for Data Intelligence
Data Catalog as the Platform for Data IntelligenceData Catalog as the Platform for Data Intelligence
Data Catalog as the Platform for Data IntelligenceAlation
 
The data quality challenge
The data quality challengeThe data quality challenge
The data quality challengeLenia Miltiadous
 
data-analytics-strategy-ebook.pptx
data-analytics-strategy-ebook.pptxdata-analytics-strategy-ebook.pptx
data-analytics-strategy-ebook.pptxMohamedHendawy17
 
Data Lake Architecture – Modern Strategies & Approaches
Data Lake Architecture – Modern Strategies & ApproachesData Lake Architecture – Modern Strategies & Approaches
Data Lake Architecture – Modern Strategies & ApproachesDATAVERSITY
 
07. Analytics & Reporting Requirements Template
07. Analytics & Reporting Requirements Template07. Analytics & Reporting Requirements Template
07. Analytics & Reporting Requirements TemplateAlan D. Duncan
 
How to Build & Sustain a Data Governance Operating Model
How to Build & Sustain a Data Governance Operating Model How to Build & Sustain a Data Governance Operating Model
How to Build & Sustain a Data Governance Operating Model DATUM LLC
 
Data Lakehouse, Data Mesh, and Data Fabric (r1)
Data Lakehouse, Data Mesh, and Data Fabric (r1)Data Lakehouse, Data Mesh, and Data Fabric (r1)
Data Lakehouse, Data Mesh, and Data Fabric (r1)James Serra
 
Managing Data Integration Initiatives
Managing Data Integration InitiativesManaging Data Integration Initiatives
Managing Data Integration InitiativesAllinConsulting
 
Achieving a Single View of Business – Critical Data with Master Data Management
Achieving a Single View of Business – Critical Data with Master Data ManagementAchieving a Single View of Business – Critical Data with Master Data Management
Achieving a Single View of Business – Critical Data with Master Data ManagementDATAVERSITY
 
What is a Data Warehouse and How Do I Test It?
What is a Data Warehouse and How Do I Test It?What is a Data Warehouse and How Do I Test It?
What is a Data Warehouse and How Do I Test It?RTTS
 

What's hot (20)

Business Value Through Reference and Master Data Strategies
Business Value Through Reference and Master Data StrategiesBusiness Value Through Reference and Master Data Strategies
Business Value Through Reference and Master Data Strategies
 
Data Architecture Strategies: Data Architecture for Digital Transformation
Data Architecture Strategies: Data Architecture for Digital TransformationData Architecture Strategies: Data Architecture for Digital Transformation
Data Architecture Strategies: Data Architecture for Digital Transformation
 
Data Governance Best Practices
Data Governance Best PracticesData Governance Best Practices
Data Governance Best Practices
 
Building a Data Governance Strategy
Building a Data Governance StrategyBuilding a Data Governance Strategy
Building a Data Governance Strategy
 
Data Quality & Data Governance
Data Quality & Data GovernanceData Quality & Data Governance
Data Quality & Data Governance
 
Capability Model_Data Governance
Capability Model_Data GovernanceCapability Model_Data Governance
Capability Model_Data Governance
 
Data Migration Steps PowerPoint Presentation Slides
Data Migration Steps PowerPoint Presentation Slides Data Migration Steps PowerPoint Presentation Slides
Data Migration Steps PowerPoint Presentation Slides
 
Enterprise Data Architecture Deliverables
Enterprise Data Architecture DeliverablesEnterprise Data Architecture Deliverables
Enterprise Data Architecture Deliverables
 
Data Catalog as the Platform for Data Intelligence
Data Catalog as the Platform for Data IntelligenceData Catalog as the Platform for Data Intelligence
Data Catalog as the Platform for Data Intelligence
 
Data Migration to Azure
Data Migration to AzureData Migration to Azure
Data Migration to Azure
 
The data quality challenge
The data quality challengeThe data quality challenge
The data quality challenge
 
data-analytics-strategy-ebook.pptx
data-analytics-strategy-ebook.pptxdata-analytics-strategy-ebook.pptx
data-analytics-strategy-ebook.pptx
 
Data Lake Architecture – Modern Strategies & Approaches
Data Lake Architecture – Modern Strategies & ApproachesData Lake Architecture – Modern Strategies & Approaches
Data Lake Architecture – Modern Strategies & Approaches
 
07. Analytics & Reporting Requirements Template
07. Analytics & Reporting Requirements Template07. Analytics & Reporting Requirements Template
07. Analytics & Reporting Requirements Template
 
How to Build & Sustain a Data Governance Operating Model
How to Build & Sustain a Data Governance Operating Model How to Build & Sustain a Data Governance Operating Model
How to Build & Sustain a Data Governance Operating Model
 
Data Lakehouse, Data Mesh, and Data Fabric (r1)
Data Lakehouse, Data Mesh, and Data Fabric (r1)Data Lakehouse, Data Mesh, and Data Fabric (r1)
Data Lakehouse, Data Mesh, and Data Fabric (r1)
 
Managing Data Integration Initiatives
Managing Data Integration InitiativesManaging Data Integration Initiatives
Managing Data Integration Initiatives
 
Achieving a Single View of Business – Critical Data with Master Data Management
Achieving a Single View of Business – Critical Data with Master Data ManagementAchieving a Single View of Business – Critical Data with Master Data Management
Achieving a Single View of Business – Critical Data with Master Data Management
 
What is a Data Warehouse and How Do I Test It?
What is a Data Warehouse and How Do I Test It?What is a Data Warehouse and How Do I Test It?
What is a Data Warehouse and How Do I Test It?
 
Data Governance
Data GovernanceData Governance
Data Governance
 

Similar to Data Quality Technical Architecture

Vinay-Resume
Vinay-ResumeVinay-Resume
Vinay-Resumesagarv48
 
Ws Soa V6 Theory And Practice
Ws Soa V6 Theory And PracticeWs Soa V6 Theory And Practice
Ws Soa V6 Theory And PracticePini Cohen
 
SharePoint 2010 Global Deployment
SharePoint 2010 Global DeploymentSharePoint 2010 Global Deployment
SharePoint 2010 Global DeploymentJoel Oleson
 
SoftwareONE Oracle Licensing Introduction 18.02.14
SoftwareONE Oracle Licensing Introduction 18.02.14SoftwareONE Oracle Licensing Introduction 18.02.14
SoftwareONE Oracle Licensing Introduction 18.02.14SoftwareONEPresents
 
Insurance Vertical Presentation
Insurance Vertical PresentationInsurance Vertical Presentation
Insurance Vertical PresentationMurty NSN
 
Soa Business And Technical Overview Presentation (Reed003707)
Soa Business And Technical Overview Presentation (Reed003707)Soa Business And Technical Overview Presentation (Reed003707)
Soa Business And Technical Overview Presentation (Reed003707)miteshisheth
 
Soa Business And Technical Overview Presentation (Reed003707)
Soa Business And Technical Overview Presentation (Reed003707)Soa Business And Technical Overview Presentation (Reed003707)
Soa Business And Technical Overview Presentation (Reed003707)miteshisheth
 
Soa Business And Technical Overview Presentation (Reed003707)
Soa Business And Technical Overview Presentation (Reed003707)Soa Business And Technical Overview Presentation (Reed003707)
Soa Business And Technical Overview Presentation (Reed003707)miteshisheth
 
Soa Business And Technical Overview Presentation (Reed003707)
Soa Business And Technical Overview Presentation (Reed003707)Soa Business And Technical Overview Presentation (Reed003707)
Soa Business And Technical Overview Presentation (Reed003707)miteshisheth
 
Soa Business And Technical Overview Presentation (Reed003707)
Soa Business And Technical Overview Presentation (Reed003707)Soa Business And Technical Overview Presentation (Reed003707)
Soa Business And Technical Overview Presentation (Reed003707)miteshisheth
 
Soa Business And Technical Overview Presentation (Reed003707)
Soa Business And Technical Overview Presentation (Reed003707)Soa Business And Technical Overview Presentation (Reed003707)
Soa Business And Technical Overview Presentation (Reed003707)miteshisheth
 
STL LItigation Services
STL LItigation ServicesSTL LItigation Services
STL LItigation Servicesguestc7f86
 
On-Demand: Is It Right For Your Company?
On-Demand: Is It Right For Your Company?On-Demand: Is It Right For Your Company?
On-Demand: Is It Right For Your Company?Callidus Software
 
SoftwareONE Oracle Licensing Introduction 18.02.14
SoftwareONE Oracle Licensing Introduction 18.02.14SoftwareONE Oracle Licensing Introduction 18.02.14
SoftwareONE Oracle Licensing Introduction 18.02.14SoftwareONEIndia
 
A "First Time Right" Start with Data Virtualization by Bart De Groeve, Practi...
A "First Time Right" Start with Data Virtualization by Bart De Groeve, Practi...A "First Time Right" Start with Data Virtualization by Bart De Groeve, Practi...
A "First Time Right" Start with Data Virtualization by Bart De Groeve, Practi...Patrick Van Renterghem
 
Service Analysis And Design
Service Analysis And DesignService Analysis And Design
Service Analysis And DesignRody Middelkoop
 

Similar to Data Quality Technical Architecture (20)

Data Quality Services
Data Quality ServicesData Quality Services
Data Quality Services
 
Vinay-Resume
Vinay-ResumeVinay-Resume
Vinay-Resume
 
Yongsan presentation 3
Yongsan presentation 3Yongsan presentation 3
Yongsan presentation 3
 
Ws Soa V6 Theory And Practice
Ws Soa V6 Theory And PracticeWs Soa V6 Theory And Practice
Ws Soa V6 Theory And Practice
 
SharePoint 2010 Global Deployment
SharePoint 2010 Global DeploymentSharePoint 2010 Global Deployment
SharePoint 2010 Global Deployment
 
SoftwareONE Oracle Licensing Introduction 18.02.14
SoftwareONE Oracle Licensing Introduction 18.02.14SoftwareONE Oracle Licensing Introduction 18.02.14
SoftwareONE Oracle Licensing Introduction 18.02.14
 
Insurance Vertical Presentation
Insurance Vertical PresentationInsurance Vertical Presentation
Insurance Vertical Presentation
 
chapter 2.pdf
chapter 2.pdfchapter 2.pdf
chapter 2.pdf
 
Soa Business And Technical Overview Presentation (Reed003707)
Soa Business And Technical Overview Presentation (Reed003707)Soa Business And Technical Overview Presentation (Reed003707)
Soa Business And Technical Overview Presentation (Reed003707)
 
Soa Business And Technical Overview Presentation (Reed003707)
Soa Business And Technical Overview Presentation (Reed003707)Soa Business And Technical Overview Presentation (Reed003707)
Soa Business And Technical Overview Presentation (Reed003707)
 
Soa Business And Technical Overview Presentation (Reed003707)
Soa Business And Technical Overview Presentation (Reed003707)Soa Business And Technical Overview Presentation (Reed003707)
Soa Business And Technical Overview Presentation (Reed003707)
 
Soa Business And Technical Overview Presentation (Reed003707)
Soa Business And Technical Overview Presentation (Reed003707)Soa Business And Technical Overview Presentation (Reed003707)
Soa Business And Technical Overview Presentation (Reed003707)
 
Soa Business And Technical Overview Presentation (Reed003707)
Soa Business And Technical Overview Presentation (Reed003707)Soa Business And Technical Overview Presentation (Reed003707)
Soa Business And Technical Overview Presentation (Reed003707)
 
Soa Business And Technical Overview Presentation (Reed003707)
Soa Business And Technical Overview Presentation (Reed003707)Soa Business And Technical Overview Presentation (Reed003707)
Soa Business And Technical Overview Presentation (Reed003707)
 
STL LItigation Services
STL LItigation ServicesSTL LItigation Services
STL LItigation Services
 
On-Demand: Is It Right For Your Company?
On-Demand: Is It Right For Your Company?On-Demand: Is It Right For Your Company?
On-Demand: Is It Right For Your Company?
 
SoftwareONE Oracle Licensing Introduction 18.02.14
SoftwareONE Oracle Licensing Introduction 18.02.14SoftwareONE Oracle Licensing Introduction 18.02.14
SoftwareONE Oracle Licensing Introduction 18.02.14
 
A "First Time Right" Start with Data Virtualization by Bart De Groeve, Practi...
A "First Time Right" Start with Data Virtualization by Bart De Groeve, Practi...A "First Time Right" Start with Data Virtualization by Bart De Groeve, Practi...
A "First Time Right" Start with Data Virtualization by Bart De Groeve, Practi...
 
Service Analysis And Design
Service Analysis And DesignService Analysis And Design
Service Analysis And Design
 
Venkataraman Raghunathan
Venkataraman RaghunathanVenkataraman Raghunathan
Venkataraman Raghunathan
 

Data Quality Technical Architecture

  • 1. Data Quality Technical Architecture Components
  • 2.
  • 3.
  • 4.
  • 5. DQ Technical Architecture – Core Functional Components Data Storage Data Replication Reference Data Data Flow Data Management Data Access Workflow Access Business Rule Access Packaged Application Management Service Definition Service Execution Service Choreography (Web) Service Management Service Registry Workflow Definition Workflow Execution Workflow Management Business Rule Definition Business Rule Automation Business Rule Management Gateway External Invocation Management Public Service Registry DQ External Interface Management Service Publication Case Management Policy Management
  • 6. DQ Technical Architecture – Core Functional Components Data Storage Data Replication Reference Data Data Flow Data Management Data Access Workflow Access Business Rule Access Packaged Application Management Service Definition Service Execution Service Choreography (Web) Service Management Service Registry Workflow Definition Workflow Execution Workflow Management Business Rule Definition Business Rule Automation Business Rule Management Gateway External Invocation Management Public Service Registry DQ External Interface Management Service Publication Case Management Policy Management Fundamental Pillars of Production Systems
  • 7. DQ Technical Architecture – Web Service Management System
  • 8. DQ Technical Architecture – Workflow Management System
  • 9. DQ Technical Architecture – Business Rule Management System
  • 10. DQ Technical Architecture – Data Management System
  • 11. DQ Technical Architecture – Packaged Application Management Integration of Crystal Report Software Integration of BO/DQ Software
  • 12. DQ Technical Architecture – External Invocation Management
  • 13. DQ Services - Components Member Provider Claim Subject Area: Other Service Area: Enrichment Profiling Other De-duping Data Area: Address Name Phone SSN Other TIN Tool Area: BO/DQ Project Crystal Report Other Rules Area: Mandated Rule Business Rule Technology Rule Industry Standard Rule Software Area: Shell Script JAVA Wrapper Other Method Area: Set() Throw() Get() Run()
  • 14.
  • 15.
  • 16.
  • 17.
  • 18. DQ Services – EEP E1 Invocation (As-is) requests executes using processes for has applies invokes invokes Conceptual Member Subject Area: Enrichment Service Area: Data Area: Address Method Area: Run() Tool Area: BO/DQ Project Tool End User Workspace Driver Rules Area: Mandated Rule Software Area: JAVA Wrapper Software Area: Shell Script
  • 19. DQ Services – Address Standardization (To-be) Address standardization for member data to match zip code with city and state. Conceptual
  • 20. DQ Services – Address Standardization (To-be) Address standardization for member data to match zip code with city and state. executes using processes for has applies requests invokes Conceptual Member Subject Area: Profiling Service Area: Data Area: Address Method Area: Get() Tool Area: Crystal Report Tool End User Workspace Driver Rules Area: Industry Standard Rule Software Area: Other (SQL)
  • 21. DQ Services – Mixed Case (To-be) Convert all upper case name to mixed case name Conceptual Draft
  • 22. DQ Services – Mixed Case (To-be) Convert all upper case name to mixed case name executes using processes for has applies requests invokes Conceptual Draft Provider Subject Area: Enrichment Service Area: Data Area: Name Method Area: Set() Tool Area: BO/DQ Project Tool End User Workspace Driver Rules Area: Business Rules Software Area: Shell/JAVA
  • 23.
  • 24.
  • 25.
  • 26.

Editor's Notes

  1. EDQ Web Service Management enables to query multiple web services in a unified and transparent fashion through a SQL-like interface. Metadata Component deals with the registration of new web services, and mapping their schemas to the global view exported in the client interface. Query Processing and Optimization is to take a user-input query and find the optimal "query plan", i.e., the most efficient way to process the query by issuing calls to the relevant web services. The execution engine then executes this query plan by making actual web service calls and efficiently managing the data exchange between the WSMS and the various web services. Profiling and Statistics component profiles the web services for response time and maintain the relevant statistics over the web service data. It is used to optimize the Query processing.
  2. The workflow service can be called asynchronously from BPEL to perform operations like adding, updating, completing, renewing, routing and escalating tasks. User applications, on the other hand, can communicate with the Workflow Service to acquire the list of tasks for selected users, render appropriate user interfaces, and return results to the workflow service, which forwards them to the BPEL process. BPEL Process - Business Process Execution Language for Web Services provides a means to formally specify business processes and interaction protocols.
  3. A BRMS or Business Rule Management System is to manage and support the EDQ business rules. Policy Management - Business Rule Definition – It defines the business constraint and asserts business structure or influence the behavior of business. Business Rule Automation - It is a technology components substituting / Supplementing the manual processes to manage information flow. Business Rule Engine – A system which manage and automate business rules.
  4. MDM – Master Data Management Federated MDM - cross references key identifying information from participating systems to implement a registry-style solution. The main benefit of a federated solution is non-intrusiveness on participating systems that maintain their original context. Integrated MDM - stores all master data information from all participating systems in a centralized MDM repository. This centralized repository houses the "gold copy" of all master data information. The main benefit of the integrated approach is that it provides the most complete, accurate, and consistent single view of master data. Hybrid MDM - stores common data elements from participating systems creating a "light gold copy" of the master data, while disparate elements are referenced from their original system of record. The benefit and drawback of the hybrid solution is the partial combination of the federated and integrated benefits. MDM and SOA evolved separately but share many design principles. "Contract first" applies to the interfaces in MDM and the service definition in SOA "Reusability" applies to data through conformance in MDM and services through SOA principles in SOA "Discoverability" applies to data through the master data repository in MDM and services through registry in SOA "Abstraction" applies to source system complexity and MDM and underlying service complexity under SOA.
  5. Data Access - Workflow Access Business Rule Access