Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Albel pres mdm implementation
1. Master Data Management
From Assessment, Design up to operation
Ali BELCAID – Managing Consultant
2. Master Data Management : An Overview
Information is a Priority
Quality and actionable information is fundamental to deliver many business strategies.
Enterprise Operations Enterprise Information
Management & Capabilities Management & Capabilities
MDM is the glue that
Solutions
M Solutions blends operational and
• ERP • BI/DW information
• CRM
D • BPM management solutions
• Supply Chain M • Portals
Operational MDM Analytical MDM
3. Master Data Management : An Overview
MDM Requires Both IT and Business
MDM is a component that promotes process efficiency, simplicity, and data quality,
improving the value IT brings to business.
Impact of MDM Initiative on Business and IT
Global
Business Master Data IT
Management
Enterprise Centralized, Efficient Data
• Avoid data redundancies
Wide Storage
• Assure data consistency
Consistency
Cost • Centralize data Improved System
Effectiveness distribution (one source) Integration
Reliable • Provide unique identifier Minimal Data
Analytics and • Create global hierarchies
Conversions
Reporting and attributes
4. Master Data Management : An Overview
MDM Implementation Styles
Implementation Style Description
Third party suppliers and managers of domain specific
External Databases master data
(Service Provider) Examples: database marketing, government service
bureaus
Master information file/database, system of record (SOR)
Persistent Operational data store, active data warehouse
(Database) Relational DBMS + extract-transform-load (ETL) + data
quality (DQ)
Metadata layer + distributed query (e.g., EII)
Registry Enterprise application integration (e.g., EAI), distributed
(Virtual) system
Portal
Ability to fine-tune performance and availability by altering
Composite
amount of master data persisted
(Hybrid)
XML, web services, service-oriented architecture (SOA)
5. Master Data Management An Overview
“Persistent” Master Data Repository
(Illustrative Scenario)
This Customer Data Integration (CDI) solution architecture illustrates how process and technology work
together through a centralized “persistent” master data repository
Operational Systems Master Data Management Business Process
Initiate Evaluate Approve Initiate
Entry/Update Request Request Entry/Update
System Owners Data Stewards Business Analysts
Customer Master Data Repository
Workflow Business Rules Mapping Rules Catalogue/Index
Automated Entry Updates
Customer Care
Extract Transform Load
SAP Care
Enterprise Application Integration
Reporting
DIM Data Mart CRM
(ETL)
Siebel Operational DIM Campaign
DIM
Data Store Management
(ODS) Customer
Reporting Contract
FACT Data Mart
Negotiations
IMS
Customer Financial
Aggregate DIM Consolidation
DIM
Financial
Monthly End
(EAI)
DIM Data Mart Reporting
PBMS Close
Enterprise Warehouse Data Marts
DATA INFORMATION
6. Approach to MDM Implementation
Business Assessment & Technology Selection
Current State Future State Develop Roadmap
Activities
Organization &
Current State Governance
Details
Process & Implementation
Quick scan Gap Analysis
Methodology Roadmap
Requirements
Technology
Selection
Deliverables
Project Initiation Current State : Data Mgt. Future State : Data Mgt. Prioritization
Gap Analysis
Current Sate : Organization Future Sate : Organization Roadmap
Current State : Architecture Future State : Architecture
4 to 6 weeks varies with scope
7. Approach to MDM Implementation
Business Assessment & Technology Selection
Topic What to Do ? What to Deliver ?
• Maturity Assessment
• Business Direction, objectives, …
• Scope of the Project : Which MDM should
Quick Scan • Engagement Management (project
be implemented ?
management, change management, quality
management and risk management )
• workshops with key client stakeholders to
Business Requirement Analysis identify business issues with master data
(Data quality, Duplicity, Incoherence, …)
• workshops with key client stakeholders to
identify technology issues that could delay
Technical Requirement Analysis the delivery of accurate and reliable master
data to consumers (multi-systems, duplicity, • MDM Finding & Assessment
non-synchronization, …)
• based on business and technical findings and
Gap Analysis
requirements
• that consists of business, technology and
Future State Recommendations
data architecture
Roadmap Definition • for attaining the future state • Roadmap Definition & Planning
8. Approach to MDM Implementation
Business Assessment & Technology Selection
Maturity Assessment
9. Approach to MDM Implementation
MDM Implementation Framework
Continues Implementation Phases
This part is done once
(Part of the Assessment Phase)
Design
Kick Off of the MDM Initiative
Build
Technology Roadmap &
Business
Assessment & Foundation
Requirement
Software Selection Activities
Integrate
Begin next Operate
Iteration
Define & Validate the data governance & operating model Implement the data governance & Operating model
10. Approach to MDM Implementation
Roadmap & Foundation Activities
The Roadmap provides the detailed requirements and solution definition Meta Data Management
that applies to the continuous implementation. It has the following
objectives:
Master Data Master Data Master Data
Modelling Migration Integration
Refine strategic business requirements to a detailed level for iterative
design
Master Data Master Data
Establish standards and develop solutions to common problems Re-engineering Profiling
Define the development and delivery environments
Detailed planning for this cycle of the implementation
Master Data Architecture
The Roadmap can be summarized as providing the Plan, the Solution
Requirements and the Solution Definition for the continuous Major deliverables and points to be addressed when setting
up the roadmap & foundation activities :
implementation part.
Detailed Project Roadmap
Foundation Activities focus on aspects of each of the streams of Testing and Deployment plans
development. These activities are : Detailed Information Modeling
Detailed Migration Plan (historical Data)
Recommended process and system changes for
Meta Data Management improved Data Governance
Data Modelling Identification of root causes leading to Data Governance
Data Migration issues
Data Integration Data Governance Metrics
Quantitative Data Investigation
Data Reengineering Improved Data Quality
Data Profiling Create/Revise Solution Architecture
Data Solution Architecture Ensure the availability of Software Development
Environment
11. Approach to MDM Implementation
MDM Work streams
Design Build Integrate Operate
MDM Program Management
Change/Issue Management
Operations Management Meta Data Master Data Master Data Master Data
Management Modelling Migration Integration
Training and Support
Master Data Master Data Master Data
Re-engineering Profiling Architecture Iteration
12. Approach to MDM Implementation
Meta Data Management
Significant metadata artifacts are produced related to data definition, business rules, transformation logic and data quality. This information
should be stored in a metadata repository; getting this repository in place from the early stages of the MDM project.
Model Management is the capability to manage Versioning of metadata provides the ability for looking back into
structures and processes used to describe the metadata history to gain a more comprehensive understanding of the
in a system. current state
Metadata Integration capability provides a basic ability to Configuration Management is a fundamental process for
build metadata flows into and out of a managed metadata developing metadata. It is the role that process and governance
environment. plays in the development and operations of a managed
metadata environment.
Identity Matching as a foundation capability ensures consistent
and accurate reuse of metadata. a system must have the ability Model Query provides the fundamental ability for publication of
to identify metadata uniquely so that the metadata may be metadata. Its capabilities form the foundation of providing
reused, validated and versioned within the managed metadata Metadata Reporting Packages
environment.
Metadata Access Control is a capability for providing a control
Validation capabilities ensure the quality and consistency of layer over metadata models. Metadata can often be sensitive
metadata flowing through the managed metadata environment information that should have restrictive controls to prevent
unauthorized access
13. Approach to MDM Implementation
Master Data Modelling
The data modeling process is used as an intermediary data store to bring data together from multiple systems in
a hub fashion. This data store provides a common, integrated model where data may undergo significant re-
engineering.
Design Logical Master Implement Physical
Data Model Master Data Model
Input: Input:
Conceptual Data Model Logical Data Model
Data Specification Standards Solution Architecture
Data Modeling Standards Data Specification Standards
Data Security Standards Data Modeling Standards
Detailed Business Requirements for Data Security Standards
each Iteration Detailed Business Requirements for each iteration
Output: Output:
Logical Data Model Physical Data Model
Database Definition Language (DDL) Scripts
Sizing Estimates
14. Approach to MDM Implementation
Master Data Integration
Dependencies: Data Integration is one of the Foundation Capabilities of MDM Development. It provides a mechanism
Metadata Management
for bringing together information from a number of distributed systems by interfacing into sources,
Data Profiling
Data Re-engineering providing a capability to transform data between the systems, enforcing business rules and being able
Data Modeling to load data into a different types of target areas.
Data Migration
ETL ETL ETL flows &
Logical Design Physical Design jobs Testing
Input: Input: Input:
Business requirements ETL Logical Design Test scenarios
Designed Process Flow Solution Architecture Data Sampling
Source & Target interfaces Data Specification Standards
load dependencies and integration with Data Modeling Standards Output:
metadata processes Data Security Standards Tested flows and jobs
Source & Target Data models
Output:
Output: ETL flows and Jobs
ETL Logical Design
15. Approach to MDM Implementation
Master Data Re-engineering
Data Re-Engineering is a term used to describe a number of related functions for standardizing data to a common format,
correcting data quality issues, removing duplicate information/building linkages between records that did not exist previously, or
enriching data with supplementary information.
Data standardization brings data into a common format for In the Data Matching and Consolidation task, data is
migrating into target environment. It addresses problems associated with other records to identify matching
related to: sets. Matching records can then either be
consolidated to remove duplications or linked to
Redundant domain values another to form new associations.
Formatting problems
Non-atomic data from complex fields
Embedded meaning in data Data Enrichment provide an organisation’s internal data
with data from external sources like :
Data Correction typically addresses problems related to: Personal data such as date-of-birth and gender codes
Geographical data
Missing data Postal Data, such as Delivery Point Identifiers (DPID)
Value issues due to range problems Demographic information
Value issues related non-unique fields Economic data
Temporal or state issues World event information
Name and address data that can be referenced against
existing reference sets
16. Approach to MDM Implementation
Master Data Profiling
Data Profiling focuses on conducting an assessment of actual data and data structures. It helps provide the following:
Identifies data quality issues - measurements are taken against a number of dimensions, to help identify issues at the individual
attribute level, at the table-level and between tables.
Captures metadata.
Identifies business rules – The next step is to perform the data mapping. Data profiling will assist in gaining an understanding of
the data held in the system and in identifying business rules for handling the data. This will feed into the future data mapping
exercise.
Assesses the source system data to satisfy the business requirements. The focus is on gaining a very detailed understanding of the
source data that will feed the MDM target system, to ensure that the quality level is sufficient to meet the requirements.
Perform Table Perform Multi
Perform Column Finalize Data
Profiling Tables Profiling
Profiling Quality Report
(Analyze Data (Analyze redundancy
(Analysis of single (Signoff of Data
across rows in and referential
or complex field) Quality Report)
single table) integrity issues)
Major Deliverables
Data Quality Assessment Report (per Source System)
Data Quality Metrics updated to Metadata Repository
Mapping Rules and Business Rules updated to Metadata Repository
17. Approach to MDM Implementation
Master Data Profiling
1.Column Input: 3.Multi-Table Input:
Profiling Profiling Completion of Table Profiling
Information Requirements for column-level data Information Requirements for multi-table level data
analysis analysis
Relevant data extracts Relevant data extracts
Output:
Output: Completion of Multi-Table Profiling
Redundancy Analysis will identify:
Completion of Column Profiling Potential relationships with fields in other tables
Understanding all the fields and document their Redundant data between tables
descriptions in the profiling tool Potential referential integrity issues eg.
Completion of the relevant sections of the Data Quality Identification of orphans records
Assessment Report Completion of the relevant sections of the Data Quality
Updates to metadata repository Assessment Report
Updates to metadata repository
2.Table Input:
Profiling 4. Quality Input:
Completion of Column Profiling Report Completion of Column Profiling
Information Requirements for table-level data analysis Completion of Table Profiling
Relevant data extracts Completion of Multi-Table Profiling
Output: Output:
Completion of the Data Quality Assessment Report
Completion of Table Profiling
Understand all the fields and document their
descriptions in the profiling tool
Primary keys for each table
Completion of the relevant sections of the Data Quality
Assessment Report
Updates to metadata repository
18. Approach to MDM Implementation
Master Data Migration
An MDM program will typically involve a migration of historical data across systems, into or through a centralized
hub. This is where many of the data quality issues are resolved in a progressive fashion before operationalizing
some of these rule-sets for the ongoing implementation.
Prod
Target 7
Data Producers
(ERP, CRM, Test
Legacy, …) Target 6
Data Integration
Migration Staging Integrated Data Store 5
Transformations
• Attribute Scan • Common Data Model
1
• Tables Scan • Detailed Data
• Assessment • Apply Re-engineering rules
• Reporting
4
2
Data Profiling Data Data Re-
Integration engineering Master Data
Modelling
Metadata Management
3
19. Approach to MDM Implementation
Master Data Migration
The key activities in the MDM migration process include:
1. Extraction of data from producers (ERP, CRM, Legacy systems, …) into a staging area.
2. The data in the staging area will be profiled to measure down columns, across rows and between tables. This information will
be used to determine which business rules and transformations need to be invoked early in the process.
3. Metadata such as data mapping rules will begin to be established at this time. Data Standards will be agreed to and invoked at
this stage in preparation for data movement. All source attributes will be mapped into the target attributes within the
metadata management environment.
4. All agreed to transformations and standardizations required to move the data into the staging area for testing and production
are implemented. The data is moved into the Integrated Data Store.
5. Data Profiling is done again and measured against the agreed upon move success criteria for all steps up to this point.
Additional data standardizations are performed in to assist in the data matching and generally measure data quality against
agreed upon criteria. After the standardizations the rules for which records can not or should not be moved are applied. It
expected that this step will require considerable analysis.
6. This step involves the actual move of the data into either the testing environment
7. Data is loaded into the production system where some further data quality cleanup may be required. Production Verification
Testing is conducted, which should also include functional testing of features that are environment specific. After testing is
complete, the system is activated as a live production system.
20. Approach to MDM Implementation
Master Data Architecture
The Master Data Architecture defines in detail the Solution Architecture for the MDM environment. The Solution Architecture provides
the overall technology solution for a specific increment and ties together the overall approach.
Define ETL conceptual Design Define SDLC conceptual Design Define Security conceptual Design
- List of sources - Testing Strategy - Security Standards
- List of targets - SDLC Procedures -Security Requirements
- Testing Plans for Applications & Infrastructure
- Major Transformations Deliverables:
-Deployment Plan
- Estimate volumes • Security Implementation Document
- Timing Deliverables:
• SDLC procedures document
Deliverables: • Testing Plan Define Infrastructure Management
• ETL Design architecture • Deployment Plan conceptual Design
• ETL Implementation Software -Backup & Recovery
-Archiving
Documents
-Controlling & Monitoring
• ETL Technical architecture document - Environments (dev, test, prod) setup
Define Metadata Management
conceptual Design Deliverables:
- Business definition of the data • Configuration Management Document
- Physical data models
Define Data Quality Processes - Data Re-Engineering metadata
- Data model -Data Quality metadata formulas used to MDM Software Implementation
-Profiling derive data -Software Implementation Planning
- Re-engineering - Parameterization/Configuration
Deliverables: - Software Testing and deployment
Deliverables: • Metadata Design architecture
• Data Quality Design architecture • Metadata Implementation Software Deliverables:
Documents • Software Installation and Configuration
• Data Quality Implementation
• Metadata Technical architecture Document
Software Documents
document
• Data Quality Technical architecture
document
Master Data Solution Architecture
21. Approach to MDM Implementation
Prototyping the Architecture
Prototyping the architecture helps to :
• test some of the major technology risk areas for the proposed MDM Solution Architecture
• gain a better understanding of how the solution will work before moving into a more formalized
design process.
• Prototyping the proposed solution should provide an end-to-end approach that includes each
of the major components of the architecture.
22. Approach to MDM Implementation
MDM - Key Lessons Learned
In an MDM implementation, there are some key lessons learned that should be considered
when initiating an MDM program.
Key Lessons
Joint business and IT team
Make the case for change
Data as a common good
Think big but start small
Measure and communicate success
Processes first, technology last
Business ownership of data
Roles and responsibilities
Data cleanliness and migration
Communicate, communicate, communicate !
23. Knowledge, is quite simply question of sharing.
http://intelligenteenterprise.blogspot.com/
http://www.linkedin.com/in/albel
Notas del editor
Major deliverables and points to be addressed when setting up the roadmap & foundation activities :Detailed Project Roadmap Testing and Deployment plans Detailed Information ModellingDetailed Migration Plan (historical Data) Recommended process and system changes for improved Data Governance Identification of root causes leading to Data Governance issues Data Governance Metrics Quantitative Data Investigation Improved Data Quality Create/Revise Solution Architecture Ensure the availability of Software Development Environment