SlideShare una empresa de Scribd logo
1 de 17
Data Quality Architecture
 Phase 1 – Account Verification

          Art Nicewick
Project Scope
• Define a Architectural flow diagram that
  provides for basis for data governance, data
  quality and impact analysis

• Create a framework to report on
  inconsistencies in data (Initial emphasis on
  Accounts)
FISMA
• The architecture provides a foundation for verifying
  that Accounts are deleted after the employees leave
  the Gallery
• The Exceptions Facility, Provides the ability for a
  application administrator to request that an Non-AD
  account be left on file
   – Audit trails
   – Non Standard accounts (e.g. TDP as Custodian)
   – CIO can approvedeny and give timelines for resolutions
• Focus of first phase of the initiative
Why Consistency Reports
• Common Practice (Asset Inventory, …)
• Ensures that data is corrected in the correct
  manner
• Re-validates automated processes
• Some changes need to be informed to system
  manager (e.g. They should know if someone
  has a new last name)
• Links into existing manual pratices
General Data Quality Process
1. Identify data stores (Based on priority)
2. Identify authoritative data
3. Identify Interfaces  replicated  redundant
   data
4. Identify consistency analysis process
5. Correct and continuous monitoring
Identify data stores
• 1.1. Create list of all know data applications
   – Define the name of the data application
   – Define the contacts related to the application
      • TDP Contact
      • Application Administrator
   – Categorize the application
Identify data stores
• 1.2. Link data into data flow representation
  for a visual analysis on enterprise data flows
Identify authoritative data
• 2.1. Review Application data to determine
  – What type of data is supported
  – Is data authoritative
Identify Interfaces  replicated 
                 redundant data
• 2.1. Review Application data to determine
   –   Where the data is sent
   –   Where the data is received from
   –   Data Quality
   –   Note: Source assumed by reverse lookup of target definitions
Identify Interfaces  replicated 
           redundant data
• Diagram linkages between data stores for
  visual review and impact analysis
Identify consistency analysis process
            Review participating data sources and
            determine how to define consistency




* At this point only “SQL” methods are used.
Correct and continuous monitoring

• Inconsistencies are periodically sent to end users for “correction” or
  “exceptions”
• Valid exceptions may be
    – “Supervisor Accounts Outside Active Directory” (e.g. TMSAdmin)
    – Ex-Employees with data attached to userid
    – Contractor or testing userid
Correct and continuous monitoring
• Users can review and update exceptions
  online
Correct and continuous monitoring
• Administrators can create schedules and Email
  recipients
Correct and continuous monitoring
• Email can be sent to
  as many people as
  desired and as
  frequently (or
  infrequently) as
  desired.
Target Data
•   Userids (First Phase and Proof of concept)
•   Object Data
•   Location data
•   Employee Names and Titles
•   Other ..
Challenges
•   Object data (Portfolio)
•   Non-SQL Data (Filemaker)
•   Secure Data (Tradewin)
•   Desktop Data (Excel)
•   Offsite data (FMS)
•   Other …

Más contenido relacionado

La actualidad más candente

Data quality overview
Data quality overviewData quality overview
Data quality overview
Alex Meadows
 

La actualidad más candente (20)

DAS Slides: Data Quality Best Practices
DAS Slides: Data Quality Best PracticesDAS Slides: Data Quality Best Practices
DAS Slides: Data Quality Best Practices
 
Data quality overview
Data quality overviewData quality overview
Data quality overview
 
Data Quality
Data QualityData Quality
Data Quality
 
Data Architecture Strategies: Data Architecture for Digital Transformation
Data Architecture Strategies: Data Architecture for Digital TransformationData Architecture Strategies: Data Architecture for Digital Transformation
Data Architecture Strategies: Data Architecture for Digital Transformation
 
Data quality and data profiling
Data quality and data profilingData quality and data profiling
Data quality and data profiling
 
Data Profiling, Data Catalogs and Metadata Harmonisation
Data Profiling, Data Catalogs and Metadata HarmonisationData Profiling, Data Catalogs and Metadata Harmonisation
Data Profiling, Data Catalogs and Metadata Harmonisation
 
Data quality metrics infographic
Data quality metrics infographicData quality metrics infographic
Data quality metrics infographic
 
Introduction to Data Governance
Introduction to Data GovernanceIntroduction to Data Governance
Introduction to Data Governance
 
DAS Slides: Data Quality Best Practices
DAS Slides: Data Quality Best PracticesDAS Slides: Data Quality Best Practices
DAS Slides: Data Quality Best Practices
 
Data Governance
Data GovernanceData Governance
Data Governance
 
DMBOK - Chapter 1 Summary
DMBOK - Chapter 1 SummaryDMBOK - Chapter 1 Summary
DMBOK - Chapter 1 Summary
 
Data Quality - Standards and Application to Open Data
Data Quality - Standards and Application to Open DataData Quality - Standards and Application to Open Data
Data Quality - Standards and Application to Open Data
 
Data Governance Maturity Model Thesis
Data Governance Maturity Model ThesisData Governance Maturity Model Thesis
Data Governance Maturity Model Thesis
 
Data Governance Powerpoint Presentation Slides
Data Governance Powerpoint Presentation SlidesData Governance Powerpoint Presentation Slides
Data Governance Powerpoint Presentation Slides
 
Data Quality
Data QualityData Quality
Data Quality
 
Data Quality Dashboards
Data Quality DashboardsData Quality Dashboards
Data Quality Dashboards
 
Data Quality Presentation
Data Quality PresentationData Quality Presentation
Data Quality Presentation
 
Data Monetization
Data MonetizationData Monetization
Data Monetization
 
BI Consultancy - Data, Analytics and Strategy
BI Consultancy - Data, Analytics and StrategyBI Consultancy - Data, Analytics and Strategy
BI Consultancy - Data, Analytics and Strategy
 
Building a Data Strategy – Practical Steps for Aligning with Business Goals
Building a Data Strategy – Practical Steps for Aligning with Business GoalsBuilding a Data Strategy – Practical Steps for Aligning with Business Goals
Building a Data Strategy – Practical Steps for Aligning with Business Goals
 

Similar a Data quality architecture

Building the enterprise data architecture
Building the enterprise data architectureBuilding the enterprise data architecture
Building the enterprise data architecture
Costa Pissaris
 
Mis system analysis and system design
Mis   system analysis and system designMis   system analysis and system design
Mis system analysis and system design
Rahul Hedau
 
Data Governance Overview - Doreen Christian
Data Governance Overview - Doreen ChristianData Governance Overview - Doreen Christian
Data Governance Overview - Doreen Christian
Doreen Christian
 

Similar a Data quality architecture (20)

Data flow ii extract
Data flow   ii extractData flow   ii extract
Data flow ii extract
 
AIS PPt.pptx
AIS PPt.pptxAIS PPt.pptx
AIS PPt.pptx
 
DATA WAREHOUSE -- ETL testing Plan
DATA WAREHOUSE -- ETL testing PlanDATA WAREHOUSE -- ETL testing Plan
DATA WAREHOUSE -- ETL testing Plan
 
Various Applications of Data Warehouse.ppt
Various Applications of Data Warehouse.pptVarious Applications of Data Warehouse.ppt
Various Applications of Data Warehouse.ppt
 
DW (1).ppt
DW (1).pptDW (1).ppt
DW (1).ppt
 
System design
System designSystem design
System design
 
Building the enterprise data architecture
Building the enterprise data architectureBuilding the enterprise data architecture
Building the enterprise data architecture
 
Mis system analysis and system design
Mis   system analysis and system designMis   system analysis and system design
Mis system analysis and system design
 
Data Governance Overview - Doreen Christian
Data Governance Overview - Doreen ChristianData Governance Overview - Doreen Christian
Data Governance Overview - Doreen Christian
 
Soft requirement
Soft requirementSoft requirement
Soft requirement
 
22-REQUIREMENT.ppt
22-REQUIREMENT.ppt22-REQUIREMENT.ppt
22-REQUIREMENT.ppt
 
Overview of Function Points Analysis
Overview of Function Points Analysis Overview of Function Points Analysis
Overview of Function Points Analysis
 
Function Points
Function PointsFunction Points
Function Points
 
5.Developing IT Solution.pptx
5.Developing IT Solution.pptx5.Developing IT Solution.pptx
5.Developing IT Solution.pptx
 
Systems Development and Documentation Techniques
Systems Development and Documentation TechniquesSystems Development and Documentation Techniques
Systems Development and Documentation Techniques
 
Topic5 - IT Implementation & Challenges.pptx
Topic5 - IT Implementation & Challenges.pptxTopic5 - IT Implementation & Challenges.pptx
Topic5 - IT Implementation & Challenges.pptx
 
rough-work.pptx
rough-work.pptxrough-work.pptx
rough-work.pptx
 
Webinar: 10-Step Guide to Creating a Single View of your Business
Webinar: 10-Step Guide to Creating a Single View of your BusinessWebinar: 10-Step Guide to Creating a Single View of your Business
Webinar: 10-Step Guide to Creating a Single View of your Business
 
ITFT- Dbms
ITFT- DbmsITFT- Dbms
ITFT- Dbms
 
An Introduction to Clinical Study Migrations
An Introduction to Clinical Study MigrationsAn Introduction to Clinical Study Migrations
An Introduction to Clinical Study Migrations
 

Más de anicewick

Más de anicewick (6)

Semantic web2
Semantic web2Semantic web2
Semantic web2
 
Defining conservation taxonomy
Defining conservation taxonomyDefining conservation taxonomy
Defining conservation taxonomy
 
Creating an RAD Authoratative Data Environment
Creating an RAD Authoratative Data EnvironmentCreating an RAD Authoratative Data Environment
Creating an RAD Authoratative Data Environment
 
FISMA Compliance
FISMA ComplianceFISMA Compliance
FISMA Compliance
 
User Interface Patterns and Nuxeo
User Interface Patterns and NuxeoUser Interface Patterns and Nuxeo
User Interface Patterns and Nuxeo
 
Understanding Document Managment Systems and Nuxeo
Understanding Document Managment Systems and NuxeoUnderstanding Document Managment Systems and Nuxeo
Understanding Document Managment Systems and Nuxeo
 

Último

Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
 
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Victor Rentea
 

Último (20)

ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
 
DBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor PresentationDBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor Presentation
 
[BuildWithAI] Introduction to Gemini.pdf
[BuildWithAI] Introduction to Gemini.pdf[BuildWithAI] Introduction to Gemini.pdf
[BuildWithAI] Introduction to Gemini.pdf
 
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost SavingRepurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
 
Mcleodganj Call Girls 🥰 8617370543 Service Offer VIP Hot Model
Mcleodganj Call Girls 🥰 8617370543 Service Offer VIP Hot ModelMcleodganj Call Girls 🥰 8617370543 Service Offer VIP Hot Model
Mcleodganj Call Girls 🥰 8617370543 Service Offer VIP Hot Model
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
 
ICT role in 21st century education and its challenges
ICT role in 21st century education and its challengesICT role in 21st century education and its challenges
ICT role in 21st century education and its challenges
 
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
Elevate Developer Efficiency & build GenAI Application with Amazon Q​
Elevate Developer Efficiency & build GenAI Application with Amazon Q​Elevate Developer Efficiency & build GenAI Application with Amazon Q​
Elevate Developer Efficiency & build GenAI Application with Amazon Q​
 
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
 
MS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectorsMS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectors
 
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ..."I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
 
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
 
Vector Search -An Introduction in Oracle Database 23ai.pptx
Vector Search -An Introduction in Oracle Database 23ai.pptxVector Search -An Introduction in Oracle Database 23ai.pptx
Vector Search -An Introduction in Oracle Database 23ai.pptx
 
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a Fresher
 
Artificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : UncertaintyArtificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : Uncertainty
 
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
 
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
 

Data quality architecture

  • 1. Data Quality Architecture Phase 1 – Account Verification Art Nicewick
  • 2. Project Scope • Define a Architectural flow diagram that provides for basis for data governance, data quality and impact analysis • Create a framework to report on inconsistencies in data (Initial emphasis on Accounts)
  • 3. FISMA • The architecture provides a foundation for verifying that Accounts are deleted after the employees leave the Gallery • The Exceptions Facility, Provides the ability for a application administrator to request that an Non-AD account be left on file – Audit trails – Non Standard accounts (e.g. TDP as Custodian) – CIO can approvedeny and give timelines for resolutions • Focus of first phase of the initiative
  • 4. Why Consistency Reports • Common Practice (Asset Inventory, …) • Ensures that data is corrected in the correct manner • Re-validates automated processes • Some changes need to be informed to system manager (e.g. They should know if someone has a new last name) • Links into existing manual pratices
  • 5. General Data Quality Process 1. Identify data stores (Based on priority) 2. Identify authoritative data 3. Identify Interfaces replicated redundant data 4. Identify consistency analysis process 5. Correct and continuous monitoring
  • 6. Identify data stores • 1.1. Create list of all know data applications – Define the name of the data application – Define the contacts related to the application • TDP Contact • Application Administrator – Categorize the application
  • 7. Identify data stores • 1.2. Link data into data flow representation for a visual analysis on enterprise data flows
  • 8. Identify authoritative data • 2.1. Review Application data to determine – What type of data is supported – Is data authoritative
  • 9. Identify Interfaces replicated redundant data • 2.1. Review Application data to determine – Where the data is sent – Where the data is received from – Data Quality – Note: Source assumed by reverse lookup of target definitions
  • 10. Identify Interfaces replicated redundant data • Diagram linkages between data stores for visual review and impact analysis
  • 11. Identify consistency analysis process Review participating data sources and determine how to define consistency * At this point only “SQL” methods are used.
  • 12. Correct and continuous monitoring • Inconsistencies are periodically sent to end users for “correction” or “exceptions” • Valid exceptions may be – “Supervisor Accounts Outside Active Directory” (e.g. TMSAdmin) – Ex-Employees with data attached to userid – Contractor or testing userid
  • 13. Correct and continuous monitoring • Users can review and update exceptions online
  • 14. Correct and continuous monitoring • Administrators can create schedules and Email recipients
  • 15. Correct and continuous monitoring • Email can be sent to as many people as desired and as frequently (or infrequently) as desired.
  • 16. Target Data • Userids (First Phase and Proof of concept) • Object Data • Location data • Employee Names and Titles • Other ..
  • 17. Challenges • Object data (Portfolio) • Non-SQL Data (Filemaker) • Secure Data (Tradewin) • Desktop Data (Excel) • Offsite data (FMS) • Other …