SlideShare a Scribd company logo
1 of 22
CONNECT. TRANSFORM. AUTOMATE.
Welcome.
Berk Bayer
The Application of Feature Manipulation Engine (FME) to Maintain
Quality Control (QC) over Standardized Spatial Data
Introduction
 I am a Data Management Specialist with the
Northrop Grumman Corporation working at the
Bureau of Land Management National Operations
Center (NOC) in Denver, Colorado.
 The NOC harvests and maintains standardized
spatial data from the twelve administrative states
in the Bureau.
Today’s Agenda
 Advantages of Data Standardization
 FME & Data Quality
 Geometry Validator Transformer
 Building Geometry Checks
 Tester Transformer
 Building Attribute Checks
 Interpreting The Error Table
 Maintaining QC
Advantages of Data
Standardization
 Data standardization ensures common
understanding of content and quality.
 Data standardization serves to resolve data
anomalies and data conflicts.
 Data standardization optimizes resources needed
for organizational data calls.
FME & Data Quality
 Quality Control (QC) over standardized data:
Improves data integrity.
Builds confidence in critical business decisions.
Builds trust in the organization.
 FME helps to develop an effective QC plan.
FME & Data Quality
 FME tools provide robust data management
capabilities:
Geometry Validator transformer
Tester transformer
Geometry Validator Transformer
 As features are harvested from the states, they
undergo a geometry validation process:
FME attempts to repair any features with null,
degenerate/corrupt or self-intersecting geometries.
An error report is created showing all features and
it’s status as passing, repaired, or not repaired.
Building Geometry Checks
 Our organization uses a similar outline of the
diagram below to validate feature geometry:
Building Geometry Checks
Building Geometry Checks
Building Geometry Checks
Building Geometry Checks
Tester Transformer
 Our organization uses the tester transformer to
evaluate the attribute integrity of harvested
features.
 The tester transformer ensures that attribute
values meet the established design constructs.
 Attributes that violate design considerations are
noted in the error report.
Building Attribute Checks
Building Attribute Checks
Building Attribute Checks
Building Attribute Checks
Building Attribute Checks
Building Attribute Checks
Interpreting The Error Table
Maintaining QC
 Data is periodically harvested from the states.
 Once the data satisfies QC measures, as
established by the business/program area, it may
be published for external customers.
Thank You!
 Questions?
 For more information:
 Berk Bayer (bbayer@blm.gov)
 Tom Chatfield (tchatfie@blm.gov)
 Jeff Safran (jsafran@blm.gov)
 Northrop Grumman Corporation (www.ngc.com)
 Bureau of Land Management (www.blm.gov)

More Related Content

Similar to The Application of Feature Manipulation Engine (FME) to Maintain Quality Control (QC) over Standardized Spatial Data

The Application of FME to Maintain Quality Control (QC) over Standardized Spa...
The Application of FME to Maintain Quality Control (QC) over Standardized Spa...The Application of FME to Maintain Quality Control (QC) over Standardized Spa...
The Application of FME to Maintain Quality Control (QC) over Standardized Spa...Safe Software
 
Top 50 Performance Testing Interview Questions | Edureka
Top 50 Performance Testing Interview Questions | EdurekaTop 50 Performance Testing Interview Questions | Edureka
Top 50 Performance Testing Interview Questions | EdurekaEdureka!
 
JMeter Interview Questions | Performance Testing Interview Questions | Edureka
JMeter Interview Questions | Performance Testing Interview Questions | EdurekaJMeter Interview Questions | Performance Testing Interview Questions | Edureka
JMeter Interview Questions | Performance Testing Interview Questions | EdurekaEdureka!
 
Malcolm burnside res9
Malcolm burnside res9Malcolm burnside res9
Malcolm burnside res9malcolm18
 
Jeremy Gold\'s Resume
Jeremy Gold\'s ResumeJeremy Gold\'s Resume
Jeremy Gold\'s ResumeJGold02
 
Software QA Metrics Dashboard Benchmarking
Software QA Metrics Dashboard BenchmarkingSoftware QA Metrics Dashboard Benchmarking
Software QA Metrics Dashboard BenchmarkingJohn Carter
 
Lalit Kumar_Quality Asscociate_3 years
Lalit Kumar_Quality Asscociate_3 yearsLalit Kumar_Quality Asscociate_3 years
Lalit Kumar_Quality Asscociate_3 yearsLalit Kumar
 
dataR Corporate Overview
dataR Corporate OverviewdataR Corporate Overview
dataR Corporate Overviewfpipolo
 
Mca se chapter_07_software_validation
Mca se chapter_07_software_validationMca se chapter_07_software_validation
Mca se chapter_07_software_validationAman Adhikari
 
Doors Quality Center Integration
Doors Quality Center IntegrationDoors Quality Center Integration
Doors Quality Center IntegrationBill Duncan
 
Delvin bolling resume
Delvin bolling   resumeDelvin bolling   resume
Delvin bolling resumeDelvinBolling
 
Completing the Data Equation: Test Data + Data Validation = Success
Completing the Data Equation: Test Data + Data Validation = SuccessCompleting the Data Equation: Test Data + Data Validation = Success
Completing the Data Equation: Test Data + Data Validation = SuccessRTTS
 
The Complete Guide to Building an Effective Enterprise Testing Strategy.pdf
The Complete Guide to Building an Effective Enterprise Testing Strategy.pdfThe Complete Guide to Building an Effective Enterprise Testing Strategy.pdf
The Complete Guide to Building an Effective Enterprise Testing Strategy.pdfkalichargn70th171
 
Performance Testing
Performance TestingPerformance Testing
Performance Testingrockden
 
Kent Dyer Resume
Kent Dyer ResumeKent Dyer Resume
Kent Dyer ResumeKent Dyer
 
Software Project Planning IV
Software Project Planning IVSoftware Project Planning IV
Software Project Planning IVGagan Deep
 

Similar to The Application of Feature Manipulation Engine (FME) to Maintain Quality Control (QC) over Standardized Spatial Data (20)

The Application of FME to Maintain Quality Control (QC) over Standardized Spa...
The Application of FME to Maintain Quality Control (QC) over Standardized Spa...The Application of FME to Maintain Quality Control (QC) over Standardized Spa...
The Application of FME to Maintain Quality Control (QC) over Standardized Spa...
 
Top 50 Performance Testing Interview Questions | Edureka
Top 50 Performance Testing Interview Questions | EdurekaTop 50 Performance Testing Interview Questions | Edureka
Top 50 Performance Testing Interview Questions | Edureka
 
JMeter Interview Questions | Performance Testing Interview Questions | Edureka
JMeter Interview Questions | Performance Testing Interview Questions | EdurekaJMeter Interview Questions | Performance Testing Interview Questions | Edureka
JMeter Interview Questions | Performance Testing Interview Questions | Edureka
 
Malcolm burnside res9
Malcolm burnside res9Malcolm burnside res9
Malcolm burnside res9
 
Jeremy Gold\'s Resume
Jeremy Gold\'s ResumeJeremy Gold\'s Resume
Jeremy Gold\'s Resume
 
Anthony Bordley Resume
Anthony Bordley ResumeAnthony Bordley Resume
Anthony Bordley Resume
 
Software QA Metrics Dashboard Benchmarking
Software QA Metrics Dashboard BenchmarkingSoftware QA Metrics Dashboard Benchmarking
Software QA Metrics Dashboard Benchmarking
 
Lalit Kumar_Quality Asscociate_3 years
Lalit Kumar_Quality Asscociate_3 yearsLalit Kumar_Quality Asscociate_3 years
Lalit Kumar_Quality Asscociate_3 years
 
dataR Corporate Overview
dataR Corporate OverviewdataR Corporate Overview
dataR Corporate Overview
 
Mca se chapter_07_software_validation
Mca se chapter_07_software_validationMca se chapter_07_software_validation
Mca se chapter_07_software_validation
 
Doors Quality Center Integration
Doors Quality Center IntegrationDoors Quality Center Integration
Doors Quality Center Integration
 
Delvin bolling resume
Delvin bolling   resumeDelvin bolling   resume
Delvin bolling resume
 
Completing the Data Equation: Test Data + Data Validation = Success
Completing the Data Equation: Test Data + Data Validation = SuccessCompleting the Data Equation: Test Data + Data Validation = Success
Completing the Data Equation: Test Data + Data Validation = Success
 
The Complete Guide to Building an Effective Enterprise Testing Strategy.pdf
The Complete Guide to Building an Effective Enterprise Testing Strategy.pdfThe Complete Guide to Building an Effective Enterprise Testing Strategy.pdf
The Complete Guide to Building an Effective Enterprise Testing Strategy.pdf
 
Performance Testing
Performance TestingPerformance Testing
Performance Testing
 
Kent Dyer Resume
Kent Dyer ResumeKent Dyer Resume
Kent Dyer Resume
 
Syed_E_Res
Syed_E_ResSyed_E_Res
Syed_E_Res
 
Services in detail
Services in detailServices in detail
Services in detail
 
System testing
System testingSystem testing
System testing
 
Software Project Planning IV
Software Project Planning IVSoftware Project Planning IV
Software Project Planning IV
 

More from Safe Software

Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FMESafe Software
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FMESafe Software
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationSafe Software
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action:  Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action:  Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationSafe Software
 
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationBeyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationSafe Software
 
Powering Real-Time Decisions with Continuous Data Streams
Powering Real-Time Decisions with Continuous Data StreamsPowering Real-Time Decisions with Continuous Data Streams
Powering Real-Time Decisions with Continuous Data StreamsSafe Software
 
The Critical Role of Spatial Data in Today's Data Ecosystem
The Critical Role of Spatial Data in Today's Data EcosystemThe Critical Role of Spatial Data in Today's Data Ecosystem
The Critical Role of Spatial Data in Today's Data EcosystemSafe Software
 
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataCloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataSafe Software
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsSafe Software
 
The Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightThe Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightSafe Software
 
Mastering MicroStation DGN: How to Integrate CAD and GIS
Mastering MicroStation DGN: How to Integrate CAD and GISMastering MicroStation DGN: How to Integrate CAD and GIS
Mastering MicroStation DGN: How to Integrate CAD and GISSafe Software
 
Geospatial Synergy: Amplifying Efficiency with FME & Esri
Geospatial Synergy: Amplifying Efficiency with FME & EsriGeospatial Synergy: Amplifying Efficiency with FME & Esri
Geospatial Synergy: Amplifying Efficiency with FME & EsriSafe Software
 
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdfIntroducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdfSafe Software
 
Breaking Barriers & Leveraging the Latest Developments in AI Technology
Breaking Barriers & Leveraging the Latest Developments in AI TechnologyBreaking Barriers & Leveraging the Latest Developments in AI Technology
Breaking Barriers & Leveraging the Latest Developments in AI TechnologySafe Software
 
Best Practices to Navigating Data and Application Integration for the Enterpr...
Best Practices to Navigating Data and Application Integration for the Enterpr...Best Practices to Navigating Data and Application Integration for the Enterpr...
Best Practices to Navigating Data and Application Integration for the Enterpr...Safe Software
 
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataCloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataSafe Software
 
New Year's Fireside Chat with Safe Software’s Founders
New Year's Fireside Chat with Safe Software’s FoundersNew Year's Fireside Chat with Safe Software’s Founders
New Year's Fireside Chat with Safe Software’s FoundersSafe Software
 
Taking Off with FME: Elevating Airport Operations to New Heights
Taking Off with FME: Elevating Airport Operations to New HeightsTaking Off with FME: Elevating Airport Operations to New Heights
Taking Off with FME: Elevating Airport Operations to New HeightsSafe Software
 
Initiating and Advancing Your Strategic GIS Governance Strategy
Initiating and Advancing Your Strategic GIS Governance StrategyInitiating and Advancing Your Strategic GIS Governance Strategy
Initiating and Advancing Your Strategic GIS Governance StrategySafe Software
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows Safe Software
 

More from Safe Software (20)

Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action:  Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action:  Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
 
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationBeyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
 
Powering Real-Time Decisions with Continuous Data Streams
Powering Real-Time Decisions with Continuous Data StreamsPowering Real-Time Decisions with Continuous Data Streams
Powering Real-Time Decisions with Continuous Data Streams
 
The Critical Role of Spatial Data in Today's Data Ecosystem
The Critical Role of Spatial Data in Today's Data EcosystemThe Critical Role of Spatial Data in Today's Data Ecosystem
The Critical Role of Spatial Data in Today's Data Ecosystem
 
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataCloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
 
The Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightThe Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and Insight
 
Mastering MicroStation DGN: How to Integrate CAD and GIS
Mastering MicroStation DGN: How to Integrate CAD and GISMastering MicroStation DGN: How to Integrate CAD and GIS
Mastering MicroStation DGN: How to Integrate CAD and GIS
 
Geospatial Synergy: Amplifying Efficiency with FME & Esri
Geospatial Synergy: Amplifying Efficiency with FME & EsriGeospatial Synergy: Amplifying Efficiency with FME & Esri
Geospatial Synergy: Amplifying Efficiency with FME & Esri
 
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdfIntroducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
 
Breaking Barriers & Leveraging the Latest Developments in AI Technology
Breaking Barriers & Leveraging the Latest Developments in AI TechnologyBreaking Barriers & Leveraging the Latest Developments in AI Technology
Breaking Barriers & Leveraging the Latest Developments in AI Technology
 
Best Practices to Navigating Data and Application Integration for the Enterpr...
Best Practices to Navigating Data and Application Integration for the Enterpr...Best Practices to Navigating Data and Application Integration for the Enterpr...
Best Practices to Navigating Data and Application Integration for the Enterpr...
 
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataCloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
 
New Year's Fireside Chat with Safe Software’s Founders
New Year's Fireside Chat with Safe Software’s FoundersNew Year's Fireside Chat with Safe Software’s Founders
New Year's Fireside Chat with Safe Software’s Founders
 
Taking Off with FME: Elevating Airport Operations to New Heights
Taking Off with FME: Elevating Airport Operations to New HeightsTaking Off with FME: Elevating Airport Operations to New Heights
Taking Off with FME: Elevating Airport Operations to New Heights
 
Initiating and Advancing Your Strategic GIS Governance Strategy
Initiating and Advancing Your Strategic GIS Governance StrategyInitiating and Advancing Your Strategic GIS Governance Strategy
Initiating and Advancing Your Strategic GIS Governance Strategy
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
 

Recently uploaded

TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc
 
MS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectorsMS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectorsNanddeep Nachan
 
DBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor PresentationDBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor PresentationDropbox
 
Apidays Singapore 2024 - Modernizing Securities Finance by Madhu Subbu
Apidays Singapore 2024 - Modernizing Securities Finance by Madhu SubbuApidays Singapore 2024 - Modernizing Securities Finance by Madhu Subbu
Apidays Singapore 2024 - Modernizing Securities Finance by Madhu Subbuapidays
 
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...DianaGray10
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoffsammart93
 
Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native ApplicationsWSO2
 
GenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdfGenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdflior mazor
 
Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)wesley chun
 
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ..."I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...Zilliz
 
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...apidays
 
Ransomware_Q4_2023. The report. [EN].pdf
Ransomware_Q4_2023. The report. [EN].pdfRansomware_Q4_2023. The report. [EN].pdf
Ransomware_Q4_2023. The report. [EN].pdfOverkill Security
 
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...apidays
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024The Digital Insurer
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerThousandEyes
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Scriptwesley chun
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Miguel Araújo
 
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProduct Anonymous
 
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherRemote DBA Services
 

Recently uploaded (20)

TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
 
MS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectorsMS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectors
 
DBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor PresentationDBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor Presentation
 
Apidays Singapore 2024 - Modernizing Securities Finance by Madhu Subbu
Apidays Singapore 2024 - Modernizing Securities Finance by Madhu SubbuApidays Singapore 2024 - Modernizing Securities Finance by Madhu Subbu
Apidays Singapore 2024 - Modernizing Securities Finance by Madhu Subbu
 
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
 
Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native Applications
 
GenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdfGenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdf
 
Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)
 
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ..."I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
 
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
 
Ransomware_Q4_2023. The report. [EN].pdf
Ransomware_Q4_2023. The report. [EN].pdfRansomware_Q4_2023. The report. [EN].pdf
Ransomware_Q4_2023. The report. [EN].pdf
 
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Script
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
 
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
 
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a Fresher
 

The Application of Feature Manipulation Engine (FME) to Maintain Quality Control (QC) over Standardized Spatial Data

  • 1. CONNECT. TRANSFORM. AUTOMATE. Welcome. Berk Bayer The Application of Feature Manipulation Engine (FME) to Maintain Quality Control (QC) over Standardized Spatial Data
  • 2. Introduction  I am a Data Management Specialist with the Northrop Grumman Corporation working at the Bureau of Land Management National Operations Center (NOC) in Denver, Colorado.  The NOC harvests and maintains standardized spatial data from the twelve administrative states in the Bureau.
  • 3. Today’s Agenda  Advantages of Data Standardization  FME & Data Quality  Geometry Validator Transformer  Building Geometry Checks  Tester Transformer  Building Attribute Checks  Interpreting The Error Table  Maintaining QC
  • 4. Advantages of Data Standardization  Data standardization ensures common understanding of content and quality.  Data standardization serves to resolve data anomalies and data conflicts.  Data standardization optimizes resources needed for organizational data calls.
  • 5. FME & Data Quality  Quality Control (QC) over standardized data: Improves data integrity. Builds confidence in critical business decisions. Builds trust in the organization.  FME helps to develop an effective QC plan.
  • 6. FME & Data Quality  FME tools provide robust data management capabilities: Geometry Validator transformer Tester transformer
  • 7. Geometry Validator Transformer  As features are harvested from the states, they undergo a geometry validation process: FME attempts to repair any features with null, degenerate/corrupt or self-intersecting geometries. An error report is created showing all features and it’s status as passing, repaired, or not repaired.
  • 8. Building Geometry Checks  Our organization uses a similar outline of the diagram below to validate feature geometry:
  • 13. Tester Transformer  Our organization uses the tester transformer to evaluate the attribute integrity of harvested features.  The tester transformer ensures that attribute values meet the established design constructs.  Attributes that violate design considerations are noted in the error report.
  • 21. Maintaining QC  Data is periodically harvested from the states.  Once the data satisfies QC measures, as established by the business/program area, it may be published for external customers.
  • 22. Thank You!  Questions?  For more information:  Berk Bayer (bbayer@blm.gov)  Tom Chatfield (tchatfie@blm.gov)  Jeff Safran (jsafran@blm.gov)  Northrop Grumman Corporation (www.ngc.com)  Bureau of Land Management (www.blm.gov)

Editor's Notes

  1. Just a brief introduction about me and the organization. I am a Data Management Specialist with the Northrop Grumman Corporation working at the Bureau of Land Management National Operations Center in Denver, Colorado. The Bureau is one of the agencies in the Department of the Interior that help manage federal lands. One of the many key functions of my position is exercising quality control over data and helping our internal customers meet and improve their data workflows. Our internal customers are primarily the twelve administrative states in the Bureau that have been delegated management responsibility over areas of federal land. The state data is standardized through a data cross-walk scheme and replicated to the NOC where it begins the QC process.
  2. Here is an overview of items that we will be covering today. We will be discussing key advantages of data standardization, raising awareness of data quality and how our organization incorporates FME with the QC workflow. We will be discussing the FME transformers that handle the data quality checks and how these checks are translated to an error report. We will also touch on the basic objectives of an effective QC plan.
  3. The initiative driving data standardization is the fact that standardized data ensures a common understanding of content and quality across the states. Data standardization serves to resolve challenges such as data anomalies and data conflicts. Data standardization aims to better support national business directives by expediting the process with which decisions may be made while helping to maintain organized data structure and improving query of data themes. Data standardization also optimizes available resources needed to successfully complete national data calls.
  4. Maintaining consistent and effective QC practices improves data integrity. Customers are able to effectively plan objectives and execute decisions when they are provided with clean data. Acknowledgment of quality data from our customers builds trust and partnership in the organization. FME helps users develop a QC plan that will ensure effective management of spatially enabled standardized data.
  5. FME tools host a wide range of capability that can allow users to effectively manage data. The Geometry Validator transformer detects spatial issues in features and attempts to repair these issues. The Tester transformer evaluates features based on the design constructs of a particular attribute field.
  6. When features are replicated from the states, they undergo the following geometry validation process where a network of Geometry Validator transformers are employed to execute spatial QC checks. Those features that may present null, corrupt, or self intersecting geometries are attempted to be repaired by FME. Features are assigned a geometry validation status as passing, repaired, or unable to be repaired and written to an error report.
  7. This is a conceptual diagram that captures how self intersections are handled by FME. The features that are channeled through all ports are assigned a status and the outputs are written to the error report. In the following, we will explore the properties of this diagram:
  8. This diagram captures the properties of the Geometry Validator that allow us to check for self intersecting features. The potential issue is selected in the corresponding checkbox and the repair option is enabled down below.
  9. In our example, the column in the error report that will hold the status of the spatial checks is declared GeometryError. The status of features without self intersecting geometry, in other words features that Pass the check, will be declared n/a, or not applicable.
  10. Features that may initially fail the self intersections check and are not repairable by FME are assigned the status of “Unable to repair” under the GeometryError column in the error table.
  11. Features that may initially fail the self intersections check but are successfully repaired by FME are assigned the status “Repaired by FME” and held under the GeometryError column in the error report.
  12. As the Geometry Validator helps us track spatial integrity, the Tester transformer helps us track attribute integrity. Each attribute in our database schema is intended to capture the design considerations realized for a particular business need. Tracking and maintaining this type of attribute integrity is central to the flow of the business process. Similar to Geometry errors, attributes that violate the established constructs are noted in the error report.
  13. This conceptual diagram illustrates the State Allotment Attribute denoted by “ST underscore ALLOT”. This particular model is comprised of two Tester transformers that will query the attribute design constructs of the input features. Similar to the spatial checks, the attribute status of the features are recorded in similar fashion in the error report as null or does not exist, invalid, or passing. Let’s explore this model in more detail:
  14. In the first tester transformer, we check to see whether a value for the “state allotment” attribute exists in the input features, using the operator “Attribute Exists”.
  15. Any feature that does not contain a value for the State Allotment attribute is written to the error report under the ST_ALLOT_ERROR column and assigned a tracking status indicating that the value was “NULL” or “did not exist”.
  16. If it is determined that an attribute exists in the first Tester transformer, then test clauses are developed in the second tester transformer to reflect the business rules that govern this particular attribute field. In this particular case, the appropriate attribute value must not equal the value noted in the first clause AND the attribute must be populated as a concatenation of two other attribute field values in respective order, as noted.
  17. Features that contain a value for the State Allotment attribute field yet fail the attribute design constructs are captured in the error report and assigned a tracking status of “ST_ALLOT is Invalid” under the ST_ALLOT_ERROR column .
  18. Features that contain a State Allotment attribute value and that the value passes the parameters of the design constructs are assigned a passing tracking status under the ST_ALLOT_ERROR column in the error table.
  19. The features are compiled from the state geodatabases (noted by the Unique Identifiers on the left) and then they are automatically input through the spatial and attribute checks in the FME workbench. This is a conceptual automated error report that is generated and emailed to the states once the FME workbench translation is executed. The states then load this table and their associated dataset into GIS software and spatially link this table to their dataset based on the feature GlobalID. The states then perform updates to their data according to the status of each feature in the error report. For instance, this particular feature has a ST_ALLOT value that is invalid but it does not have any Geometry Errors.
  20. Data is periodically harvested from the states to help sustain QC efforts. Once it is determined that the internal data meets a satisfactory level of quality, the data must then pass a certification process by the business area, before it may become available for our external customers.