SlideShare una empresa de Scribd logo
1 de 20
MGCP4LCSS Workflow
Engineering Section
Mapping & Charting Establishment
1
Martin de Zuviria
Amanda Chiprout
BattleView Overview
5
MGCP4LCSS OVERVIEW
The main GOAL of the workflow (MGCP4LCSS) is to provide
the LCSS user with the ability to view and interact with
Symbolized MGCP vector data through multiple Web Services.
The Geospatial Data Manager will need three items to execute
this process:
- MGCP vector data in Shapefile format
- FME Desktop
- ArcGIS software applications
MGCP4LCSS WORKFLOW
 The MGCP4LCSS workflow performs:
– an extraction of selected features contained in the MGCP
vector data
– transform them to meet the schema and
– visualization requirements for LCSS Battleview.
– The final output was a FGDB using representations as
symbology to be consumed in a WMS/WFS.
 The workflow created by Martin provides a step-by-
step description on how a user may proceed from
beginning to end.
 The workflow has been proven to work in ArcGIS Data
Interoperability.
Web Optimization
 MGCP data is rich with content
 MGCP not optimized for Web Services
 Schema Optimized for Web:
– Cut feature count in half
– ‘Human Readable’
– Scaled to GRLS
(Global/Regional/Local/Specialized)
– Symbolized using Topographic Line Maps
8
Blue is data conditioning team
Green is the Geo Field Teams
MCE MGCP4LCSS Production Process
Input: MGCP
Shapefiles
IGW
Filter Shapefiles &
categorization
Pre-Conversion
Check – QC/QA
Aggregate Features
Segregate Features Feature Description Symbology Coding Load & Calculate
FGDB
Battleview
schema with
representations
FME 2013 FME 2013 FME 2013
FME 2013 FME 2013 FME 2013 Arc GIS 10.1
IMO/Chief
Geo
Receives
FGDB
DMAN Team Ingest/
load on Geo Server
WFS Services built
and tested
Live on LCSS in
operational theatre
FME Workspace
An FME workspace contains source and destination types of data and attributes, as
well as transformers that manipulate data.
Bookmarks have been added to easily find and re-focus the main display on any
destination feature class (e.g. Helipad_P)
Custom transformers, represented in green color, have also been added to allow an
easy update of the settings of specific transformers that are of common use within
the same workspace (e.g. ‘Source Name’)
10
Step 1 : Feature Filter
11
There are different ways to filter data with FME:
1) Readers
2) Transformers such as ‘TestFilters’ and ‘GeometryFilters’;
3) Enabling/disabling components of the FME workspace (e.g. by
enabling only all objects within the Road_L bookmark)
Step 2: QA/QC Workspace
12
QA/QC results are:
- re-directed results to an Inspector
- using other reference layer(s),
- analyzing log files.
Note: ‘Data Quality’ is regarded here as the degree of excellence of data to satisfy
a given objective. Any tile of MGCP released has to pass extensive QA.
MGCP data has to meet new requirements in its transformation to a FGDB
format/schema and Web Mapping Services (OGC Standards) and therefore has to
be re-tested for these new requirements, regardless of previous QA performed.
Steps 3,5,6: Feature Merge/Aggregate Output
13
Feature Merging and aggregation to meet the target schema architecture:
- merging (ie ‘Extraction Mine Areas’ and ‘Quarry Areas’)
- managing schema
- coding symbology
Example Below: In the legend of the output map document, the ‘Extraction Mine
Areas’ and ‘Quarry Areas’ have been merged into a single feature class.
The bottom table shows the descriptive attributes replaced several coded values
14
Steps 3,5,6: Feature Merge/Aggregate Output
Example in FME
Example:
- Merge ‘Extraction Mine Areas’ and ‘Quarry Areas’,
- manage schema and
- Descriptive names and coding symbology using the ‘RepID’ field
Usefulness of Custom Transformers
15
Custom transformers were used to replace a sequence of transformers used in the
workflow. Some of these transformers contained long LUTs used to convert numeric
values to descriptive names. Since these LUTs may be regularly updated by having
new source values like in the example shown on the slide, the ability to update them
only once in the workflow saves time and reduces the risk of errors.
16
Steps 4,5,6: Feature Segregation and
Descriptive Names:
Building points are segregated into a new feature class and given
‘Human Readable Names’
i.e. ‘Transportation_Station’ and ‘Vertical_Obstruction’
17
Steps 4,5,6: Feature Segregation and Descriptive
Names:
Building points are segregated into a new feature class and given
‘Human Readable Names’
i.e. ‘Transportation_Station’ and ‘Vertical_Obstruction’
Step 7: Feature Loading to the Destination Empty FGDB
and Calculation of Symbology Representation Values
18
FME Output data is loaded into an empty FGDB with identical schema plus the
desired Representations, and symbology IDs are linked to Representation Rules
(values from ‘RepID’ are copied to the ‘RuleID’ field, tied to representation rules)
Example: ArcGIS ETL Detail (partial – created using ArcGIS Model Builder)
MGCP4LCSS V3 Workflow: Information Structure
Step 1:
Create Parent Folder Structure
Step 2:
Create Sub Folder where MGCP input
SHPs must be copied
Step 3:
Subfolder Output FGDBs:
- The contents of subfolders with empty
FGDBs must be copied here
Step 4:
Run Workbenches and BatchRunner ETL
for a ‘single click’ processing of steps 1 to
6
Step 6:
Subfolder ‘ArcGIS Output
Map’ with an ArcGIS Map for
visualization of output
Step 5:
‘ArcGIS Toolboxes’ containing
ArcGIS ETL Tools for
processing of step 7
20
MGCP4LCSS Workflow
Pre-conditions
The designated Geospatial Data Manager has copied the ‘MGCP4LCSS Package’ to his/her
local C:/ drive and has FME Desktop (2013 release or later) and ArcGIS (10.1 release or later)
installed on his/her computer
Trigger
The user wants to covert MGCP tiles provided as shapefiles to the FGDB schema and
representations defined by DND MCE Engineering, for subsequent use in any LCSS
application
Main Flow
1. Copy the subfolder /MGCP of the ‘MGCP4LCSS’ package on the local C:/ drive
2. Copy the MGCP tiles to the C:/MGCP/INPUT MGCP TILES folder
3. Open FME Desktop and run each of the individual eight workspaces contained in the
subfolder ‘FME Workbenches’
4. Open ArcGIS – ArcCatalog and runs the six tools contained in the Toolbox
C:/MGCP/ArcGIS Toolboxes/Battleview.tbx, starting with the three ‘Append’ tools
5. Open ArcMAP MXD BVoutputFGDB.mxd to visualize the results from the output FGDB
MGCP4LCSS SOP
21

Más contenido relacionado

Destacado

1.2 workflow and_process_redesign
1.2 workflow and_process_redesign1.2 workflow and_process_redesign
1.2 workflow and_process_redesignTCT
 
Workflow and Clinical Informatics Assessment (CIA)
Workflow and Clinical Informatics Assessment (CIA)Workflow and Clinical Informatics Assessment (CIA)
Workflow and Clinical Informatics Assessment (CIA)Edward Stern
 
Simple Process Mapping Techniques
Simple Process Mapping TechniquesSimple Process Mapping Techniques
Simple Process Mapping TechniquesStephen Deas
 
HH Centralized Maintenance-Work Order Processing Cross Functional Workflow
HH Centralized Maintenance-Work Order Processing Cross Functional WorkflowHH Centralized Maintenance-Work Order Processing Cross Functional Workflow
HH Centralized Maintenance-Work Order Processing Cross Functional WorkflowTimothy L. Butler, H.C.C.P.
 
Process mapping with flowcharts
Process mapping with flowchartsProcess mapping with flowcharts
Process mapping with flowchartsSteven Bonacorsi
 

Destacado (7)

1.2 workflow and_process_redesign
1.2 workflow and_process_redesign1.2 workflow and_process_redesign
1.2 workflow and_process_redesign
 
Workflow and Clinical Informatics Assessment (CIA)
Workflow and Clinical Informatics Assessment (CIA)Workflow and Clinical Informatics Assessment (CIA)
Workflow and Clinical Informatics Assessment (CIA)
 
Simple Process Mapping Techniques
Simple Process Mapping TechniquesSimple Process Mapping Techniques
Simple Process Mapping Techniques
 
Cerebrum
CerebrumCerebrum
Cerebrum
 
HH Centralized Maintenance-Work Order Processing Cross Functional Workflow
HH Centralized Maintenance-Work Order Processing Cross Functional WorkflowHH Centralized Maintenance-Work Order Processing Cross Functional Workflow
HH Centralized Maintenance-Work Order Processing Cross Functional Workflow
 
Process mapping with flowcharts
Process mapping with flowchartsProcess mapping with flowcharts
Process mapping with flowcharts
 
Process Mapping
Process MappingProcess Mapping
Process Mapping
 

Similar a MGCP4LCSS Workflow

PRETZEL: Opening the Black Box of Machine Learning Prediction Serving Systems
PRETZEL: Opening the Black Box of Machine Learning Prediction Serving SystemsPRETZEL: Opening the Black Box of Machine Learning Prediction Serving Systems
PRETZEL: Opening the Black Box of Machine Learning Prediction Serving SystemsNECST Lab @ Politecnico di Milano
 
MetaConfig driven FeatureStore : MakeMyTrip | Presented at Data Con LA 2019 b...
MetaConfig driven FeatureStore : MakeMyTrip | Presented at Data Con LA 2019 b...MetaConfig driven FeatureStore : MakeMyTrip | Presented at Data Con LA 2019 b...
MetaConfig driven FeatureStore : MakeMyTrip | Presented at Data Con LA 2019 b...Piyush Kumar
 
Data Con LA 2019 - MetaConfig driven FeatureStore with Feature compute & Serv...
Data Con LA 2019 - MetaConfig driven FeatureStore with Feature compute & Serv...Data Con LA 2019 - MetaConfig driven FeatureStore with Feature compute & Serv...
Data Con LA 2019 - MetaConfig driven FeatureStore with Feature compute & Serv...Data Con LA
 
CAD STANDARDS - SMART MANUFACTURING MECH
CAD STANDARDS - SMART MANUFACTURING MECHCAD STANDARDS - SMART MANUFACTURING MECH
CAD STANDARDS - SMART MANUFACTURING MECHRAJESHS631800
 
Using Spark Mllib Models in a Production Training and Serving Platform: Exper...
Using Spark Mllib Models in a Production Training and Serving Platform: Exper...Using Spark Mllib Models in a Production Training and Serving Platform: Exper...
Using Spark Mllib Models in a Production Training and Serving Platform: Exper...Databricks
 
Big data unit iv and v lecture notes qb model exam
Big data unit iv and v lecture notes   qb model examBig data unit iv and v lecture notes   qb model exam
Big data unit iv and v lecture notes qb model examIndhujeni
 
Blue Phoenix Idms Migration
Blue Phoenix Idms MigrationBlue Phoenix Idms Migration
Blue Phoenix Idms MigrationGilShalit
 
Traffic Simulator
Traffic SimulatorTraffic Simulator
Traffic Simulatorgystell
 
Power-Efficient Programming Using Qualcomm Multicore Asynchronous Runtime Env...
Power-Efficient Programming Using Qualcomm Multicore Asynchronous Runtime Env...Power-Efficient Programming Using Qualcomm Multicore Asynchronous Runtime Env...
Power-Efficient Programming Using Qualcomm Multicore Asynchronous Runtime Env...Qualcomm Developer Network
 
DUAL FIELD DUAL CORE SECURE CRYPTOPROCESSOR ON FPGA PLATFORM
DUAL FIELD DUAL CORE SECURE CRYPTOPROCESSOR ON FPGA PLATFORMDUAL FIELD DUAL CORE SECURE CRYPTOPROCESSOR ON FPGA PLATFORM
DUAL FIELD DUAL CORE SECURE CRYPTOPROCESSOR ON FPGA PLATFORMVLSICS Design
 
Spatial Data Integrator - Software Presentation and Use Cases
Spatial Data Integrator - Software Presentation and Use CasesSpatial Data Integrator - Software Presentation and Use Cases
Spatial Data Integrator - Software Presentation and Use Casesmathieuraj
 
Understanding DB2 Optimizer
Understanding DB2 OptimizerUnderstanding DB2 Optimizer
Understanding DB2 Optimizerterraborealis
 
Understanding NonStop SQLMX SDA and its impact on performance
Understanding NonStop SQLMX SDA and its impact on performanceUnderstanding NonStop SQLMX SDA and its impact on performance
Understanding NonStop SQLMX SDA and its impact on performanceFrans Jongma
 
Multi-Utility Infrastructure Management
Multi-Utility Infrastructure Management Multi-Utility Infrastructure Management
Multi-Utility Infrastructure Management Gilbert Madrid
 

Similar a MGCP4LCSS Workflow (20)

PRETZEL: Opening the Black Box of Machine Learning Prediction Serving Systems
PRETZEL: Opening the Black Box of Machine Learning Prediction Serving SystemsPRETZEL: Opening the Black Box of Machine Learning Prediction Serving Systems
PRETZEL: Opening the Black Box of Machine Learning Prediction Serving Systems
 
MetaConfig driven FeatureStore : MakeMyTrip | Presented at Data Con LA 2019 b...
MetaConfig driven FeatureStore : MakeMyTrip | Presented at Data Con LA 2019 b...MetaConfig driven FeatureStore : MakeMyTrip | Presented at Data Con LA 2019 b...
MetaConfig driven FeatureStore : MakeMyTrip | Presented at Data Con LA 2019 b...
 
Data Con LA 2019 - MetaConfig driven FeatureStore with Feature compute & Serv...
Data Con LA 2019 - MetaConfig driven FeatureStore with Feature compute & Serv...Data Con LA 2019 - MetaConfig driven FeatureStore with Feature compute & Serv...
Data Con LA 2019 - MetaConfig driven FeatureStore with Feature compute & Serv...
 
What's new in pscad v4.6.1
What's new in pscad v4.6.1What's new in pscad v4.6.1
What's new in pscad v4.6.1
 
CAD STANDARDS - SMART MANUFACTURING MECH
CAD STANDARDS - SMART MANUFACTURING MECHCAD STANDARDS - SMART MANUFACTURING MECH
CAD STANDARDS - SMART MANUFACTURING MECH
 
Using Spark Mllib Models in a Production Training and Serving Platform: Exper...
Using Spark Mllib Models in a Production Training and Serving Platform: Exper...Using Spark Mllib Models in a Production Training and Serving Platform: Exper...
Using Spark Mllib Models in a Production Training and Serving Platform: Exper...
 
Big data unit iv and v lecture notes qb model exam
Big data unit iv and v lecture notes   qb model examBig data unit iv and v lecture notes   qb model exam
Big data unit iv and v lecture notes qb model exam
 
Parallel Processor for Graphics Acceleration
Parallel Processor for Graphics AccelerationParallel Processor for Graphics Acceleration
Parallel Processor for Graphics Acceleration
 
Blue Phoenix Idms Migration
Blue Phoenix Idms MigrationBlue Phoenix Idms Migration
Blue Phoenix Idms Migration
 
Traffic Simulator
Traffic SimulatorTraffic Simulator
Traffic Simulator
 
Power-Efficient Programming Using Qualcomm Multicore Asynchronous Runtime Env...
Power-Efficient Programming Using Qualcomm Multicore Asynchronous Runtime Env...Power-Efficient Programming Using Qualcomm Multicore Asynchronous Runtime Env...
Power-Efficient Programming Using Qualcomm Multicore Asynchronous Runtime Env...
 
J045075661
J045075661J045075661
J045075661
 
DUAL FIELD DUAL CORE SECURE CRYPTOPROCESSOR ON FPGA PLATFORM
DUAL FIELD DUAL CORE SECURE CRYPTOPROCESSOR ON FPGA PLATFORMDUAL FIELD DUAL CORE SECURE CRYPTOPROCESSOR ON FPGA PLATFORM
DUAL FIELD DUAL CORE SECURE CRYPTOPROCESSOR ON FPGA PLATFORM
 
E031201032036
E031201032036E031201032036
E031201032036
 
Apache Crunch
Apache CrunchApache Crunch
Apache Crunch
 
Spatial Data Integrator - Software Presentation and Use Cases
Spatial Data Integrator - Software Presentation and Use CasesSpatial Data Integrator - Software Presentation and Use Cases
Spatial Data Integrator - Software Presentation and Use Cases
 
Understanding DB2 Optimizer
Understanding DB2 OptimizerUnderstanding DB2 Optimizer
Understanding DB2 Optimizer
 
Compose in Theory
Compose in TheoryCompose in Theory
Compose in Theory
 
Understanding NonStop SQLMX SDA and its impact on performance
Understanding NonStop SQLMX SDA and its impact on performanceUnderstanding NonStop SQLMX SDA and its impact on performance
Understanding NonStop SQLMX SDA and its impact on performance
 
Multi-Utility Infrastructure Management
Multi-Utility Infrastructure Management Multi-Utility Infrastructure Management
Multi-Utility Infrastructure Management
 

Más de Safe Software

From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationSafe Software
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action:  Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action:  Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationSafe Software
 
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationBeyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationSafe Software
 
Powering Real-Time Decisions with Continuous Data Streams
Powering Real-Time Decisions with Continuous Data StreamsPowering Real-Time Decisions with Continuous Data Streams
Powering Real-Time Decisions with Continuous Data StreamsSafe Software
 
The Critical Role of Spatial Data in Today's Data Ecosystem
The Critical Role of Spatial Data in Today's Data EcosystemThe Critical Role of Spatial Data in Today's Data Ecosystem
The Critical Role of Spatial Data in Today's Data EcosystemSafe Software
 
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataCloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataSafe Software
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsSafe Software
 
The Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightThe Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightSafe Software
 
Mastering MicroStation DGN: How to Integrate CAD and GIS
Mastering MicroStation DGN: How to Integrate CAD and GISMastering MicroStation DGN: How to Integrate CAD and GIS
Mastering MicroStation DGN: How to Integrate CAD and GISSafe Software
 
Geospatial Synergy: Amplifying Efficiency with FME & Esri
Geospatial Synergy: Amplifying Efficiency with FME & EsriGeospatial Synergy: Amplifying Efficiency with FME & Esri
Geospatial Synergy: Amplifying Efficiency with FME & EsriSafe Software
 
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdfIntroducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdfSafe Software
 
Breaking Barriers & Leveraging the Latest Developments in AI Technology
Breaking Barriers & Leveraging the Latest Developments in AI TechnologyBreaking Barriers & Leveraging the Latest Developments in AI Technology
Breaking Barriers & Leveraging the Latest Developments in AI TechnologySafe Software
 
Best Practices to Navigating Data and Application Integration for the Enterpr...
Best Practices to Navigating Data and Application Integration for the Enterpr...Best Practices to Navigating Data and Application Integration for the Enterpr...
Best Practices to Navigating Data and Application Integration for the Enterpr...Safe Software
 
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataCloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataSafe Software
 
New Year's Fireside Chat with Safe Software’s Founders
New Year's Fireside Chat with Safe Software’s FoundersNew Year's Fireside Chat with Safe Software’s Founders
New Year's Fireside Chat with Safe Software’s FoundersSafe Software
 
Taking Off with FME: Elevating Airport Operations to New Heights
Taking Off with FME: Elevating Airport Operations to New HeightsTaking Off with FME: Elevating Airport Operations to New Heights
Taking Off with FME: Elevating Airport Operations to New HeightsSafe Software
 
Initiating and Advancing Your Strategic GIS Governance Strategy
Initiating and Advancing Your Strategic GIS Governance StrategyInitiating and Advancing Your Strategic GIS Governance Strategy
Initiating and Advancing Your Strategic GIS Governance StrategySafe Software
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows Safe Software
 
Geospatial Synergy: Amplifying Efficiency with FME & Esri ft. Peak Guest Spea...
Geospatial Synergy: Amplifying Efficiency with FME & Esri ft. Peak Guest Spea...Geospatial Synergy: Amplifying Efficiency with FME & Esri ft. Peak Guest Spea...
Geospatial Synergy: Amplifying Efficiency with FME & Esri ft. Peak Guest Spea...Safe Software
 
Mastering DevOps-Driven Data Integration with FME
Mastering DevOps-Driven Data Integration with FMEMastering DevOps-Driven Data Integration with FME
Mastering DevOps-Driven Data Integration with FMESafe Software
 

Más de Safe Software (20)

From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action:  Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action:  Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
 
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationBeyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
 
Powering Real-Time Decisions with Continuous Data Streams
Powering Real-Time Decisions with Continuous Data StreamsPowering Real-Time Decisions with Continuous Data Streams
Powering Real-Time Decisions with Continuous Data Streams
 
The Critical Role of Spatial Data in Today's Data Ecosystem
The Critical Role of Spatial Data in Today's Data EcosystemThe Critical Role of Spatial Data in Today's Data Ecosystem
The Critical Role of Spatial Data in Today's Data Ecosystem
 
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataCloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
 
The Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightThe Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and Insight
 
Mastering MicroStation DGN: How to Integrate CAD and GIS
Mastering MicroStation DGN: How to Integrate CAD and GISMastering MicroStation DGN: How to Integrate CAD and GIS
Mastering MicroStation DGN: How to Integrate CAD and GIS
 
Geospatial Synergy: Amplifying Efficiency with FME & Esri
Geospatial Synergy: Amplifying Efficiency with FME & EsriGeospatial Synergy: Amplifying Efficiency with FME & Esri
Geospatial Synergy: Amplifying Efficiency with FME & Esri
 
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdfIntroducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
 
Breaking Barriers & Leveraging the Latest Developments in AI Technology
Breaking Barriers & Leveraging the Latest Developments in AI TechnologyBreaking Barriers & Leveraging the Latest Developments in AI Technology
Breaking Barriers & Leveraging the Latest Developments in AI Technology
 
Best Practices to Navigating Data and Application Integration for the Enterpr...
Best Practices to Navigating Data and Application Integration for the Enterpr...Best Practices to Navigating Data and Application Integration for the Enterpr...
Best Practices to Navigating Data and Application Integration for the Enterpr...
 
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataCloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
 
New Year's Fireside Chat with Safe Software’s Founders
New Year's Fireside Chat with Safe Software’s FoundersNew Year's Fireside Chat with Safe Software’s Founders
New Year's Fireside Chat with Safe Software’s Founders
 
Taking Off with FME: Elevating Airport Operations to New Heights
Taking Off with FME: Elevating Airport Operations to New HeightsTaking Off with FME: Elevating Airport Operations to New Heights
Taking Off with FME: Elevating Airport Operations to New Heights
 
Initiating and Advancing Your Strategic GIS Governance Strategy
Initiating and Advancing Your Strategic GIS Governance StrategyInitiating and Advancing Your Strategic GIS Governance Strategy
Initiating and Advancing Your Strategic GIS Governance Strategy
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
 
Geospatial Synergy: Amplifying Efficiency with FME & Esri ft. Peak Guest Spea...
Geospatial Synergy: Amplifying Efficiency with FME & Esri ft. Peak Guest Spea...Geospatial Synergy: Amplifying Efficiency with FME & Esri ft. Peak Guest Spea...
Geospatial Synergy: Amplifying Efficiency with FME & Esri ft. Peak Guest Spea...
 
Mastering DevOps-Driven Data Integration with FME
Mastering DevOps-Driven Data Integration with FMEMastering DevOps-Driven Data Integration with FME
Mastering DevOps-Driven Data Integration with FME
 

Último

EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptxEIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptxEarley Information Science
 
IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsIAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsEnterprise Knowledge
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc
 
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptxHampshireHUG
 
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsHandwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsMaria Levchenko
 
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processorsdebabhi2
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationRadu Cotescu
 
CNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of ServiceCNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of Servicegiselly40
 
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking MenDelhi Call girls
 
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Igalia
 
Factors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptxFactors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptxKatpro Technologies
 
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure serviceWhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure servicePooja Nehwal
 
The Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxThe Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxMalak Abu Hammad
 
How to convert PDF to text with Nanonets
How to convert PDF to text with NanonetsHow to convert PDF to text with Nanonets
How to convert PDF to text with Nanonetsnaman860154
 
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...Neo4j
 
A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024Results
 
Slack Application Development 101 Slides
Slack Application Development 101 SlidesSlack Application Development 101 Slides
Slack Application Development 101 Slidespraypatel2
 
Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)Allon Mureinik
 
Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024The Digital Insurer
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerThousandEyes
 

Último (20)

EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptxEIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
 
IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsIAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI Solutions
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
 
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
 
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsHandwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed texts
 
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processors
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organization
 
CNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of ServiceCNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of Service
 
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
 
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
 
Factors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptxFactors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptx
 
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure serviceWhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
 
The Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxThe Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptx
 
How to convert PDF to text with Nanonets
How to convert PDF to text with NanonetsHow to convert PDF to text with Nanonets
How to convert PDF to text with Nanonets
 
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
 
A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024
 
Slack Application Development 101 Slides
Slack Application Development 101 SlidesSlack Application Development 101 Slides
Slack Application Development 101 Slides
 
Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)
 
Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 

MGCP4LCSS Workflow

  • 1. MGCP4LCSS Workflow Engineering Section Mapping & Charting Establishment 1 Martin de Zuviria Amanda Chiprout
  • 2.
  • 3.
  • 5. 5 MGCP4LCSS OVERVIEW The main GOAL of the workflow (MGCP4LCSS) is to provide the LCSS user with the ability to view and interact with Symbolized MGCP vector data through multiple Web Services. The Geospatial Data Manager will need three items to execute this process: - MGCP vector data in Shapefile format - FME Desktop - ArcGIS software applications
  • 6. MGCP4LCSS WORKFLOW  The MGCP4LCSS workflow performs: – an extraction of selected features contained in the MGCP vector data – transform them to meet the schema and – visualization requirements for LCSS Battleview. – The final output was a FGDB using representations as symbology to be consumed in a WMS/WFS.  The workflow created by Martin provides a step-by- step description on how a user may proceed from beginning to end.  The workflow has been proven to work in ArcGIS Data Interoperability.
  • 7. Web Optimization  MGCP data is rich with content  MGCP not optimized for Web Services  Schema Optimized for Web: – Cut feature count in half – ‘Human Readable’ – Scaled to GRLS (Global/Regional/Local/Specialized) – Symbolized using Topographic Line Maps
  • 8. 8 Blue is data conditioning team Green is the Geo Field Teams MCE MGCP4LCSS Production Process Input: MGCP Shapefiles IGW Filter Shapefiles & categorization Pre-Conversion Check – QC/QA Aggregate Features Segregate Features Feature Description Symbology Coding Load & Calculate FGDB Battleview schema with representations FME 2013 FME 2013 FME 2013 FME 2013 FME 2013 FME 2013 Arc GIS 10.1 IMO/Chief Geo Receives FGDB DMAN Team Ingest/ load on Geo Server WFS Services built and tested Live on LCSS in operational theatre
  • 9. FME Workspace An FME workspace contains source and destination types of data and attributes, as well as transformers that manipulate data. Bookmarks have been added to easily find and re-focus the main display on any destination feature class (e.g. Helipad_P) Custom transformers, represented in green color, have also been added to allow an easy update of the settings of specific transformers that are of common use within the same workspace (e.g. ‘Source Name’) 10
  • 10. Step 1 : Feature Filter 11 There are different ways to filter data with FME: 1) Readers 2) Transformers such as ‘TestFilters’ and ‘GeometryFilters’; 3) Enabling/disabling components of the FME workspace (e.g. by enabling only all objects within the Road_L bookmark)
  • 11. Step 2: QA/QC Workspace 12 QA/QC results are: - re-directed results to an Inspector - using other reference layer(s), - analyzing log files. Note: ‘Data Quality’ is regarded here as the degree of excellence of data to satisfy a given objective. Any tile of MGCP released has to pass extensive QA. MGCP data has to meet new requirements in its transformation to a FGDB format/schema and Web Mapping Services (OGC Standards) and therefore has to be re-tested for these new requirements, regardless of previous QA performed.
  • 12. Steps 3,5,6: Feature Merge/Aggregate Output 13 Feature Merging and aggregation to meet the target schema architecture: - merging (ie ‘Extraction Mine Areas’ and ‘Quarry Areas’) - managing schema - coding symbology Example Below: In the legend of the output map document, the ‘Extraction Mine Areas’ and ‘Quarry Areas’ have been merged into a single feature class. The bottom table shows the descriptive attributes replaced several coded values
  • 13. 14 Steps 3,5,6: Feature Merge/Aggregate Output Example in FME Example: - Merge ‘Extraction Mine Areas’ and ‘Quarry Areas’, - manage schema and - Descriptive names and coding symbology using the ‘RepID’ field
  • 14. Usefulness of Custom Transformers 15 Custom transformers were used to replace a sequence of transformers used in the workflow. Some of these transformers contained long LUTs used to convert numeric values to descriptive names. Since these LUTs may be regularly updated by having new source values like in the example shown on the slide, the ability to update them only once in the workflow saves time and reduces the risk of errors.
  • 15. 16 Steps 4,5,6: Feature Segregation and Descriptive Names: Building points are segregated into a new feature class and given ‘Human Readable Names’ i.e. ‘Transportation_Station’ and ‘Vertical_Obstruction’
  • 16. 17 Steps 4,5,6: Feature Segregation and Descriptive Names: Building points are segregated into a new feature class and given ‘Human Readable Names’ i.e. ‘Transportation_Station’ and ‘Vertical_Obstruction’
  • 17. Step 7: Feature Loading to the Destination Empty FGDB and Calculation of Symbology Representation Values 18 FME Output data is loaded into an empty FGDB with identical schema plus the desired Representations, and symbology IDs are linked to Representation Rules (values from ‘RepID’ are copied to the ‘RuleID’ field, tied to representation rules) Example: ArcGIS ETL Detail (partial – created using ArcGIS Model Builder)
  • 18. MGCP4LCSS V3 Workflow: Information Structure Step 1: Create Parent Folder Structure Step 2: Create Sub Folder where MGCP input SHPs must be copied Step 3: Subfolder Output FGDBs: - The contents of subfolders with empty FGDBs must be copied here Step 4: Run Workbenches and BatchRunner ETL for a ‘single click’ processing of steps 1 to 6 Step 6: Subfolder ‘ArcGIS Output Map’ with an ArcGIS Map for visualization of output Step 5: ‘ArcGIS Toolboxes’ containing ArcGIS ETL Tools for processing of step 7
  • 19. 20 MGCP4LCSS Workflow Pre-conditions The designated Geospatial Data Manager has copied the ‘MGCP4LCSS Package’ to his/her local C:/ drive and has FME Desktop (2013 release or later) and ArcGIS (10.1 release or later) installed on his/her computer Trigger The user wants to covert MGCP tiles provided as shapefiles to the FGDB schema and representations defined by DND MCE Engineering, for subsequent use in any LCSS application Main Flow 1. Copy the subfolder /MGCP of the ‘MGCP4LCSS’ package on the local C:/ drive 2. Copy the MGCP tiles to the C:/MGCP/INPUT MGCP TILES folder 3. Open FME Desktop and run each of the individual eight workspaces contained in the subfolder ‘FME Workbenches’ 4. Open ArcGIS – ArcCatalog and runs the six tools contained in the Toolbox C:/MGCP/ArcGIS Toolboxes/Battleview.tbx, starting with the three ‘Append’ tools 5. Open ArcMAP MXD BVoutputFGDB.mxd to visualize the results from the output FGDB

Notas del editor

  1. The MGCP4LCSS workflow performs an extraction of selected features contained in the MGCP vector data, and to transform them to meet the schema and visualization requirements for LCSS Battleview. The final output was a FGDB using representations as symbology to be consumed in a WMS/WFS.This data transformation process: - initial geometry and schema validation, - feature merging or segregation - assignment of descriptive names to replace numeric codes. - load the transformed data into an empty file geodatabase (FGDB) containing symbology rules required to represent data using the cartographic style defined by Topographic Line Maps (TLM).The workflow created by Martin provides a step-by-step description on how a user may proceed from beginning to end. The workflow has been proven to work in ArcGIS Data Interoperability.
  2. There are different ways to filter data with FME: 1) readers; 2) transformers such as ‘TestFilters’ and ‘GeometryFilters’; 3) enabling/disabling components of the FME workspace (e.g. by enabling only all objects within the Road_L bookmark)QA/QC may be done by re-directing results to Inspector and using another reference layer(s), analyzing log files. In the next slide we show an example
  3. Note: ‘Data Quality’ is regarded here as the degree of excellence of data to satisfy a given objective. Any tile of MGCP released has to pass QAs, but MGCP data has to meet new requirements of a transformation to a FGDB format/schema and Web Mapping Services and therefore has to be re-tested for these new requirements, regardless of previous QAs performed before generating the input SHPs.OGC standards for different types of Web Mapping Services are outlined at http://www.opengeospatial.org/standards/wfsThese standards may be different for different types of WMS/WFSs but there are general validations that may be considered the most relevant for most of them, and OGC standards are widely accepted by the international community.One of these standards related to geometry validations is to test for ‘self-intersections’ of linear features (relevant since a point can not be a vertex and a node simultaneously in the output FGDB). A tester could be tunned to only detect or to detect and repair, and in this case it would split lines and copy the attribute values to each resulting line. It is in general a practice I would recommend to define the types of repairs accepted for features that have to undergo future transformations afterwards. Another valid example in this case of roads relates to attributes: there were a couple of road segments that were not mapped as any of the valid classes defining when applying the rules provided to define feature classes for ‘Roads’ and this segments were output as “Road types not properly defined”. If no action would have been implemented at this point in the ETL workflow, these two segments would have been missing in the destination FGDB
  4. In the legend of the output map document, we may see that ‘Extraction Mine Areas’ and ‘Quarry Areas’ have been merged into a single feature class. At the bottom we may see that descriptive attributes have replaced several coded valuesFeature Merging along with modifying schema to meet the target architecture (merging ‘Extraction Mine Areas’ and ‘Quarry Areas’, managing schema and coding symbology – Steps 3,5,6)
  5. along with modifying schema to meet the target architecture (merging ‘Extraction Mine Areas’ and ‘Quarry Areas’, managing schema and use of descriptive names and coding symbology using the ‘RepID’ field – Steps 3,5,6)
  6. Custom transformers were used to replace a sequence of transformers used in the workflow. Some of these transformers contained long LUTs used to convert numeric values to descriptive names. Since these LUTs may be regularly updated by having new source values like in the example shown on the slide, the ability to update them only once in the workflow saves time and reduces the risk of errors.
  7. FME Output FGDB is loaded into an empty FGDB with identical schema plus the desired Representations, and symbology IDs are linked to Representation Rules (values from ‘RepID’ are copied to the ‘RuleID’ field, tied to representation rules)