SlideShare a Scribd company logo
1 of 34
The FME Oven – Never Too Many Ingredients Kurt Hartman Director of Technology, Accurate Assessment Group Ltd.
Agenda Introduction to Accurate Assessment Group Case Study – Video Logging Case Study – ERCB Data Questions
Introduction – Client Map
Introduction – Webmap Clients Urban Clients City of Wetaskiwin Town of Barrhead *Town of Beaverlodge *Town of Edson *Town of Fox Creek  Town of High Level Town of Peace River Town of Redwater *Town of Sexsmith Town of Stettler *Town of Two Hills *Town of Wembley *Town of Valleyview  *Village of Derwent *Village of Myrnam *Village of Willingdon * Regional Sites Rural Clients Brazeau County County of Athabasca Camrose County *County of Grande Prairie No. 1 County of Minburn County of St. Paul County of Stettler *County of Two Hills No. 21 County of Wetaskiwin Kneehill County Lamont County *Municipal District of Greenview No. 16 Municipal District of Opportunity No. 17 Rural Municipality of Wood Buffalo Smoky Lake County Westlock County Wheatland County Woodlands County *Yellowhead County
Introduction – Municipal Information Integration
Video Logging Video logging: a method of displaying video data within a GIS Video is captured using a vehicle equipped with digital video cameras, precision GPS and on-board computers
Video Logging Precise digital images are captured at regular intervals from GPS-equipped vehicles traveling at regular road speeds up to 100 km/ hr. ,[object Object]
Assets can be located 80-100 meters from the vehicle.,[object Object]
Video Logging – Out of the oven... 4,800,000 images 1,600,000 linear referenced events Linear reference values/keys Direction of travel Image name Image path Batch files to create necessary directories Batch files to rename and move image files
Video Logging – Challenge 1 Attaching points to the correct road
Video Logging – Solution 1 Use Labeller to determine the orientation of the road in the vicinity of the point ,[object Object],[object Object],[object Object]
Video Logging – Challenge 2 More than one pass on the same road Blue and black points are going the same direction
Video Logging – Solution 2 Use StatisticsCalculator to determine for each video clip/road combination: Smallest linear reference value Largest linear reference value Total number of points
Video Logging – Solution 2 (Cont’d) Use ExpressionEvaluator to determine the coverage that each clip has per road
Video Logging – Solution 2 (Cont’d) Use a series of 3 Testers to validate which records should be included in the final dataset: Test 1: If the point is part of the only video clip on that road and it covers more than 10% of the road If it passes, include it If it fails, forward it on to Test 2
Video Logging – Solution 2 (Cont’d) Test 2 If the point is part of a clip that covers more than 25% of the road and the total coverage on the road is less than 110% This would handle scenarios where more than one clip is needed to cover a road If it passes, include it If it fails, forward it on to Test 3
Video Logging – Solution 2 (Cont’d) Test 3 If the point is part of a clip that covers more than 75% of the road If it reaches this test, then it is likely a road that has more than one pass To determine which of the passes gets included we include additional variables  Largest amount of coverage Most images Most recent date If it fails, forward it on to the Unused feature
Video Logging – Solution 2 (Overview)
Video Logging – Challenge 3 Around 5,000,000 images (about 1.4 TB) requires intelligent file management Developed a file structure that takes into account: Year of image Road name Alberta Township Survey township identifier Which camera (front, side, rear)
Video Logging – Solution 3 Points that are to be included in the final dataset are also forwarded to the Create Batch File process Using a series of Testers, Concatenators and StringReplacers the Create Batch File: Create batch files that make the necessary directory structure Creates batch files that move and rename the image files Pushes the new image name and path back into the final dataset
Video Logging – Solution 3 (Overview)
Video Logging – Final Translation
Video Logging – Finished Product
Video Logging – Benefits Predictable result Reproducible result 36 person hours to create translation 1 person hour to run and validate 170 steps completed with 1 mouse click
ERCB Data ERCB – Energy Resources Conservation Board Maintains Oil & Gas data for Alberta Wells Pipelines Facilities
ERCB Data – Ingredients… 2 shape files 9 text files Cryptic field names Uses a lot of codes and abbreviations Eg: Pipeline material type = “G”
ERCB Data – Out of the Oven… 4 feature classes Meaningful field names User-friendly data structure Replace codes and abbreviations with “English” descriptions Eg: Pipeline material type = “Composite”
ERCB Data Transformers used (118 in total): Joiner StringConcatenator SubstringExtractor FeatureMerger AttributeValueMapper PointConnector AttributeRenamer Tester
ERCB Data - Overview
ERCB Data - Benefits Create user-friendly dataset Predictable result Reproducible result 118 steps completed with 1 mouse click
ERCB Data – Finished Product
Thank You! Questions? For more information: Kurt Hartman Kurt@aag-gis.com Accurate Assessment Group Ltd. www.aag-gis.com
The FME Oven: Never Too Many Ingredients
The FME Oven: Never Too Many Ingredients

More Related Content

Similar to The FME Oven: Never Too Many Ingredients

Video Quality Measurements
Video Quality MeasurementsVideo Quality Measurements
Video Quality MeasurementsYoss Cohen
 
HTTP Adaptive Streaming State of the Art and Challenges Ahead
HTTP Adaptive StreamingState of the Art and Challenges AheadHTTP Adaptive StreamingState of the Art and Challenges Ahead
HTTP Adaptive Streaming State of the Art and Challenges AheadAlpen-Adria-Universität
 
A Video Processing based System for Counting Vehicles
A Video Processing based System for Counting VehiclesA Video Processing based System for Counting Vehicles
A Video Processing based System for Counting VehiclesIRJET Journal
 
2010 Ohmi Users Group And Ogrip
2010 Ohmi Users Group And Ogrip2010 Ohmi Users Group And Ogrip
2010 Ohmi Users Group And Ogripfjudson
 
Video smart cropping web application
Video smart cropping web applicationVideo smart cropping web application
Video smart cropping web applicationVasileiosMezaris
 
Survey Paper for Different Video Stabilization Techniques
Survey Paper for Different Video Stabilization TechniquesSurvey Paper for Different Video Stabilization Techniques
Survey Paper for Different Video Stabilization TechniquesIRJET Journal
 
Quality-delay Tradeoff Optimization in Multi-Bitrate Adaptive Streaming
Quality-delay Tradeoff Optimization in Multi-Bitrate Adaptive StreamingQuality-delay Tradeoff Optimization in Multi-Bitrate Adaptive Streaming
Quality-delay Tradeoff Optimization in Multi-Bitrate Adaptive StreamingDuc Nguyen
 
FPD-Link III Serializers & Deserializers
FPD-Link III Serializers & DeserializersFPD-Link III Serializers & Deserializers
FPD-Link III Serializers & DeserializersPremier Farnell
 
Webinar on Environmental Footprint Data requirements in PEFCR
Webinar on Environmental Footprint Data requirements in PEFCRWebinar on Environmental Footprint Data requirements in PEFCR
Webinar on Environmental Footprint Data requirements in PEFCRMarisa Vieira
 
2010 Usdot Asset Management Presentation
2010 Usdot Asset Management Presentation2010 Usdot Asset Management Presentation
2010 Usdot Asset Management Presentationfjudson
 
its project
its projectits project
its projectp71089
 
FME User Stories from Around the World
FME User Stories from Around the WorldFME User Stories from Around the World
FME User Stories from Around the WorldSafe Software
 
Traffic Sign recognition And Auto Indication System
Traffic Sign recognition And Auto Indication SystemTraffic Sign recognition And Auto Indication System
Traffic Sign recognition And Auto Indication SystemIRJET Journal
 
Hydraulic Modelling with GIS Data
Hydraulic Modelling with GIS DataHydraulic Modelling with GIS Data
Hydraulic Modelling with GIS DataSafe Software
 
Announcement of Results of xDR Challenge 2018
Announcement of Results of xDR Challenge 2018 Announcement of Results of xDR Challenge 2018
Announcement of Results of xDR Challenge 2018 Ryosuke Ichikari
 
Announcement of Results of xDR Challenge 2018
Announcement of Results of xDR Challenge 2018 Announcement of Results of xDR Challenge 2018
Announcement of Results of xDR Challenge 2018 Ryosuke Ichikari
 
IRJET- Smart Traffic Control System using Image Processing
IRJET- Smart Traffic Control System using Image ProcessingIRJET- Smart Traffic Control System using Image Processing
IRJET- Smart Traffic Control System using Image ProcessingIRJET Journal
 
Status report6
Status report6Status report6
Status report6Team 02
 

Similar to The FME Oven: Never Too Many Ingredients (20)

Video Quality Measurements
Video Quality MeasurementsVideo Quality Measurements
Video Quality Measurements
 
HTTP Adaptive Streaming State of the Art and Challenges Ahead
HTTP Adaptive StreamingState of the Art and Challenges AheadHTTP Adaptive StreamingState of the Art and Challenges Ahead
HTTP Adaptive Streaming State of the Art and Challenges Ahead
 
A Video Processing based System for Counting Vehicles
A Video Processing based System for Counting VehiclesA Video Processing based System for Counting Vehicles
A Video Processing based System for Counting Vehicles
 
2010 Ohmi Users Group And Ogrip
2010 Ohmi Users Group And Ogrip2010 Ohmi Users Group And Ogrip
2010 Ohmi Users Group And Ogrip
 
Video smart cropping web application
Video smart cropping web applicationVideo smart cropping web application
Video smart cropping web application
 
Survey Paper for Different Video Stabilization Techniques
Survey Paper for Different Video Stabilization TechniquesSurvey Paper for Different Video Stabilization Techniques
Survey Paper for Different Video Stabilization Techniques
 
Quality-delay Tradeoff Optimization in Multi-Bitrate Adaptive Streaming
Quality-delay Tradeoff Optimization in Multi-Bitrate Adaptive StreamingQuality-delay Tradeoff Optimization in Multi-Bitrate Adaptive Streaming
Quality-delay Tradeoff Optimization in Multi-Bitrate Adaptive Streaming
 
FPD-Link III Serializers & Deserializers
FPD-Link III Serializers & DeserializersFPD-Link III Serializers & Deserializers
FPD-Link III Serializers & Deserializers
 
Webinar on Environmental Footprint Data requirements in PEFCR
Webinar on Environmental Footprint Data requirements in PEFCRWebinar on Environmental Footprint Data requirements in PEFCR
Webinar on Environmental Footprint Data requirements in PEFCR
 
2010 Usdot Asset Management Presentation
2010 Usdot Asset Management Presentation2010 Usdot Asset Management Presentation
2010 Usdot Asset Management Presentation
 
its project
its projectits project
its project
 
Gated-ViGAT
Gated-ViGATGated-ViGAT
Gated-ViGAT
 
FME User Stories from Around the World
FME User Stories from Around the WorldFME User Stories from Around the World
FME User Stories from Around the World
 
Traffic Sign recognition And Auto Indication System
Traffic Sign recognition And Auto Indication SystemTraffic Sign recognition And Auto Indication System
Traffic Sign recognition And Auto Indication System
 
Hydraulic Modelling with GIS Data
Hydraulic Modelling with GIS DataHydraulic Modelling with GIS Data
Hydraulic Modelling with GIS Data
 
Announcement of Results of xDR Challenge 2018
Announcement of Results of xDR Challenge 2018 Announcement of Results of xDR Challenge 2018
Announcement of Results of xDR Challenge 2018
 
Announcement of Results of xDR Challenge 2018
Announcement of Results of xDR Challenge 2018 Announcement of Results of xDR Challenge 2018
Announcement of Results of xDR Challenge 2018
 
Npma Final
Npma FinalNpma Final
Npma Final
 
IRJET- Smart Traffic Control System using Image Processing
IRJET- Smart Traffic Control System using Image ProcessingIRJET- Smart Traffic Control System using Image Processing
IRJET- Smart Traffic Control System using Image Processing
 
Status report6
Status report6Status report6
Status report6
 

More from Safe Software

Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationBeyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationSafe Software
 
Powering Real-Time Decisions with Continuous Data Streams
Powering Real-Time Decisions with Continuous Data StreamsPowering Real-Time Decisions with Continuous Data Streams
Powering Real-Time Decisions with Continuous Data StreamsSafe Software
 
The Critical Role of Spatial Data in Today's Data Ecosystem
The Critical Role of Spatial Data in Today's Data EcosystemThe Critical Role of Spatial Data in Today's Data Ecosystem
The Critical Role of Spatial Data in Today's Data EcosystemSafe Software
 
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataCloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataSafe Software
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsSafe Software
 
The Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightThe Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightSafe Software
 
Mastering MicroStation DGN: How to Integrate CAD and GIS
Mastering MicroStation DGN: How to Integrate CAD and GISMastering MicroStation DGN: How to Integrate CAD and GIS
Mastering MicroStation DGN: How to Integrate CAD and GISSafe Software
 
Geospatial Synergy: Amplifying Efficiency with FME & Esri
Geospatial Synergy: Amplifying Efficiency with FME & EsriGeospatial Synergy: Amplifying Efficiency with FME & Esri
Geospatial Synergy: Amplifying Efficiency with FME & EsriSafe Software
 
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdfIntroducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdfSafe Software
 
Breaking Barriers & Leveraging the Latest Developments in AI Technology
Breaking Barriers & Leveraging the Latest Developments in AI TechnologyBreaking Barriers & Leveraging the Latest Developments in AI Technology
Breaking Barriers & Leveraging the Latest Developments in AI TechnologySafe Software
 
Best Practices to Navigating Data and Application Integration for the Enterpr...
Best Practices to Navigating Data and Application Integration for the Enterpr...Best Practices to Navigating Data and Application Integration for the Enterpr...
Best Practices to Navigating Data and Application Integration for the Enterpr...Safe Software
 
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataCloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataSafe Software
 
New Year's Fireside Chat with Safe Software’s Founders
New Year's Fireside Chat with Safe Software’s FoundersNew Year's Fireside Chat with Safe Software’s Founders
New Year's Fireside Chat with Safe Software’s FoundersSafe Software
 
Taking Off with FME: Elevating Airport Operations to New Heights
Taking Off with FME: Elevating Airport Operations to New HeightsTaking Off with FME: Elevating Airport Operations to New Heights
Taking Off with FME: Elevating Airport Operations to New HeightsSafe Software
 
Initiating and Advancing Your Strategic GIS Governance Strategy
Initiating and Advancing Your Strategic GIS Governance StrategyInitiating and Advancing Your Strategic GIS Governance Strategy
Initiating and Advancing Your Strategic GIS Governance StrategySafe Software
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows Safe Software
 
Geospatial Synergy: Amplifying Efficiency with FME & Esri ft. Peak Guest Spea...
Geospatial Synergy: Amplifying Efficiency with FME & Esri ft. Peak Guest Spea...Geospatial Synergy: Amplifying Efficiency with FME & Esri ft. Peak Guest Spea...
Geospatial Synergy: Amplifying Efficiency with FME & Esri ft. Peak Guest Spea...Safe Software
 
Mastering DevOps-Driven Data Integration with FME
Mastering DevOps-Driven Data Integration with FMEMastering DevOps-Driven Data Integration with FME
Mastering DevOps-Driven Data Integration with FMESafe Software
 
Identification of landscape features eligible for EU subsidy - Analysis of la...
Identification of landscape features eligible for EU subsidy - Analysis of la...Identification of landscape features eligible for EU subsidy - Analysis of la...
Identification of landscape features eligible for EU subsidy - Analysis of la...Safe Software
 
Utilizing FME as an API Test Framework
Utilizing FME as an API Test Framework Utilizing FME as an API Test Framework
Utilizing FME as an API Test Framework Safe Software
 

More from Safe Software (20)

Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationBeyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
 
Powering Real-Time Decisions with Continuous Data Streams
Powering Real-Time Decisions with Continuous Data StreamsPowering Real-Time Decisions with Continuous Data Streams
Powering Real-Time Decisions with Continuous Data Streams
 
The Critical Role of Spatial Data in Today's Data Ecosystem
The Critical Role of Spatial Data in Today's Data EcosystemThe Critical Role of Spatial Data in Today's Data Ecosystem
The Critical Role of Spatial Data in Today's Data Ecosystem
 
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataCloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
 
The Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightThe Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and Insight
 
Mastering MicroStation DGN: How to Integrate CAD and GIS
Mastering MicroStation DGN: How to Integrate CAD and GISMastering MicroStation DGN: How to Integrate CAD and GIS
Mastering MicroStation DGN: How to Integrate CAD and GIS
 
Geospatial Synergy: Amplifying Efficiency with FME & Esri
Geospatial Synergy: Amplifying Efficiency with FME & EsriGeospatial Synergy: Amplifying Efficiency with FME & Esri
Geospatial Synergy: Amplifying Efficiency with FME & Esri
 
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdfIntroducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
 
Breaking Barriers & Leveraging the Latest Developments in AI Technology
Breaking Barriers & Leveraging the Latest Developments in AI TechnologyBreaking Barriers & Leveraging the Latest Developments in AI Technology
Breaking Barriers & Leveraging the Latest Developments in AI Technology
 
Best Practices to Navigating Data and Application Integration for the Enterpr...
Best Practices to Navigating Data and Application Integration for the Enterpr...Best Practices to Navigating Data and Application Integration for the Enterpr...
Best Practices to Navigating Data and Application Integration for the Enterpr...
 
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataCloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
 
New Year's Fireside Chat with Safe Software’s Founders
New Year's Fireside Chat with Safe Software’s FoundersNew Year's Fireside Chat with Safe Software’s Founders
New Year's Fireside Chat with Safe Software’s Founders
 
Taking Off with FME: Elevating Airport Operations to New Heights
Taking Off with FME: Elevating Airport Operations to New HeightsTaking Off with FME: Elevating Airport Operations to New Heights
Taking Off with FME: Elevating Airport Operations to New Heights
 
Initiating and Advancing Your Strategic GIS Governance Strategy
Initiating and Advancing Your Strategic GIS Governance StrategyInitiating and Advancing Your Strategic GIS Governance Strategy
Initiating and Advancing Your Strategic GIS Governance Strategy
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
 
Geospatial Synergy: Amplifying Efficiency with FME & Esri ft. Peak Guest Spea...
Geospatial Synergy: Amplifying Efficiency with FME & Esri ft. Peak Guest Spea...Geospatial Synergy: Amplifying Efficiency with FME & Esri ft. Peak Guest Spea...
Geospatial Synergy: Amplifying Efficiency with FME & Esri ft. Peak Guest Spea...
 
Mastering DevOps-Driven Data Integration with FME
Mastering DevOps-Driven Data Integration with FMEMastering DevOps-Driven Data Integration with FME
Mastering DevOps-Driven Data Integration with FME
 
Identification of landscape features eligible for EU subsidy - Analysis of la...
Identification of landscape features eligible for EU subsidy - Analysis of la...Identification of landscape features eligible for EU subsidy - Analysis of la...
Identification of landscape features eligible for EU subsidy - Analysis of la...
 
Utilizing FME as an API Test Framework
Utilizing FME as an API Test Framework Utilizing FME as an API Test Framework
Utilizing FME as an API Test Framework
 

Recently uploaded

Bun (KitWorks Team Study 노별마루 발표 2024.4.22)
Bun (KitWorks Team Study 노별마루 발표 2024.4.22)Bun (KitWorks Team Study 노별마루 발표 2024.4.22)
Bun (KitWorks Team Study 노별마루 발표 2024.4.22)Wonjun Hwang
 
SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024Lorenzo Miniero
 
What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024Stephanie Beckett
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsSergiu Bodiu
 
Search Engine Optimization SEO PDF for 2024.pdf
Search Engine Optimization SEO PDF for 2024.pdfSearch Engine Optimization SEO PDF for 2024.pdf
Search Engine Optimization SEO PDF for 2024.pdfRankYa
 
Scanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsScanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsRizwan Syed
 
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmaticsKotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmaticscarlostorres15106
 
DevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache MavenDevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache MavenHervé Boutemy
 
The Future of Software Development - Devin AI Innovative Approach.pdf
The Future of Software Development - Devin AI Innovative Approach.pdfThe Future of Software Development - Devin AI Innovative Approach.pdf
The Future of Software Development - Devin AI Innovative Approach.pdfSeasiaInfotech2
 
Gen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfGen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfAddepto
 
Vertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering TipsVertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering TipsMiki Katsuragi
 
"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii SoldatenkoFwdays
 
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
"Federated learning: out of reach no matter how close",Oleksandr LapshynFwdays
 
Install Stable Diffusion in windows machine
Install Stable Diffusion in windows machineInstall Stable Diffusion in windows machine
Install Stable Diffusion in windows machinePadma Pradeep
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 
Unraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfUnraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfAlex Barbosa Coqueiro
 
Commit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easyCommit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easyAlfredo García Lavilla
 
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 
Streamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupStreamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupFlorian Wilhelm
 
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage Cost
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage CostLeverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage Cost
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage CostZilliz
 

Recently uploaded (20)

Bun (KitWorks Team Study 노별마루 발표 2024.4.22)
Bun (KitWorks Team Study 노별마루 발표 2024.4.22)Bun (KitWorks Team Study 노별마루 발표 2024.4.22)
Bun (KitWorks Team Study 노별마루 발표 2024.4.22)
 
SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024
 
What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platforms
 
Search Engine Optimization SEO PDF for 2024.pdf
Search Engine Optimization SEO PDF for 2024.pdfSearch Engine Optimization SEO PDF for 2024.pdf
Search Engine Optimization SEO PDF for 2024.pdf
 
Scanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsScanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL Certs
 
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmaticsKotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
 
DevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache MavenDevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache Maven
 
The Future of Software Development - Devin AI Innovative Approach.pdf
The Future of Software Development - Devin AI Innovative Approach.pdfThe Future of Software Development - Devin AI Innovative Approach.pdf
The Future of Software Development - Devin AI Innovative Approach.pdf
 
Gen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfGen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdf
 
Vertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering TipsVertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering Tips
 
"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko
 
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
 
Install Stable Diffusion in windows machine
Install Stable Diffusion in windows machineInstall Stable Diffusion in windows machine
Install Stable Diffusion in windows machine
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 
Unraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfUnraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdf
 
Commit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easyCommit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easy
 
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 
Streamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupStreamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project Setup
 
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage Cost
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage CostLeverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage Cost
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage Cost
 

The FME Oven: Never Too Many Ingredients

  • 1. The FME Oven – Never Too Many Ingredients Kurt Hartman Director of Technology, Accurate Assessment Group Ltd.
  • 2. Agenda Introduction to Accurate Assessment Group Case Study – Video Logging Case Study – ERCB Data Questions
  • 4. Introduction – Webmap Clients Urban Clients City of Wetaskiwin Town of Barrhead *Town of Beaverlodge *Town of Edson *Town of Fox Creek Town of High Level Town of Peace River Town of Redwater *Town of Sexsmith Town of Stettler *Town of Two Hills *Town of Wembley *Town of Valleyview *Village of Derwent *Village of Myrnam *Village of Willingdon * Regional Sites Rural Clients Brazeau County County of Athabasca Camrose County *County of Grande Prairie No. 1 County of Minburn County of St. Paul County of Stettler *County of Two Hills No. 21 County of Wetaskiwin Kneehill County Lamont County *Municipal District of Greenview No. 16 Municipal District of Opportunity No. 17 Rural Municipality of Wood Buffalo Smoky Lake County Westlock County Wheatland County Woodlands County *Yellowhead County
  • 5. Introduction – Municipal Information Integration
  • 6. Video Logging Video logging: a method of displaying video data within a GIS Video is captured using a vehicle equipped with digital video cameras, precision GPS and on-board computers
  • 7.
  • 8.
  • 9. Video Logging – Out of the oven... 4,800,000 images 1,600,000 linear referenced events Linear reference values/keys Direction of travel Image name Image path Batch files to create necessary directories Batch files to rename and move image files
  • 10. Video Logging – Challenge 1 Attaching points to the correct road
  • 11.
  • 12. Video Logging – Challenge 2 More than one pass on the same road Blue and black points are going the same direction
  • 13. Video Logging – Solution 2 Use StatisticsCalculator to determine for each video clip/road combination: Smallest linear reference value Largest linear reference value Total number of points
  • 14. Video Logging – Solution 2 (Cont’d) Use ExpressionEvaluator to determine the coverage that each clip has per road
  • 15. Video Logging – Solution 2 (Cont’d) Use a series of 3 Testers to validate which records should be included in the final dataset: Test 1: If the point is part of the only video clip on that road and it covers more than 10% of the road If it passes, include it If it fails, forward it on to Test 2
  • 16. Video Logging – Solution 2 (Cont’d) Test 2 If the point is part of a clip that covers more than 25% of the road and the total coverage on the road is less than 110% This would handle scenarios where more than one clip is needed to cover a road If it passes, include it If it fails, forward it on to Test 3
  • 17. Video Logging – Solution 2 (Cont’d) Test 3 If the point is part of a clip that covers more than 75% of the road If it reaches this test, then it is likely a road that has more than one pass To determine which of the passes gets included we include additional variables Largest amount of coverage Most images Most recent date If it fails, forward it on to the Unused feature
  • 18. Video Logging – Solution 2 (Overview)
  • 19. Video Logging – Challenge 3 Around 5,000,000 images (about 1.4 TB) requires intelligent file management Developed a file structure that takes into account: Year of image Road name Alberta Township Survey township identifier Which camera (front, side, rear)
  • 20. Video Logging – Solution 3 Points that are to be included in the final dataset are also forwarded to the Create Batch File process Using a series of Testers, Concatenators and StringReplacers the Create Batch File: Create batch files that make the necessary directory structure Creates batch files that move and rename the image files Pushes the new image name and path back into the final dataset
  • 21. Video Logging – Solution 3 (Overview)
  • 22. Video Logging – Final Translation
  • 23. Video Logging – Finished Product
  • 24. Video Logging – Benefits Predictable result Reproducible result 36 person hours to create translation 1 person hour to run and validate 170 steps completed with 1 mouse click
  • 25. ERCB Data ERCB – Energy Resources Conservation Board Maintains Oil & Gas data for Alberta Wells Pipelines Facilities
  • 26. ERCB Data – Ingredients… 2 shape files 9 text files Cryptic field names Uses a lot of codes and abbreviations Eg: Pipeline material type = “G”
  • 27. ERCB Data – Out of the Oven… 4 feature classes Meaningful field names User-friendly data structure Replace codes and abbreviations with “English” descriptions Eg: Pipeline material type = “Composite”
  • 28. ERCB Data Transformers used (118 in total): Joiner StringConcatenator SubstringExtractor FeatureMerger AttributeValueMapper PointConnector AttributeRenamer Tester
  • 29. ERCB Data - Overview
  • 30. ERCB Data - Benefits Create user-friendly dataset Predictable result Reproducible result 118 steps completed with 1 mouse click
  • 31. ERCB Data – Finished Product
  • 32. Thank You! Questions? For more information: Kurt Hartman Kurt@aag-gis.com Accurate Assessment Group Ltd. www.aag-gis.com

Editor's Notes

  1. Linear referenced values are added in a previous step