SlideShare una empresa de Scribd logo
1 de 34
Descargar para leer sin conexión
Using FME to Overcome
General GIS Software
Limitations




Alicia Foose, DCP Midstream
DCP Midstream Overview


  DCP Midstream, LLC, a 50-50 joint venture
   between Spectra Energy and ConocoPhillips,
   is headquartered in Denver, Colorado.
  The Company leads the midstream segment
   as one of the nation’s largest natural gas
   gatherers and processors in the United
   States.
  DCP Midstream is the largest natural gas
   liquids (NGLs) producers in the nation.
DCP Midstream Overview


  The Company owns or operates 58 plants, 10
   fractionating facilities, and approximately
   60,000 miles of gathering and transmission
   pipeline with connections to approximately
   38,000 active receipt points.
  Visit https://www.dcpmidstream.com for
   more details.
What Does This Look Like
Close Up Look
GIS Environment


  Oracle database
  ESRI SDE
  Pipeline Open Database Standard (PODS)
   4.02 database model
    http://pods.org/
  The volume and complexity of data can create
   challenges for GIS analysis.
A Recent Project


  DCP Midstream went through an evaluation
   process to find a software solution to manage
   the One Call (Call before you dig) process.
  One of the solutions only accepted polygon
   features. Because all of the pipelines in the
   PODS database are polylines features a buffer
   needed to be created for the pilot.
  To keep the comparisons similar we decided
   to buffer the pipeline by one foot.
  Only the location of the pipelines were of
   interest.
Remember The 60,000 Miles
Volume Of Data


  We were only interested in the pipelines we
   operated so a query was necessary.
  The pipeline layer being used has 164,535
   polylines in the database.
        SQL> select count(*) from PODS.REGULATORY_SEGMENT;

          COUNT(*)
        ----------
           164535

  Laptop processing capacity along with
   memory limits can become an issue when
   buffering this volume of data.
Creating The Buffer In FME


  The buffer was created in FME because
      It   is easy to set up
      It   can run in the background
      It   doesn’t seem to use as many resources
      It   tends to run faster on my environment
      It   has an aggregate feature
      It   can filter attributes
Creating The Buffer In FME


  The goal was to:
    Query for only DCP Midstream Operated
     pipelines.
    Simplify the data by eliminating most of the
     columns.
       I chose to keep Region because there are only 10
        regions (Regions have a logical geographical area)
    Buffer the pipelines by 1 foot.
    Aggregate the data.
    Export the Polygon feature to an ESRI shape
     file format.
Query For DCP Operated


  The data was queried directly
   from the SDE connection in
   FME – this filters the data on
   the fly.
  The 164,535 rows were
   reduced by 2,829 to total
   161,706 records to buffer.
Transformers Used


  The AttributeKeeper was used to reduce the
   number of columns from 51 down to 6
   keeping only REGION_NAME from the SDE
   layer. Because only the location of a pipeline
   was required, the associated attributes were
   not needed.
Transformers Used


  The Reprojector was used to project the data
   from NAD 83 to a projection with a unit of
   measure in feet.
  US48-DUKE was chosen because the
   projection was created for the continental US
   and has relatively little overall distortion.
Transformers Used


  The Bufferer was used to buffer by 1 foot.
Transformers Used


  The Aggregator was used to aggregate the
   data using the REGION_NAME to group by.
  Aggregating the data reduced the number of
   records from 161,706 to 10.
Transformers Used


  The Reprojector was used to project the data
   back to NAD83.
  Finally, the destination dataset was set to a
   shape file format. A visualizer was used so
   the output could be viewed right away.
  A dissolver transformer was not used because
   the aggregate combined all of the polygons
   into Regions and the overlaps were not a
   concern for the end use.
FME Workspace
Final Results


Total Features Written                                10
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Translation was SUCCESSFUL with 0 warning(s) (10 feature(s)/5295013
coordinate(s) output)
FME Session Duration: 7 minutes 9.2 seconds. (CPU: 141.9s user, 10.6s system)
END - ProcessID: 1480, peak process memory usage: 208120 kB, current process
memory usage: 53868 kB.
A Side By Side Comparison


  A 1 foot buffer was run in Arc Info using the
   same query on the same layer.
  Dissolve by field - REGION_NAME was
   selected because it is the closest option to the
   FME Aggregate .
A Side By Side Comparison


Executing (Buffer_2): Buffer PODS.REGULATORY_SEGMENT Server
   1_ft_Buffer.shp "1 Feet" FULL ROUND ALL #
Start Time: Thu Mar 11 08:29:01 2010
Dissolving...
Output feature 0 cannot be dissolved into other inputs because of memory
   limitations
Output feature 1 cannot be dissolved into other inputs because of memory
   limitations
…
…
Output feature 15 cannot be dissolved into other inputs because of memory
   limitations
Executed (Buffer_2) successfully.
End Time: Thu Mar 11 11:12:32 2010 (Elapsed Time: 2 hours 43 minutes 31
   seconds)
A Side By Side Comparison


  The FME translation ran in 7 minutes 9.2
   seconds with no memory errors.
  The Arc Info Buffer wizard ran in 2 hours 43
   minutes and 31 seconds with memory
   limitations errors.
  The results from either process were
   acceptable.
Annual Tax Project


  Benjamin Franklin once said that “In this
   world nothing is certain but death and taxes”.
  So lets talk about taxes, specifically property
   taxes. You might be asking yourself what on
   earth does FME have to do with property
   taxes. Well here is your answer-
  Each year companies with tangible assets pay
   property taxes. Pipelines are not excluded.
The Challenge


  Every State has unique taxing districts by
   which they collect and distribute property
   taxes.
  Tax districts can change from year to year
   although most remain the same.
  Population shifts and demographics are the
   most common cause of tax boundary
   changes.
  DCP Midstream operates primarily in 17
   States so tax boundary maintenance is a
   fairly large undertaking.
Tax Project


  Each year the GIS department provides the
   Tax department with a report of how many
   feet of each pipeline is in what tax district by
   install year, diameter and so on.
  The first step, for States with electronic data,
   is to download the current tax boundary files
   and update the SDE layer with the changes.
  The SDE Layer has to be topologically clean.
  Neighboring states do not tend to use the
   exact same state line. This creates gaps and
   overlaps which are ugly to clean up
   particularly along rivers.
2008 Oklahoma Tax Districts
2009 Oklahoma Tax Districts
Oklahoma Tax Districts


    Who can tell me what changed?
    Going once
    Going twice
    Going three time
    How are you going to find out?
    FME has a transformer named Matcher which
     detects both geometry and attribute changes
     from two files.
Lets See What Changed


  The 2008 Oklahoma tax districts are added as
   one source.
  The 2009 Oklahoma tax districts are added as
   another source.
  Both are run through the Matcher as input.
  The Not_Matched features are output to a
   visualizer so they can be looked at.
  The Not_Matched features are output to a
   shape file to be used in ArcMap for updating
   the SDE layer.
FME Workspace
This Is What Changed
Why So Many Changes – River
Correction
A Closer Look
Thank You!


  Questions?

  For more information:
    Alicia Foose apfoose@dcpmidstream.com
    DCP Midstream

    https://www.dcpmidstream.com
    http://pods.org/

Más contenido relacionado

Destacado

2 (14092012) dasar-dasar perencanaan jalan raya
2  (14092012) dasar-dasar perencanaan jalan raya2  (14092012) dasar-dasar perencanaan jalan raya
2 (14092012) dasar-dasar perencanaan jalan raya
mrtunsyiah
 
Pile foundation design
Pile foundation designPile foundation design
Pile foundation design
UTHM
 

Destacado (7)

FME Applications in New York City GIS
FME Applications in New York City GISFME Applications in New York City GIS
FME Applications in New York City GIS
 
2 (14092012) dasar-dasar perencanaan jalan raya
2  (14092012) dasar-dasar perencanaan jalan raya2  (14092012) dasar-dasar perencanaan jalan raya
2 (14092012) dasar-dasar perencanaan jalan raya
 
Perencanaan perkerasan jalan raya
Perencanaan perkerasan jalan rayaPerencanaan perkerasan jalan raya
Perencanaan perkerasan jalan raya
 
Kuliah minggu ke 9 struktur jembatan,06 nopb2012
Kuliah minggu ke 9 struktur jembatan,06 nopb2012Kuliah minggu ke 9 struktur jembatan,06 nopb2012
Kuliah minggu ke 9 struktur jembatan,06 nopb2012
 
Contoh Desain Perkerasan Jalan
Contoh Desain Perkerasan JalanContoh Desain Perkerasan Jalan
Contoh Desain Perkerasan Jalan
 
Pile foundation design
Pile foundation designPile foundation design
Pile foundation design
 
Perkerasan Kaku dan Lentur (Perkerasan Jalan Raya)
Perkerasan Kaku dan Lentur (Perkerasan Jalan Raya)Perkerasan Kaku dan Lentur (Perkerasan Jalan Raya)
Perkerasan Kaku dan Lentur (Perkerasan Jalan Raya)
 

Similar a FME to the Rescue

A Few of My Favorite Things: The Magic that FME Brings to My Life
A Few of My Favorite Things:The Magic that FME Brings to My LifeA Few of My Favorite Things:The Magic that FME Brings to My Life
A Few of My Favorite Things: The Magic that FME Brings to My Life
Safe Software
 
Load testing of HELIDEM geo-portal: an OGC open standards interoperability ex...
Load testing of HELIDEM geo-portal: an OGC open standards interoperability ex...Load testing of HELIDEM geo-portal: an OGC open standards interoperability ex...
Load testing of HELIDEM geo-portal: an OGC open standards interoperability ex...
Massimiliano Cannata
 
HDT for Mainframe Considerations: Simplified Tiered Storage
HDT for Mainframe Considerations: Simplified Tiered StorageHDT for Mainframe Considerations: Simplified Tiered Storage
HDT for Mainframe Considerations: Simplified Tiered Storage
Hitachi Vantara
 
Geospatial Synergy: Amplifying Efficiency with FME & Esri ft. Peak Guest Spea...
Geospatial Synergy: Amplifying Efficiency with FME & Esri ft. Peak Guest Spea...Geospatial Synergy: Amplifying Efficiency with FME & Esri ft. Peak Guest Spea...
Geospatial Synergy: Amplifying Efficiency with FME & Esri ft. Peak Guest Spea...
Safe Software
 

Similar a FME to the Rescue (20)

FME Around the World
FME Around the WorldFME Around the World
FME Around the World
 
A Few of My Favorite Things: The Magic that FME Brings to My Life
A Few of My Favorite Things:The Magic that FME Brings to My LifeA Few of My Favorite Things:The Magic that FME Brings to My Life
A Few of My Favorite Things: The Magic that FME Brings to My Life
 
FME Around the World
FME Around the WorldFME Around the World
FME Around the World
 
FME Around the World (FME Trek Part 1): Ken Bragg - Safe Software FME World T...
FME Around the World (FME Trek Part 1): Ken Bragg - Safe Software FME World T...FME Around the World (FME Trek Part 1): Ken Bragg - Safe Software FME World T...
FME Around the World (FME Trek Part 1): Ken Bragg - Safe Software FME World T...
 
Hadoop World 2011: Hadoop Network and Compute Architecture Considerations - J...
Hadoop World 2011: Hadoop Network and Compute Architecture Considerations - J...Hadoop World 2011: Hadoop Network and Compute Architecture Considerations - J...
Hadoop World 2011: Hadoop Network and Compute Architecture Considerations - J...
 
Load testing of HELIDEM geo-portal: an OGC open standards interoperability ex...
Load testing of HELIDEM geo-portal: an OGC open standards interoperability ex...Load testing of HELIDEM geo-portal: an OGC open standards interoperability ex...
Load testing of HELIDEM geo-portal: an OGC open standards interoperability ex...
 
FME World Tour 2015 - Around the World - Ken Bragg
FME World Tour 2015 - Around the World - Ken BraggFME World Tour 2015 - Around the World - Ken Bragg
FME World Tour 2015 - Around the World - Ken Bragg
 
Designing a Fault-Tolerant Channel Extension Network for Internal Recovery
Designing a Fault-Tolerant Channel Extension Network for Internal RecoveryDesigning a Fault-Tolerant Channel Extension Network for Internal Recovery
Designing a Fault-Tolerant Channel Extension Network for Internal Recovery
 
Multiscale Dataflow Computing: Competitive Advantage at the Exascale Frontier
Multiscale Dataflow Computing: Competitive Advantage at the Exascale FrontierMultiscale Dataflow Computing: Competitive Advantage at the Exascale Frontier
Multiscale Dataflow Computing: Competitive Advantage at the Exascale Frontier
 
2010 Gist
2010 Gist2010 Gist
2010 Gist
 
fmewt19 - Around the world stories master deck
fmewt19 - Around the world stories master deckfmewt19 - Around the world stories master deck
fmewt19 - Around the world stories master deck
 
MySQL Cluster Local Checkpoint (LCP) evolution in 7.6 (2019)
MySQL Cluster Local Checkpoint (LCP) evolution in 7.6 (2019)MySQL Cluster Local Checkpoint (LCP) evolution in 7.6 (2019)
MySQL Cluster Local Checkpoint (LCP) evolution in 7.6 (2019)
 
50509750 fundamentals-of-rf-planning
50509750 fundamentals-of-rf-planning50509750 fundamentals-of-rf-planning
50509750 fundamentals-of-rf-planning
 
Automated change detection in grass gis
Automated change detection in grass gisAutomated change detection in grass gis
Automated change detection in grass gis
 
HDT for Mainframe Considerations: Simplified Tiered Storage
HDT for Mainframe Considerations: Simplified Tiered StorageHDT for Mainframe Considerations: Simplified Tiered Storage
HDT for Mainframe Considerations: Simplified Tiered Storage
 
Geospatial Synergy: Amplifying Efficiency with FME & Esri
Geospatial Synergy: Amplifying Efficiency with FME & EsriGeospatial Synergy: Amplifying Efficiency with FME & Esri
Geospatial Synergy: Amplifying Efficiency with FME & Esri
 
Geospatial Synergy: Amplifying Efficiency with FME & Esri ft. Peak Guest Spea...
Geospatial Synergy: Amplifying Efficiency with FME & Esri ft. Peak Guest Spea...Geospatial Synergy: Amplifying Efficiency with FME & Esri ft. Peak Guest Spea...
Geospatial Synergy: Amplifying Efficiency with FME & Esri ft. Peak Guest Spea...
 
Super Computer
Super ComputerSuper Computer
Super Computer
 
Overview of the Data Processing Error Analysis System (DPEAS)
Overview of the Data Processing Error Analysis System (DPEAS)Overview of the Data Processing Error Analysis System (DPEAS)
Overview of the Data Processing Error Analysis System (DPEAS)
 
Blue Phoenix Idms Migration
Blue Phoenix Idms MigrationBlue Phoenix Idms Migration
Blue Phoenix Idms Migration
 

Más de Safe Software

Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
 
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataCloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
Safe Software
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Safe Software
 
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdfIntroducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
Safe Software
 
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataCloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
Safe Software
 
Taking Off with FME: Elevating Airport Operations to New Heights
Taking Off with FME: Elevating Airport Operations to New HeightsTaking Off with FME: Elevating Airport Operations to New Heights
Taking Off with FME: Elevating Airport Operations to New Heights
Safe Software
 
Initiating and Advancing Your Strategic GIS Governance Strategy
Initiating and Advancing Your Strategic GIS Governance StrategyInitiating and Advancing Your Strategic GIS Governance Strategy
Initiating and Advancing Your Strategic GIS Governance Strategy
Safe Software
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Safe Software
 

Más de Safe Software (20)

Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action:  Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action:  Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
 
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationBeyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
 
Powering Real-Time Decisions with Continuous Data Streams
Powering Real-Time Decisions with Continuous Data StreamsPowering Real-Time Decisions with Continuous Data Streams
Powering Real-Time Decisions with Continuous Data Streams
 
The Critical Role of Spatial Data in Today's Data Ecosystem
The Critical Role of Spatial Data in Today's Data EcosystemThe Critical Role of Spatial Data in Today's Data Ecosystem
The Critical Role of Spatial Data in Today's Data Ecosystem
 
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataCloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
 
The Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightThe Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and Insight
 
Mastering MicroStation DGN: How to Integrate CAD and GIS
Mastering MicroStation DGN: How to Integrate CAD and GISMastering MicroStation DGN: How to Integrate CAD and GIS
Mastering MicroStation DGN: How to Integrate CAD and GIS
 
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdfIntroducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
 
Breaking Barriers & Leveraging the Latest Developments in AI Technology
Breaking Barriers & Leveraging the Latest Developments in AI TechnologyBreaking Barriers & Leveraging the Latest Developments in AI Technology
Breaking Barriers & Leveraging the Latest Developments in AI Technology
 
Best Practices to Navigating Data and Application Integration for the Enterpr...
Best Practices to Navigating Data and Application Integration for the Enterpr...Best Practices to Navigating Data and Application Integration for the Enterpr...
Best Practices to Navigating Data and Application Integration for the Enterpr...
 
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataCloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
 
New Year's Fireside Chat with Safe Software’s Founders
New Year's Fireside Chat with Safe Software’s FoundersNew Year's Fireside Chat with Safe Software’s Founders
New Year's Fireside Chat with Safe Software’s Founders
 
Taking Off with FME: Elevating Airport Operations to New Heights
Taking Off with FME: Elevating Airport Operations to New HeightsTaking Off with FME: Elevating Airport Operations to New Heights
Taking Off with FME: Elevating Airport Operations to New Heights
 
Initiating and Advancing Your Strategic GIS Governance Strategy
Initiating and Advancing Your Strategic GIS Governance StrategyInitiating and Advancing Your Strategic GIS Governance Strategy
Initiating and Advancing Your Strategic GIS Governance Strategy
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
 
Mastering DevOps-Driven Data Integration with FME
Mastering DevOps-Driven Data Integration with FMEMastering DevOps-Driven Data Integration with FME
Mastering DevOps-Driven Data Integration with FME
 
Identification of landscape features eligible for EU subsidy - Analysis of la...
Identification of landscape features eligible for EU subsidy - Analysis of la...Identification of landscape features eligible for EU subsidy - Analysis of la...
Identification of landscape features eligible for EU subsidy - Analysis of la...
 

Último

Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsArtificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and Myths
Joaquim Jorge
 

Último (20)

A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt Robison
 
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?
 
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdfUnderstanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
 
GenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdfGenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdf
 
[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century education
 
Real Time Object Detection Using Open CV
Real Time Object Detection Using Open CVReal Time Object Detection Using Open CV
Real Time Object Detection Using Open CV
 
Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024
 
AWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of Terraform
 
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
 
HTML Injection Attacks: Impact and Mitigation Strategies
HTML Injection Attacks: Impact and Mitigation StrategiesHTML Injection Attacks: Impact and Mitigation Strategies
HTML Injection Attacks: Impact and Mitigation Strategies
 
What Are The Drone Anti-jamming Systems Technology?
What Are The Drone Anti-jamming Systems Technology?What Are The Drone Anti-jamming Systems Technology?
What Are The Drone Anti-jamming Systems Technology?
 
Advantages of Hiring UIUX Design Service Providers for Your Business
Advantages of Hiring UIUX Design Service Providers for Your BusinessAdvantages of Hiring UIUX Design Service Providers for Your Business
Advantages of Hiring UIUX Design Service Providers for Your Business
 
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, AdobeApidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
 
Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsArtificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and Myths
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
 
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
 
Boost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityBoost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivity
 
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
 

FME to the Rescue

  • 1. Using FME to Overcome General GIS Software Limitations Alicia Foose, DCP Midstream
  • 2. DCP Midstream Overview   DCP Midstream, LLC, a 50-50 joint venture between Spectra Energy and ConocoPhillips, is headquartered in Denver, Colorado.   The Company leads the midstream segment as one of the nation’s largest natural gas gatherers and processors in the United States.   DCP Midstream is the largest natural gas liquids (NGLs) producers in the nation.
  • 3. DCP Midstream Overview   The Company owns or operates 58 plants, 10 fractionating facilities, and approximately 60,000 miles of gathering and transmission pipeline with connections to approximately 38,000 active receipt points.   Visit https://www.dcpmidstream.com for more details.
  • 4. What Does This Look Like
  • 6. GIS Environment   Oracle database   ESRI SDE   Pipeline Open Database Standard (PODS) 4.02 database model   http://pods.org/   The volume and complexity of data can create challenges for GIS analysis.
  • 7. A Recent Project   DCP Midstream went through an evaluation process to find a software solution to manage the One Call (Call before you dig) process.   One of the solutions only accepted polygon features. Because all of the pipelines in the PODS database are polylines features a buffer needed to be created for the pilot.   To keep the comparisons similar we decided to buffer the pipeline by one foot.   Only the location of the pipelines were of interest.
  • 9. Volume Of Data   We were only interested in the pipelines we operated so a query was necessary.   The pipeline layer being used has 164,535 polylines in the database. SQL> select count(*) from PODS.REGULATORY_SEGMENT; COUNT(*) ---------- 164535   Laptop processing capacity along with memory limits can become an issue when buffering this volume of data.
  • 10. Creating The Buffer In FME   The buffer was created in FME because   It is easy to set up   It can run in the background   It doesn’t seem to use as many resources   It tends to run faster on my environment   It has an aggregate feature   It can filter attributes
  • 11. Creating The Buffer In FME   The goal was to:   Query for only DCP Midstream Operated pipelines.   Simplify the data by eliminating most of the columns.   I chose to keep Region because there are only 10 regions (Regions have a logical geographical area)   Buffer the pipelines by 1 foot.   Aggregate the data.   Export the Polygon feature to an ESRI shape file format.
  • 12. Query For DCP Operated   The data was queried directly from the SDE connection in FME – this filters the data on the fly.   The 164,535 rows were reduced by 2,829 to total 161,706 records to buffer.
  • 13. Transformers Used   The AttributeKeeper was used to reduce the number of columns from 51 down to 6 keeping only REGION_NAME from the SDE layer. Because only the location of a pipeline was required, the associated attributes were not needed.
  • 14. Transformers Used   The Reprojector was used to project the data from NAD 83 to a projection with a unit of measure in feet.   US48-DUKE was chosen because the projection was created for the continental US and has relatively little overall distortion.
  • 15. Transformers Used   The Bufferer was used to buffer by 1 foot.
  • 16. Transformers Used   The Aggregator was used to aggregate the data using the REGION_NAME to group by.   Aggregating the data reduced the number of records from 161,706 to 10.
  • 17. Transformers Used   The Reprojector was used to project the data back to NAD83.   Finally, the destination dataset was set to a shape file format. A visualizer was used so the output could be viewed right away.   A dissolver transformer was not used because the aggregate combined all of the polygons into Regions and the overlaps were not a concern for the end use.
  • 19. Final Results Total Features Written 10 =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Translation was SUCCESSFUL with 0 warning(s) (10 feature(s)/5295013 coordinate(s) output) FME Session Duration: 7 minutes 9.2 seconds. (CPU: 141.9s user, 10.6s system) END - ProcessID: 1480, peak process memory usage: 208120 kB, current process memory usage: 53868 kB.
  • 20. A Side By Side Comparison   A 1 foot buffer was run in Arc Info using the same query on the same layer.   Dissolve by field - REGION_NAME was selected because it is the closest option to the FME Aggregate .
  • 21. A Side By Side Comparison Executing (Buffer_2): Buffer PODS.REGULATORY_SEGMENT Server 1_ft_Buffer.shp "1 Feet" FULL ROUND ALL # Start Time: Thu Mar 11 08:29:01 2010 Dissolving... Output feature 0 cannot be dissolved into other inputs because of memory limitations Output feature 1 cannot be dissolved into other inputs because of memory limitations … … Output feature 15 cannot be dissolved into other inputs because of memory limitations Executed (Buffer_2) successfully. End Time: Thu Mar 11 11:12:32 2010 (Elapsed Time: 2 hours 43 minutes 31 seconds)
  • 22. A Side By Side Comparison   The FME translation ran in 7 minutes 9.2 seconds with no memory errors.   The Arc Info Buffer wizard ran in 2 hours 43 minutes and 31 seconds with memory limitations errors.   The results from either process were acceptable.
  • 23. Annual Tax Project   Benjamin Franklin once said that “In this world nothing is certain but death and taxes”.   So lets talk about taxes, specifically property taxes. You might be asking yourself what on earth does FME have to do with property taxes. Well here is your answer-   Each year companies with tangible assets pay property taxes. Pipelines are not excluded.
  • 24. The Challenge   Every State has unique taxing districts by which they collect and distribute property taxes.   Tax districts can change from year to year although most remain the same.   Population shifts and demographics are the most common cause of tax boundary changes.   DCP Midstream operates primarily in 17 States so tax boundary maintenance is a fairly large undertaking.
  • 25. Tax Project   Each year the GIS department provides the Tax department with a report of how many feet of each pipeline is in what tax district by install year, diameter and so on.   The first step, for States with electronic data, is to download the current tax boundary files and update the SDE layer with the changes.   The SDE Layer has to be topologically clean.   Neighboring states do not tend to use the exact same state line. This creates gaps and overlaps which are ugly to clean up particularly along rivers.
  • 26. 2008 Oklahoma Tax Districts
  • 27. 2009 Oklahoma Tax Districts
  • 28. Oklahoma Tax Districts   Who can tell me what changed?   Going once   Going twice   Going three time   How are you going to find out?   FME has a transformer named Matcher which detects both geometry and attribute changes from two files.
  • 29. Lets See What Changed   The 2008 Oklahoma tax districts are added as one source.   The 2009 Oklahoma tax districts are added as another source.   Both are run through the Matcher as input.   The Not_Matched features are output to a visualizer so they can be looked at.   The Not_Matched features are output to a shape file to be used in ArcMap for updating the SDE layer.
  • 31. This Is What Changed
  • 32. Why So Many Changes – River Correction
  • 34. Thank You!   Questions?   For more information:   Alicia Foose apfoose@dcpmidstream.com   DCP Midstream   https://www.dcpmidstream.com   http://pods.org/