SlideShare una empresa de Scribd logo
1 de 27
APDM 101
(ArcGIS Pipeline Data Model)
  or "What did you do to my As-Builts
  and Alignment sheets and how am i
  supposed to use this new acronym
                thingy?"
First, an introduction to GIS
Geographic Information System

 • A geographic information system (GIS) integrates hardware,
   software, and data for capturing, managing, analyzing, and
   displaying all forms of geographically referenced
   information.
GIS allows us to view, understand, question, interpret, and
visualize data in many ways that reveal relationships, patterns,
and trends in the form of maps, globes, reports, and charts.
A GIS helps you answer questions and solve problems by
looking at your data in a way that is quickly understood and
easily shared.
 • System of :
    o Data, Software, Hardware, People and Processes
GIS - Data makes the world go round
Geographic Information System

• Data
  o Raster - Aerial Photography
      A continuous grid of Values at X,Y coordinates
  o Tabular - "Object Classes"
      Rows and columns of attribute values
  o Vector - "Feature Classes"
      Points,
         X,Y coordinate (with attribute values)
      Lines,
         Collections of X,Y coordinates (with attributes)
      Polygons
         Area within a closed loop consisting of a collection
          of X,Y cordinates (with attributes)
What GIS Software do we have

• ESRI - ArcGIS
  o ArcSDE
      Database application which manages geospatial data
       inside of Oracle
  o ArcCatalog
      Explorer for Geospatial data
  o ArcMap
      Data Editing
      Data Visualization
      Data Analysis
      Reporting
      Map Generation
  o ArcFM Viewer
      Data Viewing
What is this Geodatabase you speak of?

• The Geodatabase -
   o A geo-spatially enabled database.
   o Database stores tables in columns and rows of attributes
   o Geodatabase adds the ability to store spatial locations
     along with the other attributes in the tables.

• Geodatabase "Flavors"
  o Enterprise - Multi User, Versioned, Stored in Oracle
  o Personal - Single user, non-Versioned, Stored in .MDB
  o File - Multi-User w/ limitations, Stored in folder of files.
What's in the Geodatabase?

• The Geodatabase contains
   o Object Classes - Tables with attribute fields and rows
   o Feature Classes - Tables which also store Geometry
   o Relationship Classes - relate Objects & Features
• Inside the Geodatabase, Feature Datasets contain related
  FeatureClasses in the same coordinate system.
       APDM < --- Geodatabase
          Site < --- Object Class
          Transmission < --- Feature Dataset (like a folder)
          ControlPoints < --- Feature Class
          StationSeries < --- Feature Class
          StationSeriesControlPoints < -- Relationship Class
So the data can be one of these types?
• Pipeline Data Examples
  o Tables
      Site
      StructureOrIDSite
      AllLeakInfo
  o Points
      Valves - "Online" - "on the pipeline centerline"
      CP Test Station - "Offline" - not on the centerline
  o Lines
      Pipe Segments - "Online" - "on the centerline"
      Casing - "Online" - "on the centerline"
      Access Roads - "Offline" - not on the centerline
  o Polygons
      Identified Sites - "Offline"
      Storage Field Pool and Buffer - "Offline"
Attribute Fields?
• What Kinds of attributes
  o Date       - "10/12/11"
  o Text      - "CW1234"
  o Integer   - "4"
  o Double    - "12.75"
  o GUID      - "{21EC2020-3AEA-1069-A2DD-08002B30309D}"
       More about GUIDs later....
• Defaults - value inserted when a feature or object is created.
• Allow Null - Y/N
• Domains
  o Coded Domains - AKA: Pick Lists or Valid Value Lists
       Valve Class,
       Structure Type,
  o Ranges - "Minimum and Maximum"
       0-360 - degrees of rotation
APDM
Attribute
Fields
Examples
Joins
GIS - Geographic Information System

• Records in one Object Class or Feature Class can be
  connected to records in another Object Class or Feature
  Class.
   o Joins
       Appending the fields of one table to those of another
        through an attribute or field common to both tables. A
        join is usually used to attach more attributes to the
        attribute table of a geographic layer.
   o Primary Key - the unique identifier of a row or feature.
   o Foreign Key - the unique identifier of another row or
     feature which can be joined to the first row or feature.
Joins : an example
GIS - Geographic Information System
Relationship Classes
Permanent Relates, stored inside the Geodatabase.

• StructureOrIdentifiedSite
  o IdentifiedSite
  o StructureOutline
  o NearestPointToLine


• InLine Inspection runs
   o Metal Loss Anomaly


• Identified Site
   o HCA Segment


• Assessment Project
  o Assessment Region
      Assessment Region Range
GUIDS

Globally Unique IDentifier -
• Represented as a 32-character hexadecimal string, such as
  {21EC2020-3AEA-1069-A2DD-08002B30309D}
• Applied to all records and features in the APDM as the
  Primary Key field: "EventID"
• Used as the Keys and Foreign Keys for Relationship
  Classes
• How to identify a foreign key?
   o It talks funny?
• No, actually it ends in "EventID" and begins (usually) with
  the name of another Feature / Object Class.
   o e.g.: StructureOrIDSiteEventID
   o e.g.: NearestPointToLineEventID
• It holds the EventID of a feature in that other class....
MetaData

• Often described as:
  o "Data about Data"
• But it is more completely described as:
  o Information that describes the content, quality, condition,
    origin, and other characteristics of data or other pieces of
    information.
  o Data to document and describe its subject matter; how,
    when, where, and by whom the data was collected;
    availability and distribution information; its projection,
    scale, resolution, and accuracy; and its reliability with
    regard to some standard.
Linear Referencing

• Stationing is the standard way pipeline operators have
  historically stored pipeline records.
• Stationing provides the mechanism to support sophisticated
  analysis of online features, also known as "events".
• GIS Implementation of Engineering Stationing
   o Locates online points and lines along a Route
   o Locates by Route Name and Station Value
   o Or a Begin Station and End Station for lines.


• The APDM Route is called "StationSeries"
• The APDM Station attribute is:
   o for Points: "Station"
   o for Lines: "BeginStation" / "EndStation"
Linear Referencing

• Highway Mile Post Example:


• You're driving your car along I-90. You can locate your car
  with a GPS or Linear referencing. What units are used?

    o    GPS: Degrees of Latitude and Longitude
- or -
    o    Stationing: I-90 & Mile Post 12.5

• Both Descriptions are valid, the stationing description may
  be easier to generate and search by.
Data Modeling and the Schema
• Data Model:
  o   A set of database design specifications for items in a GIS
      application. A data model describes the layers used in the
      application (for example, Valves, Pipes, and Identified Sites);
      their geometry type (point, line, or polygon); their attributes; and
      their data validation rules.

• Schema:
  o   The structure or design of a database or item in a database,
      such as a object class, feature class or relationship class. I
  o   The schema defines the classes, the fields in each class, the
      relationships between fields and classes, and the grouping of
      items within the database.
The APDM - A history

• Now in Version 5.0 - We're using an enhancement of Ver. 4
• In existence since 2002 (Inherits from previous models.)
• Based on Stationed Pipeline Data, Implemented with ESRI's
  measures and linear referencing

•   Intended as starting point for operator implementations
•   Intended to permit common industry wide tools
•   Designed to use ESRI Geodatabase technology
•   Intended to be 80% common with all pipeline operators
•   Expandable through use of Abstract Classes

• Developed by operator and vendor representatives.
• The APDM model and supporting documentation is freely
  available and accessible to everyone
UML - Unified Modeling Language

"Secret Language which is used by International super models
to talk to other equally super models."

• Unfortunately, while this is false, UML is somewhat of a
  secret language and does take some patience to read and
  understand.
• The APDM is distributed by the APDM committes in UML
  created in Visio. The UML enables the systematic creation,
  documentation and update of the APDM schema.
• ESRI tools permit the creation of the APDM schema in a
  Geodatabase.
• Reading the UML gives us an understanding of how our
  objects and features are constructed and related.
UML Concepts

• Abstract Classes
  o a specification for subclasses that is often shown on
    object model diagrams to help give structure to the
    diagram. An abstract class is not defined in a type library
    and cannot be instantiated.

• Inheritance
   o the acquisition of methods and properties by child classes
     or interfaces from their previously existing parent classes
     or interfaces. Inheritance is one of the defining
     characteristics of an object-oriented system.
UML Concepts

Importance?
   Allow us to create database schema with logical,
consistent features with appropriate attributes. And
to update and apply them without re-doing everything
time after time.

Example:
 Elbow and Tee are "Concrete" Classes
and Inherit from the Abstract Class "Fitting"
APDM
Abstract
Classes

and
symbols
Core Classes

"When it absolutely, positively has to be
there" in your APDM Model.......
Essentially the bare minimum to make the model work.

The most common classes and attributes which make the
APDM work.
APDM is a Data model Template

• Implement the Core Classes

• Add attributes to the core Classes if you want to or not.

• Implement the Optional Classes if you want to or not.

• Add attributes to the Optional Classes if you want to or not.

• Add more classes, based on the Abstract Classes, if you
  want to or not.

• Do all of these and make the model fit your business!
How did the data get into the APDM?

• Data from the Legacy Systems, was loaded into the Schema
  using custom tools.
• Some Legacy data was "massaged" before it could be
  loaded.
• Some Legacy data had to be cleaned before it was loaded.
• Some data had to be created from scratch in order to be
  loaded.
• StationSeries were created from Control Points created from
  previous Routes used by the integrity group.
• RAM was the primary legacy data set used to create data
  for the APDM.
• Features were loaded along the StationSeries based upon
  their Route Identifier ("Location") and their Station Value(s).
Whats next?

• Data Maintenance

• Inspection Data Loading
   o Data Calibrator


• Data Analysis
  o HCA / Class Location Calculator
  o Risk Calculator


• Data Reporting
  o Alignment Sheet Generator
  o DOT Report Generator
Need more Information?

• www.FisherAssoc.com
  o DNichter@FisherAssoc.com


• www.ESRI.com/industries/pipeline

• www.APDM.net

Más contenido relacionado

La actualidad más candente

Gis Day Presentation 2010 - ACCC - Expanded Version
Gis Day Presentation 2010 - ACCC - Expanded VersionGis Day Presentation 2010 - ACCC - Expanded Version
Gis Day Presentation 2010 - ACCC - Expanded Versionpdcaris
 
Data managing and Exchange GDB
Data managing and Exchange GDB Data managing and Exchange GDB
Data managing and Exchange GDB Esri
 
Smarter Together - Bringing Relational Algebra, Powered by Apache Calcite, in...
Smarter Together - Bringing Relational Algebra, Powered by Apache Calcite, in...Smarter Together - Bringing Relational Algebra, Powered by Apache Calcite, in...
Smarter Together - Bringing Relational Algebra, Powered by Apache Calcite, in...Julian Hyde
 
Enterprise geodatabase sql access and administration
Enterprise geodatabase sql access and administrationEnterprise geodatabase sql access and administration
Enterprise geodatabase sql access and administrationbrentpierce
 
Introduction To PostGIS
Introduction To PostGISIntroduction To PostGIS
Introduction To PostGISmleslie
 
Introduction to Tools in ArcGIS
Introduction to Tools in ArcGISIntroduction to Tools in ArcGIS
Introduction to Tools in ArcGISDaniele Baker
 
Improving Apache Spark's Reliability with DataSourceV2
Improving Apache Spark's Reliability with DataSourceV2Improving Apache Spark's Reliability with DataSourceV2
Improving Apache Spark's Reliability with DataSourceV2Databricks
 
How to Extend Apache Spark with Customized Optimizations
How to Extend Apache Spark with Customized OptimizationsHow to Extend Apache Spark with Customized Optimizations
How to Extend Apache Spark with Customized OptimizationsDatabricks
 
Getting to Know ArcGIS Pro
Getting to Know ArcGIS ProGetting to Know ArcGIS Pro
Getting to Know ArcGIS ProEsri UK
 
Getting started with GIS
Getting started with GISGetting started with GIS
Getting started with GISEsri India
 
Developing Efficient Web-based GIS Applications
Developing Efficient Web-based GIS ApplicationsDeveloping Efficient Web-based GIS Applications
Developing Efficient Web-based GIS ApplicationsSwetha A
 
Working with ArcGIS Online
Working with ArcGIS OnlineWorking with ArcGIS Online
Working with ArcGIS OnlineEsri
 
Raster data model
Raster data modelRaster data model
Raster data modelPramoda Raj
 
Graph Features in Spark 3.0: Integrating Graph Querying and Algorithms in Spa...
Graph Features in Spark 3.0: Integrating Graph Querying and Algorithms in Spa...Graph Features in Spark 3.0: Integrating Graph Querying and Algorithms in Spa...
Graph Features in Spark 3.0: Integrating Graph Querying and Algorithms in Spa...Databricks
 
Foreign Data Wrappers and You with Postgres
Foreign Data Wrappers and You with PostgresForeign Data Wrappers and You with Postgres
Foreign Data Wrappers and You with PostgresEDB
 
Introduction to Apache Spark
Introduction to Apache SparkIntroduction to Apache Spark
Introduction to Apache SparkRahul Jain
 
GeoMesa: Scalable Geospatial Analytics
GeoMesa:  Scalable Geospatial AnalyticsGeoMesa:  Scalable Geospatial Analytics
GeoMesa: Scalable Geospatial AnalyticsVisionGEOMATIQUE2014
 
Constraining terrestrial carbon balance through assimilation of remotely sens...
Constraining terrestrial carbon balance through assimilation of remotely sens...Constraining terrestrial carbon balance through assimilation of remotely sens...
Constraining terrestrial carbon balance through assimilation of remotely sens...Integrated Carbon Observation System (ICOS)
 

La actualidad más candente (20)

Gis Day Presentation 2010 - ACCC - Expanded Version
Gis Day Presentation 2010 - ACCC - Expanded VersionGis Day Presentation 2010 - ACCC - Expanded Version
Gis Day Presentation 2010 - ACCC - Expanded Version
 
Data managing and Exchange GDB
Data managing and Exchange GDB Data managing and Exchange GDB
Data managing and Exchange GDB
 
Smarter Together - Bringing Relational Algebra, Powered by Apache Calcite, in...
Smarter Together - Bringing Relational Algebra, Powered by Apache Calcite, in...Smarter Together - Bringing Relational Algebra, Powered by Apache Calcite, in...
Smarter Together - Bringing Relational Algebra, Powered by Apache Calcite, in...
 
Enterprise geodatabase sql access and administration
Enterprise geodatabase sql access and administrationEnterprise geodatabase sql access and administration
Enterprise geodatabase sql access and administration
 
Introduction To PostGIS
Introduction To PostGISIntroduction To PostGIS
Introduction To PostGIS
 
Introduction to Tools in ArcGIS
Introduction to Tools in ArcGISIntroduction to Tools in ArcGIS
Introduction to Tools in ArcGIS
 
Improving Apache Spark's Reliability with DataSourceV2
Improving Apache Spark's Reliability with DataSourceV2Improving Apache Spark's Reliability with DataSourceV2
Improving Apache Spark's Reliability with DataSourceV2
 
How to Extend Apache Spark with Customized Optimizations
How to Extend Apache Spark with Customized OptimizationsHow to Extend Apache Spark with Customized Optimizations
How to Extend Apache Spark with Customized Optimizations
 
Getting to Know ArcGIS Pro
Getting to Know ArcGIS ProGetting to Know ArcGIS Pro
Getting to Know ArcGIS Pro
 
Getting started with GIS
Getting started with GISGetting started with GIS
Getting started with GIS
 
Developing Efficient Web-based GIS Applications
Developing Efficient Web-based GIS ApplicationsDeveloping Efficient Web-based GIS Applications
Developing Efficient Web-based GIS Applications
 
Working with ArcGIS Online
Working with ArcGIS OnlineWorking with ArcGIS Online
Working with ArcGIS Online
 
Raster data model
Raster data modelRaster data model
Raster data model
 
Graph Features in Spark 3.0: Integrating Graph Querying and Algorithms in Spa...
Graph Features in Spark 3.0: Integrating Graph Querying and Algorithms in Spa...Graph Features in Spark 3.0: Integrating Graph Querying and Algorithms in Spa...
Graph Features in Spark 3.0: Integrating Graph Querying and Algorithms in Spa...
 
What Is GIS?
What Is GIS?What Is GIS?
What Is GIS?
 
introduction to GIS
introduction to GIS introduction to GIS
introduction to GIS
 
Foreign Data Wrappers and You with Postgres
Foreign Data Wrappers and You with PostgresForeign Data Wrappers and You with Postgres
Foreign Data Wrappers and You with Postgres
 
Introduction to Apache Spark
Introduction to Apache SparkIntroduction to Apache Spark
Introduction to Apache Spark
 
GeoMesa: Scalable Geospatial Analytics
GeoMesa:  Scalable Geospatial AnalyticsGeoMesa:  Scalable Geospatial Analytics
GeoMesa: Scalable Geospatial Analytics
 
Constraining terrestrial carbon balance through assimilation of remotely sens...
Constraining terrestrial carbon balance through assimilation of remotely sens...Constraining terrestrial carbon balance through assimilation of remotely sens...
Constraining terrestrial carbon balance through assimilation of remotely sens...
 

Destacado

Georeferencing
GeoreferencingGeoreferencing
GeoreferencingSBL-corp
 
Georeferencing Raster Data Using ArcGIS
Georeferencing Raster Data Using ArcGISGeoreferencing Raster Data Using ArcGIS
Georeferencing Raster Data Using ArcGISJordan Frey
 
Gis arc map& georeferencing-arcgis
Gis arc map& georeferencing-arcgisGis arc map& georeferencing-arcgis
Gis arc map& georeferencing-arcgisAshok Peddi
 
Components of Spatial Data Quality in GIS
Components of Spatial Data Quality in GISComponents of Spatial Data Quality in GIS
Components of Spatial Data Quality in GISKaium Chowdhury
 
Lec_6_Intro to geo-referencing
Lec_6_Intro to geo-referencingLec_6_Intro to geo-referencing
Lec_6_Intro to geo-referencingAtiqa khan
 
Types of maps, long, lat, satellites, time zones
Types of maps, long, lat, satellites, time zonesTypes of maps, long, lat, satellites, time zones
Types of maps, long, lat, satellites, time zonesBantay's Earth Science!
 
Introduction to MAPS,Coordinate System and Projection System
Introduction to MAPS,Coordinate System and Projection SystemIntroduction to MAPS,Coordinate System and Projection System
Introduction to MAPS,Coordinate System and Projection SystemNAXA-Developers
 
Geodetic systems (earth, ellipsoid)
Geodetic systems (earth, ellipsoid)Geodetic systems (earth, ellipsoid)
Geodetic systems (earth, ellipsoid)Md. Tamim Haider
 
Understanding Coordinate Systems and Projections for ArcGIS
Understanding Coordinate Systems and Projections for ArcGISUnderstanding Coordinate Systems and Projections for ArcGIS
Understanding Coordinate Systems and Projections for ArcGISJohn Schaeffer
 
ppt spatial data
ppt spatial datappt spatial data
ppt spatial dataRahul Kumar
 
Floodmap Desktop for ArcGIS
Floodmap Desktop for ArcGISFloodmap Desktop for ArcGIS
Floodmap Desktop for ArcGISewug
 
Data Models - GIS I
Data Models - GIS IData Models - GIS I
Data Models - GIS IJohn Reiser
 

Destacado (20)

Mapping Projections and Data conversions
Mapping Projections and Data conversionsMapping Projections and Data conversions
Mapping Projections and Data conversions
 
1 georeferencing
1 georeferencing1 georeferencing
1 georeferencing
 
Georeferencing
GeoreferencingGeoreferencing
Georeferencing
 
Georeferencing Raster Data Using ArcGIS
Georeferencing Raster Data Using ArcGISGeoreferencing Raster Data Using ArcGIS
Georeferencing Raster Data Using ArcGIS
 
Raster
RasterRaster
Raster
 
Gis arc map& georeferencing-arcgis
Gis arc map& georeferencing-arcgisGis arc map& georeferencing-arcgis
Gis arc map& georeferencing-arcgis
 
Components of Spatial Data Quality in GIS
Components of Spatial Data Quality in GISComponents of Spatial Data Quality in GIS
Components of Spatial Data Quality in GIS
 
Geodesy
GeodesyGeodesy
Geodesy
 
Lec_6_Intro to geo-referencing
Lec_6_Intro to geo-referencingLec_6_Intro to geo-referencing
Lec_6_Intro to geo-referencing
 
Basic of Geodesy
Basic of GeodesyBasic of Geodesy
Basic of Geodesy
 
Types of maps, long, lat, satellites, time zones
Types of maps, long, lat, satellites, time zonesTypes of maps, long, lat, satellites, time zones
Types of maps, long, lat, satellites, time zones
 
Spatial Data Model
Spatial Data ModelSpatial Data Model
Spatial Data Model
 
GIS data structure
GIS data structureGIS data structure
GIS data structure
 
Introduction to MAPS,Coordinate System and Projection System
Introduction to MAPS,Coordinate System and Projection SystemIntroduction to MAPS,Coordinate System and Projection System
Introduction to MAPS,Coordinate System and Projection System
 
Geodetic systems (earth, ellipsoid)
Geodetic systems (earth, ellipsoid)Geodetic systems (earth, ellipsoid)
Geodetic systems (earth, ellipsoid)
 
Understanding Coordinate Systems and Projections for ArcGIS
Understanding Coordinate Systems and Projections for ArcGISUnderstanding Coordinate Systems and Projections for ArcGIS
Understanding Coordinate Systems and Projections for ArcGIS
 
ppt spatial data
ppt spatial datappt spatial data
ppt spatial data
 
Floodmap Desktop for ArcGIS
Floodmap Desktop for ArcGISFloodmap Desktop for ArcGIS
Floodmap Desktop for ArcGIS
 
Data Models - GIS I
Data Models - GIS IData Models - GIS I
Data Models - GIS I
 
Causes Of Great Depression
Causes Of Great DepressionCauses Of Great Depression
Causes Of Great Depression
 

Similar a Apdm 101 Arc Gis Pipeline Data Model (1)

Arc gis introduction-ppt
Arc gis introduction-pptArc gis introduction-ppt
Arc gis introduction-pptAshok Peddi
 
2016 workshops: understanding the language of gis and cad by warren geissler
2016 workshops:  understanding  the language  of gis and cad by warren geissler2016 workshops:  understanding  the language  of gis and cad by warren geissler
2016 workshops: understanding the language of gis and cad by warren geisslerGIS in the Rockies
 
Arches Getty Brownbag Talk
Arches Getty Brownbag TalkArches Getty Brownbag Talk
Arches Getty Brownbag Talkbenosteen
 
A full Machine learning pipeline in Scikit-learn vs in scala-Spark: pros and ...
A full Machine learning pipeline in Scikit-learn vs in scala-Spark: pros and ...A full Machine learning pipeline in Scikit-learn vs in scala-Spark: pros and ...
A full Machine learning pipeline in Scikit-learn vs in scala-Spark: pros and ...Jose Quesada (hiring)
 
Evolution of Esri Data Formats Seminar
Evolution of Esri Data Formats SeminarEvolution of Esri Data Formats Seminar
Evolution of Esri Data Formats SeminarEsri South Africa
 
Data Analytics with R and SQL Server
Data Analytics with R and SQL ServerData Analytics with R and SQL Server
Data Analytics with R and SQL ServerStéphane Fréchette
 
Graph Databases in the Microsoft Ecosystem
Graph Databases in the Microsoft EcosystemGraph Databases in the Microsoft Ecosystem
Graph Databases in the Microsoft EcosystemMarco Parenzan
 
Composable Parallel Processing in Apache Spark and Weld
Composable Parallel Processing in Apache Spark and WeldComposable Parallel Processing in Apache Spark and Weld
Composable Parallel Processing in Apache Spark and WeldDatabricks
 
An architecture for federated data discovery and lineage over on-prem datasou...
An architecture for federated data discovery and lineage over on-prem datasou...An architecture for federated data discovery and lineage over on-prem datasou...
An architecture for federated data discovery and lineage over on-prem datasou...DataWorks Summit
 
Making the Most of Raster Analysis with Living Atlas Data - Esri UC 2018
Making the Most of Raster Analysis with Living Atlas Data - Esri UC 2018Making the Most of Raster Analysis with Living Atlas Data - Esri UC 2018
Making the Most of Raster Analysis with Living Atlas Data - Esri UC 2018Aileen Buckley
 
Big data analytics with R tool.pptx
Big data analytics with R tool.pptxBig data analytics with R tool.pptx
Big data analytics with R tool.pptxsalutiontechnology
 
How To Model and Construct Graphs with Oracle Database (AskTOM Office Hours p...
How To Model and Construct Graphs with Oracle Database (AskTOM Office Hours p...How To Model and Construct Graphs with Oracle Database (AskTOM Office Hours p...
How To Model and Construct Graphs with Oracle Database (AskTOM Office Hours p...Jean Ihm
 
A machine learning and data science pipeline for real companies
A machine learning and data science pipeline for real companiesA machine learning and data science pipeline for real companies
A machine learning and data science pipeline for real companiesDataWorks Summit
 
Etosha - Data Asset Manager : Status and road map
Etosha - Data Asset Manager : Status and road mapEtosha - Data Asset Manager : Status and road map
Etosha - Data Asset Manager : Status and road mapDr. Mirko Kämpf
 

Similar a Apdm 101 Arc Gis Pipeline Data Model (1) (20)

Essentials of R
Essentials of REssentials of R
Essentials of R
 
Arc gis introduction-ppt
Arc gis introduction-pptArc gis introduction-ppt
Arc gis introduction-ppt
 
2016 workshops: understanding the language of gis and cad by warren geissler
2016 workshops:  understanding  the language  of gis and cad by warren geissler2016 workshops:  understanding  the language  of gis and cad by warren geissler
2016 workshops: understanding the language of gis and cad by warren geissler
 
Arches Getty Brownbag Talk
Arches Getty Brownbag TalkArches Getty Brownbag Talk
Arches Getty Brownbag Talk
 
A full Machine learning pipeline in Scikit-learn vs in scala-Spark: pros and ...
A full Machine learning pipeline in Scikit-learn vs in scala-Spark: pros and ...A full Machine learning pipeline in Scikit-learn vs in scala-Spark: pros and ...
A full Machine learning pipeline in Scikit-learn vs in scala-Spark: pros and ...
 
Evolution of Esri Data Formats Seminar
Evolution of Esri Data Formats SeminarEvolution of Esri Data Formats Seminar
Evolution of Esri Data Formats Seminar
 
Data Analytics with R and SQL Server
Data Analytics with R and SQL ServerData Analytics with R and SQL Server
Data Analytics with R and SQL Server
 
System analyst and design
System analyst and designSystem analyst and design
System analyst and design
 
Velocity cubes of galaxies
Velocity cubes of galaxiesVelocity cubes of galaxies
Velocity cubes of galaxies
 
Graph Databases in the Microsoft Ecosystem
Graph Databases in the Microsoft EcosystemGraph Databases in the Microsoft Ecosystem
Graph Databases in the Microsoft Ecosystem
 
Composable Parallel Processing in Apache Spark and Weld
Composable Parallel Processing in Apache Spark and WeldComposable Parallel Processing in Apache Spark and Weld
Composable Parallel Processing in Apache Spark and Weld
 
An architecture for federated data discovery and lineage over on-prem datasou...
An architecture for federated data discovery and lineage over on-prem datasou...An architecture for federated data discovery and lineage over on-prem datasou...
An architecture for federated data discovery and lineage over on-prem datasou...
 
Informatica slides
Informatica slidesInformatica slides
Informatica slides
 
Making the Most of Raster Analysis with Living Atlas Data - Esri UC 2018
Making the Most of Raster Analysis with Living Atlas Data - Esri UC 2018Making the Most of Raster Analysis with Living Atlas Data - Esri UC 2018
Making the Most of Raster Analysis with Living Atlas Data - Esri UC 2018
 
Big data analytics with R tool.pptx
Big data analytics with R tool.pptxBig data analytics with R tool.pptx
Big data analytics with R tool.pptx
 
How To Model and Construct Graphs with Oracle Database (AskTOM Office Hours p...
How To Model and Construct Graphs with Oracle Database (AskTOM Office Hours p...How To Model and Construct Graphs with Oracle Database (AskTOM Office Hours p...
How To Model and Construct Graphs with Oracle Database (AskTOM Office Hours p...
 
DA_MAP
DA_MAPDA_MAP
DA_MAP
 
A machine learning and data science pipeline for real companies
A machine learning and data science pipeline for real companiesA machine learning and data science pipeline for real companies
A machine learning and data science pipeline for real companies
 
Geoprocessing
GeoprocessingGeoprocessing
Geoprocessing
 
Etosha - Data Asset Manager : Status and road map
Etosha - Data Asset Manager : Status and road mapEtosha - Data Asset Manager : Status and road map
Etosha - Data Asset Manager : Status and road map
 

Apdm 101 Arc Gis Pipeline Data Model (1)

  • 1. APDM 101 (ArcGIS Pipeline Data Model) or "What did you do to my As-Builts and Alignment sheets and how am i supposed to use this new acronym thingy?"
  • 2. First, an introduction to GIS Geographic Information System • A geographic information system (GIS) integrates hardware, software, and data for capturing, managing, analyzing, and displaying all forms of geographically referenced information. GIS allows us to view, understand, question, interpret, and visualize data in many ways that reveal relationships, patterns, and trends in the form of maps, globes, reports, and charts. A GIS helps you answer questions and solve problems by looking at your data in a way that is quickly understood and easily shared. • System of : o Data, Software, Hardware, People and Processes
  • 3. GIS - Data makes the world go round Geographic Information System • Data o Raster - Aerial Photography  A continuous grid of Values at X,Y coordinates o Tabular - "Object Classes"  Rows and columns of attribute values o Vector - "Feature Classes"  Points,  X,Y coordinate (with attribute values)  Lines,  Collections of X,Y coordinates (with attributes)  Polygons  Area within a closed loop consisting of a collection of X,Y cordinates (with attributes)
  • 4. What GIS Software do we have • ESRI - ArcGIS o ArcSDE  Database application which manages geospatial data inside of Oracle o ArcCatalog  Explorer for Geospatial data o ArcMap  Data Editing  Data Visualization  Data Analysis  Reporting  Map Generation o ArcFM Viewer  Data Viewing
  • 5. What is this Geodatabase you speak of? • The Geodatabase - o A geo-spatially enabled database. o Database stores tables in columns and rows of attributes o Geodatabase adds the ability to store spatial locations along with the other attributes in the tables. • Geodatabase "Flavors" o Enterprise - Multi User, Versioned, Stored in Oracle o Personal - Single user, non-Versioned, Stored in .MDB o File - Multi-User w/ limitations, Stored in folder of files.
  • 6. What's in the Geodatabase? • The Geodatabase contains o Object Classes - Tables with attribute fields and rows o Feature Classes - Tables which also store Geometry o Relationship Classes - relate Objects & Features • Inside the Geodatabase, Feature Datasets contain related FeatureClasses in the same coordinate system.  APDM < --- Geodatabase  Site < --- Object Class  Transmission < --- Feature Dataset (like a folder)  ControlPoints < --- Feature Class  StationSeries < --- Feature Class  StationSeriesControlPoints < -- Relationship Class
  • 7. So the data can be one of these types? • Pipeline Data Examples o Tables  Site  StructureOrIDSite  AllLeakInfo o Points  Valves - "Online" - "on the pipeline centerline"  CP Test Station - "Offline" - not on the centerline o Lines  Pipe Segments - "Online" - "on the centerline"  Casing - "Online" - "on the centerline"  Access Roads - "Offline" - not on the centerline o Polygons  Identified Sites - "Offline"  Storage Field Pool and Buffer - "Offline"
  • 8. Attribute Fields? • What Kinds of attributes o Date - "10/12/11" o Text - "CW1234" o Integer - "4" o Double - "12.75" o GUID - "{21EC2020-3AEA-1069-A2DD-08002B30309D}"  More about GUIDs later.... • Defaults - value inserted when a feature or object is created. • Allow Null - Y/N • Domains o Coded Domains - AKA: Pick Lists or Valid Value Lists  Valve Class,  Structure Type, o Ranges - "Minimum and Maximum"  0-360 - degrees of rotation
  • 10. Joins GIS - Geographic Information System • Records in one Object Class or Feature Class can be connected to records in another Object Class or Feature Class. o Joins  Appending the fields of one table to those of another through an attribute or field common to both tables. A join is usually used to attach more attributes to the attribute table of a geographic layer. o Primary Key - the unique identifier of a row or feature. o Foreign Key - the unique identifier of another row or feature which can be joined to the first row or feature.
  • 11. Joins : an example GIS - Geographic Information System
  • 12. Relationship Classes Permanent Relates, stored inside the Geodatabase. • StructureOrIdentifiedSite o IdentifiedSite o StructureOutline o NearestPointToLine • InLine Inspection runs o Metal Loss Anomaly • Identified Site o HCA Segment • Assessment Project o Assessment Region  Assessment Region Range
  • 13. GUIDS Globally Unique IDentifier - • Represented as a 32-character hexadecimal string, such as {21EC2020-3AEA-1069-A2DD-08002B30309D} • Applied to all records and features in the APDM as the Primary Key field: "EventID" • Used as the Keys and Foreign Keys for Relationship Classes • How to identify a foreign key? o It talks funny? • No, actually it ends in "EventID" and begins (usually) with the name of another Feature / Object Class. o e.g.: StructureOrIDSiteEventID o e.g.: NearestPointToLineEventID • It holds the EventID of a feature in that other class....
  • 14. MetaData • Often described as: o "Data about Data" • But it is more completely described as: o Information that describes the content, quality, condition, origin, and other characteristics of data or other pieces of information. o Data to document and describe its subject matter; how, when, where, and by whom the data was collected; availability and distribution information; its projection, scale, resolution, and accuracy; and its reliability with regard to some standard.
  • 15. Linear Referencing • Stationing is the standard way pipeline operators have historically stored pipeline records. • Stationing provides the mechanism to support sophisticated analysis of online features, also known as "events". • GIS Implementation of Engineering Stationing o Locates online points and lines along a Route o Locates by Route Name and Station Value o Or a Begin Station and End Station for lines. • The APDM Route is called "StationSeries" • The APDM Station attribute is: o for Points: "Station" o for Lines: "BeginStation" / "EndStation"
  • 16. Linear Referencing • Highway Mile Post Example: • You're driving your car along I-90. You can locate your car with a GPS or Linear referencing. What units are used? o GPS: Degrees of Latitude and Longitude - or - o Stationing: I-90 & Mile Post 12.5 • Both Descriptions are valid, the stationing description may be easier to generate and search by.
  • 17. Data Modeling and the Schema • Data Model: o A set of database design specifications for items in a GIS application. A data model describes the layers used in the application (for example, Valves, Pipes, and Identified Sites); their geometry type (point, line, or polygon); their attributes; and their data validation rules. • Schema: o The structure or design of a database or item in a database, such as a object class, feature class or relationship class. I o The schema defines the classes, the fields in each class, the relationships between fields and classes, and the grouping of items within the database.
  • 18. The APDM - A history • Now in Version 5.0 - We're using an enhancement of Ver. 4 • In existence since 2002 (Inherits from previous models.) • Based on Stationed Pipeline Data, Implemented with ESRI's measures and linear referencing • Intended as starting point for operator implementations • Intended to permit common industry wide tools • Designed to use ESRI Geodatabase technology • Intended to be 80% common with all pipeline operators • Expandable through use of Abstract Classes • Developed by operator and vendor representatives. • The APDM model and supporting documentation is freely available and accessible to everyone
  • 19. UML - Unified Modeling Language "Secret Language which is used by International super models to talk to other equally super models." • Unfortunately, while this is false, UML is somewhat of a secret language and does take some patience to read and understand. • The APDM is distributed by the APDM committes in UML created in Visio. The UML enables the systematic creation, documentation and update of the APDM schema. • ESRI tools permit the creation of the APDM schema in a Geodatabase. • Reading the UML gives us an understanding of how our objects and features are constructed and related.
  • 20. UML Concepts • Abstract Classes o a specification for subclasses that is often shown on object model diagrams to help give structure to the diagram. An abstract class is not defined in a type library and cannot be instantiated. • Inheritance o the acquisition of methods and properties by child classes or interfaces from their previously existing parent classes or interfaces. Inheritance is one of the defining characteristics of an object-oriented system.
  • 21. UML Concepts Importance? Allow us to create database schema with logical, consistent features with appropriate attributes. And to update and apply them without re-doing everything time after time. Example: Elbow and Tee are "Concrete" Classes and Inherit from the Abstract Class "Fitting"
  • 23. Core Classes "When it absolutely, positively has to be there" in your APDM Model....... Essentially the bare minimum to make the model work. The most common classes and attributes which make the APDM work.
  • 24. APDM is a Data model Template • Implement the Core Classes • Add attributes to the core Classes if you want to or not. • Implement the Optional Classes if you want to or not. • Add attributes to the Optional Classes if you want to or not. • Add more classes, based on the Abstract Classes, if you want to or not. • Do all of these and make the model fit your business!
  • 25. How did the data get into the APDM? • Data from the Legacy Systems, was loaded into the Schema using custom tools. • Some Legacy data was "massaged" before it could be loaded. • Some Legacy data had to be cleaned before it was loaded. • Some data had to be created from scratch in order to be loaded. • StationSeries were created from Control Points created from previous Routes used by the integrity group. • RAM was the primary legacy data set used to create data for the APDM. • Features were loaded along the StationSeries based upon their Route Identifier ("Location") and their Station Value(s).
  • 26. Whats next? • Data Maintenance • Inspection Data Loading o Data Calibrator • Data Analysis o HCA / Class Location Calculator o Risk Calculator • Data Reporting o Alignment Sheet Generator o DOT Report Generator
  • 27. Need more Information? • www.FisherAssoc.com o DNichter@FisherAssoc.com • www.ESRI.com/industries/pipeline • www.APDM.net