3. CONNECT. TRANSFORM. AUTOMATE.
Snow Plows, ArcGIS Online, an
iPhone, and FME
Eric Abrams, Iowa Department of Transportation
901 Snow Plows
32 – 34 inches of snow
9400 miles of road
15 million gallons of brine
120,000 tons of salt on 45
inches of snowfall in 2013
ROI - Every dollar spent on AVL returns $6.40
A 10% reduction of salt is $1.4 million dollars in savings
5. CONNECT. TRANSFORM. AUTOMATE.
Automatic Vehicle Location
Invisible to driver
Real-time flow of data
– position, status,
material usage,
conditions
Data uploading to
Amazon cloud
FME moves data to
Oracle Spatial for
internal usage, then to
AGOL for public
viewing
6. CONNECT. TRANSFORM. AUTOMATE.
Dashcams
Dash-mounted
iPhones send image
stream when
vehicle is in motion
FME handles KML
generation and
upload to Windows
Azure
8. CONNECT. TRANSFORM. AUTOMATE.
Dashcam Feeds
Public can also see
what the driver is
seeing for better
awareness of road
and weather
conditions – making
winter driving safer.
11. CONNECT. TRANSFORM. AUTOMATE.
Sharing Public Data
Ide Rioja committed to sharing and
collaborating on public data.
Spatial Data Sharing taken to the next level
Creative Commons License
Enter GitHub
12. CONNECT. TRANSFORM. AUTOMATE.
Why GitHub
GitHub is a web-based Version Control System
(VCS) which records changes to a file or set of
files over time.
Allows:
commit files to a public repository
revert files back to a previous state
review changes made over time
see who last modified something, and more...
15. CONNECT. TRANSFORM. AUTOMATE.
How does FME Help?
Of course FME translates data from Oracle
Spatial to GeoJSON for GitHub
But first!
FME reads the layer list from GitHub using Python
Scripted Parameter – git pull
And after!
FME commits updated GeoJSON to GitHub in Shut
Down Script – git push
Scheduled Job on FME Server
17. CONNECT. TRANSFORM. AUTOMATE.
The Beauty of GeoJSON in GitHub
GitHub supports automatic rendering of
GeoJSON repositories using Leaflet.js
Looking ahead
geojson.io a Chrome extension for
editing
IDE Rioja plans open collaboration
on spatial data with GitHub
FME can include links to image data
when writing GeoJSON (automatic
download service)
18. CONNECT. TRANSFORM. AUTOMATE.
Learn More at FME User Conference
Extended Version of this topic will be
presented at the FME International User
Conference
20. CONNECT. TRANSFORM. AUTOMATE.
FME Server and the Gävle Data
Portal
Peter Jäderkvist, GIS
Developer & FME Certified
Professional, Community
Development Gävle
Provides services and
centralized workspace
organization, FME usage
tracking
Dynamic forms via client
communication with FME
Server REST API
An evolving UI: Peter recently added upload an irregular polygon to clip
21. CONNECT. TRANSFORM. AUTOMATE.
Various Maps to DWG
Most popular workspace
Map type (5), contours,
metadata, AutoCAD version
XML geometry + parameters
triggers FeatureReaders
SchemaMapper, clip &
output
Example output DWG basemap
22. CONNECT. TRANSFORM. AUTOMATE.
Specialty DWG Requests
Custom workspaces
generate specialty DWG
output for other users
Water & sewer mains for
local water company
Power distribution grid for
local provider
23. CONNECT. TRANSFORM. AUTOMATE.
3D Model to PDF, Sketchup or
DWG
Output: Sketchup 8, 3D-PDF and DWG
Add streets and water, yes/no
Drape roof tops with aerial photography, yes/no
Drape elevation model with aerial photography,
yes/no
Add roof models if they exist, yes/no
Some buildings don’t have heights, a parameter
decides how to treat those, e.g. “extrude by 7
meters”
24. CONNECT. TRANSFORM. AUTOMATE.
1. all parameters set to yes
except for “add roof models”.
2. all parameters set to no.
3. streets water and roof models
set to yes
Example output sketchup files
1 2
3
28. CONNECT. TRANSFORM. AUTOMATE.
Linear Referencing and Pipe
Video with FME
Amanda Graf and Raymond Kinser, FME
Certified Professionals, California CAD Solutions
Challenge: Map and share non-spatial inspection
video footage of all sewer lines for the City of Los
Altos.
Approach: Use FME and linear referencing
methods to QA and position video, creating an
automated, repeatable process.
29. CONNECT. TRANSFORM. AUTOMATE.
Data QA Issues
No spatial coordinates or geometry
Data inconsistencies across video data vendors
and databases
Differences between measured pipe lengths from
the vendors and the City
Inconsistent data entry of defect types
30. CONNECT. TRANSFORM. AUTOMATE.
Data Cleanup &
Homogenization
Filter for unwanted and bad
data
Time stamp formatting
Defect notation
standardization
Match to best known good
City records
QA for flow direction
Catch issues for manual
intervention
31. CONNECT. TRANSFORM. AUTOMATE.
Geometry Creation
Adjust video session data for
best pipe length
Adjust for directionality
(video with/against flow)
Create geometry using linear
measures, chopper, and
NeighborFinder
33. CONNECT. TRANSFORM. AUTOMATE.
Results
Easy access to data for all
Future processing of new
observation video
automated
City saves money on
future contracts
35. CONNECT. TRANSFORM. AUTOMATE.
Tableau Dataset Creation
Dami Sonoiki, FME Certified
Professional, Dotted Eyes
(Miso)
Problem: Tableau does great
data visualization, but lacks
good mapping capabilities
Solution: Use FME to break
down polygons
36. CONNECT. TRANSFORM. AUTOMATE.
Geometry Manipulation
Separate individual boundaries with Deaggregator
Generalize and reduce vertices
Deal with donuts
Produce OGC Well Known Text values for polys
37. CONNECT. TRANSFORM. AUTOMATE.
String Manipulation
Format WKT values to
extract coordinate strings
StringConcatenator
appends _part_number
supplied by Deaggregator
with Code_Count to
provide unique ID
PolygonPart
PolygonPart defines Detail
for Tableau reconstruction
40. CONNECT. TRANSFORM. AUTOMATE.
Address Point Frontage
Movement
Rajesh Dhull, FME Certified
Professional & Senior Data
Engineer, Data Development
Asia – Pacific, Pitney Bowes
Software
Problem: Addresses are
pinpointed by lot centroids,
but services are provided at
the street.
Solution: Create a value-
added, dynamic geocoding
dataset with addresses located
at the front of the property. It’s (almost) always best that your taxi arrives at
the front door rather than the living room.
41. CONNECT. TRANSFORM. AUTOMATE.
Requirements
Close to 14 million address points need to be moved to a
new position – property frontage.
The process should be robust, reliable and repeatable every
quarter.
The process should be able to handle heavy datasets.
The process should be able to fit in the existing processes
smoothly and should not lead to extra times or delays in
the product releases.
Source address points in Oracle, referential data
(boundaries, streets) in MapInfo Tab files
42. CONNECT. TRANSFORM. AUTOMATE.
The Approach
1. Create state-wise views in oracle as handling 14
million records in 1 process is not desirable.
2. Create single FME workspace for frontage
movement process for states with smaller
datasets.
3. Split this process in smaller manageable
processes for states with bigger datasets as FME
performance varies greatly based on the size of
the datasets.
43. CONNECT. TRANSFORM. AUTOMATE.
FME Workflow Overview
Filtered,
Buffered
Roads
Lot
Boundary
Polygons
Candidates for movement Pull address centroids from Oracle
Update Oracle
46. CONNECT. TRANSFORM. AUTOMATE.
Railway Platform Profiling
Rudolf Stastny, FME Certified Professional,
CSmap, s.r.o.
Challenge: Process hundreds of railway platform
profile DXF files derived from laser scans to look
for areas outside tolerances (preventing collisions)
Solution: Automate it with FME
52. CONNECT. TRANSFORM. AUTOMATE.
Telco Spatial Data Portal
Business Requirement: A Telecom customer
wanted a web portal for secure internal data
sharing/downloading.
Layer and coordinate system selections needed
Sources included imagery, vectors, and
proprietary data, updated daily by external
contractors
55. CONNECT. TRANSFORM. AUTOMATE.
FME Server Processing
Single workspace with Custom Transformers
Geodatabase Reader
Bounding Box
create
Clip
Write to choice
of format and
projection
57. CONNECT. TRANSFORM. AUTOMATE.
Laser Scanning Roads and FME
Gyula Sz. Fekete, Head of GIS Development and Data
Production, BKK Közút Zrt. (a company of the Municipality
of Budapest)
1. Management of Mobile Laser Scanning (MLS) missions
and post-processing
2. Data conversion from CAD-based data capture system
to Oracle ArcSDE GDB
3. Point cloud data analysis
58. CONNECT. TRANSFORM. AUTOMATE.
MLS Mission Management
Aim
visualize MLS (Mobile Laser Scanning) trajectories
locations of MLS projects
attribute information of each scanning project – project metadata
(scanner, driver, acquisition time, etc.)
positions of all exposed images
attribute information of each images comes from image header
information.
provide a GDB where post-processing steps can be visualized and
modified on a WebGIS GUI.
63. CONNECT. TRANSFORM. AUTOMATE.
Point Cloud Analysis
Find road surface
errors based on MLS
scanned 3D point
clouds
Generate vector data
to be used for further
spatial analysis
Read: 3D point clouds
and parameters
67. CONNECT. TRANSFORM. AUTOMATE.
Comprehensive Waterways
Data QA
Rob Vangeneugden, FME Certified Professional,
GIM nv
Challenge: Simplify and automate a complex data
validation process for waterways authorities
Solution: Create a Django user interface and use
FME Server to validate, manage results, and
perform database updates
69. CONNECT. TRANSFORM. AUTOMATE.
Custom Transformers
Uses both linked and embedded
Each data type has a specific custom transformer that identifies
Update/Insert/Delete by comparing geometry and attributes to operational
tables:
Terminal
Bridge
Bollard
Fairway
Berth
Node
Lock
Lock Chamber
70. CONNECT. TRANSFORM. AUTOMATE.
Using Parameters
Published –
Passed by Django application
Format-specific (3-GML, 9-shape) and user
credentials, verify authority
Private –
PostGIS connections
Schema parameters
Output location