SlideShare una empresa de Scribd logo
1 de 101
Descargar para leer sin conexión
smartpuffin.com | @ice_lenor | #geosmartpuffin.com | @ice_lenor | #geo
Geo Crash Course
Elena Nikolaeva
smartpuffin.com | @ice_lenor | #geo
Who am I?
● 10 years in Geo
● Started in 2GIS
● Built a distributed scalable
GIS-system called Fiji
○ Tailored to needs of
3000 cartographers
● Allowed the business to
scale world-wide
● 3 years in Geo team in
Booking
● Built GeoObject database
● Built process of data
collection between Tech
and Geo Content
departments
smartpuffin.com | @ice_lenor | #geo
Purpose:
make you familiar
with Geo
smartpuffin.com | @ice_lenor | #geo
What is an integer?
● A whole number
● 4 bytes (nowadays, mostly)
● Unsigned or signed
○ Signed range: [-2,147,483,648 .. 2,147,483,647]
○ Unsigned range: [0, 4,294,967,295]
● If you divide 2 integers, you'll get an integer
smartpuffin.com | @ice_lenor | #geo
Common wisdom
● One doesn't store money in double
● When working with strings, mind the encoding
● Theoretically, π is endless; but practically, first
~10 digits will do
● A very large or small double loses precision
● Array representation in memory depends
smartpuffin.com | @ice_lenor | #geo
Geo data: it's a
data type!
smartpuffin.com | @ice_lenor | #geosmartpuffin.com | @ice_lenor | #geo
Geometry types
smartpuffin.com | @ice_lenor | #geo
Point
Point(25 10)
smartpuffin.com | @ice_lenor | #geo
MultiPoint
1
2
point1
point2
MultiPoint( (25 10), (32 9) )
smartpuffin.com | @ice_lenor | #geo
LineString
1
2
LineString(5 10, 22 15)
● Consists of ordered points
smartpuffin.com | @ice_lenor | #geo
MultiLineString
1
2
3
4
line1 line2
MultiLineString( (10 10, 20 20), (30 15, 40 12) )
smartpuffin.com | @ice_lenor | #geo
Polygon
1
3
2
4
Polygon((0 0, 2 5, 4 0, 0 0))
● Consists of ordered points; certain direction (i.e., counterclockwise)
● Is closed: Point 1 === Point 4
Depends on
implementation
smartpuffin.com | @ice_lenor | #geo
Polygon with holes
6
7
8
9
1
2
3
4
5
Polygon((0 0, 8 0, 8 5, 0 5, 0 0), (3 2, 4 5, 6 2, 3 2))
● First set - "outer ring" (closed, counterclockwise)
● Other sets - "holes" (closed, clockwise)
smartpuffin.com | @ice_lenor | #geo
1
2
3
4
5
MultiPolygon
10
6
7 8
9
11
12 13
14
15
16
17 18
poly1
poly2
poly3
smartpuffin.com | @ice_lenor | #geo
GeometryCollection
GeometryCollection(
Polygon((...)),
LineString(...),
Point(...),
Polygon((...),(...))
)
smartpuffin.com | @ice_lenor | #geo
● Name: Amsterdam
● Type: city
● Population: 821,752
● Country: The Netherlands
GeoObject / Feature
Geometry
Attributes
+
smartpuffin.com | @ice_lenor | #geosmartpuffin.com | @ice_lenor | #geo
Coordinates
smartpuffin.com | @ice_lenor | #geo
Who thinks that
Earth is flat?
smartpuffin.com | @ice_lenor | #geo
smartpuffin.com | @ice_lenor | #geo
SHOCK!
smartpuffin.com | @ice_lenor | #geo
(x, y)
Point (0 10)
LineString (0 0, 10 0)
Flat coordinates
smartpuffin.com | @ice_lenor | #geo
(lat, long)
Point (52.373160 4.891999)
52.3680° N, 4.9036° E
Geography coordinates
smartpuffin.com | @ice_lenor | #geo
Flat coordinates
● Computer games
● Building plans
● Large scale maps of small areas
○ One city, one district
smartpuffin.com | @ice_lenor | #geo
Flat coordinates
(0, 0) 5 10 15 20 25
5
10
15
Ficus is at
(23, 8)
x
y
smartpuffin.com | @ice_lenor | #geo
Geographical coordinates
● Map of the whole planet
● Map of a large area of the planet
○ "Small scale"
smartpuffin.com | @ice_lenor | #geo
Small scale: when everything is small
smartpuffin.com | @ice_lenor | #geo
Large scale: when everything is LARGE
smartpuffin.com | @ice_lenor | #geo
Latitude
0°
90° North
-90° South
North Pole
South Pole
30th parallel North
30° N
30°
40th parallel South
40° S
-40°
Equator
0° N
smartpuffin.com | @ice_lenor | #geo
Longitude
0°
-180°
West
180°
East
40th meridian east
40° E
40°20th meridian west
40° W
-40°
Prime meridian
0°
smartpuffin.com | @ice_lenor | #geosmartpuffin.com | @ice_lenor | #geo
Datums and
Coordinate Systems
How round is Earth, exactly?
smartpuffin.com | @ice_lenor | #geo
Geoid
Earth's shape is a
smartpuffin.com | @ice_lenor | #geo
Spheroid
But we can simplify it to a
smartpuffin.com | @ice_lenor | #geo
WGS84
The latest revision is WGS 84 (also known as WGS
1984, EPSG:4326). The WGS 84 datum surface is
an oblate spheroid with equatorial radius a =
6378137 m at the equator and flattening f =
1/298.257223563. The polar semi-minor axis b
then equals a × (1 − f) = 6356752.3142 m. Currently,
WGS 84 uses the Earth Gravitational Model 1996
(EGM96) geoid, revised in 2004. This geoid defines
the nominal sea level surface by means of a
spherical harmonics series of degree 360 (which
provides about 100 km latitudinal resolution near
the Equator). The deviations of the EGM96 geoid
from the WGS 84 reference ellipsoid range from
about −105 m to about +85 m.[8] EGM96 differs
from the original WGS 84 geoid, referred to as
EGM84.
smartpuffin.com | @ice_lenor | #geo
WGS84
radius
R = 6378 km
flattening
F = 1 / 298.25
● Earth is an ellipsoid
○ flattened from the poles
○ has a certain radius
● (0,0) is specified
● Latitude is [90..-90]
● Longitude is [180..-180]
(0,0) is here
smartpuffin.com | @ice_lenor | #geo
Datum describes Earth's
general shape
(Sphere? Ellipsoid? Spheroid?)
smartpuffin.com | @ice_lenor | #geo
Coordinate system describes
where coordinates start
where they end
and what’s the unit
smartpuffin.com | @ice_lenor | #geo
You must specify
the Coordinate System
to make your math correct!
smartpuffin.com | @ice_lenor | #geo
Formula for distance: flat
Point(0 0) to Point(1 1) = 1.414 (units)
● We all learned it in school
● Derived from the Pythagorean
theorem
● Doesn’t know about Earth
● Just plain old distance.
smartpuffin.com | @ice_lenor | #geo
Formula for distance: haversine
● Uses radius
● Doesn’t use flattening
● => Calculates on sphere!
Point(0 0) to Point(1 1) = 157.2 km
smartpuffin.com | @ice_lenor | #geo
Formula for distance: Vincenty
● Uses radius
● And flattening
● => Calculates on WGS84!
○ Or any ellipsoid provided
● => Millimetre precision!
Point(0 0) to Point(1 1) = 156.899568 km
smartpuffin.com | @ice_lenor | #geo
SRID: Spatial Reference Id
● EPSG - European Petroleum Survey
Group
○ A registry of datums and projections
● WGS84's SRID is EPSG:4326
● There are other SRIDs
● You must specify SRID with your
geometry for correct math
○ I.e., when storing geodata in a database
○ Or using a library or a formula
smartpuffin.com | @ice_lenor | #geo
Why is a SRID like a toothbrush?
Because everyone has got one.
● British National Grid
○ Ellipsoid: Airy 1830
● Amersfoort / RD New -- Netherlands
○ Ellipsoid: Bessel 1841
● Geoscience Australia Lambert
○ Ellipsoid: GRS 1980
smartpuffin.com | @ice_lenor | #geo
Who still thinks
that
Earth is flat?
smartpuffin.com | @ice_lenor | #geosmartpuffin.com | @ice_lenor | #geo
Projections
smartpuffin.com | @ice_lenor | #geo
"How do we draw
a 3D-Earth
on paper?"
- Gerardus Mercator
smartpuffin.com | @ice_lenor | #geo
Projection
● A means to display a N-dimensional object
in a (N-1)-dimensional space
● For maps:
○ How do we display 3D Earth on a 2D paper?
○ Or a piece of parchment?
○ Or a tapestry?
○ Or a computer screen?
smartpuffin.com | @ice_lenor | #geo
We take a datum
and flatten it!
smartpuffin.com | @ice_lenor | #geo
smartpuffin.com | @ice_lenor | #geo
Projections
● A sphere can't be flattened without problems
○ You have to tug and stretch and shrink
● No projections are "100% accurate depictions"
● One has to choose what traits to preserve
○ direction
○ shape
○ area
○ distance
smartpuffin.com | @ice_lenor | #geo
Specific way of flattening,
shrinking, stretching, fitting
is called
a projection
smartpuffin.com | @ice_lenor | #geo
Mercator Projection
● Preserves angles and shapes of
small objects
● => Good for navigation
● Loses sizes (Greenland larger than
Africa)
● Mercator on WGS84 is popular
(EPSG:3395)
● What does Google use? Read on.
smartpuffin.com | @ice_lenor | #geo
Orthographic
● "Like I'm looking at it from
space!"
● Directions from the central
point are preserved
● The closer to the edge, the
more objects are distorted
smartpuffin.com | @ice_lenor | #geo
Plate Carrée
● longitude => y
● latitude => x
● Preserves angles
● Loses everything
else :)
smartpuffin.com | @ice_lenor | #geo
Equal-area (i.e., Mollweide, Lambert, etc)
● Preserves
relative areas
● Distorts angles
and shapes
smartpuffin.com | @ice_lenor | #geo
Winkel Tripel: A Compromise?
● Tries to preserve all at once
● Minimises distortion of area,
direction, and distance
● Cool name
● Used by
smartpuffin.com | @ice_lenor | #geo
Universal Transverse Mercator (UTM)
● Family of 60 projections ("zones")
○ Slice Earth like an orange
○ Each slice has own coordinates
● => Flat coordinates! In meters!
● Coordinate system AND projection,
in one
● Can't make map of whole planet,
only one slice
smartpuffin.com | @ice_lenor | #geo
There is no single
"best projection"
smartpuffin.com | @ice_lenor | #geo
Each projection
is good
for a specific purpose
smartpuffin.com | @ice_lenor | #geo
Web Mercator: A Cursed Projection :)
● "Invented" by Google
○ They mix sphere and ellipsoid, boo!
○ "The underlying geographic coordinates
are defined using WGS84 [ellipsoid]
(as in 3857), but projected as if they
were defined on a sphere (as in 3785)".
● Wasn't accepted for a long time
● Old unofficial code: EPSG:900913, official: EPSG:3857
● Now - alas! - is used by many online maps
smartpuffin.com | @ice_lenor | #geosmartpuffin.com | @ice_lenor | #geo
OGC Standard
smartpuffin.com | @ice_lenor | #geo
OGC Standard
● Open Geospatial Consortium: 500+ companies
● De-facto defines the geo-industry
● A set of documents about how to:
○ Define geodata
○ Store geodata (in databases)
○ Index it
○ Manipulate it (in databases and in libraries)
● => Start reading with Simple Features
smartpuffin.com | @ice_lenor | #geosmartpuffin.com | @ice_lenor | #geo
What can I do with my geodata?
Geo Math:
Spatial Operations
smartpuffin.com | @ice_lenor | #geo
Distance
Point(0 0)
Point(4, 2)
Distance = 222.4 km
(calculated on WGS84)
Distance = 4.472
(calculated on flat coords)
smartpuffin.com | @ice_lenor | #geo
Inside
smartpuffin.com | @ice_lenor | #geo
Touches
smartpuffin.com | @ice_lenor | #geo
Overlaps
smartpuffin.com | @ice_lenor | #geo
Topological operations: Union
∪
poly1
poly2
poly1
smartpuffin.com | @ice_lenor | #geo
Topological operations: Intersection
∩
poly1
poly2
smartpuffin.com | @ice_lenor | #geo
Topological operations: Difference
poly1
poly2
smartpuffin.com | @ice_lenor | #geo
Bounding box
Geometry
Bounding box:
smallest enclosing
rectangle
smartpuffin.com | @ice_lenor | #geo
Buffer(5)
Buffer
5
5
smartpuffin.com | @ice_lenor | #geo
● Coordinates must be within SRID's limits
○ I.e., 95° North latitude is a no-no
● Line:
○ Must have min 2 points
○ Length must be > 0
● Polygon:
○ Must have min 4 points
○ Area must be > 0
○ First and last points must be the same (closed ring)
○ Can't intersect itself
○ Holes must be inside
IsValid
smartpuffin.com | @ice_lenor | #geo
1
5
2
IsValid, MakeValid: Self-Intersection
3
4
● 8-shape polygon
● "Butterfly" geometry
● Is NOT valid!
● Must be stored as
MultiPolygon of 2 polygons
○ MakeValid(geom) may fix
problems
smartpuffin.com | @ice_lenor | #geosmartpuffin.com | @ice_lenor | #geo
Storing spatial data
smartpuffin.com | @ice_lenor | #geo
Well-known geometry representation
● Defined in OGC standard
● Well-known text
○ "Point(0 0)"
○ "LineString(0 0, 1 0)"
○ "Polygon((0 0, 5 0, 5 5, 0 5, 0 0), (1 1, 1 4, 4 4, 4 1, 1 1))"
● Well-known binary
○ 0x0101000000000000000000f03f0000000000000040
A hole in the
polygon
smartpuffin.com | @ice_lenor | #geo
Spatial databases
● OGC-compliant
● Store and index spatial data
○ Fast and correct querying
○ Geometry validation
● Support spatial operations: Union, Buffer, etc
● Support different SRIDs
○ => Support flat coordinates and geography
○ Support all geo math correctly
smartpuffin.com | @ice_lenor | #geo
Spatial databases
● SQL Server
● Postgres: PostGIS extension
● Oracle Spatial and Graph
● MySQL 8.0+ (perhaps, limited support)
● Column type: Geometry / Geography
smartpuffin.com | @ice_lenor | #geo
Spatial Elasticsearch?
● Supports only geographical coordinates on
WGS84
● Index: GeoHash
● Imprecise operations (only uses index, doesn't
do precise calculations)
● Slow to reindex
● Still, can be useful
smartpuffin.com | @ice_lenor | #geo
Spatial indexes
● Quickly answer questions:
○ "What is close to this place"
○ "What is inside this area"
● Usually rough answer, needs precise
calculation afterwards
● Usually already implemented in database or in
library
smartpuffin.com | @ice_lenor | #geo
Spatial index: Quadtree
● Divides space into 4
squares recursively
● Adding new objects is
easy and cheap
● Good for constantly
changing data
smartpuffin.com | @ice_lenor | #geo
Spatial index: R-tree
● Divides space into
rectangles
● Efficient for reading
● Adding new objects
(reindexing) is
expensive
smartpuffin.com | @ice_lenor | #geo
Spatial index: Z-order Curve
● Efficient for reading
● And for writing
● Harder to comprehend
○ And implement
● Though looks cool :)
smartpuffin.com | @ice_lenor | #geo
File formats
● Download your geodata into a file (like .csv)
● Then upload it somewhere else
● No indexing, no operations, simple plain file
● Standardised file formats:
○ .kml
○ .kmz - zipped kml
○ .geojson
○ shapefile
smartpuffin.com | @ice_lenor | #geo
Libraries
● OGC defines the API
● Can load a geometry from spatial db
● Can operate on it in memory
● Supports all topological operations
○ Java: Java Topology Suite
○ C#: NetTopologySuite
○ Ports to many different languages
smartpuffin.com | @ice_lenor | #geo
GIS-systems
Geographic information
systems: apps for creating
and editing geodata.
● ArcGIS from ESRI
● OpenStreetMap editor
● QGIS
● ... many of them
smartpuffin.com | @ice_lenor | #geo
GIS-systems
● Users: geo-specialists (cartographers)
● Create layers (types of objects: "Buildings", "Rivers",
"Parks")
● Create geoobjects
○ Draw geometry
○ Add attributes (names, tags, etc)
○ Do spatial operations, data validations ("buildings
should not intersect each other")
smartpuffin.com | @ice_lenor | #geosmartpuffin.com | @ice_lenor | #geo
Common Problems in Geo
smartpuffin.com | @ice_lenor | #geo
new Point (lat, lon)
OR
new Point (lon, lat)
?
● Library-dependent
● Always check!
smartpuffin.com | @ice_lenor | #geo
Units: degrees
● Units of latitude and longitude: degrees
● You CAN'T measure distance in them!
smartpuffin.com | @ice_lenor | #geo
Units: degrees
● Two points on the equator:
● Point 1: latitude 0, longitude 0.
● Point 2: latitude 0, longitude 1.
● Distance between them, measured as if they
were flat coordinates: 1. (Units: degrees.)
● Distance between them, measured as if they're
on the planet Earth: 111.19 km.
smartpuffin.com | @ice_lenor | #geo
Units: degrees
● Two points on Amsterdam's latitude (52° N):
● Point 1: latitude 52, longitude 0.
● Point 2: latitude 52, longitude 1.
● Distance between them, measured as if they
were flat coordinates: still 1. (Units: degrees.)
● Distance between them, measured as if they're
on the planet Earth: 68.46 km.
smartpuffin.com | @ice_lenor | #geo
The closer to the pole,
the less km's fit
in one degree longitude
smartpuffin.com | @ice_lenor | #geo
Degrees latitude
are always the same
in km's
smartpuffin.com | @ice_lenor | #geo
180th Meridian
smartpuffin.com | @ice_lenor | #geo
180th Meridian: Always Check With Fiji!
-16.760, -179.658
-16.712, 179.713
180th Meridian
smartpuffin.com | @ice_lenor | #geo
180th Meridian
smartpuffin.com | @ice_lenor | #geo
180th Meridian
● Fiji
● New Zealand
● Russia
● Kiribati
● United States
smartpuffin.com | @ice_lenor | #geo
Poles
● One latitude: 90° or -90°
● All longitudes meet there
=> It's all exactly the same point!
○ Point (lat=90, lon=0)
○ Point (lat=90, lon=100)
○ Point (lat=90, lon=-140)
smartpuffin.com | @ice_lenor | #geo
Poles
● Some projections don't support poles
● I.e., Mercator on WGS84:
○ "World between 80°S and 84°N."
○ Otherwise, poles would be stretched into infinity
● => If you need poles, check if your projection
and datum support them
smartpuffin.com | @ice_lenor | #geo
Spatial operations around problem areas
● Math close to 180th meridian must work
correctly!
● Same goes for poles!
smartpuffin.com | @ice_lenor | #geosmartpuffin.com | @ice_lenor | #geo
Thank you
Your questions, please!

Más contenido relacionado

Similar a Geo crash course

Similar a Geo crash course (20)

Gis for healthcare introduction
Gis for healthcare   introductionGis for healthcare   introduction
Gis for healthcare introduction
 
Coordinate systems
Coordinate systemsCoordinate systems
Coordinate systems
 
Sistem dan Transformasi Koordinat
Sistem dan Transformasi KoordinatSistem dan Transformasi Koordinat
Sistem dan Transformasi Koordinat
 
Google map and navigation
Google map and navigation Google map and navigation
Google map and navigation
 
Geek Sync | Having Fun with Spatial Data
Geek Sync | Having Fun with Spatial DataGeek Sync | Having Fun with Spatial Data
Geek Sync | Having Fun with Spatial Data
 
geog2750_15.ppt principles of grid based modelling
geog2750_15.ppt principles of grid based modellinggeog2750_15.ppt principles of grid based modelling
geog2750_15.ppt principles of grid based modelling
 
HPA*
HPA*HPA*
HPA*
 
CARTOGRAPHY – yesterday, today and tomorrow
CARTOGRAPHY – yesterday, today and tomorrowCARTOGRAPHY – yesterday, today and tomorrow
CARTOGRAPHY – yesterday, today and tomorrow
 
Introduction to GIS
Introduction to GISIntroduction to GIS
Introduction to GIS
 
Introduction to GIS
Introduction to GISIntroduction to GIS
Introduction to GIS
 
Map Skills
Map SkillsMap Skills
Map Skills
 
Get mapping with leaflet js
Get mapping with leaflet jsGet mapping with leaflet js
Get mapping with leaflet js
 
Managing GeoData with PostGIS @ KhmelnytskyiPy #1
Managing GeoData with PostGIS @ KhmelnytskyiPy #1Managing GeoData with PostGIS @ KhmelnytskyiPy #1
Managing GeoData with PostGIS @ KhmelnytskyiPy #1
 
projection.pptx
projection.pptxprojection.pptx
projection.pptx
 
Gis Concepts 3/5
Gis Concepts 3/5Gis Concepts 3/5
Gis Concepts 3/5
 
Lighting
LightingLighting
Lighting
 
coordinate systems map projections and graphical and atoms ppt group (B).pptx
coordinate systems map projections and graphical and atoms ppt group (B).pptxcoordinate systems map projections and graphical and atoms ppt group (B).pptx
coordinate systems map projections and graphical and atoms ppt group (B).pptx
 
Global positioning system (gps)
Global positioning system (gps)Global positioning system (gps)
Global positioning system (gps)
 
The Earth is not flat; but it's not round either (Geography on Boost.Geometry)
The Earth is not flat; but it's not round either (Geography on Boost.Geometry)The Earth is not flat; but it's not round either (Geography on Boost.Geometry)
The Earth is not flat; but it's not round either (Geography on Boost.Geometry)
 
Shading in OpenGL
Shading in OpenGLShading in OpenGL
Shading in OpenGL
 

Último

Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
 
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
panagenda
 
Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native Applications
WSO2
 

Último (20)

Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, AdobeApidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
 
DBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor PresentationDBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor Presentation
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt Robison
 
Real Time Object Detection Using Open CV
Real Time Object Detection Using Open CVReal Time Object Detection Using Open CV
Real Time Object Detection Using Open CV
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Script
 
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a Fresher
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
 
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
 
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodPolkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
 
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
 
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
 
FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024
 
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processors
 
Apidays Singapore 2024 - Scalable LLM APIs for AI and Generative AI Applicati...
Apidays Singapore 2024 - Scalable LLM APIs for AI and Generative AI Applicati...Apidays Singapore 2024 - Scalable LLM APIs for AI and Generative AI Applicati...
Apidays Singapore 2024 - Scalable LLM APIs for AI and Generative AI Applicati...
 
Manulife - Insurer Transformation Award 2024
Manulife - Insurer Transformation Award 2024Manulife - Insurer Transformation Award 2024
Manulife - Insurer Transformation Award 2024
 
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?
 
Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native Applications
 
Artificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : UncertaintyArtificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : Uncertainty
 
Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)
 

Geo crash course

  • 1. smartpuffin.com | @ice_lenor | #geosmartpuffin.com | @ice_lenor | #geo Geo Crash Course Elena Nikolaeva
  • 2. smartpuffin.com | @ice_lenor | #geo Who am I? ● 10 years in Geo ● Started in 2GIS ● Built a distributed scalable GIS-system called Fiji ○ Tailored to needs of 3000 cartographers ● Allowed the business to scale world-wide ● 3 years in Geo team in Booking ● Built GeoObject database ● Built process of data collection between Tech and Geo Content departments
  • 3. smartpuffin.com | @ice_lenor | #geo Purpose: make you familiar with Geo
  • 4. smartpuffin.com | @ice_lenor | #geo What is an integer? ● A whole number ● 4 bytes (nowadays, mostly) ● Unsigned or signed ○ Signed range: [-2,147,483,648 .. 2,147,483,647] ○ Unsigned range: [0, 4,294,967,295] ● If you divide 2 integers, you'll get an integer
  • 5. smartpuffin.com | @ice_lenor | #geo Common wisdom ● One doesn't store money in double ● When working with strings, mind the encoding ● Theoretically, π is endless; but practically, first ~10 digits will do ● A very large or small double loses precision ● Array representation in memory depends
  • 6. smartpuffin.com | @ice_lenor | #geo Geo data: it's a data type!
  • 7. smartpuffin.com | @ice_lenor | #geosmartpuffin.com | @ice_lenor | #geo Geometry types
  • 8. smartpuffin.com | @ice_lenor | #geo Point Point(25 10)
  • 9. smartpuffin.com | @ice_lenor | #geo MultiPoint 1 2 point1 point2 MultiPoint( (25 10), (32 9) )
  • 10. smartpuffin.com | @ice_lenor | #geo LineString 1 2 LineString(5 10, 22 15) ● Consists of ordered points
  • 11. smartpuffin.com | @ice_lenor | #geo MultiLineString 1 2 3 4 line1 line2 MultiLineString( (10 10, 20 20), (30 15, 40 12) )
  • 12. smartpuffin.com | @ice_lenor | #geo Polygon 1 3 2 4 Polygon((0 0, 2 5, 4 0, 0 0)) ● Consists of ordered points; certain direction (i.e., counterclockwise) ● Is closed: Point 1 === Point 4 Depends on implementation
  • 13. smartpuffin.com | @ice_lenor | #geo Polygon with holes 6 7 8 9 1 2 3 4 5 Polygon((0 0, 8 0, 8 5, 0 5, 0 0), (3 2, 4 5, 6 2, 3 2)) ● First set - "outer ring" (closed, counterclockwise) ● Other sets - "holes" (closed, clockwise)
  • 14. smartpuffin.com | @ice_lenor | #geo 1 2 3 4 5 MultiPolygon 10 6 7 8 9 11 12 13 14 15 16 17 18 poly1 poly2 poly3
  • 15. smartpuffin.com | @ice_lenor | #geo GeometryCollection GeometryCollection( Polygon((...)), LineString(...), Point(...), Polygon((...),(...)) )
  • 16. smartpuffin.com | @ice_lenor | #geo ● Name: Amsterdam ● Type: city ● Population: 821,752 ● Country: The Netherlands GeoObject / Feature Geometry Attributes +
  • 17. smartpuffin.com | @ice_lenor | #geosmartpuffin.com | @ice_lenor | #geo Coordinates
  • 18. smartpuffin.com | @ice_lenor | #geo Who thinks that Earth is flat?
  • 21. smartpuffin.com | @ice_lenor | #geo (x, y) Point (0 10) LineString (0 0, 10 0) Flat coordinates
  • 22. smartpuffin.com | @ice_lenor | #geo (lat, long) Point (52.373160 4.891999) 52.3680° N, 4.9036° E Geography coordinates
  • 23. smartpuffin.com | @ice_lenor | #geo Flat coordinates ● Computer games ● Building plans ● Large scale maps of small areas ○ One city, one district
  • 24. smartpuffin.com | @ice_lenor | #geo Flat coordinates (0, 0) 5 10 15 20 25 5 10 15 Ficus is at (23, 8) x y
  • 25. smartpuffin.com | @ice_lenor | #geo Geographical coordinates ● Map of the whole planet ● Map of a large area of the planet ○ "Small scale"
  • 26. smartpuffin.com | @ice_lenor | #geo Small scale: when everything is small
  • 27. smartpuffin.com | @ice_lenor | #geo Large scale: when everything is LARGE
  • 28. smartpuffin.com | @ice_lenor | #geo Latitude 0° 90° North -90° South North Pole South Pole 30th parallel North 30° N 30° 40th parallel South 40° S -40° Equator 0° N
  • 29. smartpuffin.com | @ice_lenor | #geo Longitude 0° -180° West 180° East 40th meridian east 40° E 40°20th meridian west 40° W -40° Prime meridian 0°
  • 30. smartpuffin.com | @ice_lenor | #geosmartpuffin.com | @ice_lenor | #geo Datums and Coordinate Systems How round is Earth, exactly?
  • 31. smartpuffin.com | @ice_lenor | #geo Geoid Earth's shape is a
  • 32. smartpuffin.com | @ice_lenor | #geo Spheroid But we can simplify it to a
  • 33. smartpuffin.com | @ice_lenor | #geo WGS84 The latest revision is WGS 84 (also known as WGS 1984, EPSG:4326). The WGS 84 datum surface is an oblate spheroid with equatorial radius a = 6378137 m at the equator and flattening f = 1/298.257223563. The polar semi-minor axis b then equals a × (1 − f) = 6356752.3142 m. Currently, WGS 84 uses the Earth Gravitational Model 1996 (EGM96) geoid, revised in 2004. This geoid defines the nominal sea level surface by means of a spherical harmonics series of degree 360 (which provides about 100 km latitudinal resolution near the Equator). The deviations of the EGM96 geoid from the WGS 84 reference ellipsoid range from about −105 m to about +85 m.[8] EGM96 differs from the original WGS 84 geoid, referred to as EGM84.
  • 34. smartpuffin.com | @ice_lenor | #geo WGS84 radius R = 6378 km flattening F = 1 / 298.25 ● Earth is an ellipsoid ○ flattened from the poles ○ has a certain radius ● (0,0) is specified ● Latitude is [90..-90] ● Longitude is [180..-180] (0,0) is here
  • 35. smartpuffin.com | @ice_lenor | #geo Datum describes Earth's general shape (Sphere? Ellipsoid? Spheroid?)
  • 36. smartpuffin.com | @ice_lenor | #geo Coordinate system describes where coordinates start where they end and what’s the unit
  • 37. smartpuffin.com | @ice_lenor | #geo You must specify the Coordinate System to make your math correct!
  • 38. smartpuffin.com | @ice_lenor | #geo Formula for distance: flat Point(0 0) to Point(1 1) = 1.414 (units) ● We all learned it in school ● Derived from the Pythagorean theorem ● Doesn’t know about Earth ● Just plain old distance.
  • 39. smartpuffin.com | @ice_lenor | #geo Formula for distance: haversine ● Uses radius ● Doesn’t use flattening ● => Calculates on sphere! Point(0 0) to Point(1 1) = 157.2 km
  • 40. smartpuffin.com | @ice_lenor | #geo Formula for distance: Vincenty ● Uses radius ● And flattening ● => Calculates on WGS84! ○ Or any ellipsoid provided ● => Millimetre precision! Point(0 0) to Point(1 1) = 156.899568 km
  • 41. smartpuffin.com | @ice_lenor | #geo SRID: Spatial Reference Id ● EPSG - European Petroleum Survey Group ○ A registry of datums and projections ● WGS84's SRID is EPSG:4326 ● There are other SRIDs ● You must specify SRID with your geometry for correct math ○ I.e., when storing geodata in a database ○ Or using a library or a formula
  • 42. smartpuffin.com | @ice_lenor | #geo Why is a SRID like a toothbrush? Because everyone has got one. ● British National Grid ○ Ellipsoid: Airy 1830 ● Amersfoort / RD New -- Netherlands ○ Ellipsoid: Bessel 1841 ● Geoscience Australia Lambert ○ Ellipsoid: GRS 1980
  • 43. smartpuffin.com | @ice_lenor | #geo Who still thinks that Earth is flat?
  • 44. smartpuffin.com | @ice_lenor | #geosmartpuffin.com | @ice_lenor | #geo Projections
  • 45. smartpuffin.com | @ice_lenor | #geo "How do we draw a 3D-Earth on paper?" - Gerardus Mercator
  • 46. smartpuffin.com | @ice_lenor | #geo Projection ● A means to display a N-dimensional object in a (N-1)-dimensional space ● For maps: ○ How do we display 3D Earth on a 2D paper? ○ Or a piece of parchment? ○ Or a tapestry? ○ Or a computer screen?
  • 47. smartpuffin.com | @ice_lenor | #geo We take a datum and flatten it!
  • 49. smartpuffin.com | @ice_lenor | #geo Projections ● A sphere can't be flattened without problems ○ You have to tug and stretch and shrink ● No projections are "100% accurate depictions" ● One has to choose what traits to preserve ○ direction ○ shape ○ area ○ distance
  • 50. smartpuffin.com | @ice_lenor | #geo Specific way of flattening, shrinking, stretching, fitting is called a projection
  • 51. smartpuffin.com | @ice_lenor | #geo Mercator Projection ● Preserves angles and shapes of small objects ● => Good for navigation ● Loses sizes (Greenland larger than Africa) ● Mercator on WGS84 is popular (EPSG:3395) ● What does Google use? Read on.
  • 52. smartpuffin.com | @ice_lenor | #geo Orthographic ● "Like I'm looking at it from space!" ● Directions from the central point are preserved ● The closer to the edge, the more objects are distorted
  • 53. smartpuffin.com | @ice_lenor | #geo Plate Carrée ● longitude => y ● latitude => x ● Preserves angles ● Loses everything else :)
  • 54. smartpuffin.com | @ice_lenor | #geo Equal-area (i.e., Mollweide, Lambert, etc) ● Preserves relative areas ● Distorts angles and shapes
  • 55. smartpuffin.com | @ice_lenor | #geo Winkel Tripel: A Compromise? ● Tries to preserve all at once ● Minimises distortion of area, direction, and distance ● Cool name ● Used by
  • 56. smartpuffin.com | @ice_lenor | #geo Universal Transverse Mercator (UTM) ● Family of 60 projections ("zones") ○ Slice Earth like an orange ○ Each slice has own coordinates ● => Flat coordinates! In meters! ● Coordinate system AND projection, in one ● Can't make map of whole planet, only one slice
  • 57. smartpuffin.com | @ice_lenor | #geo There is no single "best projection"
  • 58. smartpuffin.com | @ice_lenor | #geo Each projection is good for a specific purpose
  • 59. smartpuffin.com | @ice_lenor | #geo Web Mercator: A Cursed Projection :) ● "Invented" by Google ○ They mix sphere and ellipsoid, boo! ○ "The underlying geographic coordinates are defined using WGS84 [ellipsoid] (as in 3857), but projected as if they were defined on a sphere (as in 3785)". ● Wasn't accepted for a long time ● Old unofficial code: EPSG:900913, official: EPSG:3857 ● Now - alas! - is used by many online maps
  • 60. smartpuffin.com | @ice_lenor | #geosmartpuffin.com | @ice_lenor | #geo OGC Standard
  • 61. smartpuffin.com | @ice_lenor | #geo OGC Standard ● Open Geospatial Consortium: 500+ companies ● De-facto defines the geo-industry ● A set of documents about how to: ○ Define geodata ○ Store geodata (in databases) ○ Index it ○ Manipulate it (in databases and in libraries) ● => Start reading with Simple Features
  • 62. smartpuffin.com | @ice_lenor | #geosmartpuffin.com | @ice_lenor | #geo What can I do with my geodata? Geo Math: Spatial Operations
  • 63. smartpuffin.com | @ice_lenor | #geo Distance Point(0 0) Point(4, 2) Distance = 222.4 km (calculated on WGS84) Distance = 4.472 (calculated on flat coords)
  • 66. smartpuffin.com | @ice_lenor | #geo Overlaps
  • 67. smartpuffin.com | @ice_lenor | #geo Topological operations: Union ∪ poly1 poly2 poly1
  • 68. smartpuffin.com | @ice_lenor | #geo Topological operations: Intersection ∩ poly1 poly2
  • 69. smartpuffin.com | @ice_lenor | #geo Topological operations: Difference poly1 poly2
  • 70. smartpuffin.com | @ice_lenor | #geo Bounding box Geometry Bounding box: smallest enclosing rectangle
  • 71. smartpuffin.com | @ice_lenor | #geo Buffer(5) Buffer 5 5
  • 72. smartpuffin.com | @ice_lenor | #geo ● Coordinates must be within SRID's limits ○ I.e., 95° North latitude is a no-no ● Line: ○ Must have min 2 points ○ Length must be > 0 ● Polygon: ○ Must have min 4 points ○ Area must be > 0 ○ First and last points must be the same (closed ring) ○ Can't intersect itself ○ Holes must be inside IsValid
  • 73. smartpuffin.com | @ice_lenor | #geo 1 5 2 IsValid, MakeValid: Self-Intersection 3 4 ● 8-shape polygon ● "Butterfly" geometry ● Is NOT valid! ● Must be stored as MultiPolygon of 2 polygons ○ MakeValid(geom) may fix problems
  • 74. smartpuffin.com | @ice_lenor | #geosmartpuffin.com | @ice_lenor | #geo Storing spatial data
  • 75. smartpuffin.com | @ice_lenor | #geo Well-known geometry representation ● Defined in OGC standard ● Well-known text ○ "Point(0 0)" ○ "LineString(0 0, 1 0)" ○ "Polygon((0 0, 5 0, 5 5, 0 5, 0 0), (1 1, 1 4, 4 4, 4 1, 1 1))" ● Well-known binary ○ 0x0101000000000000000000f03f0000000000000040 A hole in the polygon
  • 76. smartpuffin.com | @ice_lenor | #geo Spatial databases ● OGC-compliant ● Store and index spatial data ○ Fast and correct querying ○ Geometry validation ● Support spatial operations: Union, Buffer, etc ● Support different SRIDs ○ => Support flat coordinates and geography ○ Support all geo math correctly
  • 77. smartpuffin.com | @ice_lenor | #geo Spatial databases ● SQL Server ● Postgres: PostGIS extension ● Oracle Spatial and Graph ● MySQL 8.0+ (perhaps, limited support) ● Column type: Geometry / Geography
  • 78. smartpuffin.com | @ice_lenor | #geo Spatial Elasticsearch? ● Supports only geographical coordinates on WGS84 ● Index: GeoHash ● Imprecise operations (only uses index, doesn't do precise calculations) ● Slow to reindex ● Still, can be useful
  • 79. smartpuffin.com | @ice_lenor | #geo Spatial indexes ● Quickly answer questions: ○ "What is close to this place" ○ "What is inside this area" ● Usually rough answer, needs precise calculation afterwards ● Usually already implemented in database or in library
  • 80. smartpuffin.com | @ice_lenor | #geo Spatial index: Quadtree ● Divides space into 4 squares recursively ● Adding new objects is easy and cheap ● Good for constantly changing data
  • 81. smartpuffin.com | @ice_lenor | #geo Spatial index: R-tree ● Divides space into rectangles ● Efficient for reading ● Adding new objects (reindexing) is expensive
  • 82. smartpuffin.com | @ice_lenor | #geo Spatial index: Z-order Curve ● Efficient for reading ● And for writing ● Harder to comprehend ○ And implement ● Though looks cool :)
  • 83. smartpuffin.com | @ice_lenor | #geo File formats ● Download your geodata into a file (like .csv) ● Then upload it somewhere else ● No indexing, no operations, simple plain file ● Standardised file formats: ○ .kml ○ .kmz - zipped kml ○ .geojson ○ shapefile
  • 84. smartpuffin.com | @ice_lenor | #geo Libraries ● OGC defines the API ● Can load a geometry from spatial db ● Can operate on it in memory ● Supports all topological operations ○ Java: Java Topology Suite ○ C#: NetTopologySuite ○ Ports to many different languages
  • 85. smartpuffin.com | @ice_lenor | #geo GIS-systems Geographic information systems: apps for creating and editing geodata. ● ArcGIS from ESRI ● OpenStreetMap editor ● QGIS ● ... many of them
  • 86. smartpuffin.com | @ice_lenor | #geo GIS-systems ● Users: geo-specialists (cartographers) ● Create layers (types of objects: "Buildings", "Rivers", "Parks") ● Create geoobjects ○ Draw geometry ○ Add attributes (names, tags, etc) ○ Do spatial operations, data validations ("buildings should not intersect each other")
  • 87. smartpuffin.com | @ice_lenor | #geosmartpuffin.com | @ice_lenor | #geo Common Problems in Geo
  • 88. smartpuffin.com | @ice_lenor | #geo new Point (lat, lon) OR new Point (lon, lat) ? ● Library-dependent ● Always check!
  • 89. smartpuffin.com | @ice_lenor | #geo Units: degrees ● Units of latitude and longitude: degrees ● You CAN'T measure distance in them!
  • 90. smartpuffin.com | @ice_lenor | #geo Units: degrees ● Two points on the equator: ● Point 1: latitude 0, longitude 0. ● Point 2: latitude 0, longitude 1. ● Distance between them, measured as if they were flat coordinates: 1. (Units: degrees.) ● Distance between them, measured as if they're on the planet Earth: 111.19 km.
  • 91. smartpuffin.com | @ice_lenor | #geo Units: degrees ● Two points on Amsterdam's latitude (52° N): ● Point 1: latitude 52, longitude 0. ● Point 2: latitude 52, longitude 1. ● Distance between them, measured as if they were flat coordinates: still 1. (Units: degrees.) ● Distance between them, measured as if they're on the planet Earth: 68.46 km.
  • 92. smartpuffin.com | @ice_lenor | #geo The closer to the pole, the less km's fit in one degree longitude
  • 93. smartpuffin.com | @ice_lenor | #geo Degrees latitude are always the same in km's
  • 94. smartpuffin.com | @ice_lenor | #geo 180th Meridian
  • 95. smartpuffin.com | @ice_lenor | #geo 180th Meridian: Always Check With Fiji! -16.760, -179.658 -16.712, 179.713 180th Meridian
  • 96. smartpuffin.com | @ice_lenor | #geo 180th Meridian
  • 97. smartpuffin.com | @ice_lenor | #geo 180th Meridian ● Fiji ● New Zealand ● Russia ● Kiribati ● United States
  • 98. smartpuffin.com | @ice_lenor | #geo Poles ● One latitude: 90° or -90° ● All longitudes meet there => It's all exactly the same point! ○ Point (lat=90, lon=0) ○ Point (lat=90, lon=100) ○ Point (lat=90, lon=-140)
  • 99. smartpuffin.com | @ice_lenor | #geo Poles ● Some projections don't support poles ● I.e., Mercator on WGS84: ○ "World between 80°S and 84°N." ○ Otherwise, poles would be stretched into infinity ● => If you need poles, check if your projection and datum support them
  • 100. smartpuffin.com | @ice_lenor | #geo Spatial operations around problem areas ● Math close to 180th meridian must work correctly! ● Same goes for poles!
  • 101. smartpuffin.com | @ice_lenor | #geosmartpuffin.com | @ice_lenor | #geo Thank you Your questions, please!