GDAL & WebGL: European Commission, Joint Research Center, Ispra
1. Joint Research Centre - European Commision, November 22, Ispra, Italy.
Research & Development with
WebGL, Closure, GDAL & MapRank
Petr Pridal <petr.pridal@klokantech.com>
2.
3.
4. WebGL Earth Closure Tools
3D digital globe. JavaScript Toolset.
MapTiler Cluster
GDAL2Tiles on steroids. GeoSearch with ranking.
5. !
beta
WebGL Earth
Free and open source 3D digital globe
for web and mobile devices
6. HTML5 & WebGL
• HTML5 <canvas> WebGL extension
• Based on OpenGL ES
KLOKAN TECHNOLOGIES - Museo 300
• Supported by NVidia, ATI, Intel on desktop 500
KLOKAN TECHNOLOGIES - Museo
KLOKAN TECHNOLOGIES - Museo 700
• HW acceleration on mobile devices
• Firefox 4+, Chrome 9+, Opera, Safari
• Internet Explorer: with Chrome Frame
7. WebGL: Shaders in GLSL
KLOKAN TECHNOLOGIES - Museo 300
KLOKAN TECHNOLOGIES - Museo 500
KLOKAN TECHNOLOGIES - Museo 700
8. KLOKAN TECHNOLOGIES - Museo 300
KLOKAN TECHNOLOGIES - Museo 500
KLOKAN TECHNOLOGIES - Museo 700
9. KLOKAN TECHNOLOGIES - Museo 300
KLOKAN TECHNOLOGIES - Museo 500
KLOKAN TECHNOLOGIES - Museo 700
10. KLOKAN TECHNOLOGIES - Museo 300
KLOKAN TECHNOLOGIES - Museo 500
KLOKAN TECHNOLOGIES - Museo 700
11.
12. KLOKAN TECHNOLOGIES - Museo 300
KLOKAN TECHNOLOGIES - Museo 500
KLOKAN TECHNOLOGIES - Museo 700
Inspired by OpenLayers.Layer.GoogleNG
(done by Andreas Hocevar)
Compliant to Google TOS?
13. Try the demo application:
KLOKAN TECHNOLOGIES - Museo 300
www.webglearth.com
KLOKAN TECHNOLOGIES - Museo 500
KLOKAN TECHNOLOGIES - Museo 700
14. Mission of the project
• User friendly (dragging, zooming, ...)
• Keep it simple & maximally web oriented
KLOKAN TECHNOLOGIES - Museo 300
• Raster data from prerendered tiles, WMS, - Museo 500
KLOKAN TECHNOLOGIES
KLOKAN TECHNOLOGIES - Museo 700
existing tile APIs and TileCache, GeoServer...
• “NoServer” = o ine usage & cloud
• Calculations from CPU to GPU. Preprocess!
• Open-Source (GPLv3), Community powered
15. Open Source project
• Simple JavaScript API
• Embed <script src=“...../api.js”>
• Mashup with your own tiles, custom WMS, 500
KLOKAN TECHNOLOGIES - Museo 300
KLOKAN TECHNOLOGIES - Museo
overlays, markers and popups TECHNOLOGIES - Museo 700
KLOKAN
• The full codebase + demo app sources on
GitHub, object oriented, documented code,
manual describing internals available
• Contributors: KlokanTech, CampToCamp,
AI2 Spain, BIMServer.org, NOAA
16. JavaScript API: Demo
KLOKAN TECHNOLOGIES - Museo 300
KLOKAN TECHNOLOGIES - Museo 500
KLOKAN TECHNOLOGIES - Museo 700
17. JavaScript API
• Embedding of the 3D globe
• Custom base maps and overlays with
adjustable transparency (tilesKLOKAN TECHNOLOGIES - Museo 300
+ WMS)
KLOKAN TECHNOLOGIES - Museo 500
• Camera API, flyTo, bbox KLOKAN TECHNOLOGIES - Museo 700
• User interaction and events
• Markers & popups
• <canvas> is part of HTML DOM
18.
19.
20. Codebase: we, weapp
• Demo application (www.webglearth.com)
completely open-source, API exports too
• Compiled with Closure Compiler with types: 300
KLOKAN TECHNOLOGIES - Museo
KLOKAN TECHNOLOGIES - Museo 500
http://blog.klokantech.com/2010/12/closure-compiler-for-openlayers-3x.html
KLOKAN TECHNOLOGIES - Museo 700
• Documented (google style guideline)
• Technically, anything is possible :-)
• Manual describing the implementation
details: http://data.webglearth.com/doc/
21.
22. Object hierarchy
KLOKAN TECHNOLOGIES - Museo 300
KLOKAN TECHNOLOGIES - Museo 500
KLOKAN TECHNOLOGIES - Museo 700
23. “Keep it simple” design
• Sphere with diameter 1.0
• WGS84 coordinates used without ellipsoid
transformation (a la Google Maps, OSM, etc.)
KLOKAN TECHNOLOGIES - Museo 300
KLOKAN TECHNOLOGIES - Museo 500
KLOKAN TECHNOLOGIES - Museo 700
• Textures in Spherical Mercator (EPSG:3857,
EPSG:900913)
• Clip stack for megatextures with lookup
• Segmented plane as the model
24. Segmented plane
KLOKAN TECHNOLOGIES - Museo 300
KLOKAN TECHNOLOGIES - Museo 500
KLOKAN TECHNOLOGIES - Museo 700
25. Terrain
KLOKAN TECHNOLOGIES - Museo 300
KLOKAN TECHNOLOGIES - Museo 500
KLOKAN TECHNOLOGIES - Museo 700
26. KLOKAN TECHNOLOGIES - Museo 300
KLOKAN TECHNOLOGIES - Museo 500
KLOKAN TECHNOLOGIES - Museo 700
27. KLOKAN TECHNOLOGIES - Museo 300
KLOKAN TECHNOLOGIES - Museo 500
KLOKAN TECHNOLOGIES - Museo 700
28. Vector data (direct rendering)
• Now: rasterized only (WMS, tiles with
Mapnik, ...). Raster tiles must be there for
aerial data.
KLOKAN TECHNOLOGIES - Museo 300
• Future: direct rendering (from GeoJSON,
KLOKAN TECHNOLOGIES - Museo 500
KLOKAN TECHNOLOGIES - Museo 700
KML, WFS, ...)
• Sharing code for loading data formats with
OpenLayers 3, Leaflet, Polymaps?
• Vector Tiles - Google MapsGL
29. Future: Google MapsGL
KLOKAN TECHNOLOGIES - Museo 300
KLOKAN TECHNOLOGIES - Museo 500
KLOKAN TECHNOLOGIES - Museo 700
30. Future: Mobile devices
KLOKAN TECHNOLOGIES - Museo 300
KLOKAN TECHNOLOGIES - Museo 500
KLOKAN TECHNOLOGIES - Museo 700
31. Problems: drivers + CORS
• Long list of blacklisted graphic cards
• June 2011: Introduced restrictions on cross-
domain textures for WebGL :-(
KLOKAN TECHNOLOGIES - Museo 300
• CORS HTTP Header: Access-Control-Allow-Origin: * - Museo 500
KLOKAN TECHNOLOGIES
KLOKAN TECHNOLOGIES - Museo 700
• http://enable-cors.org/ or chrome --disable-web-security
• Support only in Chrome 15+ (beta), Firefox 8 (11/2011)
• Workaround for now: HTTP proxy
• Tiles hosted on your own server runs always!
32. Try it: 10 minut cookbook
• JavaScript API: check examples
• Prepare data (one of):
KLOKAN TECHNOLOGIES - Museo 300
• WMS / TMS / WMTS in EPSG:900913 - Museo 500
KLOKAN TECHNOLOGIES
KLOKAN TECHNOLOGIES - Museo 700
• Convert any GeoTIFF with GDAL2Tiles /
MapTiler
• Copy everything on your web server
• Display in any modern web browser
33. Try it at:
http://www.webglearth.com/
Project website, mailing-list, documentation:
http://www.webglearth.org/
KLOKAN TECHNOLOGIES - Museo 300
KLOKAN TECHNOLOGIES - Museo 500
KLOKAN TECHNOLOGIES - Museo 700
Plans for integration with OpenLayers 3.0
Support, open-source contracted development:
Klokan Technologies: http://www.klokantech.com/
34. WebGL Earth Closure Tools
3D digital globe. JavaScript Toolset.
MapTiler Cluster
GDAL2Tiles on steroids. GeoSearch with ranking.
37. It is all about compiler...
1. Compiles readable JavaScript into compressed
machine-readable JavaScript: dead code removal,
function inlining, etc.
KLOKAN TECHNOLOGIES - Museo 300
KLOKAN TECHNOLOGIES - Museo 500
2. JSDoc Tags: typos, wrong use of a @constructor,
KLOKAN TECHNOLOGIES - Museo 700
wrong type of a variable, misuse of a field
annotated with @private and @protected, etc.
3. A reusable JavaScript library, such as OpenLayers,
exports public API - and the compiler optimizes
the internal code.
http://goo.gl/H6twe
38. It is all about compiler...
4. Unused parts of the library stripped. Application
dependencies solved during compilation.
5. Constants to remove unwanted functionality: - Museo 300
KLOKAN TECHNOLOGIES
Mobile WebKit only, no Quirks mode support, - Museo 500
KLOKAN TECHNOLOGIES
KLOKAN TECHNOLOGIES - Museo 700
compilation without support of IE6
6. Debugging with FireBug possible even for the
compiled version
7. Dynamically loadable modules, additional
functionality loaded only when required.
http://goo.gl/H6twe
39. Hello world
goog.require('goog.dom');
function sayHi() { KLOKAN TECHNOLOGIES - Museo 300
var newHeader = goog.dom.createDom('h1', TECHNOLOGIES - Museo 500
KLOKAN {'style':
KLOKAN TECHNOLOGIES - Museo 700
'background-color:#EEE'},
'Hello world!');
goog.dom.appendChild(document.body, newHeader);
}
goog.exportSymbol('sayHi', sayHi);
40. How to start?
• Easiest is to get plovr - where all the tools
are packed into one .jar application.
• Run java -jar
KLOKAN TECHNOLOGIES - Museo 300
plovr.jar serve config.json and use
KLOKAN TECHNOLOGIES - Museo 500
it with HTML as <script src="http://localhost: - Museo 700
KLOKAN TECHNOLOGIES
9810/compile?id=hello"></script>
• Plovr: Hello world + skeletons at GitHub:
https://github.com/klokantech/closure-library-plovr-hello-world-skeletons/
• Closure Library websites:
http://code.google.com/closure/library/
41.
42. Where it excels?
• Complex online applications - any serious
work done in JS
• More developers working on KLOKANsame code 300
the TECHNOLOGIES - Museo
KLOKAN TECHNOLOGIES - Museo 500
• KLOKAN TECHNOLOGIES - Museo 700
High peformance app + mobile apps
• Libraries and APIs
• :-( no fast copy & paste hacking
• :-( steep learning curve, smaller community
43. Read more at:
http://code.google.com/closure/
Plovr.jar - the “packed” toolset:
KLOKAN TECHNOLOGIES - Museo 300
http://plovr.com/KLOKAN TECHNOLOGIES - Museo 500
KLOKAN TECHNOLOGIES - Museo 700
The book: Closure Library, The Definitive Guide
Blog post: http://goo.gl/H6twe
44. WebGL Earth Closure Tools
3D digital globe. JavaScript Toolset.
MapTiler Cluster
GDAL2Tiles on steroids. GeoSearch with ranking.
70. NECESSARY TRANSFORMATIONS AND WARPING
GEODETIC COORDINATES SPHERICAL MERCATOR PYRAMID COORDINATES TILE INDEX
LATITUDE LONGITUDE METERS XYZ PIXELS / ZOOM XYZ TILE / ZOOM
WGS 84 (EPSG:4326) EPSG:3785 / EPSG:900913 WEB VIEWERS TILE MAP SERVICE
71. SPHERICAL MERCATOR:
TILE PROJECTION AND COORDINATES
NOT ONLY IN GOOGLE MAPS!
• Popular on the web: Google Maps, Microsoft Bing
Maps, Yahoo Maps, OpenStreetMap, ...
• Lat/Long WGS84 (GPS devices) datum coordinates
always displayed to users - KML
• Map rasters are projected by a global Mercator
projection: “Spherical Mercator”. EPSG:900913 /
EPSG:3857 (meters)
• Conformal, cylindrical. Cropped to square (maximal
latitude ~ 85.05 degrees), WGS84 ellipsoid is sphere!
72. OPEN-SOURCE ALTERNATIVES:
PREGENERATING X CACHING TILES
• Generating from raster input: • Caching tiles from WMS servers
(MapServer, GeoServer):
• MapTiler & GDAL2Tiles
• TileCache.org (Python)
• MSR MapCruncher
• GeoWebCache.org (Java Servlet)
• Generating from features (vectors)
• Mapnik
• GMapCreator, • Commercial alternatives
• ... • License per server (ArcGIS Server)
73. EXTREMELY FAST ONLINE MAPS
FOR THOUSANDS OF PEOPLE
• Easy hosting: only static files / or raster blobs in a database
• Extremely fast delivery, highest-quality output possible (you
have more time for rendering), SLA for up to 99.99% availability
74. Our software is used ...
• ... in: web applications, mashups, native
mobile applications (iPhone/iPad/Android),
hardware appliances and 3D visualizations
• ... for: standard base maps, aerial imagery,
KLOKAN TECHNOLOGIES - Museo 300
KLOKAN TECHNOLOGIES - Museo 500
coverage maps, thematic maps, marine
KLOKAN TECHNOLOGIES - Museo 700
navigation, by pilots in airplains, on the
ground during disaster events, ...
• ... by: Google, NOAA, NASA, CERN,
Vodafone, T-Mobile, Orange, Three, United
Nations, and thousands of developers
75. PARALLELIZED RENDERING:
MAPTILER CLUSTER
• Parallelized rendering on Multi-Core processors or even on
computer clusters (Amazon EC2 supported)
• Speed-up the rendering on modern computers
• Rendering on a computer cluster (Amazon EC2) - price: 0.1 USD / hour / core
• Publishing of tiles on a world-wide CDNs (Amazon S3/CloudFront, Google App
Engine, Azure, SimpleCDN) - fast multi-threaded upload for tiles possible
• Merging of huge raster datasets into a seamless map a la
Google Maps on a computer cluster:
• Fast parallelized rendering and merging of large data collections
s
nt
• Cluster parallelization based on MPI
lie
rc
• Test of MapReduce approach (via Nokia labs Disco project)
fo
76. MapTiler Cluster vs
GDAL2Tiles
• Fast rendering: multi-core CPU & MPI cluster
• Direct merging of several input files
• Excellent visual quality
• Heavily optimized PNGs/JPEGs tiles to speed
up the delivery and save disk space
• Precise "data over-zooming"
• Various coordinate systems supported
s
nt
lie
rc
fo
77.
78.
79.
80.
81.
82.
83.
84.
85.
86.
87. WWW . MAPTILER . ORG
Runs under Windows, Linux and Mac OS X + GDAL2Tiles
You can try it on the FOSS4G LiveDVD!
Community: MapTiler User Group
maptiler@googlegrups.com
MapTiler Cluster, commercial support, integration,
customization
http://www.maptiler.com/
88. WebGL Earth Closure Tools
3D digital globe. JavaScript Toolset.
MapTiler Cluster
GDAL2Tiles on steroids. GeoSearch with ranking.
90. GEOGRAPHICAL SEARCHING
• Research, scientific articles, work on Ph.D. thesis (Petr Přidal),
Computer Science + Geodesy and Cartography
• Alternative projects tested in OldMapsOnline.org (R&D project in MZK):
• Lucene with GeoTemporal Ext. (LGTE) from DigMap.eu
• Alexandria Digital Library
• TimeMap.net (ECAI.org)
• Google GeoSearch API
• Knowledge of traditional GIS search methods (PostGIS, ...)
Result: Development of a new geographical search engine
for map collections.
91. CORE FEATURES
• Map-driven interface: Simple. Intuitive. Web 2.0.
Users love it!
• Combined queries: Where, When, Who & What
• Results sorted geographically with an advanced
spatial ranking algorithm.
• Instant search. Immediate delivery of the results by
moving the map or typing.
• It is fast. Even for large databases (million+ records).
92. METADATA & CUSTOMIZATION
• Indexing metadata in MARC, DublinCore, ISO
19139 formats, ...
• Batch import or harvesting: OAI-PMH or OGC CWS
• Numerical geographical boundaries necessary!
(MARC 034, DublinCore DCMI) - enrichment with
Geoparser or Georeferencer tools
• Possible customization of the user interface,
integration into products or websites
• For developers: REST API (JSON format), queries
following OpenSearch.org
97. MAPRANK RANKING &
SCALABILITY
• MapRank ranking: an advanced algorithm calculating
similarity of geographical area of query and area covered
by every document and combining it with map scale, time
range, predefined categories and traditional full text
search.
• Data indexing implemented specifically to provide ranked
real-time results even for large databases. Traditional R-
Tree indexing or Lucene are failing.
• Server component: FastCGI, C++
• Default user interface developed with Closure JavaScript
Library + Closure Compiler.
98. ENRICHMENT OF METADATA
Geoparser
Text parsing and semi-automatic assignment of
bounding boxes to map records with detectable
geonames (gazetteer), known physical size and map
scale.
Georeferencer
Crowdsourced georeferencing. Higher accuracy, more
outputs beside geometadata such as visualization, 3D
view, comparing of maps, OGC WMS for use in GIS.
104. Thank you!
Questions, ideas?
Cooperation on OpenSource and R&D?
Coding week for OpenLayers3 & WebGL Earth?
Interest in applications of MapTiler / MapRank?
Petr Pridal <petr.pridal@klokantech.com>