Standard version of the GeoSolutions presentation @ FOSS4g 2009.

1. GeoServer, GeoTools and GeoBatch:
supporting operational
Meteorology and Oceanography
Ing Simone Giannecchini
FOSS4G 2009, Sydney
23rd September 2009

2. Outline
 GeoSolutions Facts
 Reference Scenario/Domain
 Enterprise SDI
 SDI Building Blocks explained
 GeoBatch
 GeoServer
 BUDDATA ebRIM
 geoSDI-ERA
 Relevant Use cases
FOSS4G 2009, Sydney
23rd September 2009

3. GeoSolutions - Facts
 Funded mid 2006 in Italy
 Staff of 7 engineers (2 consultants)
 Expertise
 Image Processing, GeoSpatial Data Fusion
 Java, Java Enterprise, C++, Python
 JPEG2000, JPIP, Advanced 2D visualization
 Supporting/Developing FOSS4G projects
 GeoTools, GeoServer, GeoBatch, ImageIO-Ext, uDig
 Focus on
 Consultancy (agencies, large private companies, etc…)
 Building stack of geospatial Open Source products
FOSS4G 2009, Sydney
23rd September 2009

4. Reference Scenario/Domain
Enterprise Boundaries
REP/COP
FOSS4G 2009, Sydney
23rd September 2009

5. Enterprise Spatial Data Infrastructure
Reporting A&A&A GeoPortal
Data Collection Data Management Layer
Monitoring
Agents
Batch
Processing
ESB
Map
Accelerator
Online Data Dissemination
Index & Search
Processing Server
Data Management Layer
Spatial Net
DBMS Storage
FOSS4G 2009, Sydney
23rd September 2009

6. Enterprise Spatial Data Infrastructure
GeoBI A&A&A geoSDI-ERA OpenLayers
Data Collection GeoTools
Monitoring ESB Agents
GeoWebCache
GeoBatch GeoServer Buddata ebRIM
GeoTools
Net
PostGIS Storage
FOSS4G 2009, Sydney
23rd September 2009

7. Ingestion/Preprocessing
 Real-time produced data
 Remote Sensing, In situ, Models, etc..
 Recognizable, isolate data streams
 Flow
 Incoming series of geospatial data
 Specific format
 Defined rules for preprocessing
 Defined rules for ingestion and/or exploitation
 Automatic Ingestion and preprocessing  No manual intervention
 Example, flow of EO raster
GeoWebCache
Producer Tile Pyramid Publish
GeoServer
FOSS4G 2009, Sydney
23rd September 2009

8. GeoBatch: Basics
 Event-based geospatial-aware batch ingestion/processing
system
 Basic tools for automatic collection, processing and
publication of data.
 Open Source (leverages on GeoTools, Apache FTP, Spring,
Xstream, etc…)
 Embedded FTP Server
 Publish to GeoServer
 Publish to GeoWebCache (ongoing)
 Web based
 XML configuration
 http://docs.codehaus.org/display/GEOBATCH
FOSS4G 2009, Sydney
23rd September 2009

9. GeoBatch:FlowManager
FileSystemMonitor
Directory
Mailbox
Event
Monitoring
Action
Action
Action
EventDispatcher
Event Event
Consumer Consumer
FlowManager
FOSS4G 2009, Sydney
23rd September 2009

10. GeoBatch:Sample Flow
- <FlowConfiguration>
<workingDirectory>AISAnomalies</workingDirectory>
<autorun>true</autorun>
- <EventConsumerConfiguration>
<workingDirectory>AISAnomalies</workingDirectory>
<performBackup>true</performBackup>
- <FileEventRule>
<optional>false</optional>
<originalOccurrencies>1</originalOccurrencies>
<regex>.*.shp</regex>
<id>r1</id>
<description>description</description>
<name>test</name>
</FileEventRule>
….
- <GeoServerActionConfiguration>
<crs>EPSG:4326</crs>
<envelope />
<dataTransferMethod>URL</dataTransferMethod>
<geoserverPWD>geoserver</geoserverPWD>
<geoserverUID>admin</geoserverUID>
<geoserverURL>http://localhost:8080/geoserver</geoserverURL>
<wmsPath>/</wmsPath>
<defaultStyle>polygon</defaultStyle>
<styles />
</GeoServerActionConfiguration>
FOSS4G 2009, Sydney
23rd September 2009

11. GeoBatch:Sample Flow
- <GeoServerActionConfiguration>
<crs>EPSG:4326</crs>
<envelope />
<dataTransferMethod>URL</dataTransferMethod>
<geoserverPWD>geoserver</geoserverPWD>
<geoserverUID>admin</geoserverUID>
<geoserverURL>http://localhost:8080/geoserver</geoserverURL>
<wmsPath>/</wmsPath>
<defaultStyle>polygon</defaultStyle>
<serviceID>aisAnomaliesGeoServerGeneratorService</serviceID>
<styles />
</GeoServerActionConfiguration>
</EventConsumerConfiguration>
<eventGeneratorConfiguration class="FsEventGeneratorConfiguration">
<wildCard>*.*</wildCard>
<watchDirectory>AISAnomalies/in</watchDirectory>
<keepFiles>true</keepFiles>
<osType>OS_UNDEFINED</osType>
<eventType>FILE_ADDED</eventType>
</eventGeneratorConfiguration>
<id>flowAISAnomalies</id>
<description>flowAISAnomalies: testing injection of AISAnomalies shapefiles </description>
<name>flowAISAnomalies</name>
</FlowConfiguration>
FOSS4G 2009, Sydney
23rd September 2009

12. GeoBatch: New UI
FOSS4G 2009, Sydney
23rd September 2009

13. Management/Dissemination/Exploitation
 Raw Data Storage and Dissemination
 Metadata Storage and Dissemination
 Index & Search Capabilities for data
 Service Discovery
 Index & Search Capabilities for services
 Raw Data Portrayal
 Coverage  Maps
 Feature  Maps
 Processing
FOSS4G 2009, Sydney
23rd September 2009

14. GeoServer
 GeoSpatial enterprise gateway
 Management raster and vector data
 Standards compliant
 OGC WCS 1.0 - 1.1.1 (RI)
 OGC WFS 1.0 - 1.1 (RI)
 OGC WMS 1.1.1
 OGC WPS 1.0
 Google support
 KML, GeoSearch, etc..
 Enterprise ready (almost!)
FOSS4G 2009, Sydney
23rd September 2009

15. GeoServer: Limits/Shortfalls
 In-memory internal catalog
 Non transactional, Scale sub-optimally
 Replication is problematic
 File Based Configuration
 Non transactional, Scale sub-optimally
 Replications is problematic
 File based Coverage support (can be overcome easily)
 1 CoverageStore  1 Coverage (can be overcome, not easily)
 Embedding/Controlling GeoServer can be difficult
 REST uses HTTP always
 REST is non transactional
 No Time and Height/Depth support for raster
 No CSW or ebRIM support (yet!)
 WCS support needs to be improved
FOSS4G 2009, Sydney
23rd September 2009

16. GeoServer: Hibernate catalog
Web REST WMS WFS WCS ...  Configuration stored in DB
Catalog Interface  Hibernate
Hibernate Based Catalog  Spring based Dao
 Transactions via Spring/EJB3
ServiceDAO CatalogDAO
 Caching with EhCache
Hibernate  Same catalog interface, different
2°L cache implementation
 Catalog interface needs tweaking
 GeoServer UI still needs tweaking
 Transaction demarcation needs
Net DB tweaking
Storage
FOSS4G 2009, Sydney
23rd September 2009

17. GeoServer: Hibernate catalog
Web REST WMS WFS WCS ... Web REST WMS WFS WCS ...
Catalog Interface Catalog Interface
Hibernate Based Catalog Hibernate Based Catalog
ServiceDAO CatalogDAO ServiceDAO CatalogDAO
Hibernate Hibernate
2° L Distributed Cache
Net
DB DB DB Storage
FOSS4G 2009, Sydney
23rd September 2009

18. GeoServer: Transactional API
JVM 1
 In-JVM communication
GeoServer
 Transactional
 No HTTP, No Serialization
Master
Application  Spring, EJB3, JMX
 Intra JVM
 Transactional
 Serialization is involved
 Separate mechanism for
JVM 2 data upload
 HTTP, TCP, etc…
Master  Spring Remoting, EJB3,
Application
JMX
FOSS4G 2009, Sydney
23rd September 2009

19. GeoServer: Advanced Raster
FOSS4G 2009, Sydney
23rd September 2009

20. GeoServer: RasterVault
FOSS4G 2009, Sydney
23rd September 2009

21. GeoServer: RasterVault
Web REST WMS WFS WCS ...
GeoServer
Catalog Interface
CoverageStore Interface CoverageStore
RasterLayerDAO RangeDAO
Hibernate Spatial
2°L cache RasterVault
Net DB
Storage
FOSS4G 2009, Sydney
23rd September 2009

22. Index & Search Capabilities: ebRIM
 Registry versus Repository
Trader Authority
 Flexible Information Model
 Object Lifecycle Management
 Content Management Service Provider
 Event Notification
 Protocol Bindings
 Publish-Find-Subscribe(Consume) Consumer
 Federation
Registry
Broker
Publish Find
Subscribe
Provider Consume Consumer
FOSS4G 2009, Sydney
23rd September 2009

23. ERGO Buddata EbRR
 Open Source ebXML Registry/Repository (or ebRR in short)
e brr
buddata
 ESA founded ERGO project lead by Intecs
 OGC specifications implemented
 07-110r4: CSW-ebRIM Registry Service - Part 1: ebRIM profile of
CSW (1.0.1)
 07-144r4: CSW-ebRIM Registry Service – Part 2: Basic extension
package (1.0.1)
 07-038: OGC Cataloguing of ISO Metadata (CIM) using the ebRIM
profile of CS-W (0.1.11)
 06-131r6: EO Products Extension Package for ebRIM (ISO/TS 15000-
3) Profile of CSW 2.0
 Code on Google Code (http://code.google.com/p/buddata-
ebxml-registry/)
 Wiki ERGO project (ESA HMA wiki)
FOSS4G 2009, Sydney
23rd September 2009

24. EbRR: Architecture overview
Web Service Interface Layer
e brr
buddata
HTTP GetRepositoryItem
Interface
HTTP GetRepositoryItem SOAP Interfaces
Interface HTTP GetRepositoryItem
Interface
API Layer CSW SOAP Client
Registry APIs
CSW Backend Client
Deployer
Lifecycle Manager
Business Layer
Translator
Transaction Manager
Core Registry Add-on
Functions Functions Harvester
Query Manager
Validator
Repository Manager
Data Layer Postgresql Database
Persistence
PostGIS Functions
FOSS4G 2009, Sydney
23rd September 2009

25. Buddata EbRR @ Work
 Metadata published with data contextually
e brr
buddata
 Use of custom profiles
Profiles
Registry
Publish
Metadata Find
Metadata
GeoBatch Consumer
Publish Consume
Data GeoServer Data
---------
---------
-----
FOSS4G 2009, Sydney
23rd September 2009

26. GeoPortal: geoSDI - ERA
 Modular/Extensible Rapid-mapping Web Application
 EXT-GWT + Openlayers + Hibernate Spatial + Spring
 Core components to
perform basic actions
 OpenLayers integration
 Layers Wide search
 Layer Tree
 Security subsystem
 Style editor (ongoing)
 Plugin Engine to perform
specific tasks
 Routing, geocoding, etc..
 GEOGWT?
 GEOEXT + EXT-GWT
FOSS4G 2009, Sydney
23rd September 2009

27. geoSDI – ERA Modular
 GeoSDI-ERA Architecture Schema
GeoSDI-ERA GeoSDI-ERA
CORE WEB
OGC- Map Map
WMS Context
Plugins Map
Engine Widgets
GeoSDI-ERA
SLD Editor I-Mobility
Plug-Ins
Tools
Security
Subsystem PostGIS
-Raster Symbolizer -Geo-Coding
DB -Classifier -Routing
-SLD Load/Save -Traffic Density Maps
FOSS4G 2009, Sydney
23rd September 2009

28. Use Case: AIS data management
 Automated Identification System (AIS)
 WHF emitter/receiver
 GPS
 Continuously recording AIS traffic
 Mediterranean Sea
 Black Sea
 Providing support for storage and publishing of the results
of the AIS processing
 AIS Coverage
 AIS Forecasts
 AIS Anomalies
 More in the near future, like Radar contacts and Fuse d Tracks
FOSS4G 2009, Sydney
23rd September 2009

29. Use Case: AIS data management
 Fully automatic AIS coverage and anomaly detection ingestion and
publishing process.
AIS
Data
Stream GeoBatch
Standard OGC WMS/WFS clients
GeoServer
PostGIS
Google Earth/KML compliant clients
FOSS4G 2009, Sydney
23rd September 2009

30. Use case: SAS Mission Manager
 SAS: Synthetic Aperture Sonar
 Sea bed analysis and detection of features/artifacts
 Automatic Ingestion of SAS data
and metadata
 SAS Data preprocessing
 SAS mission data management
and access
 SAS mission data visualization
FOSS4G 2009, Sydney
23rd September 2009

31. Use case: SAS Mission Manager
SAS Mission Manager
Web interface Presentation Tier
Data and Metadata GeoServer
Management
Registry
ebRIM
WFS WCS WMS
Middleware Tier
Network GeoBatch
Area
Storage Mission Data DataIngestion
Package Tier
FOSS4G 2009, Sydney
23rd September 2009

32. Use case: SAS Mission Manager
Sonar images are Curtesy of NURC
FOSS4G 2009, Sydney
23rd September 2009

33. Use Case: Super Ensemble Modeling
 Optimal MetOc models mixing
 Same model, different boundary conditions
 Different model same boundary coinditions
 System training (Kalman filter)  historycal data
 Output  Series of weight for optimal MetOc model mixing
 Result  optimal MetOc model as fusion of weighted
models
FOSS4G 2009, Sydney
23rd September 2009

34. Use Case: Super Ensemble Modeling
Discover Buddata
EbRR
Toolbox WPS
Invoke
Harvest
MATLAB
Ensemble GUI
Publish Results
WCS::getCoverage
WMS::getMap
GeoServer
Input Data Deploy
GeoBatch
FOSS4G 2009, Sydney
23rd September 2009

35. Use Case: Super Ensemble Modeling
 Current solution
 Split 3D hyper-cubes of data into a set of 2D slices
 Use registry for registering T and Z
. .. . . GeoBatch Registry
. .. . . ... .. .
Publish
. . .. . . .  Vertical Metadata
. . . .... . . ..
Interpolation
. ... . . . .  Geo-Rectification
. . .  Resampling
Publish
Input 3D Data GeoServer
Cubes
• Next Step
• Handle multidim coverages through GeoServer + RasterVault
FOSS4G 2009, Sydney
23rd September 2009

36. Use Case: Super Ensemble Modeling
FOSS4G 2009, Sydney
23rd September 2009

37. Use Case: TDA
Satellite data
Value
added
Oceanographic/Meteo Fusion products
forecast
On-site observations
FOSS4G 2009, Sydney
23rd September 2009

38. Use Case: TDA
Discover Buddata
EbRR
Toolbox WPS
Harvest Invoke
FUZZY
TDA logic
Explorer
Publish
WCS:getCoverage
WMS:getMap
GeoServer
Input Data Publish
GeoBatch
FOSS4G 2009, Sydney
23rd September 2009

39. Use Case: TDA
Search page
WPS invocation
FOSS4G 2009, Sydney
23rd September 2009

40. Use Case: LAMMA
WRF Postproc Postproc
NCEP-GFS
NMM
ECMWF
FOSS4G 2009, Sydney
23rd September 2009

41. Use Case: LAMMA
WRF Postproc
NCEP-GFS
NMM
GeoBatch
Publish
Data Publish
GeoServer Metadata
ECMWF RasterVault Registry
FOSS4G 2009, Sydney
23rd September 2009

42. Use Case: FAO FIGIS
 Dynamic Statistical Query Engine
- Compute GIS Layer Intersections
- Download statistics outcomes
UI - Monitoring the Engine status
HTTP HTTP
Provider Consumer
WFS-T
Enterprise Service BUS GeoServer
(Apache ServiceMix) WFS
Scheduler GeomCompare
Service Service
The Service Units interact
The Dynamic Query directly with FIGIS
Engine consists of a set GeoServer instances
of Service Units capable through the OGC WFS-T
of generating statistics protocol.
on a periodic interval
fashion.
DB (ORACLE)
FOSS4G 2009, Sydney
23rd September 2009

43. Use Case: FAO FIGIS
The Service Units interact
directly with FIGIS
GeoServer instances
through the OGC WFS-T
protocol.
FOSS4G 2009, Sydney
23rd September 2009