Grid Enabling Satellite Data using OWS and GRASS
•Descargar como PPT, PDF•
1 recomendación•2,535 vistas
Presented at the FOSS4G Conference, Victoria, Canada
Recomendados
Recomendados
Más contenido relacionado
Último
Último (20)
Destacado
Destacado (20)
Grid Enabling Satellite Data using OWS and GRASS
- 1. landmap.mimas.ac.uk Grid Enabling Satellite Data using OWS and GRASS Gail Millin Geodata Services Development Officer Pascal Ekin Software Developer Victoria, Canada FOSS4G 2007 24th – 27th September 2007
- 2. FOSS4G 2007 Victoria Canada landmap.mimas.ac.uk Mimas Data Centre
- 3. FOSS4G 2007 Victoria Canada landmap.mimas.ac.uk Main Activities Satellite Data Provision to UK academia Geospatial Support Services Educational Materials Research and Development New Data Delivery Infrastructure Applications of Radar & Optical Data Dissemination & Outreach
- 4. FOSS4G 2007 Victoria Canada landmap.mimas.ac.uk Who Uses Landmap? First 1000 Registered Users of the Landmap Service
- 5. FOSS4G 2007 Victoria Canada landmap.mimas.ac.uk Data Archive Optical Collection Landsat 5, Landsat 7, SPOT, MERIS Radar Collection ASAR (AP, WS, IM), ERS 1, ERS 2 Elevation Collection SRTM, 25m DEM, KGPS Available to academics attending an institute that has signed a licensing agreement. Free service since 1st August 2007
- 6. FOSS4G 2007 Victoria Canada landmap.mimas.ac.uk Optical Collection - Landsat Landsat 7 Bands 3,2,1Landsat 7 Bands 7,4,3
- 7. FOSS4G 2007 Victoria Canada landmap.mimas.ac.uk Optical Collection - SPOT
- 8. FOSS4G 2007 Victoria Canada landmap.mimas.ac.uk Radar Collection - ASAR
- 9. FOSS4G 2007 Victoria Canada landmap.mimas.ac.uk Elevation Collection – DEM & KGPS
- 10. FOSS4G 2007 Victoria Canada landmap.mimas.ac.uk Landmap - Image Streaming Image Streaming – ER Mappers Image Web Server 8.1 Landmap Image Streaming Viewer Free downloadable heavy clients ArcGIS Explorer Google Earth GetCapabilities request for IWS http://ims1.landmap.ac.uk/ecwp/ecw_wms.dll?request=G
- 11. FOSS4G 2007 Victoria Canada landmap.mimas.ac.uk
- 12. FOSS4G 2007 Victoria Canada landmap.mimas.ac.uk
- 13. FOSS4G 2007 Victoria Canada landmap.mimas.ac.uk WMS – web application which provides a portrayal of geographic data which is stored on the server. WFS – is an interface allowing requests for geographical features across the web using the XML based GML for data exchange. WCS – defines how to provide satellite data over the internet. This differs from WMS as actual data values are sent. WMS/WCS via OGC Landmap viewer or OGC compliant software UDig Cadcorp ArcGIS Landmap OWS
- 14. FOSS4G 2007 Victoria Canada landmap.mimas.ac.uk Requires a variety of supporting libraries PROJ 4 for on-the-fly projection Geospatial Data Abstraction Library (GDAL) a translator library for raster data GetCapabilities requests for GEMS II work Landsat 5 - http://ims2.landmap.ac.uk/cgi-bin/mapserv.exe? map=..chameleonsatellitemapl5_wcs.map&SERVICE=WCS&R EQUEST=GetCapabilities&WMTVER=1.0 Landsat 7 - http://ims2.landmap.ac.uk/cgi-bin/mapserv.exe? map=..chameleonsatellitemapl7_wcs.map&SERVICE=WCS&R EQUEST=GetCapabilities&WMTVER=1.0 GetCoverage request http://ims2.landmap.ac.uk/cgi- bin/mapserv.exe? map=..chameleonsatellitemapl7_wcs.map&SERVICE=WCS&REQ UEST=GetCoverage&Coverage=orthol7_20123xs190600&BBOX=48 6600.00,252275.00,699975.00,468500.00&CRS=EPSG:27700&RES X=25&RESY=25&FORMAT=GEOTIFF MapServer
- 15. FOSS4G 2007 Victoria Canada landmap.mimas.ac.uk
- 16. FOSS4G 2007 Victoria Canada landmap.mimas.ac.uk GEMS I Aim: To Provide Grid-enabled access to the aggregate statistics from the 2001 Census via OGSA-DAI on the National Grid Service Connect SQL Server database directly to the Grid via NGS Grid-enable current data access system Casweb to facilitate ease of use A fully working version was deployed on the NGS in November 2006 http://casweb.mimas.ac.uk/
- 17. FOSS4G 2007 Victoria Canada landmap.mimas.ac.uk GEMS II Aim and Objectives Grid enable a subset of the Landmap Service Data archive Landsat 4/5 Thematic Mapper Scenes 1988 - 1992 Landsat 7 1999 - 2002 25m Digital Elevation Model GEMS II provide a data processing service on the National Grid Service using OWS & Grid technologies Make use of 128 CPU/compute node High processing power
- 18. FOSS4G 2007 Victoria Canada landmap.mimas.ac.uk GEMS II Use Case Urban Change Detection for the UK Satellite Image Processing renown for computer intensive tasks. “There are many studies that take place in the remote sensing community focused on change detection of urban areas; however these studies can be limited to local areas due to the lack of computer power to enable much larger areas to be assessed” (Comert, 2004) Subset from the Data Archive for Use Case Landsat 4/5 Thematic Mapper Scenes 1988 - 1992 Landsat 7 1999 - 2002 25m Digital Elevation Model
- 19. FOSS4G 2007 Victoria Canada landmap.mimas.ac.uk GEMS II Architecture - Tier 1 Provide the Visualisation of the WCS data results and querying of the CSW using OGC compliant clients on the Users System Clients Evaluated for this Role: Cadcorp SIS MapBrowser Free to download but not open source Browse OGC Data services Supports WMS, WFS, WCS MPGC1.0 Data menu allows users to add/remove WCS, WFS and WMS Client offers image processing capabilities Quantum GIS Open source, free of charge GIS supports WMS and WCS results (GeoTiffs). GRASS layers can be imported into the interface using a GRASS plug- in
- 20. FOSS4G 2007 Victoria Canada landmap.mimas.ac.uk GEMS II Architecture - Tier 1/2 Developments of a GEMS GridRelay Client/Server Proxy GEMS GridRelay client proxy - install on the user’s system GEMS GridRelay proxy server – installed on Mimas Server (Linux OS) Benefits of GridRelay GridRelay allows user accreditation (Athens, UK eScience certificates) verifiable of the service side Deployment of the client proxy is seamless Works at the TCP level GridRelay can be extended to include other authentication/authorisation protocols such as Shibboleth
- 21. FOSS4G 2007 Victoria Canada landmap.mimas.ac.uk GridRelay – Further Benefits Dependent on traditional internet protocols & tools OS independent as it is Java based Very small (a jar file with few dependencies) Configurable through an internet browser (developed) Can be combined with existing Web Service protocols REST, SOAP, WSDL, existing Grid Security (GSI)
- 22. FOSS4G 2007 Victoria Canada landmap.mimas.ac.uk Overview of GEMS II Architecture
- 23. FOSS4G 2007 Victoria Canada landmap.mimas.ac.uk GRASS - Scripts Change Detection of Urban Areas i.cluster – Clustering algorithm Pixel clusters = imagery categories that can be related to land cover types on the ground Output = Cluster spectral signatures, cluster means and covariance matrices i.maxlik – Classifies the cell spectral reflectances in imagery data based on the spectral signatures generated by i.cluster Output = Raster map layer of classified image in which each cell has been assigned a spectral class.
- 24. FOSS4G 2007 Victoria Canada landmap.mimas.ac.uk GEMS II Outcomes Exploit GRID Technologies Expand on OWS work initiated by the JISC funded Interoperability Project Use OWS in a Grid Context Users can benefit from high CPU computer power of the NGS Create the use case demonstrator using Landsat 4/5, Landsat 7 and DEM
- 25. FOSS4G 2007 Victoria Canada landmap.mimas.ac.uk Future Developments New Website – due to be released in January 2008 New Release of ASAR data – January 2008 Monthly acquisitions of TOPSAT data from QinetiQ 10 images per month International locations 2m/3m resolution 3 band optical imagery Extend the Image Streaming Viewer to display the recently updated metadata
- 26. FOSS4G 2007 Victoria Canada landmap.mimas.ac.uk Register Institutes must sign a licensing agreement form to use the service after July 31st 2007 93 sites are already registered The service is now free! Further information available at http:// landmap.mimas.ac.uk Users require Athens Username and Password Access to download area Access to the online learning materials e.g. Image Processing Course
- 27. FOSS4G 2007 Victoria Canada landmap.mimas.ac.uk Thanks for Listening Enquiries spatial@mimas.ac.uk Website landmap.mimas.ac.uk
Notas del editor
- This data is from the first 1000 SIDS registered users Shows 31% of users come from disciplines that would traditionally use satellite data e.g. IT, GIS, Geomatics, Environment, Geography, Agricultural studies and Marine/Hydrology studies 27% come from unconventional subject areas such as social sciences and economics 22% are from the biological and physical sciences with the largest user group not from geomatic but from the disciplines of biology, physics and engineering Less than a third of the total users are from conventional spatial science disciplines over 2 thirds are from a wide range of other academic disciplines including engineering, languages, history, finance and physics. 20% was classified in the other or learning resources category and therefore is from a more generic user group.
- Landsat 5 (late 80s early 90s) Landsat 7 (1999 -2000) SPOT (mid 1990s) Mediterranean datasets available for Landsat imagery only ASAR (2004 – present) ERS 1 & 2 (1995 – 1999) Landmap DEM derived from ERS
- Orthorectified and Projected to British National Grid 64 Scenes - Landsat 5 and Landsat 7 Formats JPEG, GeoTiff, Imagine Image 1 – Band combination 3,2,1 enables information about water sediment patterns and water features to be highlighted – area is Isle of Wight and Portsmouth Image 2 – Band combination 7,4,3 highlights the urban areas of Southampton, Totton, Cowes, Gosport and Lymington in pink, forested areas are represented as a bright vivid green e.g The New Forest and Parkhurst Forest on the Isle of Wight.
- Image 1: SPOT image of Greenwich and Isle of Dogs acquired 13 th September 1991 Image 2: 15m Panchromatic Landsat 7 image acquired 19 th June 2000 – can detect urban change addition of Ed-Excel Exhibition Centre and the Millennium Dome Projected to British National Grid 150 Panchromatic images available 10m resolution 60km x 60km coverage ECW & GeoTIFF format
- Image 1: August 04 Alternating Polarisation during a low tide Image 2: Solway viewed through ASAR Image Mode data on two dates: 18th January 2005 and 16 August 2005 Image 3: Solway viewed through ASAR Alternative Polarisation (AP) on 8th August 2004
- 25m DEM created using interferometric techniques with ERS 1 and ERS 2 data KGPS Survey provides accurate high-resolution locational data of about 6,400km of roads in the British Isles.
- ER Mapper - Provides ECW high speed streaming for all Landmap imagery and is also OGC WMS compliant
- New approaches are being developed so that our users can easily take advantage of the desktop visualisation capabilities of ArcGIS.
- New approaches are being developed so that our users can easily take advantage of the desktop visualisation capabilities of Google Earth. This example shows how the Image Streaming Service run from ER Mappers Image Web Server has been linked into the Google Earth interface using KML
- Open Geospatial Consortium (OGC) - a non-profit, international, voluntary standards organization that is leading the development of standards for geospatial services Their aim is to advance geo-processing interoperability across the GIS industry
- MapServer is a freely available, open source development environment, offering support of WMS and WCS http://mapserver.gis.umn.edu/
- MIMAS is now a member of the OGC organisation Many WMS have been created at Landmap so that all data within the archive is accessible using an OGC compliant software client. Additionally this interface has been created for the Landmap website where sample WCS can be accessed to download the data if required This makes the Landmap Service highly interoperable and accessible in a variety of ways
- Holding 2001 aggregate census data The system allowed users to Upload query results to a Grid/File Transfer Protocol (FTP) server. View SQL generated by user interface for further integration into an OGSA-DAI client. Redirect query results to a grid service/ OGSA-DAI activity for further processing. Bulk upload query results to a user specified OGSA-DAI enabled database Integration of table metadata into query results. Transform query result into a variety of formats (CSV, HTML, etc...) by employing built in Java/SAX transformation code. Convert query results from a target to a user selected source geography.
- The overall aim of GEMS II Project is to Grid Enabling Mimas Services (GEMS) in particular a subset of the Landmap Service Data Archive. The GEMS II project would allow large amounts of satellite imagery available from the Landmap Service to be transferred to a National Grid Service (NGS) compute node for large scale, CPU intensive analysis. GEMS II will provide a data processing service through an OGC defined Web Processing Service
- The limitations in using desktop for high CPU intensive processing tasks like mosaicing or image classification is justification for trying to incorporate the power of the grid into a satellite image processing workflow. This use case is a proof of concept to test out combining the different grid technologies and OWS
- Proposed software architecture in GEMS II would be built around several of the Open Geospatial Consortium (OGC) web standards. Web Coverage Service - Web Processing Service Catalogue Service for the Web
- The GridRelay software component developed by Pascal Ekin at Mimas relays client/server connections through proxies This is necessary to track who is accessing the data and reporting usage to JISC The more seamless the architecture is by avoiding customised coding to services ultimately means that the GridRelay client can connect to any service using HTTP protocol By working at the Transmission Control Protocol level GridRelay is compatible with most internet protocols e.g. native database connection through JDBC as well as providing secure non local communication between proxies through SSL (Secure Sockets Layer)
- Some problems Not a full grid solution Not as efficient as some specific Grid Solutions as there is an efficiency penalty incurred by employing proxy components to relay information between nodes, clients, servers.
- Visualisation Clients – OGC compliant GEMS GridRelay client proxy – allow users to employ any known OWS client with GEMS using a Secure Socket Layer to the GEMS GridRelay Proxy here the users credentials can be checked using ConvertGrid for Athens Authentication/authorisation. If ok can use PyWPS to launch Globus GRASS jobs on the NGS HPC system. Globus calls MyProxy which enables users to safely store a proxy of their UK e-science certificate permitting proxy delegation the GRASS jobs can then be launched employing the users credentials. In a similar fashion to the WPS the WCS will be deployed on the Mimas Linux server. WCS is provided using MapServer The catalogue service bypasses the GEMS GridRelay proxy server as authentication of metadata is not required and will be used for data discovery.
- GRASS – Geographic Resource Analysis Support System is used for data management, image processing, graphics production, spatial modelling and visualisation of many types of data. Free software and open source containing over 350 programs. i.Cluster – Reads through the raster imagery data and builds pixel clusters based on the spectral reflectances of the pixels. i.Maxlik – The maximum likelihood classifier is being used for unsupervised classification the cluster means and covariance matrices from i.cluster signature file are used to determine which category each cell in the image has the highest probability of belonging . r.Kappa – Calculate error matrix and kappa parameter for accuracy assessment of classification results.
- The NGS data node utilised for this project consists of 128 CPU system accessible to UK e-Science certificate holders. The system currently runs on GLOBUS 2.
- Image Streaming Viewer Information Interoperable with all ESRI ArcGIS software including the new 3 D viewer ArcGIS Explorer Provides users with real time zoom and roam capabilities for exploring the SIDS data archive Is in the process of being extended further with a dynamic data download service using WCS and incorporating the metadata.