Talk about PyWPS-4.0.0 at ISPRS 2016, Prague, related paper is published in conference proceedings.

1. PyWPS 4.0.0

2. Introduction

3. What is PyWPS
• PyWPS is an implementation of the Web Processing Service
standard from the Open Geospatial Consortium
• PyWPS is written in Python
• Started in the Spring of 2006
• Supports all available tools in Python for geospatial operations
• http://pywps.org

4. What PyWPS is NOT
• complicated
• a client
• a GUI
• a server with pre-installed processes

5. OGC Web Processing Service

6. The OGC Web Processing Service
• OGC open web standard for remote geo-spatial processing.
• Integrated with web data services: WFS, WCS.
• Three basic requests:
• GetCapabilities
• DescribeProcess
• Execute
• Three basic input/output classes:
• Literal
• Complex - for geo-spatial data and services
• BouningBox - for geo-spatial data extent

7. The OGC Web Processing Service
http://www.slideshare.net/TheodorFoerster/restful-web-processing-service

8. Essential PyWPS Functionality
• Communication bridge with WPS.
• Fetches input data referenced in Execute requests.
• Creates a container for the process instance.
• Manage processes: communication, reporting, logging.
• Data storage for final data outputs.
• Client notification, status reporting

9. History

10. 2006
• Jachym Cepicky starts working on a project to link GRASS with UMN
Mapserver.
• Scholarship funded by the German Foundation for Environment.
• September: PyWPS is presented at FOSS4G 2006.
• October: Project homepage hosted by Intevation GmbH.
• November: Version 1.0.0 is released; other developers join the
project.

11. 2007 - 2010
• Version 2.0.0 released in October of 2007
• Introduced the Process class;
• WPS plugin for OpenLayers;
• Improved stability.
• Version 3.0.0 released in November of 2008
• New implementation of the WPS 1.0.0 standard.
• JavaScript WPS client introduced in 2009 with PyWPS 3.1.0.

12. 2011
• The WPS Cookbook is published.
• A deliverable of the NETMAR project.
• Introduces the WPS standard.
• Presents PyWPS as implementation.
• Still today a valuable introduction to WPS.
• Practical examples of SOAP envelopes and Taverna
Workflow management system
• Staple GIS software in Industry and Academia.
• PyWPS featuring regularly in scientific publications.

13. 2012-2014
• 2013 FOSS4G-CEE:
• New implementation starts being discussed;
• First lines of code are written.
• 2014 WPS Workshop organised by the Luxembourg Institute of
Science and Technology (LIST):
• Broad architecture for PyWPS-4 devised;
• Move to a new Licence discussed (MIT);
• Application to OSGeo

14. 2015
• Development picks ups throughout 2015:
• Student at LIST;
• Google Summer of Code (GSoC).
• PyWPS enters the OSGeo incubation process in April of 2015.

15. 2016
• Big push from OSGeo mentorship (Tom Kralidis)
• Consolidation of documentation and websites.
• PyWPS.org goes live.
• More successful GSoC applications.
• Official release of PyWPS 4.0.0 some time this Summer ...

16. PyWPS-4

17. Motivation for PyWPS-4
• PyWPS is now a decade old.
• Python version 3 taking over.
• New bindings for other libraries (e.g PyGRASS).
• New data formats (e.g. GeoJSON, KML, TopoJSON).
• Version 2.0 of the WPS standard.
• Less restrictive licence (MIT).

18. New Technologies
• lxml
• Flask / Werkzeug
• WSGI
• Multiprocessing
• PyPy
• Jython
• OWSLib

19. Data Validation
• A four level approach:
• No validation - always considered valid.
• Simple validation - uses just the mimeType.
• Strict validation - attempts to open Complex inputs with
GDAL, comparing with mimeType.
• Very strict validation - uses an XML schema.
• Plus custom validation, already including: ESRI Shapefile, GeoJSON,
GML, GeoTIFF.

20. Process Containerising
• WPS server should run Execute requests concurrently.
• but in totally insulated fashion;
• no shared resources or data.
• PyWPS-3:
• a temporary folder is created for each Execute;
• resulting data is moved to publishing folder;
• temporary folder is deleted at execution end.
• PyWPS-4 aims at a safer approach (e.g Docker, vagrant)

21. Asynchronous Execution
• PyWPS can process various Execute requests in parallel:
• configurable number;
• plus a queue of waiting requests.
• PyWPS-4 now uses the Multiprocessing module:
• os.fork() abandoned;
• PyWPS should now run on Windows too.
• Process metadata now stored in a database (SQLite, PostgreSQL -
logging, request queueing, etc). - GSoC project
• Local database to support WPS 2.0.

22. Future Work
• New requests in WPS 2.0: Pause, Release, Delete.
• Improved security with process containerising.
• Administrative web interface.
• External services to publish outputs.
• REST API
• Support for other languages beyond English.

23. Administration interface and REST API

24. OSGeo Incubation

25. Application to OSGeo
• Motion to enter incubation approved in April of 2015.
• Process mentored by Tom Kralidis.
• PyWPS-3 to feature in the next OSGeoLive release.

26. Governance
• PyWPS now has a Project Steering Committee (PSC)
• currently composed by 5 members
• PSC sets the roadmap, technical standards, release schedule, code
review, ...
• Monthly meetings (IRC)

27. Community
• Mailing list https://lists.osgeo.org/mailman/listinfo/pywps-dev
• 35 members
• IRC #geopython at irc.freenode.net server
• Gitter https://gitter.im/PyWPS
• Attending code sprints, conferences, ...

28. Example

29. Installing
# Checkout *pywps*
git clone git@github.com:geopython/pywps.git pywps
# Checkout *pywps-demo*
git clone git@github.com:geopython/pywps-demo.git pywps-demo
# Install dependencies (*requirements.txt*)
cd pywps && pip install -r requirements-dev.txt
# Install pywps
sudo python setup.py install
# Run the demo server
cd ../pywps-demo && python demo.py

30. Folder structure
pywps
pywps-demo
+-- demo.py
+-- LICENCE.txt
+-- processes <- working folder
| +-- area.py
| +-- bboxinout.py
| `-- ...
+-- pywps.cfg <- configurations
+-- requirements.txt
+-- server.py <- run server
+-- setup.py
+-- static
+-- templates
`-- tests

31. Process structure
from pywps import Process, ComplexInput, LiteralOutput, ...
class Area(Process):
def __init__(self):
inputs = [ComplexInput('layer', 'Layer', ...
outputs = [LiteralOutput('area', 'Area', ...
super(Area, self).__init__(
self._handler,
identifier='area',
title='Process Area',
inputs=inputs,
outputs=outputs,
store_supported=True,
status_supported=True)
def _handler(self, request, response):

32. The handler method
def _handler(self, request, response):
from shapely.geometry import shape
with temp_dir() as tmp:
input_gml = request.inputs['layer'][0].file
input_geojson = os.path.join(tmp, 'input.geojson')
subprocess.check_call(['ogr2ogr', '-f', 'geojson',
str(input_geojson), input_gml])
with open(input_geojson, 'rb') as f:
data = json.loads(f.read())
for feature in data['features']:
geom = shape(feature['geometry'])
feature['area'] = geom.area
response.outputs['area'].data = [feature['area'] ...
return response

33. The PyWPS Philosophy

34. Bikes
Jachym's Luís'

35. Bikes

36. Bikes

37. Bikes

38. The PyWPS Philosophy
• Versatility: PyWPS can be whatever you want.
• Ease of use: you only need to known how to pedal.

39. Thank you !
Homepage: http://pywps.org/
Source code, download: https://github.com/geopython/pywps
Twitter: @PyWPS

40. Licence
• Presenstation licence CC-Attribution-ShareAlike
• http://creativecommons.org/licenses/by-sa/2.5/