GEOGRAPHIC SCRIPTING IN GVSIG HALFWAY BETWEEN USER AND DEVELOPER held at the University of Potsdam in 2016 - PART 5: RASTER DATA

1. GEOGRAPHIC SCRIPTING IN GVSIG
HALFWAY BETWEEN USER AND DEVELOPER
Geoinformation Research Group,
Department of Geography
University of Potsdam
21-25 November 2016
Andrea Antonello
PART 5: RASTER DATA

2. RASTER DATA
Raster data can usually be seen as 2 main types: imagery (ex. ortofoto)
and physical data (ex. elevation model).
While imagery is something you mostly use as nice background data, with
an elevation model there are tons of analyses one can do.
In this document we will mostly refer to raster as the latter case.
Loading raster data in a view is quite simple:
rasters = [
"/home/hydrologis/data/potsdam/dsm_test.asc",
"/home/hydrologis/data/potsdam/dtm_test.asc"
]
epsg = "EPSG:32632"
newview = currentProject().createView("Raster view")
newview.setProjection(getCRS(epsg))
newview.showWindow()
for raster in rasters:
loadRasterFile(raster)

3. GET RASTER METADATA
To work with raster data it is necessary to get some metadata first.
The most used are: rows and cols, boundaries, resolution, bands
dtmLayer = currentView().getLayer("dtm_test")
print "Raster information: "
print "=============================="
print "Columns:", dtmLayer.getWidth()
print "Rows:", dtmLayer.getHeight()
print "NoData value:", dtmLayer.getNoDataValue().getValue()
print "Resolution:", dtmLayer.cellSize
print "Bands count: ", dtmLayer.getBandsCount()
print "Bounds:"
print " north: ", dtmLayer.maxX
print " south: ", dtmLayer.minX
print " west: ", dtmLayer.minY
print " east: ", dtmLayer.maxY
# values can be read through the getData(band, row, column) method
print "Random value: ", dtmLayer.getData(0, 10, 10)

4. THE SIMPLEST RASTER ANALYSIS
We now have all the necessary info to loop through the raster data and
calculate max, min, average and valid cells
dtmLayer = currentView().getLayer("dtm_small")
count = 0
sum = 0
min = 10000000 # initialize to something high
max = 0 # initialize to something low
cols = int(dtmLayer.getWidth())
rows = int(dtmLayer.getHeight())
novalue = -9999.0
# now loop over rows and columns
for row in range(0, rows):
if row % 10 == 0: print row, " of ", rows, " are done"
for col in range(0, cols):
value = dtmLayer.getData(0, row, col)
if value != novalue: # active cells are only the valid ones
count += 1
sum += value
if value > max: max = value
if value < min: min = value
avg = sum/count
print "nnMax elevation: ", max
print "Min elevation: ", min
print "Average elevation: ", avg
print "Valid cells: %i of %i" % (count, cols*rows)

5. PUTTING THE RASTER MIN/MAX IN A SHAPEFILE
Let's put together what we learned until now to extract the position of
the max and min elevation of the raster and place it in a shapefile. Most of
the first part is the same as the simple min/max/avg example seen before.
In order to put the points in a shapefile we will need the positions. A
simple way to get them, is to start in the center of the first cell of the
col/row and move pixel by pixel incrementing the X/Y coordinates:
cellSize = dtmLayer.cellSize
runningY = dtmLayer.maxY - dtmLayer.cellSize / 2.0
for row in xrange(0, rows):
runningX = dtmLayer.minX + dtmLayer.cellSize / 2.0
for col in xrange(0, cols):
value = dtmLayer.getData(0, row, col)
# max/min calculation here
runningX += cellSize
runningY -= cellSize

6. PUTTING THE RASTER MIN/MAX IN A SHAPEFILE
The whole piece will look like:
dtmLayer = currentView().getLayer("dtm_small")
count = 0; sum = 0
min = 10000000; minX = 0; minY = 0
max = 0; maxX = 0; maxY = 0
cols = dtmLayer.getWidth()
rows = dtmLayer.getHeight()
novalue = -9999.0
cellSize = dtmLayer.cellSize
runningY = dtmLayer.maxY - dtmLayer.cellSize / 2.0
for row in xrange(0, rows):
runningX = dtmLayer.minX + dtmLayer.cellSize / 2.0
for col in xrange(0, cols):
value = dtmLayer.getData(0, row, col)
if value != novalue:
count += 1
sum += value
if value > max:
max = value
maxX = runningX
maxY = runningY
if value < min:
min = value
minX = runningX
minY = runningY
runningX += cellSize
runningY -= cellSize

7. PUTTING THE RASTER MIN/MAX IN A SHAPEFILE
Now that we have positions and values, we can create the shapefile:
schema = createFeatureType()
schema.append("name", "STRING", 20)
schema.append("value", "DOUBLE", 8)
schema.append("x", "DOUBLE", 8)
schema.append("y", "DOUBLE", 8)
schema.append("GEOMETRY", "GEOMETRY")
schema.get("GEOMETRY").setGeometryType(POINT, D2)
shape = createShape(schema, prefixname="min_max", CRS="EPSG:3003")
and insert the data:
shape.edit()
minPoint = createPoint2D(minX, minY)
shape.append(name='MIN', value=min, x=minX, y=minY, GEOMETRY=minPoint)
maxPoint = createPoint2D(maxX, maxY)
shape.append(name='MAX', value=max, x=maxX, y=maxY, GEOMETRY=maxPoint)
shape.commit()
currentView().addLayer(shape)

8. PUTTING THE RASTER MIN/MAX IN A SHAPEFILE
and since we are at it, let's add some style:
pointsLegend = shape.getLegend()
pointSymbol = simplePointSymbol()
pointSymbol.setSize(15);
pointSymbol.setColor(Color.BLUE);
pointSymbol.setStyle(3);
pointsLegend.setDefaultSymbol(pointSymbol)
shape.setLegend(pointsLegend)
which should then produce
something like:

9. CREATING A NEW ARC/INFO ASCII GRID
The Arc/Info ASCII Grid raster format is quite simple to understand.
The file starts with a header:
NCOLS 355 # number of columns
NROWS 361 # number of rows
XLLCORNER 1637140.0 # x coordinate of the lower left corner
YLLCORNER 5110830.0 # y coordinate of the lower left corner
CELLSIZE 10.0 # size of the cell or resolution
NODATA_VALUE -9999 # novalue placeholder
and after that, the data - space separated - are inserted row by row.
First, prepare the input and output data:
# open the new file in write mode
newRasterPath = "/home/hydrologis/data/potsdam_data/dtm_small_1400_1600.asc"
newRasterFile = open(newRasterPath, "w")
# get the start raster
dtmLayer = currentView().getLayer("dtm_small")
cols = dtmLayer.getWidth()
rows = dtmLayer.getHeight()
novalue = -9999.0

10. CREATING A NEW ASC RASTER
We have all the information to write the header:
newRasterFile.write("NCOLS " + str(cols))
newRasterFile.write("nNROWS " + str(rows))
newRasterFile.write("nXLLCORNER " + str(dtmLayer.minX))
newRasterFile.write("nYLLCORNER " + str(dtmLayer.minY))
newRasterFile.write("nCELLSIZE " + str(dtmLayer.cellSize))
newRasterFile.write("nNODATA_VALUE " + str(novalue))
Then loop through all the values and filter away the ones outside our
range, i.e. set them to novalue:
for row in xrange(0, rows):
newRasterFile.write("n")
for col in xrange(0, cols):
value = dtmLayer.getData(0, row, col)
if value != novalue and value > 1400 and value < 1600:
newRasterFile.write(str(value) + " ")
else:
newRasterFile.write(str(novalue) + " ")
newRasterFile.close()
loadRasterFile(newRasterPath)

11. CREATING A NEW ASC RASTER
The result, overlayed in greyscale on the original dtm, should look like:

12. NEIGHBOUR OPERATIONS: EXTRACT PITS
A pit is a cell in the raster that is lower in elevation than its surrounding
cells. Pits are important in hydrology, since "programmatically speaking",
the water doesn't flow out of those pits.
Let's start by creating the shapefile that will hold the pit points and put it
in editing mode:
schema = createFeatureType()
schema.append("value", "DOUBLE", 8)
schema.append("GEOMETRY", "GEOMETRY")
schema.get("GEOMETRY").setGeometryType(POINT, D2)
shape = createShape(schema, prefixname="pits", CRS="EPSG:3003")
shape.edit()
and gather the usual information about the raster:
# get necessary info from layer
dtmLayer = currentView().getLayer("dtm_small")
cols = dtmLayer.getWidth()
rows = dtmLayer.getHeight()
novalue = -9999.0

13. NEIGHBOUR OPERATIONS: EXTRACT PITS
The core part loops through the raster values and looks for cells with
higher elevation values around them:
cellSize = dtmLayer.cellSize
runningY = dtmLayer.maxY - dtmLayer.cellSize / 2.0
for row in xrange(0, rows):
runningX = dtmLayer.minX + dtmLayer.cellSize / 2.0
for col in xrange(0, cols):
if row == 0 or row == rows-1 or col == 0 or col == cols-1:
continue
v = dtmLayer.getData(0, row, col)
if v == novalue: continue
v11 = dtmLayer.getData(0, row-1, col-1)
v12 = dtmLayer.getData(0, row-1, col)
v13 = dtmLayer.getData(0, row-1, col+1)
v21 = dtmLayer.getData(0, row, col-1)
v23 = dtmLayer.getData(0, row, col+1)
v31 = dtmLayer.getData(0, row+1, col-1)
v32 = dtmLayer.getData(0, row+1, col)
v33 = dtmLayer.getData(0, row+1, col+1)

14. NEIGHBOUR OPERATIONS: EXTRACT PITS
If the cell is a pit, add it directly to the shapefile:
if v<v11 and v<v12 and v<v13 and v<v21 and v<v23
and v<v31 and v<v32 and v<v33:
pitPoint = createPoint2D(runningX, runningY)
shape.append(value=v, GEOMETRY=pitPoint)
runningX += cellSize
runningY -= cellSize
shape.commit()
currentView().addLayer(shape)
And never forget to add some style:
pointsLegend = shape.getLegend()
pointSymbol = simplePointSymbol()
pointSymbol.setSize(6);
pointSymbol.setColor(Color.RED);
pointSymbol.setStyle(0);
pointsLegend.setDefaultSymbol(pointSymbol)
shape.setLegend(pointsLegend)

15. NEIGHBOUR OPERATIONS: EXTRACT PITS
The result should look like:

16. <license>
This work is released under Creative Commons Attribution Share Alike (CC-BY-SA).
</license>
<sources>
Much of the knowledge needed to create this training material has been produced
by the sparkling knights of the
<a href="http:www.osgeo.org">Osgeo</a>,
<a href="http://tsusiatsoftware.net/">JTS</a>,
<a href="http://www.jgrasstools.org">JGrasstools</a> and
<a href="http:www.gvsig.org">gvSIG</a> communities.
Their websites are filled up with learning material that can be use to grow
knowledge beyond the boundaries of this set of tutorials.
Another essential source has been the Wikipedia project.
</sources>
<acknowledgments>
Particular thanks go to those friends that directly or indirectly helped out in
the creation and review of this series of handbooks.
Thanks to Antonio Falciano for proofreading the course and Oscar Martinez for the
documentation about gvSIG scripting.
</acknowledgments>
<footer>
This tutorial is brought to you by <a href="http:www.hydrologis.com">HydroloGIS</a>.
<footer>