Explains the features of the Meteo I/O library dedicated to treat meteo data

1. MeteoIO
A meteo data I/O library

2. Before MeteoIO...
• Each application doing its IO
– Insufficient robustness
– IO functions spread out all over the code
– Input data very strictly formatted, several
steps in preparing the data
– Hard to adapt to new protocols/formats
→IO a source of weakness...
2

3. MeteoIO: goals
• Remove all IO functions out of the
physics engine
• Make IO easy to use
• Make IO robust
• Make IO flexible
• Greatly simplify data preparation
• Allow unmonitored, automatic operation
• Allow end user to fine tune his IO...
3

4. MeteoIO: ini files
[Parameters]
DEMSRC = ARC
DEMFILE = ../input/surface-grids/zwischberg.dem
LANDUSESRC = GRASS
LANDUSEFILE = ../input/surface-grids/zwischberg.lus plugins
METEOSRC = A3D
METEOPATH = ../input/meteo
SPECIALPTSSRC = A3D
SPECIALPTSFILE = ../input/surface-grids/zwischberg.pts
OUTPUT = ARC
OUTPATH = ../output/
4

5. MeteoIO: primary data structures
• MeteoData: • StationData:
– Date object – Longitude
– Ta – Latitude
– Iswr – Altitude
– Vw – Northing (metric grid)
– Rh – Easting (metric grid)
– Lwr – Name
– Nswc – Supports equality comparison
– Ts0
– P
– ...
– Supports equality comparison
5

6. MeteoIO: example of use
6

7. What is it doing?
• Parsing io.ini
• Loading plugins supporting requested data sources
• Reading data (across files, network, protocols and
supporting comments, different EOL, etc)
• Converts stations' coordinates if necessary
• Filtering data, marking invalid data as “no data”
• Conversion of units to SI
7

8. Example: output
[i] meteoio/A3DIO.cc:86: Loading dynamic plugins:
[i] Number of 2D meteo stations: 15
---------- Station: 1 / 16
Name:
Air Temperature: 266.25 This station has no name
---------- Station: 2 / 16
Name: ANET-ZER
Air Temperature: 266.45
---------- Station: 3 / 16
Name: ANET-VIS
Air Temperature: 268.75
(etc)
8

9. Data sources
• Currently supported:
– Meteo data: Alpine3D's Ascii, Boschung's
XML, Gsn, Imis Oracle database, GeoTop
– Grids: ARC, GRASS
9

10. Writing a new IO plugin
• Provide implementation of some or all of:
ReadMeteoData (const Date_IO& dateStart, const Date_IO& dateEnd,
std::vector< std::vector<MeteoData> >& vecMeteo, std::vector<
std::vector<StationData> >& vecStation, const unsigned int&
stationindex=IOUtils::npos)
Read2DGrid (Grid2DObject& grid_out, const string& parameter="")
ReadDEM (DEMObject& dem_out)
ReadLanduse (Grid2DObject& landuse_out)
ReadAssimilationData (const Date_IO& date_in, Grid2DObject& da_out)
ReadSpecialPoints (CSpecialPTSArray& pts)
Write2DGrid (const Grid2DObject& grid_in, const string& name="")
10

11. Filters
• Available:
– Min, max
– Max observed changed
– No observed change
• To be added:
– Median Average Deviation
• Special processing
– Resampling
– accumulation
• Easy to add new filters 11

12. MeteoIO: example of use II
12

13. What does it change?
• IO are now buffered -> efficient on slow
access
• Automatic resampling for missing data
– Using linear interpolation between two
neighbors
– To be added: cyclic interpolation, etc
13

14. Userland addition
In the user's code: add a function checking
that the input requirements are met
(ie: nb of lwr, nb of ta, etc)
14

15. Data structures/methods
• Array, Array2D, Array3D
– Using vectors for a safe and efficient
implementation
• Grid2DObject, Grid3DObject
– Geolocalized array
• DEMObject
– Computes slope, aspect, normal's
components (Corripio, Hicks, etc)
15

16. MeteoIO: example of use III
16

17. MeteoIO: example of use IV
17

18. TA, 0 to 16° RH, 62% to 90%
Gotthard, 2008­10­01 12:00, 19 stations 18

19. Spatial interpolations
• Available methods:
– Std pressure
– Constant value
– Constant with lapse rate
– IDW
– IDW with lapse rate
– For Rh, Td IDWK with lapse rate and back to Rh
– Terrain influenced VW and DW
• One method defined for only 1 data input, one for
multiple data input (fallback)
• Regression:
– Linear regression for the moment, removes the worst data point if the
regression is too poor
19

20. Spatial interpolations 2
Planned additions:
– Physical wind interpolation (quick Bernoulli
resolution?)
– More regression models (dynamically
adjusted to the data)
– Smarter choice of which method to use
(dynamic, including data quality)
20

21. Geographic projections
• Conversion to/from WGS84 from any
coordinate system
– Through own implementation (CH1903, ...)
– Through proj4 library
• Conversion to/from local grid
MapProj local_proj(lat_ref, long_ref);
local_proj.convert_from_WGS84(latitude, longitude, easting,
northing);
21

22. MeteoIO: on the side
• Date handling (Date_IO class)
• Config files handling (ConfigReader
class)
• Data storage (Arrays & grids classes,
sparse matrix (coming))
• Ported to Pop-C++ for parallel execution
• Application specific IOs classes
22

23. Thank you!
Mathias Bavay (SLF)
Thomas Egger (Egger Consulting)
Laurent Winkler (HES-SO)
Florian Hof (SLF)
Moustapha Mbengue (SLF)
Gael Rosset (HES-SO)
Julien Jeanneret (HES-SO)
Nora Helbig (SLF)
23