CCAFS workshop titled "Using Climate Scenarios and Analogues for Designing Adaptation Strategies in Agriculture," 19-23 September in Kathmandu, Nepal.

1. Generating weather data for agricultural applications
September 2011

2. Why do we sometimes need synthetic daily
weather?
• Synthetic: weather records that are statistically the same as (or
similar to) historical records
• Needed to fill data gaps, for interpolation, for long-term
simulations, for short-term simulations, for help in simulating future
scenarios
• To drive models of agricultural processes: growth and
development of plants, of animals, pests & diseases, …

3. Weather generation
Which variables, and how often?
• Plant growth models
Daily: rainfall, max and min temperatures, solar
radiation (or sunshine hours)
• Livestock and ecosystem models
Weekly, monthly
• Special purpose models
< 1 day time step: wind speed, leaf wetness, etc

4. Rainfall generation
• Rainfall is a major determinant of agriculture globally
• Difficult to model well; most rainfall generators severely under-
estimate monthly and annual variance
• In risk studies, tails of the distributions (especially the lower) are
of particular importance: impacts on household incomes, food
security, …
 Need for representative rainfall generation

5. How do weather generators work?
• Very many different weather generators have been built and
applied, often using similar principles
• They use sequences of pseudo-random numbers to operate; a
sequence that is “seeded” will eventually repeat itself
Same seed  same random number sequence  same weather
sequence

6. A common method of generating rainfall
The chance of rain depends on whether it rained yesterday (a “first-
order Markov chain”):
Pi (w | w) = 1 – Pi (d | w)
Pi (w | d) = 1 – Pi (d | d)
Y esterday T oday S tate
wet 1 wet 1 1
001101000101110 … dry 0 2
dry 0 wet 1 3
4 2 3 1 dry 0 4

7. Is today wet?
Yesterday was dry: Pi (w | d) = 0.3
Generate a random number R, distributed
uniformly between 0 and 1
Y esterday T oday S tate
If R < 0.3, today is wet wet 1 wet 1 1
dry 0 2
dry 0 wet 1 3
dry 0 4

8. If today is wet, how much rain falls?
Use a two-parameter gamma
probability distribution, defined by
• a shape parameter p
• a scale (or location) parameter av

9. What about temperatures and solar
radiation?
For example, WGEN (a common generator) takes into account:
• serial correlation
• cross correlations on the same day,
and on one day and the previous day
Mean SR
Generated from monthly means
and monthly standard deviations dry
(northern hemisphere)
wet
1 365
Day of year

10. Weather generator parameters
Depending on the weather generator, a set of parameters for any
site might include long-term means for
• Monthly rainfall amounts
• Number of rain days per month
• Monthly max and min temperatures, averages and
standard deviations
• Monthly solar radiation, averages and standard deviations
Where might these come from, for a site?
• Historical daily data for several years for these variables
• Climate surfaces based on station data, with interpolation
techniques used to fill in the “grid”

11. What is the “minimum” set of climate
variables for generating daily data?
Depends on the methods used, but could include:
• Monthly rainfall amounts – from climate grids
• Number of rain days per month
• Monthly average max and min temperatures – from climate
grids, standard deviations from regressions
• Monthly solar radiation – can be estimated from
• sunshine hours, day length, and 2 empirical parameters, or
• daily maximum and minimum air temperatures, latitude,
longitude, and various empirical parameters

12. MarkSim is a tool that combines a daily
weather generator with climate surfaces of
the common variables (rainfall, tmax and
tmin), and uses climate typing to estimate
the full parameter set of the model

13. Climate surfaces in MarkSim v1
About 10,000 stations for Latin America 7,000 stations for Africa
4,500 stations for Asia Different resolutions
10’ pixel
2.5’ pixel
2.5’ pixel
10’ pixel
10’ pixel 0
Climate
270 Phase 90
Angle
180

14. A Few Pixels of METGRID for Africa: Near Limuru
-1.083 36.583 1981
54 44 84 210 177 42 18 23 24 54 118 83
17.7 18.3 18.5 17.9 16.6 15.2 14.5 14.8 16.1 17.4 17.2 17.1
13.0 13.3 12.0 9.8 9.2 10.1 10.2 10.9 12.8 12.3 10.2 11.0
-1.083 36.750 2072
52 46 97 224 182 45 18 27 26 62 139 85
17.2 18.0 18.3 17.7 16.6 15.2 14.2 14.6 16.1 17.2 16.9 16.7
13.3 14.0 12.5 9.8 9.4 9.8 10.3 10.8 12.9 12.1 10.1 10.8
-1.083 36.917 1676
47 42 101 233 181 43 21 30 29 67 148 79
19.0 19.7 20.1 19.8 19.0 17.4 16.5 16.8 18.1 19.4 18.9 18.5
13.8 15.1 13.3 10.5 10.0 10.7 11.0 10.9 13.2 12.7 10.3 11.4
-1.083 37.083 1493
36 32 96 212 137 27 15 19 18 61 147 77
19.7 20.2 21.1 21.1 20.1 18.6 17.5 18.0 19.5 20.5 20.0 19.6
14.7 16.1 14.0 11.0 10.4 11.3 11.0 11.4 13.5 13.1 10.6 12.0

15. 140 Northvil e
120
100
80
Rainfall mm
60
40
20
0
Jan Mar May Jul Sep Nov
140 Southvil e
120
100
80
Rainfall mm
60
40
20
0
Jan Mar May Jul Sep Nov

16. Northville Southville
150 JAN 150 JAN
DEC FEB DEC FEB
100 100
NOV MAR NOV MAR
50 50
OCT APR OCT APR
SEP MAY SEP MAY
AUG JUN AUG JUN
JUL JUL
Rotation of climate record based on 12-point Fourier transform
Convert 12 monthly values to a series of sine & cosine functions and subtract the first phase angle

17. Northville Southville
150 JAN 150 AUG
DEC FEB JUL SEP
100 100
NOV MAR JUN OCT
50 50
OCT APR MAY NOV
SEP MAY APR DEC
AUG JUN MAR JAN
JUL FEB

18. Standardisation of Climate Normals
Standard climate normals
tillaber 14.180 1.430 209
0. 1. 1. 3. 18. 55. 119. 181. 73. 13. 0. 0.
24.7 27.5 30.8 33.2 34.0 32.1 29.2 27.5 28.9 30.4 28.5 25.3
15.8 16.8 16.8 15.1 13.8 12.5 10.8 9.8 11.0 14.5 16.1 16.1
Standardised climate normals – rotated through the first phase angle of the Fourier-
transformed monthly rainfall data
tillaber 14.180 1.430 209 3.4554
170. 131. 19. 10. 0. 6. 0. 5. 0. 12. 36. 89.
27.9 28.0 30.1 29.6 26.5 24.5 26.1 29.6 32.4 34.0 33.1 30.3
10.0 10.2 13.1 15.8 16.1 15.9 16.2 17.2 15.8 14.3 13.1 11.5

19. MarkSim the rainfall generator
• Based on a third-order Markov process: probability of
rain on any day is dependent on occurrence of rain
on the three previous days
• To model observed rainfall variances in parts of the tropics
and subtropics, some of the parameters of the rainfall
model are randomly sampled
• The model has been fitted to more than 9000 data sets
world-wide and has been extensively tested
• May model parameters offer some insight into the nature of
long-term climate change?

20. Probability of a Wet Day
The probability of a day being wet is
-1
P(W/D1D2D3) = Φ (bi + ai-1d1 + ai-2d2 + ai-3d3)
Where:
-1
Φ is the inverse of the probit function
bi is the monthly probit of a wet day following 3 consecutive dry days
am are binary coefficients for rain (1) or no rain (0) on day m
dm are lag constants.
-1
The probability of a wet day following three dry days is Φ (bi)
-1
The probability of a wet day following three wet days is Φ (bi + d1 + d2 + d3)

21. 1.0
0.8
0.6
Cumulative Probability
0.4
0.2
0.0
-3 -2 -1 0 1 2 3
Normal Probability (Probit)

22. Sampling Model Parameters
For each year of generated rainfall required, the baseline probits are randomly
sampled from a 12-dimensional normal distribution:
*
b i = si RNi + bi, i=1,12
Where:
*
bi is the sampled value of bi, the baseline probability of rain.
si is the standard deviation of bI (from the fitting of the model).
RNi is a random normal number.
The sampled baseline monthly probabilities are correlated using the correlation
matrix of raindays per month.

23. Baseline Rainfall Probit Resampling: Tillabery, Niger
Mean Rep 1 Rep 2
(000) 001 010 011 100 101 110 111
+d1 +d2 +d1+d2 +d3 +d1+d3 +d2+d3 +d1+d2+d3
______________________________________________________________________________
J -4.76 -4.99 -5.32 :
F -2.75 -2.74 -2.01 :
M -2.49 -2.31 -2.26 :
A -2.03 -2.71 -1.66 :
M -1.31 -0.93 -1.17 :
J -0.78 -1.25 -1.00 -0.90 –0.87 –0.77 –0.91 –0.81 –0.78 –0.68
J -0.38 -0.35 -0.79 :
A -0.25 -0.39 -0.59 :
S -0.72 -0.90 -0.94 :
O -1.56 -1.83 -2.14 :
N -2.68 -2.61 -2.91 :
D -2.91 -3.03 -2.33 :
d1 0.10
d2 0.13
d3 0.09
______________________________________________________________________________

24. GAMMA DISTRIBUTIONS
SHAPE (P) AND LOCATION (AV)
0.3
P = 0.5
AV = 5.0
0.2 (Mean = 10.0)
0.1
0.0
P = 1.5
AV = 5.0
0.2 (Mean = 3.3)
0.1
0.0
PROBABILITY DENSITY
P = 5.5
AV = 5.0
0.4
(Mean = 0.9)
0.3
0.2
0.1
0.0
0 3 6 9 12 15 18 21 24 27 30
RAINFALL (mm)

25. Historical Rainfall Data: Mangalore, western India
----1988 MANGALORE/BAJPE IN 12.917 74.883 102
JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC
1 0 0 0 0 7 0 170 10 0 90 0 0
2 0 0 0 0 2 330 350 140 260 510 0 0
3 0 0 0 20 0 120 150 140 550 350 0 0
4 0 0 0 0 0 20 30 170 80 2 0 0
5 0 0 0 0 0 20 540 500 6 0 0 0
6 0 0 0 0 0 860 80 430 0 230 0 0
7 0 0 0 0 0 140 5 10 0 90 0 0
8 0 0 0 0 3 700 400 20 0 0 10 0
9 0 0 0 0 01010 70 50 540 0 0 0
10 0 0 0 0 0 520 0 430 250 0 0 0
11 0 0 0 8 0 450 350 90 1 0 0 0
12 0 0 0 10 3 710 400 220 6 0 0 0
13 0 0 0 0 0 2301000 170 0 0 0 0
14 0 0 0 0 0 5 930 10 3 0 0 0
15 0 0 0 0 0 330 270 250 110 0 0 0
16 0 0 0 30 0 130 60 140 4 0 0 4
17 0 0 0 40 0 830 690 760 290 0 0 0
18 0 0 0 0 20 18013201010 350 0 0 0
19 0 0 0 4 0 50 290 250 160 0 0 0
20 0 0 0 0 0 270 300 60 200 0 0 0
21 0 0 0 0 20 730 620 50 310 0 0 0
22 0 0 0 0 1 60 550 130 190 0 0 0
23 0 0 0 0 0 250 680 150 50 0 0 0
24 0 0 0 0 0 170 30 0 10 0 0 0
25 0 0 0 0 01150 90 3 50 10 0 0
26 0 0 0 0 90 750 380 4 370 70 0 0
27 0 0 0 0 0 820 30 20 330 0 0 0
28 0 0 0 0 20 100 10 60 50 0 0 0
29 0 0 0 0 0 250 20 50 1 0 0 0
30 0 0 0 8 210 210 0 110 0 0 0
31 0 0 70 240 20 0 0

26. Historical Rainfall Data: Antofagasta, Chile
----1988 ANTOFAGASTA/CERRO CL-23.433 -70.433 120
JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC
1 0 0 0 0 0 0 0 0 0 0 0 0
2 0 0 0 0 0 0 0 0 0 0 0 0
3 0 0 0 0 0 0 2 0 0 0 0 0
4 0 0 0 0 0 0 * 0 0 0 0 0
5 0 0 0 0 0 0 0 0 0 0 0 0
6 0 0 0 0 0 0 0 0 0 0 0 0
7 0 0 0 0 0 0 0 0 0 0 0 0
8 0 0 0 0 0 0 0 0 0 0 0 0
9 0 0 0 0 0 0 0 0 0 0 0 0
10 0 0 0 0 0 0 0 0 0 0 0 0
11 0 0 0 0 0 0 0 0 0 0 0 0
12 0 0 0 0 0 0 0 0 0 0 0 0
13 0 0 0 0 0 0 0 0 0 0 0 0
14 0 0 0 0 0 0 0 0 0 0 0 0
15 0 0 0 0 0 0 0 0 0 0 0 0
16 0 0 0 0 0 0 0 0 0 0 0 0
17 0 0 0 0 0 0 0 0 0 0 0 0
18 0 0 0 0 0 0 0 0 0 0 0 0
19 0 0 0 0 0 0 0 0 0 0 0 0
20 0 0 0 0 0 0 0 0 0 0 0 0
21 0 0 0 0 0 0 0 0 0 0 0 0
22 0 0 0 0 0 0 0 0 0 0 0 0
23 0 0 0 0 0 0 0 0 0 0 0 0
24 0 0 0 0 0 0 0 0 0 0 0 0
25 0 0 0 0 0 0 0 0 0 0 0 0
26 0 0 0 0 0 0 0 0 0 0 0 0
27 0 0 0 0 0 0 0 0 0 0 0 0
28 0 0 0 0 0 0 0 0 0 0 0 0
29 0 0 0 0 0 0 0 0 0 0 0 0
30 0 0 0 0 0 0 0 0 0 0 0
31 0 0 0 0 0 0 0

27. (1)
200
GUATEMALA
TOTAL 1175 mm
(n=38)
100
RAINFALL (mm)
0
PALMIRA
(2) TOTAL 993 mm
200 (n=52)
100
RAINFALL (mm)
0
TILLABERY
(3) TOTAL 483 mm
200 (n=35)
100
RAINFALL (mm)
0
J F M A M J J A S O N D

28. ANNUAL VARIANCE OF RAINFALL FOR THREE SITES
50
HISTORICAL
WGEN
40
MarkSim
TOMM
30
ANNUAL RAINFALL
VARIANCE X 1000
20
10
0
GUATEMALA PALMIRA, TILLABERY,
CITY COLOMBIA NIGER

29. Combining climate surfaces and the weather generator to provide
daily weather data that are characteristic of any location
• MarkSim v1 on CD-ROM, data for Latin America, Africa, Asia (2002)
• The original MarkSim calibration data station map

30. Weather typing
~10,000 weather stations with > 12 years daily rainfall data
Lat, long, elevation
Long-term monthly temp Estimate rainfall model parameters
Monthly diurnal temp range
Monthly rainfall
Cluster analysis in 36-D space, giving ~700 climate clusters
Calculate regression coefficients for rainfall
model
parameters for each cluster
Store in cluster
files

31. Generating daily weather
Pick a location: lat, long (elevation)
Read the climate surface to find long-term means
Read cluster coverage to find cluster number
Read cluster files for regression coefficients
Reconstitute rainfall model parameters
Generate daily rainfall; generate daily temps from long-term means
Generate daily solar radiation from temps
Application

32. Where do the MarkSim v1 model
parameters come from?
From climate grids, or from the user directly:
• Monthly rainfall amounts
• Monthly average max and min temperatures
From the climate typing clusters:
• Number of rain days per month
• Monthly correlation matrix of raindays per month
• Baseline normal probabilities (probits) of a wet day following
three dry days and the “lag parameters”
Derived parameters:
• Monthly solar radiation

33. Applications
Weather data to drive crop, livestock, household, ecosystem
models
providing information on
• system performance with respect to technological, climate,
policy changes
• constraint identification and characterisation
• identification of possible intervention points
• answer “what-if” questions relating to risk, sustainability,
trade-offs

34. Simulated Cereal Yields at Three Sites
1.0
Til abery
(mil et) Guatemala City
0.8
(maize)
0.6
Palmira
CUMULATIVE PROBABILITY
0.4 (maize)
0.2
Historical weather
"Interpolated" weather
0.0
0 1 2 3 4 5 6 7
GRAIN YIELD (t/ha)

35. Versions of MarkSim
Version 1.0 GIS-based CD edition (2002)
• Getting old
• Some known problems with the code and the climate grids
• Support?
Version 1.5 MarkSimGCM (2011)
• Google-Earth based web application that generates data in
relation to future climates from GCMs as well as data for
“current conditions”
• Many v1 bugs fixed, data produced in DSSAT weather file
format

36. Versions of MarkSim
Version 1.7 MarkSimGCM batch mode (late 2011)
• Run MarkSimGCM as a DLL and an EXE (depending on OS)
• Can then be run as part of the DSSAT v4.5 system or of any
other model system
Version 2 MarkSim2012
• Use an expanded gauge dataset (~54,000 stations) to refit
MarkSim globally
• Will incorporate the features of MarkSimGCM and AR5
climate data
Version 3 onwards
• Current research ideas: improve temperature simulation;
data filling; utilising satellite data in places where gauge data
are sparse; MarkSim spatial