Presented by Oswaldo Carrillo from CIFOR, at Online Workshop Capacity Building on the IPCC 2013 Wetlands Supplement, FREL Diagnostic and Uncertainty Analysis, 20-22 September 2021
3. Concluding Remarks
The Annex to decision 12 / CP.17 establishes that the
information included in the FREL should be guided by
the most recent IPCC guidelines.
According to IPCC (2006), uncertainty
estimates are an essential element of a
complete GHG inventory.
Emissions/removals estimates are based on:
(1) conceptualization,
(2) models and
(3) input data and assumptions.
Each of these three can be a source of
uncertainty
1. Context
4. Lack of knowledge of the true value
of a variable that can be described as a
probability density function (PDF)
characterising the range and likelihood
of possible values (IPCC, 2006).
What Uncertainty is?
How to combine Uncertainties?
Once the uncertainties in AD, EF or emissions
for a category have been determined, they may
be combined to provide uncertainty estimates
for the entire inventory (IPCC, 2006)
1. Context
EF
100 t C/ha
30%
of U
(70 t C/ha - 130 t C/ha)
5. Concluding Remarks
It is good practice to account, as far as
possible, for all causes of U (IPCC, 2006)
Why it is important to quantify the U of the FREL?
What is the acceptable level of U for the FREL ?
Quantification of U in the FREL is a requirement
in the methodological frameworks of several base
payment iniciatives:
REDD Early
Movers
Programme
Same as FCPF and
BCF but different
Reversal Buffer
No
threshold
for
UNFCCC
Criteria Reversal Buffer
≤ 15% 0%
> 15% and ≤ 30% 4%
> 30% and ≤ 60% 8%
> 60% and ≤ 100% 12%
> 100% 15%
Criteria Score
If U > 50 % 0
30 > U <= 50% 1
If U <= 30% 2
6. Quiz 1
1. Why it is important to quantify uncertainties of the FREL?
https://www.menti.com/58gqxbk81u
The voting code 1770 4053
• It is good practice to account, as far as possible, for all
sources of uncertainties (IPCC, 2006)
• is a requirement in the methodological frameworks of
several base payment initiatives
8. The Annex to decision 12 / CP.17 establishes that
the information included in the FREL should be
guided by the most recent IPCC guidelines and
be:
• transparent,
• consistent,
• comparable,
• complete and,
• accurate.
• Accuracy means that emission and
removal estimates should be
accurate in the sense that they are
systematically neither over nor
under true emissions or removals,
as far as can be judged, and
• that uncertainties are reduced as
far as practicable.
• Appropriate methodologies should
be used, in accordance with the
2006 IPCC Guidelines (decision 12 /
CP.17)
2. IPCC Uncertainties
Concepts
9. Concluding Remarks
2.2 Basis for Uncertainty Analysis (IPCC, 2006):
The estimation of
emissions should
prevent bias
(avoiding the use of
incorrect
conceptualizations,
models, inputs and
assumptions)
Once biases are
corrected, the
uncertainty
analysis can then
focus on
quantification of
the random errors
with respect to the
mean estimate
Once the
uncertainties have
been correctly
determined, they can
be combined to
obtain the
uncertainties of total
emissions. There are
two methods:
• Method 1 uses IPCC
equations,
• Method 2 uses the
Monte Carlo
technique
Prevent bias Quantification of U Combination of U
1. 2. 3.
10. Accuracy: Agreement between the true value and
the average of repeated estimates of a variable
2.3 Basic Terminology
Bias: Lack of accuracy
Precision: Agreement among repeated
measurements of the same variable. Better
precision means less random error. Precision is
independent of accuracy.
11. Concluding Remarks
2.4 Bias
Bias can occur because of :
• imperfections in conceptualisation, models, measurement techniques,
• failure to capture all relevant processes involved or
• the available data are not representative of all real-world situations, or
• of instrument error.
Examples of bias in AD
To estimate de AD and prevent bias, it is
necessary to estimate unbiased areas
using reference data (sample plots)
To prevent bias in EF it necessary to
use the correct statistical estimator
according to the sampling design of
the NFI
R𝑘 =
σ𝑖=1
n𝑘
y𝑖𝑘
σ𝑖=1
n𝑘
𝑎ik
ҧ
𝑥𝑗´ =
ҧ
𝑥𝑖 𝑤𝑖𝑗
𝑤
• Simple average
• Ratio estimator
• Weighted estimator
Examples of bias in EF
Mapped AD
(Bias AD)
Mapped AD +Accuracy A.
(Unbiased AD)
Bias in AD
ҧ
𝑥 =
𝑥𝑖
𝑛
12. 2.5 Uncertainties: concepts
Uncertainty:
Lack of knowledge of the true value of a variable
that can be described as a probability density
function (PDF) characterising the range and
likelihood of possible values.
Causes of U:
• Lack of
completeness
• Lack of data
• Lack of
representativen
ess of data
• Statistical
random
sampling error
• Measurement
error
• Missing data
Symmetric uncertainty
of ±30% relative to the
mean
Asymmetric uncertainty
of -50% to +100% relative
to the mean
13. Concluding Remarks
2.5 Uncertainties: AD
According to Chapter 3 of Vol. 4 of 2006 IPCC Guidelines:
• Uncertainties associated with the approaches used to
representing land use area should be quantified and reduced as
far as practicable
• Land-use area uncertainty estimates are required as an input to
estimate overall uncertainties
• In Approuch 3 “SPATIALLY-EXPLICIT LAND-USE CONVERSION
DATA” the amount of uncertainty can be estimated more
accurately because errors are mapped and can be tested
against independent data/field checked
25. Concluding Remarks
The are several paper where the estimation of unbiased
estimators of AD and its uncertainties is explained
2.5 Uncertainties: AD
26. According to Chave, there are several sources of
uncertainties in the estimations of EF
2.5 Uncertainties: EF
27. Concluding Remarks
According to Chave,
there are several
sources of uncertainties
in the estimations of EF
DBH=30cm
U= 10%
IC: 27-33
Random DBH=32
Bimass (30 cm)=100 kg
U=40%
IC: 60-140
Random Biomass: 130
Meassurement
error
Model error
Confidence Interval
2.5 Uncertainties: EF
28. 2.6 Combination of Uncertainties
Once the uncertainties of the different sources for a category have been
correctly determined, they can be combined to obtain the uncertainties of
the emissions.
According to the IPCC (2006), there are two methods to combine them:
Method 1 uses simple error propagation equations, while Method 2 uses
the Monte Carlo technique or similar
Class/Com
ponent
Emission
Factor
Uncertainty
of EF (UEF)
AD
Uncertainty
of AD (UAD)
Emission
(at component level)
Uncertainty of E (UE)
A EF1A UEF1A AD1A UAD1A E1A=EF1A*AD1A
B EF1B UEF1B AD1B UAD1B E1B=EF1B*AD1B
C EF1C UEF1C AD1C UADF1C E1C=EF1C*AD1C
E1=E1A+E1B+E1C
Total emission / Propagated uncertainty of
Transition 1
Transition 1 (FL-OU)
1 = 1
2
1
2
1 = 1
2
1
2
1 = 1
2
1
2
1 =
× × ×
Method 1: simple error propagation equations
29. Concluding Remarks
The Monte Carlo analysis is suitable
for (IPCC, 2006) :
• A detailed assessment, category
by category, of uncertainty,
• in cases where uncertainties are
large, distribution is not normal,
• there are correlations between
some of the sets of activities, AD,
EF, or both
• it is a good practice to use this
analysis instead of Method 1
Furthermore: for BPR initiatives is
mandatory to combine of U using
MC simulation
2.6 Combination of Uncertainties
Method 2: Monte Carlo simulation
30. Quiz 2
1. What are the three steps to implement the Uncertainty
Analysis?
https://www.menti.com/58gqxbk81u
The voting code 1770 4053
• Prevent bias,
• Quantify uncertainties and
• Combine uncertainties
32. 3.1 Example of M1:
Inputs:
• AD and EF from FOLU sector of the Indonesia 2nd BUR
• Uncertanties of AD and EF from de FREL 2016
Example of combining uncertainties using Method 1 in
Sumatera for the period 2016-2017
Class AD
(Ha)
U of
AD
EF
(t CO2e/Ha)
U of
EF
Emissions
(t CO2e)
U of
Emissions
(Emissions x UEmissions)2
Primary dry land
forest
14,647 12 % 463 3 % 6,786,224 (122+32)1/2=
12 %
(6,786,224 x 12)2=
7.16E+15
Secondary dry
land forest
93,599 12 % 314 4 % 29,415,945 (122+42)1/2=
13 %
(29,415,945 x 13)2=
1.36E+17
Primary
mangrove forest
1,182 12 % 455 9 % 538,101 (122+92)1/2=
15 %
(538,101 x 15)2=
6.53E+13
Primary swamp
forest
893 12 % 381 11 % 340,294 (122+112)1/2=
16 %
(340,294 x 16)2=
3.02E+13
Secondary
mangrove forest
8,214 12 % 348 12 % 2,858,136 (122+122)1/2=
17 %
(2,858,136 x 17)2=
1.78E+15
Secondary
swamp forest
39,001 12 % 261 5 % 10,185,146 (122+52)1/2=
13 %
(10,185,146 x 13)2=
7.16E+16
Sum of
Emissions
U of Total
Emissions
Som of
(Emissions x UEmissions)2
50,123,846 405,085,514
/50,123,846 =
8 %
(7.16E+15 + 1.36E + 6.53E+13 +
3.02E+13 + 1.78E+15 + 7.16E+16)1/2=
405,085,514
33. 3.2 Example of Monte Carlo simulation: Multiplication
1,040ha 1,323ha
2.5th
percentile
97.5th
percentile
-12% +12%
1,182ha
Red area acumulate 95% of the probability
414 496
2.5th
percentile
97.5th
percentile
-9% +9%
455
Red area acumulate 95% of the probability
Random Number
Normal Distribution
AD
ID iterate AD Simulated EF Simulated Emision Simulated
Random Number
Normal Distribution
EF
AD EF
34. 1,040ha 1,323ha
1,231
1,182ha
414 496
455
ID iterate AD Simulated EF Simulated Emision Simulated
1,231
1 428 526, 421
Random Number
Normal Distribution
AD Random Number
Normal Distribution
EF
428
3.2 Example of Monte Carlo simulation: Multiplication
AD EF
35. 1,040ha 1,323ha
1,231
1,182ha
414 496
455
ID iterate AD Simulated EF Simulated Emision Simulated
1,231
1 428 526, 421
Random Number
Normal Distribution
AD Random Number
Normal Distribution
EF
428 454
1,169
1,169
2 454 531, 019
3.2 Example of Monte Carlo simulation: Multiplication
AD EF
36. 1,040ha 1,323ha
1,084 1,182ha
414 496
455
ID iterate AD Simulated EF Simulated Emision Simulated
1,231
1 428 526, 421
Random Number
Normal Distribution
AD Random Number
Normal Distribution
EF
450
454
1,169
1,169
2 454 531, 019
1,084
3 450 488, 053
3.2 Example of Monte Carlo simulation: Multiplication
AD EF
37. 1,040ha 1,323ha
1,084 1,182ha
414 496
455
ID iterate AD Simulated EF Simulated Emision Simulated
1,231
1 428 526, 421
Random Number
Normal Distribution
AD Random Number
Normal Distribution
EF
450
1,169
2 454 531, 019
1,084
3 450 488, 053
4
5
6
7
8
9
10
.
.
.
.
.
.
.
.
.
.
.
.
.
1,231
10,000 433 532, 413
.
.
.
.
1,231
433
3.2 Example of Monte Carlo simulation: Multiplication
AD EF
38. 1,040ha 1,323ha
1,182ha
414 496
455
Random Number
Normal Distribution
AD Random Number
Normal Distribution
EF
ID iterate AD Simulated EF Simulated Emision Simulated
1,231
1 428 526, 421
1,169
2 454 531, 019
1,084
3 450 488, 053
4
5
6
7
8
9
10
.
.
.
.
.
.
.
.
.
.
.
.
.
1,231
10,000 433 532, 413
.
.
.
.
3.2 Example of Monte Carlo simulation: Multiplication
AD EF
39. 1,040ha 1,323ha
1,182ha
414 496
455
ID iterate AD Simulated EF Simulated Emision Simulated
1,231
1 428 526, 421
1,169
2 454 531, 019
1,084
3 450 488, 053
4
5
6
7
8
9
10
.
.
.
.
.
.
.
.
.
.
.
.
.
1,231
10,000 433 532, 413
.
.
.
.
3.2 Example of Monte Carlo simulation: Multiplication
AD EF
Percentile
2.5%
619,971
Percentile
97.5%
460,416
Average:
537,788
CI = Per 97.5-Per 2.5
CI = 460,416 - 619,971 = 159,556
U= ((0.5 x IC)/Average) x 100
U= ((0.5 x 159,556)/537,788) x 100
U= 15%
40. 3.2 Example of Monte Carlo simulation: Sum
AD EF
14,647 463
12 3
AD EF
93,599 314
12 4
AD EF
1,182 455
12 9
AD EF
893 381
12 11
AD EF
8,214 348
12 12
AD EF
39,001 261
12 5
Mean
Uncertainty (%)
ID
iterate
Primary dry
land forest
Secondary dry
land forest
Primary mangrove
forest
Primary swamp
forest
Secondary mangrove
forest
Secondary swamp
forest
Primary dry land
forest
Secondary dry land
forest
Primary mangrove
forest
Primary swamp
forest
Secondary
mangrove forest
Secondary swamp
forest
AD
sim
EF
sim
EM
AD
sim
EF
sim
EM
AD
sim
EF
sim
EM
AD
sim
EF
sim
EM
AD
sim
EF
sim
EM
AD
sim
EF
sim
EM
TOTAL
EMISSION
13,655 475
1 104,488 311 1,231 428 808 347 8,279 331 36,539 268
13,655 475 104,488 311 1,231 428 808 347 8,279 331 36,539 268
6.480 32.487 0.526 0.280 2.744 9.800 52.317
41. AD EF
14,647 463
12 3
AD EF
93,599 314
12 4
AD EF
1,182 455
12 9
AD EF
893 381
12 11
AD EF
8,214 348
12 12
AD EF
39,001 261
12 5
Mean
Uncertainty (%)
ID
iterate
Primary dry
land forest
Secondary dry
land forest
Primary mangrove
forest
Primary swamp
forest
Secondary mangrove
forest
Secondary swamp
forest
Primary dry land
forest
Secondary dry land
forest
Primary mangrove
forest
Primary swamp
forest
Secondary
mangrove forest
Secondary swamp
forest
AD
sim
EF
sim
EM
AD
sim
EF
sim
EM
AD
sim
EF
sim
EM
AD
sim
EF
sim
EM
AD
sim
EF
sim
EM
AD
sim
EF
sim
EM
TOTAL
EMISSION
13,655 475
1 104,488 311 1,231 428 808 347 8,279 331 36,539 268
13,655 475 104,488 311 1,231 428 808 347 8,279 331 36,539 268
32.487 0.526 0.280 2.744 9.800 52.317
6.480
3.2 Example of Monte Carlo simulation: Sum
42. AD EF
14,647 463
12 3
AD EF
93,599 314
12 4
AD EF
1,182 455
12 9
AD EF
893 381
12 11
AD EF
8,214 348
12 12
AD EF
39,001 261
12 5
Mean
Uncertainty (%)
ID
iterate
Primary dry
land forest
Secondary dry
land forest
Primary mangrove
forest
Primary swamp
forest
Secondary mangrove
forest
Secondary swamp
forest
Primary dry land
forest
Secondary dry land
forest
Primary mangrove
forest
Primary swamp
forest
Secondary
mangrove forest
Secondary swamp
forest
AD
sim
EF
sim
EM
AD
sim
EF
sim
EM
AD
sim
EF
sim
EM
AD
sim
EF
sim
EM
AD
sim
EF
sim
EM
AD
sim
EF
sim
EM
TOTAL
EMISSION
13,655 475
1 104,488 311 1,231 428 808 347 8,279 331 36,539 268
32.487 0.526 0.280 2.744 9.800 52.317
6.480
13,698 467
2 94,076 314 1,169 454 818 404 7,810 342 38,382 261
29.584 0.531 0.330 2.674 10.008 49.527
6.400
13,698 467 94,076 314 1,169 454 818 404 7,810 342 38,382 261
3.2 Example of Monte Carlo simulation: Sum
43. AD EF
14,647 463
12 3
AD EF
93,599 314
12 4
AD EF
1,182 455
12 9
AD EF
893 381
12 11
AD EF
8,214 348
12 12
AD EF
39,001 261
12 5
Mean
Uncertainty (%)
ID
iterate
Primary dry
land forest
Secondary dry
land forest
Primary mangrove
forest
Primary swamp
forest
Secondary mangrove
forest
Secondary swamp
forest
Primary dry land
forest
Secondary dry land
forest
Primary mangrove
forest
Primary swamp
forest
Secondary
mangrove forest
Secondary swamp
forest
AD
sim
EF
sim
EM
AD
sim
EF
sim
EM
AD
sim
EF
sim
EM
AD
sim
EF
sim
EM
AD
sim
EF
sim
EM
AD
sim
EF
sim
EM
TOTAL
EMISSION
13,655 475
1 104,488 311 1,231 428 808 347 8,279 331 36,539 268
32.487 0.526 0.280 2.744 9.800 52.317
6.480
13,698 467 94,076 314 1,169 454 818 404 7,810 342 38,382 261
29.584 0.531 0.330 2.674 10.008 49.527
6.400
13,864 461 95,622 311 1,084 450 808 388 8,181 345 40,896 264
13,864 461
3 95,622 311 1,084 450 808 388 8,181 345 40,896 264
29.713 0.488 0.313 2.822 10.810 50.536
6.390
2
3.2 Example of Monte Carlo simulation: Sum
44. AD EF
14,647 463
12 3
AD EF
93,599 314
12 4
AD EF
1,182 455
12 9
AD EF
893 381
12 11
AD EF
8,214 348
12 12
AD EF
39,001 261
12 5
Mean
Uncertainty (%)
ID
iterate
Primary dry
land forest
Secondary dry
land forest
Primary mangrove
forest
Primary swamp
forest
Secondary mangrove
forest
Secondary swamp
forest
Primary dry land
forest
Secondary dry land
forest
Primary mangrove
forest
Primary swamp
forest
Secondary
mangrove forest
Secondary swamp
forest
AD
sim
EF
sim
EM
AD
sim
EF
sim
EM
AD
sim
EF
sim
EM
AD
sim
EF
sim
EM
AD
sim
EF
sim
EM
AD
sim
EF
sim
EM
TOTAL
EMISSION
13,655 475
1 104,488 311 1,231 428 808 347 8,279 331 36,539 268
32.487 0.526 0.280 2.744 9.800 52.317
6.480
13,698 467 94,076 314 1,169 454 818 404 7,810 342 38,382 261
29.584 0.531 0.330 2.674 10.008 49.527
6.400
13,864 461 95,622 311 1,084 450 808 388 8,181 345 40,896 264
13,864 461 95,622 311 1,084 450 808 388 8,181 345 40,896 264
29.713 0.488 0.313 2.822 10.810 50.536
6.390
2
3
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
3.2 Example of Monte Carlo simulation: Sum
45. AD EF
14,647 463
12 3
AD EF
93,599 314
12 4
AD EF
1,182 455
12 9
AD EF
893 381
12 11
AD EF
8,214 348
12 12
AD EF
39,001 261
12 5
Mean
Uncertainty (%)
ID
iterate
Primary dry
land forest
Secondary dry
land forest
Primary mangrove
forest
Primary swamp
forest
Secondary mangrove
forest
Secondary swamp
forest
Primary dry land
forest
Secondary dry land
forest
Primary mangrove
forest
Primary swamp
forest
Secondary
mangrove forest
Secondary swamp
forest
AD
sim
EF
sim
EM
AD
sim
EF
sim
EM
AD
sim
EF
sim
EM
AD
sim
EF
sim
EM
AD
sim
EF
sim
EM
AD
sim
EF
sim
EM
TOTAL
EMISSION
13,655 475
1 104,488 311 1,231 428 808 347 8,279 331 36,539 268
32.487 0.526 0.280 2.744 9.800 52.317
6.480
13,698 467 94,076 314 1,169 454 818 404 7,810 342 38,382 261
29.584 0.531 0.330 2.674 10.008 49.527
6.400
15,765 462 104,007 312 1,231 433 882 353 8,328 322 37,370 263
13,864 461 95,622 311 1,084 450 808 388 8,181 345 40,896 264
29.713 0.488 0.313 2.822 10.810 50.536
6.390
2
3
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
15,765 462
10,000 104,007 312 1,231 433 882 353 8,328 322 37,370 263
32.449 0.532 0.311 2.685 9.833 53.094
7.284
3.2 Example of Monte Carlo simulation: Sum
46. AD EF
14,647 463
12 3
AD EF
93,599 314
12 4
AD EF
1,182 455
12 9
AD EF
893 381
12 11
AD EF
8,214 348
12 12
AD EF
39,001 261
12 5
Mean
Uncertainty (%)
Primary dry land
forest
Secondary dry land
forest
Primary mangrove
forest
Primary swamp
forest
Secondary
mangrove forest
Secondary swamp
forest
15,765 462 104,007 312 1,231 433 882 353 8,328 322 37,370 263
3.2 Example of Monte Carlo simulation: Sum
ID
iterate
Primary dry
land forest
Secondary dry
land forest
Primary mangrove
forest
Primary swamp
forest
Secondary mangrove
forest
Secondary swamp
forest
AD
sim
EF
sim
EM
AD
sim
EF
sim
EM
AD
sim
EF
sim
EM
AD
sim
EF
sim
EM
AD
sim
EF
sim
EM
AD
sim
EF
sim
EM
TOTAL
EMISSION
13,655 475
1 104,488 311 1,231 428 808 347 8,279 331 36,539 268
32.487 0.526 0.280 2.744 9.800 52.317
6.480
13,698 467 94,076 314 1,169 454 818 404 7,810 342 38,382 261
29.584 0.531 0.330 2.674 10.008 49.527
6.400
13,864 461 95,622 311 1,084 450 808 388 8,181 345 40,896 264
29.713 0.488 0.313 2.822 10.810 50.536
6.390
2
3
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
15,765 462
10,000 104,007 312 1,231 433 882 353 8,328 322 37,370 263
32.449 0.532 0.311 2.685 9.833 53.094
7.284
Percentile
2.5%
46,124,368
Percentile
97.5%
54,149,192
Average:
50,133,801
CI = Per 97.5-Per 2.5
CI = 54,149,192 - 46,124,368
CI= 8,024,824
U= ((0.5 x IC)/Average) x 100
U= ((0.5 x 8,024,824)/ 50,133,801) x 100
U= 8%
49. Concluding Remarks
3.5 What does it take to run MC
simulation for FREL 2020?
In summary to run MC simulation it is necessary to have:
• Unbiased mean of AD
• PDF of Unbiased mean of AD
• Uncertainties of AD
• Unbiased mean of EF
• PDF of Unbiased mean of EF
• Uncertainties of EF
per
Forest
Type
&
Island
per
REDD+ Activity:
• Deforestation
• Degradation
• Peat
Descomposition
• Conv to FL
per
Period
Carbon
Pool
per