Presented by Colas Chervier (CIFOR-ICRAF) at "Science and Policy Dialogue IV: Taking Local Context Into Account in REDD+ Policies Implementation", Bogor, Indonesia, on 14 Dec 2022

1. Colas Chervier, Javier G. Montoya-Zumaeta, Julia Naime, Cauê Carilho, Sandy Nofyanza
1
Quasi-experimental impact evaluation:
GCS REDD+ experience

2. The policy-relevant questions that
these methods can help address
And results from GCS REDD+ studies
2

3. Examples of previous research work (1/3)
3
[Peru] In the 3 years between surveys, we observed a severe decline in forest revenue.
However, by using a BACI study design and matching, we show that this decrease was
not caused by the REDD+ interventions. Thus, REDD+ “did no harm” to local people, at
least in terms of forest revenues (Solis et al, 2021)
[Brazil] We find significant but small additional conservation effects from the
implementation of the PES program. Notwithstanding, treatment effects are relatively
larger in areas with higher deforestation pressure and higher potential agricultural
income (Cisneros et al, 2022)
[Brazil] We find that it is more effective to locate parks and payments away from each
other, rather than in the same location or near each other. (Robalino et al. 2015)

4. Examples of previous research work (2/3) previous
research work (2/2)
4
[Indonesia] Contrary to the objective of the program, community titles aimed at
conservation did not decrease deforestation; if anything, they tended to increase forest
loss. In contrast, community titles in zones aimed at timber production decreased
deforestation, albeit from higher baseline forest loss rates. (Kraus et al. 2021)
[Nepal] Our results indicate that CFM has, on average, contributed to significant net
reductions in both poverty and deforestation across Nepal, and that CFM increases the
likelihood of win–win outcomes. We also find that the estimated reduced deforestation
impacts of community forests are lower where baseline poverty levels are high, and
greater where community forests are larger and have existed longer (Oldekop 2021).
[Brazil] Results indicate the REDD+ project conserved an average of 7.8% to 10.3% of forest
cover per household and increased the probability of improving enrollees' well-being by
27–44%. After the project ended, forest loss rebounded and perceived well-being
declined – yet, importantly, past saved forest was not cleared (Carrilho et al. 2022)

5. Examples of previous research work (3/3) previous
research work (2/2)
5
[Peru]. REDD+ Ucayali and Madre de Dios REDD+ had negligible impacts on
deforestation and forest degradation outcomes and negligible impacts on most
wellbeing outcomes (Naime et al.,2022; Montoya-Zumaeta et al.,2022)
https://doi.org/10.17528/cifor/008599

6. Summary of policy-relevant questions (1/2)
Directly related to the measure of impact…
• What it the additional impact of an intervention on deforestation and
community wellbeing incomes?
• How does the impact varies depending on the context or according to the
characteristics of the beneficiaries?
• Is the target intervention more effective when combined with other
interventions?
• Is it possible to achieve win-win outcomes?
• How permanent are the impacts?
• Etc…Is there any leakage effect?

7. … and beyond
• Is a policy worth replicating and if so, where?
• Is a given policy efficient i.e. compared to other management
approaches?
7
Summary of policy-relevant questions (2/2)

8. Introduction to quasi-experimental
impact evaluation
And GCS methodological approach
8

9. Overall definition (1/2)
• An assessment of if and the extent to
which an intervention affects outcomes
• Based on the identification a
counterfactual, i.e. a control group
representing what those outcomes
would have been for the program
participants in the absence of the
intervention
• Quasi-experimental because used in
cases where the target area of the
intervention has already been selected
Without payment
(non-participant)
(participant)
Intervention Outcomes
(participant)

10. Overall definition (2/2)
• By comparing counterfactual and
intervention group, we can say how
many hectares of forest have been saved
or by how many times the income of a
group of individuals has been increased
by a program, and if this result if
significant 10

11. Differences with more widespread methods
• Monitoring tools
• Following a number of indicators over time in
program area
• E.g. LTKL monitoring tool →
• Different goal :does not aim to provide
information about if an intervention/policy
affects indicators measured
• Carbon sequestration scenarios
• Based on projections of historical trends in target
areas
• E.g. EK or any other forest carbon projects →
• Biased: changes cannot be attributed to REDD+
interventions as lots of changes in the context can
influence deforestation
11
ERPD - FCPF

12. Biases associated with widespread methods and
targeted by quasi-experimental methods
Before/after comparison
• Time-varying conditions that influence the
target outcome (e.g. change in the price of
agricultural products)
« Simple » with/without comparison
• Selection bias, there are initial differences
between control group and intervention that
influence the results
12
Before/after PES
Control
Intervention (PES)

13. GCS REDD+ overall methodological approach
13
• 6 countries
• 23 initiatives
• 150 villages
• 3 time periods
(1 before, 2 after)
• Control and target
villages
• 4000 households
surveyed
• Remote-sensing
data on
deforestaion and
forest degradation

14. Matching
• Select a control group that looks
like the participant group based
on observable characteristics that
can influence the outcome.
• The purpose is to address the
selection bias (reduces initial
differences)
14

15. Difference in Difference
• Compares differences in outcomes
over time between a population
participating in a program and one
not participating.
• Control for the influence of global
contemporary factors and constant
differences
15
10
20
15
35
Simple comparison:
35-15 =20
Double comparison:
(35-15) – (20-10) = 10
Simple comparison
(contemporary effect):
(35+5) – (15+5) =20

16. Next steps: applying these methods
in Indonesia

17. Collaboration with Universitas Mulawarman
• Co-development of impact
evaluation studies of
prominent policies in East
Kalimantan
• Prioritization of policies to
evaluate according to
provincial needs (FCPF and
benefit-sharing mechanism?)
• Gathering secondary and
spatially-explicit datasets
(e.g., PODES, tanahair, etc.)

18. Using GCS REDD+ dataset
→ Sandy

19. cifor.org | worldagroforestry.org | globallandscapesforum.org | resilientlandscapes.org
The Center for International Forestry Research (CIFOR) and World Agroforestry (ICRAF) envision a more equitable world where forestry and
landscapes enhance the environment and well-being for all. CIFOR–ICRAF are CGIAR Research Centers.
cifor.org/gcs
Terima
Kasih