Más contenido relacionado

Similar a Quasi-experimental impact evaluation methods: an introduction(20)

Más de Center for International Forestry Research (CIFOR) (20)


Quasi-experimental impact evaluation methods: an introduction

  1. Colas Chervier, Javier G. Montoya-Zumaeta, Julia Naime, Cauê Carilho, Sandy Nofyanza 1 Quasi-experimental impact evaluation: GCS REDD+ experience
  2. The policy-relevant questions that these methods can help address And results from GCS REDD+ studies 2
  3. Examples of previous research work (1/3) 3 [Peru] In the 3 years between surveys, we observed a severe decline in forest revenue. However, by using a BACI study design and matching, we show that this decrease was not caused by the REDD+ interventions. Thus, REDD+ “did no harm” to local people, at least in terms of forest revenues (Solis et al, 2021) [Brazil] We find significant but small additional conservation effects from the implementation of the PES program. Notwithstanding, treatment effects are relatively larger in areas with higher deforestation pressure and higher potential agricultural income (Cisneros et al, 2022) [Brazil] We find that it is more effective to locate parks and payments away from each other, rather than in the same location or near each other. (Robalino et al. 2015)
  4. Examples of previous research work (2/3) previous research work (2/2) 4 [Indonesia] Contrary to the objective of the program, community titles aimed at conservation did not decrease deforestation; if anything, they tended to increase forest loss. In contrast, community titles in zones aimed at timber production decreased deforestation, albeit from higher baseline forest loss rates. (Kraus et al. 2021) [Nepal] Our results indicate that CFM has, on average, contributed to significant net reductions in both poverty and deforestation across Nepal, and that CFM increases the likelihood of win–win outcomes. We also find that the estimated reduced deforestation impacts of community forests are lower where baseline poverty levels are high, and greater where community forests are larger and have existed longer (Oldekop 2021). [Brazil] Results indicate the REDD+ project conserved an average of 7.8% to 10.3% of forest cover per household and increased the probability of improving enrollees' well-being by 27–44%. After the project ended, forest loss rebounded and perceived well-being declined – yet, importantly, past saved forest was not cleared (Carrilho et al. 2022)
  5. Examples of previous research work (3/3) previous research work (2/2) 5 [Peru]. REDD+ Ucayali and Madre de Dios REDD+ had negligible impacts on deforestation and forest degradation outcomes and negligible impacts on most wellbeing outcomes (Naime et al.,2022; Montoya-Zumaeta et al.,2022)
  6. Summary of policy-relevant questions (1/2) Directly related to the measure of impact… • What it the additional impact of an intervention on deforestation and community wellbeing incomes? • How does the impact varies depending on the context or according to the characteristics of the beneficiaries? • Is the target intervention more effective when combined with other interventions? • Is it possible to achieve win-win outcomes? • How permanent are the impacts? • Etc…Is there any leakage effect?
  7. … and beyond • Is a policy worth replicating and if so, where? • Is a given policy efficient i.e. compared to other management approaches? 7 Summary of policy-relevant questions (2/2)
  8. Introduction to quasi-experimental impact evaluation And GCS methodological approach 8
  9. Overall definition (1/2) • An assessment of if and the extent to which an intervention affects outcomes • Based on the identification a counterfactual, i.e. a control group representing what those outcomes would have been for the program participants in the absence of the intervention • Quasi-experimental because used in cases where the target area of the intervention has already been selected Without payment (non-participant) (participant) Intervention Outcomes (participant)
  10. Overall definition (2/2) • By comparing counterfactual and intervention group, we can say how many hectares of forest have been saved or by how many times the income of a group of individuals has been increased by a program, and if this result if significant 10
  11. Differences with more widespread methods • Monitoring tools • Following a number of indicators over time in program area • E.g. LTKL monitoring tool → • Different goal :does not aim to provide information about if an intervention/policy affects indicators measured • Carbon sequestration scenarios • Based on projections of historical trends in target areas • E.g. EK or any other forest carbon projects → • Biased: changes cannot be attributed to REDD+ interventions as lots of changes in the context can influence deforestation 11 ERPD - FCPF
  12. Biases associated with widespread methods and targeted by quasi-experimental methods Before/after comparison • Time-varying conditions that influence the target outcome (e.g. change in the price of agricultural products) « Simple » with/without comparison • Selection bias, there are initial differences between control group and intervention that influence the results 12 Before/after PES Control Intervention (PES)
  13. GCS REDD+ overall methodological approach 13 • 6 countries • 23 initiatives • 150 villages • 3 time periods (1 before, 2 after) • Control and target villages • 4000 households surveyed • Remote-sensing data on deforestaion and forest degradation
  14. Matching • Select a control group that looks like the participant group based on observable characteristics that can influence the outcome. • The purpose is to address the selection bias (reduces initial differences) 14
  15. Difference in Difference • Compares differences in outcomes over time between a population participating in a program and one not participating. • Control for the influence of global contemporary factors and constant differences 15 10 20 15 35 Simple comparison: 35-15 =20 Double comparison: (35-15) – (20-10) = 10 Simple comparison (contemporary effect): (35+5) – (15+5) =20
  16. Next steps: applying these methods in Indonesia
  17. Collaboration with Universitas Mulawarman • Co-development of impact evaluation studies of prominent policies in East Kalimantan • Prioritization of policies to evaluate according to provincial needs (FCPF and benefit-sharing mechanism?) • Gathering secondary and spatially-explicit datasets (e.g., PODES, tanahair, etc.)
  18. Using GCS REDD+ dataset → Sandy
  19. | | | The Center for International Forestry Research (CIFOR) and World Agroforestry (ICRAF) envision a more equitable world where forestry and landscapes enhance the environment and well-being for all. CIFOR–ICRAF are CGIAR Research Centers. Terima Kasih