Más contenido relacionado

Similar a Jeanne Pinay_2023 AGRODEP Annual Conference(20)



Jeanne Pinay_2023 AGRODEP Annual Conference

  1. Economist, FAO From Challenges to Solutions: A Comprehensive Guide for Conducting Impact Evaluations in Fragile Settings Jeanne Pinay Jeanne Pinay and Marco d’Errico
  2. #2023 AGRODEP CONFERENCE Motivation • 2022: 274 million people require humanitarian assistance and protection • 260 million people (95 percent) living in fragile areas (UN OCHA, 2022) Fragile countries have attracted increased attention from actors of the humanitarian and development world • 2021: total international humanitarian assistance = USD 31.3 billion (GHA, 2022) Enormous amounts of humanitarian aid have been injected into crisis countries Policymakers need to identify interventions that work best to better assist people in need, and more efficiently allocate funds Rigorous IE in fragile settings is necessary… … because rigorous IE in fragile settings is challenging! … but not well-developed Why?
  3. #2023 AGRODEP CONFERENCE Contribution and target audience • Based on the literature (esp. Puri et al., 2017; Bakrania et al., 2021) and new evidence from FAO case studies • Contributes to the literature on aid effectiveness and impact evaluation methods • Target audience: International and local researchers, practitioners, and donors involved in impact evaluations in fragile settings Challenges Implications on data quality and validity of impact evaluation Solutions Identify the challenges encountered Determine appropriate solutions to maintain design, data quality, and credibility of the IE • Provide guidance to researchers and local actors working on impact evaluation in fragile contexts to:
  4. #2023 AGRODEP CONFERENCE Definition of fragile settings and FAO case studies • OECD (2022): Fragility is the combination of exposure to risk and insufficient coping capacity of the state, system and/or communities to manage, absorb or mitigate those risks • Based on the assessment of the respective balance between risk and coping capacities, contexts are categorized as extremely fragile, fragile, or ‘rest of the world’
  5. #2023 AGRODEP CONFERENCE Challenges Practical • Complex settings or unpredictability • Burkina Faso, Mali, Niger, Nigeria, Somalia • Need for speed or time constraints • Burkina Faso, Gaza Strip • Multiplicity of actors • High co-variability or invalid counterfactual Ethical • Do no harm • Gaza Strip • Randomization and ethics • Myanmar Methodological • Selection bias • All FAO case studies • Information bias • South Sudan • Contamination bias • Somalia • Attrition • Somalia, Nigeria, … Local capacity • Misidentification of the treated • Pakistan • Misunderstanding on IE design • Madagascar • No HHID • Gaza Strip • No experience in tracking HHs • Remote trainings and lower degree of supervision
  6. #2023 AGRODEP CONFERENCE Implications Invalid counterfactual Measurement error Attrition
  7. #2023 AGRODEP CONFERENCE Solutions Invalid counterfactual • Quasi-experimental methods • IV, RDD, DD, Matching • Combining primary with secondary data • Somalia, Madagascar • Alternative designs for randomization • Phase-in randomization • Factorial design • Randomization at higher levels (e.g., village) Measurement error • Before fieldwork • Pilot testing • Reduce questionnaire • During fieldwork • CAPI • Mixed methods • Gaza Strip • After fieldwork • Correct measurement error • Secondary data • Nigeria Attrition • Before fieldwork • Regular assessment of security situation • CATI • Diversified C group • Oversampling • Clear communication • During fieldwork • Tracking techniques • Nigeria • After fieldwork • Attrition tests • Correct attrition (IPW, CiC, bounding)
  8. #2023 AGRODEP CONFERENCE Conclusions • Rigorous impact evaluation in fragile settings is challenging but possible, and necessary  cannot wait for the right setting to generate evidence! • Strike a balance between • Rigor of the evaluation vs. assistance of people in need • Rigor of the evaluation vs. costs • Collaboration among humanitarian agencies, donors and local actors • Agree upon a common research agenda • Flexible and adjustable budget • Combine data collection efforts Investing substantial public resources in programs whose effectiveness is unknown is unethical! (Gertler et al., 2016) • Program adaptability and flexibility • Focus time and resources on less but more reliable evidence
  9. #2023 AGRODEP CONFERENCE Need to identify innovative and low- cost techniques • research designs • statistical methods • survey methods Way forward Geospatial data Satellite data Remote-sensing data
  10. #2023 AGRODEP CONFERENCE References • Bakrania, S., Balvin, N., Daidone, S. & De Hoop, J. (2021). Impact Evaluation in Settings of Fragility and Humanitarian Emergency. Office of Research – Innocenti Discussion Paper DP-2021-02 February 2021 • Global Humanitarian Assistance. (2022) GHA Report 2022. • OECD. (2022). States of Fragility 2022. OECD Publishing. • Puri, J., Aladysheva, A., Iversen, V., Ghorpade, Y. and Brück, T. (2017). Can rigorous impact evaluations improve humanitarian assistance? Journal of Development Effectiveness, 9:4, 519-542, DOI: 10.1080/19439342.2017.1388267 • UN OCHA. (2022). Global Humanitarian Overview 2022, United Nations Office for the Coordination of Humanitarian Affairs (UN OCHA), New York,