ispc15

1. Standing Panel on Impact Assessment
Doug Gollin, SPIA Chair, 4th April 2017

2. Content
• Update since last ISPC meeting (Hyderabad, Sept 2016)
• Overview of conference in Nairobi (July 2017)
• Plans for second phase of Strengthening Impact Assessment in the
CGIAR (SIAC) program

3. CGIAR Standing Panel on Impact Assessment
Doug Gollin
Chair
Karen Macours
Activity leader
Erwin Bulte
Activity leader
JV Meenakshi
Member
Bob Herdt
Member
0.5 FTE (approx.) across all 5 members
3 FTE across Secretariat staff for SPIA activities

4. SPIA Chair vacancy announcement
• I have been Chair since July 2012 (joined as a member under Derek
Byerlee in late 2011)
• Vacancy announcement now out, for successful candidate to start
as Chair by January 2018
http://ispc.cgiar.org

5. Update since September meeting
• October: Tim Kelley retires from ISPC Secretariat after
career of 27 years in CGIAR
• November: James Stevenson participates in DIME
workshop at World Bank on impact evaluation
• December: Lakshmi Krishnan leads drafting and
submission of management response to IEA-managed
evaluation of SIAC
• January: Karen Macours and Secretariat participate in
IFAD / FAO / World Bank / CGIAR workshop on
evaluating progress towards SDG 1 and 2
• February: Nancy Johnson joins Secretariat as Senior
Agricultural Research Officer
• Throughout: wrapping up and synthesizing material
from 40 distinct projects in this first phase of SIAC….

6. M A M J J A S O N D J F M A M J J A S O N D J F M A M J J A S O N D J F M A M J J A S O N D J F M A M J J A S O N D
Michigan State University
Virginia Tech / CIFOR / CIP
ICRISAT / Illinois
CIMMYT
Tufts
Paris School of Economics
IPA
ILRI
CIMMYT
CIAT
COLUMBIA
QFD
Berkeley
Yale
ICARDA
CIP
IITA
UC San Diego
IFPRI
WorldFish
TANGO International
CIAT
Virginia Tech / CIFOR
IRRI / NCSU
ICRAF / Illinois
Univ Minnesota
IFMR
NMBU
CIMMYT / ICRISAT
Nong Lam Univ / UC Santa Cruz
IFPRI / GeoPoll / IITA
ICRAF
CIMMYT
ICRISAT / Illinois
FAO (EPIC)
LSMS-ISA
LSMS-ISA
LSMS-ISA
2013 2014 2015 2016 2017

7. SIAC program by the numbers
• 40 separate projects; 33 of which were competitively
commissioned following calls for proposals
• 28 different lead institutions; many more onward partners
Universities
Michigan State Univ
Virginia Tech (x2)
Tufts
Paris School of
Economics
Columbia Univ
UC Berkeley
Yale
UC San Diego
U Minnesota
NMBU (Oslo)
Nong Lam Univ
CGIAR Centers
ICRISAT (x3)
CIMMYT (x4)
ILRI
CIAT (x2)
ICARDA
CIP
IITA
IFPRI (x2)
World Fish
IRRI
ICRAF (x2)
Other institutions
Innovations for Poverty
Actions
Qué Funciona para el
Desarollo
TANGO International
Institute for Financial
Management and
Research
FAO
World Bank (Living
Standards Measurement
Study team)

8. Nairobi conference, July 6-8th 2017
• Conference to wrap up SIAC program: “Impacts of agricultural
research: Rigorous evidence for policy”
• 6 – 8th July 2017, hosted by World Agroforestry Center, Nairobi
• Being held jointly with PIM CRP, who are organizing a number of
parallel sessions and a post-conference workshop on social science
in the CGIAR
• Presenters from across SIAC program + invited external speakers
(researchers, donors, policy-makers)
• Approx 150 participants

9. Nairobi conference, July 6-8th 2017
DAY 1, 6th July 2017: Adoption of technologies
1. Plenary papers
Featuring keynote from Alain de Janvry
2. Use of evidence in an era of “alternative facts”
(Plenary panel)
3a. Why don’t farmers know which
varieties they are growing? Insights
on African seed systems from DNA
fingerprinting
3b. PIM session
4a. Farmer behavior and the impacts
of new technologies
4b. PIM session

10. Nairobi conference, July 6-8th 2017
DAY 2, 7th July 2017: Impacts on development outcomes
5. Plenary panel on research designs for measuring development outcomes
Chris Barrett, Tavneet Suri, Cheryl Doss, Tor-Gunnar Vagen
6a. Impacts of agricultural
technologies on food security,
nutrition and health: What have we
learned?
6b. Tracing out causal pathways for
the impact of policy-oriented
research
7a. Poster session 7b. Match-making: Researchers pitch
“break-through” technologies to
social scientists
8a. From productivity increases to
aggregate, long-run impacts
8b. PIM session
9. What lessons are there for how CGIAR sets priorities?

11. Nairobi conference, July 6-8th 2017
DAY 3, Saturday 8th July 2017: Post-conference workshops
1. PIM-led workshop on social sciences in the CGIAR
2. SPIA-led workshop to wrap-up studies measuring NRM
adoption at scale

12. Nairobi conference, July 6-8th 2017
We particularly welcome biophysical scientists
• If you think all economists think alike, attend one of
their seminars.
• If you think economists are especially rude to non-
economists, attend one of their seminars.
Dani Rodrik, Economics Rules

13. Critical functions for future SPIA work
• Process of consultation over future priorities for SPIA has been going
for past 12 months
• Settling on three critical functions for SPIA that are not met
elsewhere in the system
1. Filling critical data gaps: Establish a nationally-representative
monitoring system in high-priority countries, for tracking CGIAR
development outcomes
2. Evidence of development effectiveness: Portfolio of impact
evaluation studies on topics with public goods characteristics
3. Strengthening capacity: Institutionalizing impact assessment in
CGIAR

14. 1. Data gaps
1. Fill data gaps for 4 SLO targets (1.1, 2.1, 2.3, 2.4). Especially
important for 1.1 (adoption of CGIAR technologies and practices), a
key outcome in many CRP impact pathways
2. Robust, credible. Implemented by an independent entity using state
of the art methods to minimize measurement error
3. Efficient. Common framework and methodology reduce costs and
ensure comparability across CRPs and contexts
4. Proof of concept for what is happening across the countries where
CGIAR works. Need to prioritize a subset of technologies/practices
per country.

15. 1. Data gaps
Throughout 2017, we are working with:
• CGIAR global partners (World Bank; IFAD; FAO; BMGF; DFID; USAID)
• CGIAR “country coordination” process
• Big Data and Excellence in Breeding platforms
• Respective country statistical offices
to identify best opportunities for coordinated work on data
Candidate countries we are scoping:
• Ethiopia, Tanzania, Uganda, Malawi
• Nigeria, Mali
• Bangladesh, India, Vietnam

16. 1. Data gaps
First effort to do multiple large-scale surveys of CGIAR adoption
that are:
• Not based primarily on expert opinions (unlike DIIVA, TRIVSA,
Evenson and Gollin, Dalrymple, etc)
• Not primarily administered by CGIAR centers
• Starts from installed capacity in specific countries to create
capacity in national systems
• Depends on our (rapidly improving) ability to detect technology
use through objective measurement

17. CGIAR SRF target
table indicator
Data needed Minimum method Alternative data
collection methods
100 million more
farm HHs have
adopted improved
varieties, breeds or
trees, and/or
improved
management
practices
Plant varietal data for
crops grown on farm
Self-report Sampling of leaf / grain
material for DNA
fingerprinting
List of NRM
technologies and
practices applied per
plot
Questions with visual
aid protocols
Remote sensing (some
NRM practices only)
Livestock management
practices
Questions with visual
aid protocols
Remote sensing (some
practices only)
Livestock breeds Questions with visual
aid protocols
Sampling of animal
genetic material for
DNA fingerprinting
Aquaculture
management practices
Questions with visual
aid protocols
Remote sensing
Fish breeds Questions with visual
aid protocols
Sampling of animal
genetic material for
DNA fingerprinting

18. CGIAR SRF target table
indicator
Data needed Typical method Alternative data
collection methods
Improve the rate of yield
increase for major food
staples from current
<1% to 1.2-1.5% per
year
Crop production (up to
3 main crops grown on
farm, in terms of value
of production)
Self-report for
previous season
Crop-cut and/or remote
sensing
Area of crop
production
Self-report GPS measurement
and/or remote sensing
150 million more
people, of which 50%
are women, without
deficiencies essential
micronutrients
10% reduction in
women of reproductive
age who are consuming
less than the adequate
number of food groups
Data on dietary intake
for women and
children
Dietary diversity
score
Measures to be
developed in
collaboration with A4NH

19. 2. Impact evaluation portfolio
Builds on model of competitive calls that worked well in SIAC
Phase 1
Improvements planned:
• More systematic and detailed process for identifying topics
for calls
• Channel impact evaluation investments into priority countries
• Greater coordination of impact evaluation agenda across key
partners (especially: World Bank, IFAD, FAO, and 3ie)

20. • Partnerships between universities and CGIAR centers
• Long- and short-term training
• Integrating impact considerations earlier in the research
process through improving how promising innovations are
assessed
• Supporting a CGIAR impact assessment community of practice
• CRPs and Centers retain primary responsibility for impact
assessment
• Quality control?
3. Institutionalizing impact assessment