One of the presentations at the Performance Measurement masterclass at Inclusive Asia Roundtable on 23-24 September 2015 in Vietnam.

1. Performance Measurement Framework

2. Shared framework of indicators
Impact Areas Guiding Question Indicator
Livelihood and Well Being
Are the basic needs of the
farmers being met?
Food Security: Access to sufficient
food
Income
Assets
Perceived Well-Being
Gender
What are women’s roles and
benefits in this crop?
Women’s participation in crop
Equitable Access to Training
Participation in Decision-Making
Environmental
Performance
Is the land well stewarded? Adoption of conservation practices
Farm Productivity
Are farmers realizing the
potential of their farm?
Adoption of good ag practices
Estimated Productivity
Crop Revenue or Net Income
Access to Services
Do farmers have access to
services?
Access to credit, training and inputs
Are farmers using these
services?
Use of credit, training, and inputs
Trading Relationships
Are farmers experiencing
good trading relationships?
TBD

3. What is a shared approach?
• A common framework of indicators and metrics to
help guide the practitioner.
• A suite of indicators to pick what’s appropriate. Not a
single set of indicators.
• Shared approach implies committing to use the same
indicators and metrics when asking the same
questions.

4. Why a shared approach?
From shared questions about farmers to shared
approach for:
• Greater efficiency and effectiveness
• Reduced burden on suppliers and farmers
• More effective community learning

5. Criteria leading to framework
• Fewest questions (indicators) that give “sufficient”
insight into livelihoods and performance.
• Affordability and scalability vs scientific robustness
• Simplicity vs nuance
• Embedding approaches for regular monitoring and
reporting

6. Characteristics of
Performance Measurement
Indicators and metrics appropriate for:
• Surveys under 30 minutes
• Minimally trained enumerators or even self reporting
• Across a wide range of supply chain types
Yields
actionable data
Cost efficient
enough to scale

7. Grew from overlap in
a theory of change many share

8. Out of the ToC grew common learning questions

9. the Learning Questions dictated the Shared Framework of indicators
Impact Areas Guiding Question Indicator
Livelihood and Well Being
Are the basic needs of the
farmers being met?
Food Security: Access to sufficient
food
Income
Assets
Perceived Well-Being
Gender
What are women’s roles and
benefits in this crop?
Women’s participation in crop
Equitable Access to Training
Participation in Decision-Making
Environmental
Performance
Is the land well stewarded? Adoption of conservation practices
Farm Productivity
Are farmers realizing the
potential of their farm?
Adoption of good ag practices
Estimated Productivity
Crop Revenue or Net Income
Access to Services
Do farmers have access to
services?
Access to credit, training and inputs
Are farmers using these
services?
Use of credit, training, and inputs
Trading Relationships
Are farmers experiencing
good trading relationships?
TBD

11. Scope of the Shared Approach Framework
Core reporting
metrics
(a subset for
reporting &
communication)
Shared PM Framework
(detailed enough to be
“actionable”) Full Measurement
Study
(will usually address
additional questions)

12. Made-to-fit
Each organization will have it’s
own purpose and goals for
measurement, but when we
share learning questions in
common, we can align on
indicators and increase the
value of our data by gathering it
in a common way.

13. M&E in the Programme Cycle
Financing and
contracting
Final Evaluation
Implementation
Scoping
Project Design M&E
Strategy/Framework
Operational Planning M&E
Plan/Matrix
Partici
pation
Monitoring and
Evaluation

14. Impact Assessment
Year 1 Year 2 Year 3 Year 4 Year 5
M&E systems
Deep
dive
Deep
dive
Year 0
Baseline
Impact
Assessment
Interventions
Goal: Evaluate impact of specific changes (interventions) so that
outcomes can be attributed to the specific interventions. Valid control
groups (counterfactuals) are needed.

15. Performance Monitoring
Year 1 Year 2 Year 3 Year 4 Year 5
Performance Monitoring
Chain wide
indexes
Goals: Assess “status” and track change over time. Outcome change
cannot be attributed to specific causes but trends can be tracked and
correlations observed.

16. Mixing methods
Year 1 Year 2 Year 3 Year 4 Year 5
(2) Performance measurement to track broad annual
change on few key indicators (short survey by
technician, 10 min)
Baseline
(1) Baseline to key issues, get
stakeholder buy in, and tailor survey
to specific context. Research + focus
groups
Interventions
(3) Focused impact
Assessment on small sample to
answer specific causation question.
Trained interviewers plus control
group

17. Example of thinking process
 Learning questions: Are the farmers consistently
food secure in our supply chain? Is food security
improving?
 Indicators: Food security
 Specific Metric: % of farmers with 2 months or
more of food insecurity
 Approach: Youth in coop run short interviews
during annual meeting using Ipod-based system.
 Survey question: Are there times in the year
where you have to reduce meal size, skip meals,
or change diet? Which times in the year?
(number of months)
What you want to know
What you ask a farmer

18. Appropriate methodology depends on
purpose (and budget!)
 Purpose: Why do you want to know? What
will you do with the results? Who is the
audience?
 Learning questions & population: What
questions are you trying to answer? About
whom?
 Indicators, Metrics, and Questions: What
are the appropriate indicators to track? What
specific survey questions are appropriate
and effective?
 Approach: How and to whom are you going
to ask the questions?
 Embedding: How could this be part of an
ongoing system?
Purpose
Learning
Questions
ToC
Approach
Indicators/
survey
Embedding

19. Of course -- depends on purpose, specific questions, and resources
Key Methodology Questions
How do we collect the data?
From whom do we collect data?
Who collects the data?
Self reporting Group settingIndirect Household
Group gathering
point
Rough
sampling
Voluntary
individual
Statistical
sampling w/control
Coop TA YouthSelf Certifier Researcher

20. Of course -- depends on purpose, specific questions, and resources
Key methodology Questions
How do we collect the data?
Who collects the data?
Self reporting Group settingIndirect Household
Group gathering
point
Rough
sampling
Voluntary
individual
Statistical
sampling w/control
Coop TA YouthSelf Certifier Researcher
household survey
for rough baseline
From whom do we collect data?

21. Of course- depends on purpose, specific questions, and resources
Key Methodology Questions
How do we collect the data?
Who collects the data?
Self reporting Group settingIndirect Household
Group gathering
point
Rough
sampling
Voluntary
individual
Statistical
sampling w/control
Coop TA YouthSelf Certifier Researcher
From whom do we collect data?

22. Of course- depends on purpose, specific questions, and resources
Key Methodology Questions
How do we collect the data?
Who collects the data?
Self reporting Group settingIndirect Household
Group gathering
point
Rough
sampling
Voluntary
individual
Statistical
sampling w/control
Coop TA YouthSelf Certifier Researcher
From whom do we collect data?