The document provides an overview of a briefing on impact investment from Next Generation Consultants. Some key points:
1) The briefing discusses the need for an impact investment index for Africa that takes into account the complexities of development contexts on the continent. Existing global models of impact measurement are not always applicable.
2) The proposed Impact Investment Index aims to create a shared performance measurement system for social investment and community development organizations to improve coordination, reduce costs, and better assess collective impact.
3) Impact assessments should distinguish between measuring performance, outcomes, and long-term impacts. The ultimate goal is to understand the tangible and intangible effects of investments and determine what changes can be attributed to interventions.
3. Our Context
Started in 2009 – global benchmarking – 40 models currently
being applied internationally
Conclusion:
Africa needs its own solution – applicable to both the funding
and development sectors
Practitioners need capacity, support and knowledge
Practitioners don’t need complex methodologies or approaches
requiring specialist skills, licenced software or specific hardware
or expensive solutions
The industry needs a transparent, comparable, and flexible
solution that contextualises and take into consideration the
complexities, relationships and fundamental development
principles of our specific development context
4. Our Objective
The purpose of Impact Investment Index is to create a
shared performance measurement system to be utilized
by all organisations in the social investment and
community development sector
The current system lacks coordination, leading to added
expense, limited learning, and inadequate ability to assess
shared value and collective impact
5. Why the pressure to measure?
The debate on impact and return on investment are playing out
in three arenas:
In private foundations and corporate CSI divisions
Aiming to be more strategic about their philanthropy, grant making and
social/community investments
In nonprofit organisations in response to pressures from corporates,
foundations and government
To be more accountable for the resources received and program
outcomes expected
Among international development organisations such as bilateral
government agencies and intermediary organisations
Seeking to improve development effectiveness and lessen dependency
on grant/development aid
6. Why now?
Funders are increasingly asking for demonstrable results –
to understand the difference they make, directly and
indirectly
This trend is accelerating, as the development and in
particular the funding sector is increasingly looking to pay by
results – to learn from what they, and those they fund, do
It is not just donors that care about impact. In a age of
competition, transparency, recessionary economies,
there is growing competition for resources
7. Three primary applications of
Impact Assessments
Prospective
Looking forward to determine whether or not the projected costs
and benefits indicate a favorable investment
Ongoing
Testing assumptions and projections along the way in order to
aid course correction
Retrospective
Looking back to determine whether or not it was a favorable
investment given the costs incurred, and therefore to inform
future decisions
8. Variety of purposes – our school of
thought
One can and should use impact data to make funding
allocation decisions across program areas
You can compare programs once you get in the sector of health, but
you cannot compare health vs. arts vs. education vs. sport.
One can and should use impact data to make funding
decisions within program areas
It is not about building a unifying measurement across domains, but to
build a conceptual framework for understanding the biggest impact
across a Rand value unit. So it is not about comparing health to
education to sport or the arts, but to determine which program yields
the highest return for the most effective use of resources
9. The conundrum
Funders and non-profits often use the words “evaluation”
and “impact” loosely, stretching these terms to include any
type of report on the use of funds or the results they achieve
Practitioners should distinguish between measuring
performance (monitoring inputs, activities, and outputs),
measuring outcomes (near-term results), and evaluating
impact (long-term changes that can be attributed to the
investor’s activities) as well as return on investment
(benefits accrued to the investor) as a result of funds and
other resources invested
10. What is impact
“Measure of the tangible and intangible effects (consequences)
of one thing's or entity's action or influence upon another. An
impact evaluation/assessment assesses changes in the wellbeing of individuals, households, communities or firms that can
be attributed to a particular project, program or policy. The
central impact evaluation question is what would have
happened to those receiving the intervention if they had not
in fact received the program. ”
For us this means: The broad or longer-term effects of a
project or organisation’s work (also referred to as the difference
it makes). This can include effects on people who are direct
users/beneficiaries of a project or organisation’s work, effects
on those who are not direct users/indirect beneficiaries, or
effects on a wider field/aspect such as government policy,
processes, systems, infrastructure or support systems
11. What did we want to achieve?
To provide evidence
To demonstrate performance
To prove accountability
To show program/ investment
effectiveness
To demonstrate value
To empower and capacitate
communities and funders
Ultimately - to alleviate,
eliminate and eradicate
poverty
12. What we have achieved
Assessed R1 billion worth of investment but also considered
all types of input resources from money to hours, products
and services, infrastructure and skills and volunteerism
Across both socio economic and enterprise development
sectors
Including 15 focus areas
400 programmes – from flagship to donations, once off to 3-5
year programs, sponsorships and cause related marketing,
social and business enterprises
Across 15 dimensions of impact and 25 dimensions of return
Developed a library of more than 500 indicators
13. But more than that
Impact across the value chain:
Outcomes of partnerships, relationships and applied
resources (to be more sustainable and effective)
Outcomes of the initiative (policy change, organisations
working more effectively together reducing/alleviating/
eradicating poverty)
Outcomes at the community level (are we moving the needle
against the strategic objectives/mandates indicators?)
Returns for the donor/funder (are we getting value, recognition,
licence to operate, etc.)
14. Guiding Principles of our work
Impact means impact
The goal is to understand what changes as a result of the investment from
funders in communities as beneficiaries and recipients of interventions
The impact is shared
The goal is to understand who is impacted along the impact value chain –
including funders , intermediaries and beneficiaries
Impact includes and involve all stakeholders
Analysis must be comprehensive. Instead of cherry picking something that’s
working and leaving out what is not, the analysis should include all aspects
of impact and those impacted
Results must be transparent
Companies should report to their investors, and investors should aggregate
and report results. What is left out should be stated. Assumptions and
sources should be stated. It should be possible for a third party to
replicate the analysis based on the documentation of it and get the
same result.
Context matters
It is harder to create a stable job in a rural area than in a city. The qualitative
and quantitative context should be provided to inform the impact as well
as an understanding of how much of the problem may exist or remain.
15. Challenges remain
There are two main challenges to measuring impact
One: how to do it well
Two: how to do it cheap
Getting both to happen can be done in either of two ways:
Force it, by making it a requirement
Or attract it, by creating an environment in which the parties
whose efforts are necessary to make it happen want it
16. The breakthrough – a Comparative
Performance System
1
2
3
• Shared measurement system
• Provides a menu of common indicators and a common platform to report on
different outcomes and indicators
• Comparative performance system
• Involves asking participants to report on the same indicators, using the same
process and measure to compare performance
• Adaptive learning system
• Supports ongoing collaboration and learning among organisations to align
efforts and goals, and to measure common outcomes and indicators
The Outcome: The Impact Investment Index
17. Our process for impact assessment
• Share and act on
learning and results
• Seek to improve the
investment and
development practice
• Understanding the
difference/change the
funder wanted to make.
Who the intermediaries
were that was supported
and who beneficiaries
was that was affected
4
Review
3
Assess
• Identify and calculate
the difference that was
made and assess how
and why it was made
1
Plan
2
Do
• Design the assessment
• Conduct the
assessment
• Collect the information
needed about impact
18. Impact Value Chain - Our Focus
Hierarchy
Inputs
Activities and
Outputs
The activities whose
effects are to be
measured.
Goods and services
generated by the
use of inputs (shortterm)
Outputs and
Outcomes
The results from
activities.
Expected changes
in access, usage,
behaviour or
performance
(medium-term)
Impacts
Returns
Definition
Resources invested
(e.g. money, skills,
hours,
infrastructure,
services, products,
etc.)
Ultimate (long-term)
effect of the
intervention on a
key dimension of
the development
(e.g. living
standards) (longterm)
Direct or indirect
business
benefits/ value
generated by
activities/
programs/
interventions
Quantitative
Indicators
Value of investment,
number of hours,
number of books,
number of
stakeholders
Number of schools,
number of
volunteer, number
of teachers, jobs,
income, etc.
% change –
access to health
services,
education,
government
infrastructure, pass
rates or
sustainable jobs
Students finding
employment, people
with skills finding
employed due to
increased health
status
Number of
graduates hired
by company, %
change in
grievance,
complaints
received by
company
Qualitative
indicators
Stakeholders
satisfaction with the
programme and
measurable
difference in their
lives, environment,
context
Perceptions of
Beneficiaries
schools, educators, reporting benefits
learners, community and application of
skills, education,
improved health
Quality of links to
local employment
opportunities,
perceptions of
improved socio
economic status or
opportunities
Changes in
community /
customer/
employee
perceptions
attributable
directly/ indirectly
to CSI
19. The Impact Investment Index - III
Inputs
Outputs/Outcomes
Impacts
• How
• Cash, time, in kind,
management costs
• Why
• Community Relations,
community investments,
commercial initiatives
• Where
• Geographic Location
• Who
• Stakeholders Affected,
Impacted
• What
• Focus Areas, Investment
Portfolios
• Community Benefits
• Qualitative
• Quantitative
• Dimensions of impact
(15)
• Community Impacts
• Business Impacts/Return
• Business Benefits
• Qualitative
• Quantitative
• Dimensions of Impact
and Return (25)
• Shared Value
• +500 Indicators
22. Dimensions of Impact – FOR WHOM?
How do we calculate?
1. We count each and every stakeholder group
2. We count each and every impact
3. We distinguish between community and business impact
4. Get to a figure: X:Y
25. Some examples of impact (1)
Greater output and
productivity
The economic value
of increased revenue
and reduced costs.
The increase in local
economic output
could be estimated
with multiplier
analysis
Employees of
Suppliers
Accumulation of
human capital,
empowerment and self
confidence
The cost of external
skills courses could
be used. The
increase in wages
resulting from greater
levels of human
capital. Willingness to
pay for courses
Wider Local
Community
Increased welfare –
i.e. lower
unemployment,
improved health,
education
The value of reduction
in public expenditure
on health care
services and
unemployment
benefits
Local Suppliers
ED Supplier
Development Program
Supplier capacity
building and access to
banking services
26. Some examples of impact (2)
Local businesses
Greater output and productivity
The economic value of
increased revenue and reduced
costs. The increase in local
economic output could be
estimated with multiplier
analysis
Local Students
Improved quality of education,
better grades
Increased wages resulting from
improved education levels
Local Households
Increased communications, and
quality of life
Monetary value of savings
Rural electrification of village
Increased publicity
Investor/Donor
Value of increased publicity
Increased employee morale,
participation, skills development
Value of retained employees
27. Some examples of impact (3)
Farmers and their
families
Greater output and
productivity,
income and
empowerment
The economic
value of increased
revenue and
reduced costs.
Local Authorities
Increased tax
revenue
The increase of
local economic
output could be
estimated with
multiplier analysis
The value of
increased income
tax and export
duties
The value of
increased savings
Training of local
farmers
Local Consumers
and Communities
Improved quality
and quantity of
food
The value of
increased
consumption
The value of
leveraged
resources
Attraction of new
resources/donors
Intermediary
Increased income,
increased
employment
Value of salaries
Value of publicity
The value of
Increased quality
of life, in particular
health which lead
to increased
productivity
32. Testing Expectations
Focus Area
Ranking
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
Bottom 10
Education
Education
Community Development
Economic Development
Economic Development
Community Development
Education
Environment
Economic Development
Health
Community Development
Community Development
Community Development
Infrastructure
Infrastructure
Other
Community Development
Community Development
Disaster Relief
Safety and Security
Number of Responses
91
62
56
52
46
44
42
40
36
35
33
29
25
16
11
11
10
8
5
4
Top 10
Community Investment Areas
Education
Skills Development
Youth
Job Creation
Entrepreneurs
Children
Bursaries
Environment
Business Development
Health
Physically Challenged
Women
The Aged
Housing
Infrastructure
Other
Culture
Arts
Emergencies and Disasters
Safety and Security
33. Comparing and Benchmarking
Across Industry
Across Focus Areas
Across Perceptions
Including local and global benchmarks
Vision, Mission, Strategy and Policy,
guidelines, frameworks
Strategic intent, objectives, mandate
Focus Areas
Grant making criteria
Beneficiaries
Intermediaries
Operational alignment and integration
Governance and compliance
Reporting structure
Budget and resources
Monitoring and evaluation
Employee Volunteerism
Marketing, communication and
awareness
Sponsorship, donations and cause
related marketing
Partnerships
Recognition, Awards, perceptions
Future focus, GAP and SWOT analysis
36. Co-Development of impact
assessment guidelines
Section One:
Project Name:
Basic information:
Project Budget:
Section Two:
Section Three:
Stakeholders who
benefited or were
impacted by the
program
Stakeholder 1
Stakeholder 2
Stakeholder 3
Stakeholder 4
Stakeholder 5
Section Four:
Specific aspects of
Impact:
Project Description and
Specific Outcomes:
In your opinion – what do
you think Nedbank gained/
benefitted by funding this
program?
In other words what was
Nedbank’s return on
investment from funding
this program?
What do you think about
or is your opinion or
impressions of Nedbank
as a funder?
What do you think
Nedbank can do in future
to be a better funder?
Do you think the Nedbank
Foundation engage in a
respectful, honest,
approachable way?
Perceptions of Nedbank:
Project Owner:
Organisation
Project Manager:
Contact Details:
Do you know what the
Nedbank Foundation
Corporate Social
Investment strategy is and
do you think it is
applicable in the South
African context?
Do you have any other
comments or feedback or
suggestions for the
Nedbank Foundation?
Qualitative Impact
Qualitative Impact
Qualitative Impact
Qualitative Impact
Qualitative Impact
Economic Impact
Environmental Impact
Social Impact
Short Term Impact
Positive Impact
Intended Impact
Medium Term Impact
Negative Impact
Unintended Impact
Quantitative Impact
Long Term Impact
Combined Impact
Any other Impact
Section Five:
Lessons learnt, insights,
additional comments,
feedback, suggestions
Section Six:
Consultants Comments
Section Seven:
Scores:
SG:
NF:
AOI:
EC:
EN:
SO:
S:
M:
L:
P:
N:
C:
I:
UI:
OI:
T
37. Section One:
Project Name:
Basic information:
Leliebloem House Child
and Youth Care Centre
Project Budget:
Project Owner:
Organisation
Leliebloem House Child
and Youth Care Centre
Transparency in assessment
Project Manager:
R5,6m pa
Francisco G Cornelius
Contact Details:
Section Three:
Stakeholders who
benefited or were
impacted by the
program
Stakeholder 1
Leliebloem House
Child and Youth Care
Centre
Qualitative Impact
Qualitative Impact
Basic care and protection,
provided. The children’s
basic needs are provided
for.
84 children receive basic
social, health, educational
and developmental
services.
Less dependent on other
support (government)
Leverage funds of other
donors
2
Section Four:
Specific aspects of
Impact:
Qualitative Impact
Jobs provided to staff
2
021-6964947/ 0827719233
Qualitative Impact
Qualitative
Impact
Children cared for –
behaviour and motivation
changes – basic quality of
life improved
2
Economic Impact
It was a top up to the
overall budget we need to
provide these services.
1
Intended Impact
Section Five:
Lessons learnt,
insights, additional
comments, feedback,
suggestions
Section Six:
Consultants
Comments
Section Seven:
Scores:
Project
Services to Vulnerable Children in
Description and Residential Child Care.
Specific
Outcomes:
Offering various therapeutic
programmes to both the children
and their families that eventually
allows the children to be reunified
with family or community.
Environmental Impact
Yes so that we can render Unintended Impact
the services to the children.
1
Operational expenses
leveraged
1
Marketing and
publicity
exposure
1
Quantitative Impact
84 Children
Number of staff
Number of Jobs
None evident
0
Social Impact
Media exposure and
recognition as an
organisation with good
financial practices.
1
Any other
Impact
Value of funds leveraged / generated
4
It gave these children a second
chance they may not have received
under normal circumstances.
1
Support and leverage government
funds/ resources & support work of
government – providing basic
services to communities
1
We have a reputation of not saying no to opportunities. We pride ourselves in staying up to date with training for all staff and trying to do staff development programs 2 a
year, if the funding permits. We have dedicated staff that could leave any time because of the poor salary structures or lack of funding, but they stay on and work in the
life-space of these vulnerable children on a daily basis.
In our opinion – some consideration should be given to assisting organisations to leverage funds better – i.e. capacity building (training and other assistance) for
organisations to use funds not only for operational expenses but also to raise additional funds. Some organisations will never be sustainable, but similarly organisations
should not be overly dependent on a single organisation as dependencies are created and it may be considered unsustainable development.
SG: 1
EC: 1
S: 1
P: 1
I: 1
T:25
BOE: 1
EN: 0
M:1
N: 1
UI: 1
QL: 8
SO: 1
L: 1
C: 1
OI: 1
ON: 4
48. What our clients say - investors
Improved financial analysis and reporting as well as management
capacity and operational efficiency
Improved operational efficiency and information management
processes and systems
Improved board governance and oversight, understanding,
knowledge and capacity
Improved M&E; reporting; communication; stakeholder
relationships and partnerships
Improved operational processes from application requirements; due
diligence processes; reporting requirements; responsiveness;
knowledge and expertise of staff
Improved access to further investment
49. What our clients say –
intermediaries and beneficiaries
Intermediaries
We feel comfortable with the transparency of the process
The process have added value to our own work – especially M&E
and reporting practices
The processes have increased our effectiveness and own
performance; increased our learning and knowledge; built internal
capacity; and increased our credibility
We believe we were assured independently by someone who can
verify our claims – it validated our own beliefs
Beneficiaries
We had an opportunity to talk without being judged – we could be
honest
We learnt to document our own work and the contribution we
made
We feel we are being trusted, being heard and someone asks
our opinion
We had an opportunity to share and learn
50. What we have learnt (1)
One size of evaluation does not fit all
Funders should take extra time in their planning to learn which evaluation
techniques will work with indigenous populations and specific
communities.
Trying to define and measure empirical changes is difficult and
“a slippery process,”
Understanding, defining, qualifying and quantifying long-term social
change is an incremental effort and on-going process.
Although evaluations are significant for their organisational
development, requiring intermediaries to perform them is a challenge.
A suggested solution is to consider evaluation results as one – but
not the only – source of information and to couple it with knowledge,
experience, strategy, and context to obtain a fuller picture.
51. What we have learnt (2)
Impact assessments describes three relationships:
between evaluation and the funder’s approach;
between evaluation and the funder’s strategy;
and between the intermediary evaluation and the beneficiary evaluation.
Grantmakers may have to change their funding approach to
better accommodate intermediaries’ capacity to conduct evaluations,
or provide funding to help them develop it
Funders need to be clear about their overall goals and how
individual investments fit within that model.
Intermediary and funder evaluations should be linked:
Both partners face the same issues of inadequate resources,
coordination and expertise for conducting evaluations.
52. What we have learnt (3)
ANY resources CAN be measured – books, wheelchairs,
buildings, time – cash and non-cash
The same project can deliver varied results for different
funders
How and on what you spend the money (inside the program)
has a direct influence on the impact and return
The strategy and focus areas has to clearly define the
return and impact required - upfront
Sustainability has to be clearly defined for exit and
completion
Indicators have to be developed, agreed, and documented
as part of the contractual phase
Internal monitoring and external evaluation processes has to be
established and adhered too - Impact Assessment does not
replace evaluation and monitoring
53. What we have learnt (4)
You have to consider the impact of the impact assessment on
so many levels
As a result of our work we can now categorically state that most
programs:
Have only short term impact
Those that have medium term impact are not necessarily
sustainable
The long term impact is mostly social only as opposed to
economic impact which really contributes to poverty alleviation
and eradication!
It is possible to determine impact and return
The real value lies in independent, verifiable, assurance of social
investment expenditure, program results, outcomes and impact
54. What I have learnt (5)
There is so much return for business – we just need to prove it
Growth
Return on
Capital
Risk
Management
Management
Quality
New products
Price premium
/ market share
Regulatory risk
Leadership
New markets
Capital
efficiency
Reputational
risk
Employee
development
New customers
Human
efficiency
Licence to
operate
Adaptability
Supply chain
/security of
supply
Long term
strategic view
Innovation
Reputation
Differentiation
55. What I now know…
We all have impact – but it is not necessarily measurable and
sustainable impact
Do we want economic or social impact?
By implication social impact help people right now – but may not
help them in the future – which renders the project/our intervention
UNSUSTAINABLE
Sometimes it is our own (CSI Practitioner’s) fault we don’t have higher
impact as we decide what, who and how to fund/not to fund
The most sustainable projects/programs with the highest impact have
social, socio economic and ECONOMIC impacts i.e. the number of jobs
created
Sometimes there is negative impact – i.e. dependencies are created
Mostly there is only short term impact –which makes our interventions
UNSUSTAINABLE
56. Moving from Strategic to Catalytic
Responsive
Strategic
Catalytic
Responsive
to society
Strategic to
core
business
Catalytic change
for the good and
survival of mankind
57. Going Forward and Doing it better
Impact assessment can help funders, intermediaries, and
beneficiaries they support to:
Plan how their work will make a difference, and determine how
much of a difference they are making
Understand what does or does not work and why and detect
unintended consequences
Build an (scientific) evidence base to share with others, thus
influencing and informing debate, and increasing the sector’s body
of knowledge
Challenge yourself and others by looking critically at your/their work
in order to improve, to replicate good work, or to innovate and
develop new processes, products and services
Inspire and motivate staff, trustees and other stakeholders including
volunteers, beneficiaries, policy makers and other practitioners,
funders and investors to build relationships with others,
communicate added value and raise the profile of their work
58. In closing
Involve stakeholders
Establish the scope and identify the key
stakeholders impacted
Understand what changes
Map the outcomes – identify the
indicators to measure impact
Value things that matter
Community
Impact
Look for evidence, don’t forget baseline
and impact studies
If you can count
and define
indicators you
can determine
impact and
return
Only include what is material
Focus on agreed outcomes first then
incidental impact
Do not over claim
Calculate the impact based on evidence
Be transparent
Report the outcome – the good and BAD
news
Verify the result
Share the learning
Business
Return
60. Contact
Reana Rossouw
Next Generation Consultants - Specialists in Development
E-mail: rrossouw@nextgeneration.co.za
Web: www.nextgeneration.co.za
PLEASE NOTE: THIS PRESENTATION IS PART OF A
LARGER BODY OF RESEARCH! THIS INFORMATION IS
COPYWRITED AND THE INTELLECTUAL PROPERTY OF
NEXT GENERATION CONSULTANTS.