This document provides an overview of evaluation methods and processes for evaluating the impact of a program. It discusses evaluating the program's impact on residents through sampling residents and determining what factors influenced outcomes. It also discusses evaluating the impact on systems change by looking at changes to context, components, connections, infrastructure and scale. The document provides details on baselining the current system and sampling systems change over time. It concludes with discussing perspectives for evaluation, such as whether the intervention achieved its goals and measuring specific impacts.

1. Annexes
• Overview of evaluation methods & processes
• Evaluating the impact of programme on residents
• Sampling residents
• Evaluating the impact of the programme on the process of systems
change
• Baseline & sampling systems change
• Types of perspectives of evaluating interventions

2. Overview of evaluation methods &
processes

3. Evaluating the impact of programme on
residents
• What are the different actions,
behaviours & people that have
influenced the outcome / change in
behaviour (see Behaviour Change
Wheel)
• To what extent can the outcome
achieved be attributed to the
intervention or programme? (i.e.
could inactive residents have become
more active anyway, have staff
benefited from other training that has
helped them influence resident's
behaviours)
• What are the pathways that have led
to the change in behavior? Behaviour Change Wheel

4. Sampling
residents
• Randomised sampling, such as
using a residents survey
• Cluster sampling, such as getting
Activity Hubs to carry out research
• Extreme case sampling,
interviewing people who used to
be inactive but are now active, but
also getting others active
• Snowball sampling, asking
residents / community connectors
to identify people to recommend
to speak to
• Multi-stage sampling, using
different sampling methods

5. Evaluating the impact of the programme on
the process of systems change
• Context—improving the political environment that surrounds a system so it
produces the policy and funding changes needed to create and sustain it
• Components—establishing programs and services that produce results for
system stakeholders and/or participants
• Connections—creating strong and effective linkages across system
components that improve results for clients
• Infrastructure—developing support that systems need to function
effectively
• Scale—ensuring a comprehensive system is available to as many people as
possible

6. Baselining systems change
What is the system?
• Describe the system the programme/change is expected to
influence
• Describe the system’s relationships and dynamics.
• Describe the diverse goals or perspectives within the system.
What is the systems change intervention (the programme
building blocks)?
• Describe how the intervention will be governed or
implemented.
• Describe the key points of the intervention’s theories of
change and action. Describe the intervention’s intended
outcomes.
What are the goals of the systems change evaluation?
• Describe the evaluation’s users, purposes and methods.
Example

7. Sampling systems change
• As the evaluation of systems
change will be iterative, we want
the sampling methods to be
iterative and not fixed from the
start

8. Types of perspectives of evaluating
interventions
• Did the intervention do what was
intended?
• Did the intervention produce the
expected results within its
intended focus area?
• To what extent can a specific
impact be attributed to the
intervention?
• Did the intervention make a
difference?
• How has the intervention made a
difference?