Summary presentation of a study to track use of campaign evaluation findings of two organisations

1. Tracking Use of Campaign
Evaluation Findings of Two
International Organisations
12th EES Biennial Conference
Maastricht
28 September 2016
Dr Glenn O’Neil
oneil@owlre.com
www.owlre.com

2. 2
– Evaluations were carried out of two global
communication campaigns in 2009/10
– Four years later, I returned to look at how the
evalution findings were being used
Background
Evaluator
Campaign Evaluation unit

3. 3
– Qualitative approach to trace, categorise and
validate instances of use
– Interviews with campaign staff (6-ICRC, 5-OHCHR)
– Each instance of use (28-use; 6-non-use) were
coded on the basis of a conceptual framework
Methodology
Levels: Individual Interpersonal Collective
# So. An. Instance
description
Type Description Att. Description Att. Description Att. How Inf. Ver.
ICRC
1
I1,
2, 3
y Reduce
complexity of
messages and
products
In Recomm.
reviewed,
considered,
thought about,
priority given
Sa,
El
Discussed with
campaign team,
confirmed
existing
consensus
Ex,
Pe
Integrated into
next campaign
concept and
implemented in
messages/produc
ts developed
PC ALU F, P D; I1, 2,
3
Source
Anticipated
(yes/no)
Type e.g. Instrumental
Attributes, e.g.
salience, elaboration
Process category -
e.g. Anticipated linear use
Influences e.g. funding, people
Verification,
e.g. documentation

4. 4
1. Use was mostly non-linear and unanticipated
2. Use was unpredictable, opportunistic and unexpected
3. Use never occurred in a vacuum devoid of influences
4. Strongest influences on use were internal
Key findings

5. 5
The data
Instances
of use
What: Type of use
Where: Levels of use
Why: Influences on use
How: Process of use

6. 6
Instrumental
13 instances
Process
5 instances
Conceptual
9 instances
Symbolic
I instance
Non-use
6 instances
Field
People
Funding
External
context
Evaluation
policies and
institutions
Organisational
context
Communication
Goals and
ambitions
Campaign
manager
(2)
Campaign
senior /
researcher
(2)
Campaign
staff (3)
Campaign
staff
(after) (4)
24
16
7
9
1
2
4
9
6
6
8
19
5
11
1
12
22
2
11
1
8
InfluencesInstancesSources
What: Type of use
Why: Influences on use

7. 7
Where: Levels of use
Individual Interpersonal Collective
Informal change
Formal change
Four
Years

8. 8
How: Process of use
Unexpected hop Use was not anticipated and occurred
in a linear way.
Unforeseen foray Use was not anticipated and occurred
in a non-linear way.
Direct route Use was anticipated and occurred in a
linear way.
A planned ramble Use was anticipated and occurred in a
non-linear way.
Expedition
starts/stops
Use was anticipated, did not occur
and happened in a non-linear way.
Travel plans
cancelled
Use was anticipated, did not occur
and happened in a linear way.
Surprise trip deferred Use was not anticipated, did not occur
and happened in a non-linear way.
Unannounced
stop-over skipped
Use was not anticipated, did not occur
and happened in a linear way.

9. 9
1. Use was mostly non-linear and unanticipated
2. Use was unpredictable, opportunistic and unexpected
3. Use never occurred in a vacuum devoid of influences
4. Strongest influences on use were internal
Key findings – a reminder

10. 10
Contact details
View these slides on my blog:
oneil@owlre.com
glennoneil
www.owlre.com
@glenn_oneil
www.intelligentmeasurement.com
Contacts: