Patrick Ladbury, National Social Marketing Centre www.charitycomms.org.uk/events

1. Social Marketing Conference:
Changing Behaviour Through
Communications
30 November 2011
www.charitycomms.org.uk
www.twitter.com/CharityComms
www.facebook.com/CharityComms

2. Evaluating social marketing
Patrick Ladbury
www.thensmc.com

3. Evaluation - Simples!
• What behaviour do you want people to do?
• Which people do you want to do that behaviour?
• By when?
• How can we measure it?

4. An evaluation plan
Questions to ask
• What are we measuring?
– Changes in knowledge, attitudes (intention, priority)
and behaviour
• What are we measuring against?
– What are our baselines or controls?
• Who is doing the measuring?
– Do we need any independent input?
• When do we do the measuring?
• Can we turn the outcome into an economic ROI?

5. An evaluation plan
Different types of evaluation
TYPE Definition
FORMATIVE Done throughout the development of a project to pre-test (or ‘sound out’) aspects with
the audience or stakeholders. Keeps you on track in terms of what will work with the
target segment.
PROCESS A review of how the project was managed – usually involving key internal and external
stakeholders.
ECONOMIC Evaluating cost effectiveness , return on investment and value for money
IMPACT (or Measures the changes that have happened directly as a result of our intervention.
Output) These are likely to be changes in knowledge, awareness and, most importantly of all,
behaviour
OUTCOME Seeks to link the short term actions (impacts) with the longer term benefits (outcomes)
The changes in behaviour will have been encouraged in order to meet a wider, or
longer term benefit.

6. An evaluation plan
Impact and Outcome

7. An evaluation plan
Template
What are we measuring? How are we going to What needs to Who by? When
measure it be done? Resource needed by?
(method)?
IMPACT - Knowledge
IMPACT - Attitude
IMPACT – Behaviour
OUTCOME –

8. A step by step approach
Planning a behaviour change intervention using
Social marketing is a step by step process
Using the process helps to ensure more efficient and
effective interventions and better use of resources.

9. Getting started: example
A number of young people under 16 were regularly
drinking alcohol on the streets of North Tyneside at
the weekend.
The number of ASBOs handed out by police was
increasing as were the number of complaints
received by residents who were feeling less safe on
the streets.
.

10. Scoping: example
Research led to other areas that impacted on the
issue and that needed evaluating.
• Alcohol sales to young people
• Attendance and activities at youth clubs
• Young people’s drinking levels

11. Smoking cessation services
Smokers
Smokers Smokers Smokers
All who quit
who want who try who
smokers using
to quit to quit succeed
support
70%* 45% 23% 2.7%
100%
Driving Triggering Making quitting
motivation action more effective
*NB figures relate to all smokers.

12. Intervention Mix – Tobacco Control
SUPPORT DESIGN
Give people the means to change Change physical context
“Pop Up” cessation services in targeted
Better NHS Stop Smoking Services
areas e.g. 360 Asda supermarkets
“Quit Kit” for cold turkey - quit plan,
Over 200,000 people signed up to online
information about medication, details for
communities
their local stop smoking services.
INFORM/EDUCATE CONTROL/COMPEL
Provide information Incentivise/Disincentivise
Normalising NHS Stop Smoking - designed Legislation to end tobacco displays in
“for people like me” shops
Focus on fact that their children 3 times Using tax to maintain high price
more likely to start
Plain packaging of tobacco products

13. The Results
• 480,000 Quit Kits ordered
• 95% new contacts for the CRM database
• 3 million smokers attempted to quit –08/09
• Nearly 220,000 stayed quit 1 year later
• Budget of £21,115,194 (down 22%)
• 3 year payback of £73.5 million

15. Past evaluation work: example #1
Measurement Method Timing
Formative Qualitative Development
Outcome Questionnaire – pre and post Development &
Knowledge, attitudes, behaviour (intended evaluation
and actual)
Process Benchmark criteria Development,
Horizon scanning implementation &
Stakeholder interviews evaluation
Emerging key insights, opportunities and
threats
Event monitoring Number of attendees Implementation &
Impact on knowledge/awareness & evaluation
behaviour
Media analysis Content analysis – qualitative Implementation &
Impact – n evaluation
Web analysis

16. Past evaluation work: example #2
Measurement Method Timing
Awareness and Quantitative survey: Development &
experience of quitting Awareness and experiences evaluation
Process Benchmark criteria Development,
Horizon scanning implementation &
Stakeholder interviews evaluation
Emerging key insights,
opportunities and threats
Service delivery Qualitative Development &
evaluation evaluation
Customer journey Observation, including Development &
mapping assessment of sign-posting evaluation
Telephone helpline Quantitative survey Development &
monitoring evaluation

17. Take Charge – Take the Test
17

18. New HIV
diagnoses in
2006
African Americans
make up 13% of the
total population
18

19. What behaviour?
Get an HIV test within
3 months of having
unprotected sex
19

20. Key Insights
• Fear of results & stigma are main barriers
• Strong family and church ties
• Everyday burdens outweigh concerns about
HIV (e.g., children, work, bills, etc.)
• Concerns about partner’s sexual behaviour are
significant
20

21. Key Message
• Look out for yourself – Take Charge. Take the Test
• Emphasized:
– Danger from their partners risky behaviour
– Empowerment that comes with getting tested
– Availability and convenience of testing
– The “norm” that others were getting tested

22. The Marketing Mix
• Targeted advertising – radio, direct mail, outdoor
• Drive people to free HIV testing locations
• Toll free line & website
• Community outreach targeted key influencers (beauty
salon owners etc)
• Community “rapid HIV testing” events
• Focus on “normalizing” testing
22

23. The Results
• Significant increase in awareness and intention
• Increased testing rates by nearly 70%
• Calls to the HIV hot line rose 290%
• Set new records for the number of HIV tests
conducted in one day
• No significant impact on use of existing HIV
testing services
23

24. Key Lessons
• Rapid testing events in the community were the
key to success
• They helped to reduce the “Intention – Action”
barrier
• Greater support for cities to expand the number,
reach and impact of these events
24

25. Value for Money
• VfM is about getting the best health possible for
the resources available
• Short term vs. long term impact

26. Measuring Health Gain, Costs
and Savings
• Health Gain is measured as QALYs
– 1 QALY = £25,000
• Costs to all stakeholders: Local Authorities,
Government, NHS, Employer
• Savings from long term reductions in costs to
the stakeholders
• Health Gain of behaviour change is taken from WHO
Global Health Risk report
– NICE used this data, in conjunction with their evidence
base

27. Using the Tool: Data Needed
• Baseline data for behavioural goal
• Cost of intervention
• Number of people in target segment/audience
• Post intervention data
– E.g. number of quitters
– Age
• % of target audience with high levels of
disadvantage

30. Conclusion
• Continuous evaluation
• Measuring behaviour which counts
• Who needs to know
• How to sing!

31. Contact
Patrick Ladbury
p.ladbury@thensmc.com
www.thensmc.com/resources/vfm