Presented at LILAC 2019

1. PHE Knowledge and Library Service
Steph Grey & Trish Lacey
I wouldn’t believe your radio:
developing tools to critically appraise
atypical materials and improve
information and visualization literacy.
Thursday 24th April 2019
LILAC 2019, University of Nottingham

2. Public Health: Promote, Prevent, Protect
2
I wouldn’t believe your radio: developing tools to critically appraise atypical materials and improve information and
visualization literacy.
Public Health, Public Health, what the !&£$ is Public
Health? (Amateur Transplants, Finals Fantasy)
Alcohol, Drugs,
Tobacco
Mental
health
Screening
Diet and
Obesity
Health
equity
Cancer Environment
Life
expectancy

3. Public Health England
3
I wouldn’t believe your radio: developing tools to critically appraise atypical materials and improve information and
visualization literacy.
Created in 2013, bringing together over 70
organisations, including:
Our mission: to protect and improve the nation’s health and
wellbeing, and reduce health inequalities.
• Health Protection Agency
• Specialised Commissioners
• Regional Cancer Intelligence Services
• Screening Programmes
• National Infection Service
• Public Health Observatories

4. PHE: an evidence-driven organisation
4
I wouldn’t believe your radio: developing tools to critically appraise atypical materials and improve information and
visualization literacy.
Health: Public Health?
Whatever form the evidence takes, QUALITY MATTERS

5. • To provide reassurance about the trustworthiness, value
and relevance of the evidence.
• To help provide justification of why this piece of evidence
has been used.
5
I wouldn’t believe your radio: developing tools to critically appraise atypical materials and improve information and
visualization literacy.
http://www.knowledge.scot.nhs.uk/media/931648/informationsupportroleframework.pdf
Why evaluate?

6. Quality in health: critical appraisal
6
I wouldn’t believe your radio: developing tools to critically appraise atypical materials and improve information and
visualization literacy.
CASP CEBMJBI
PROBLEM FOR PH: All address traditional forms of research.

7. What can we do about quality in PH
evidence?
7
I wouldn’t believe your radio: developing tools to critically appraise atypical materials and improve information and
visualization literacy.
Two areas of focus

8. Infographics: How they’re used at PHE
8 I wouldn’t believe your radio: developing tools to critically appraise atypical materials and improve information and
visualization literacy.
Developed:
Great for conveying a message with
great impact.
https://assets.publishing.service.gov.uk/government/uploads/system/upl
oads/image_data/file/77253/Health-Matters-July-11-07.jpg
Retrieved as evidence:
What is the effect of X?
Does X work?
http://www.heart.org/HEARTORG/HealthyLiving/HealthyEating/
HealthyDietGoals/The-Effects-of-Excess-Sodium-
Infographic_UCM_454384_SubHomePage.jsp

9. Infographics: Searching
9
I wouldn’t believe your radio: developing tools to critically appraise atypical materials and improve information and
visualization literacy.
What we want: What we found:

10. Infographics: Design
10
I wouldn’t believe your radio: developing tools to critically appraise atypical materials and improve information and
visualization literacy.
Literature Search
Read articles
Extracted evidence
Warning! Reliance
on one article.

11. Infographics: Quality control
11
I wouldn’t believe your radio: developing tools to critically appraise atypical materials and improve information and
visualization literacy.
Tested by members of PHE’s Knowledge & Library Services
Team
Approved by PHE KLS SMT before being put on catalogue.

12. Infographics: Launch
12
I wouldn’t believe your radio: developing tools to critically appraise atypical materials and improve information and
visualization literacy.
Find it on our library catalogue: https://phe.koha-
ptfs.co.uk/cgi-bin/koha/opac-detail.pl?biblionumber=54912
Soft launch: distributed to stakeholders

13. Infographics: General questions
13
I wouldn’t believe your radio: developing tools to critically appraise atypical materials and improve information and
visualization literacy.
1. Does the infographic provide enough information to
base a decision on?
2. Do you need to undertake further research?
3. Is the population group similar to yours?
4. Will it work in your setting?
5. Is this really the only piece of evidence you can find for
what you are doing?

14. Infographics: Demo 1
14 v
https://3jbq2ynuxa-flywheel.netdna-ssl.com/wp-content/uploads/2013/05/91.png
Does the infographic
provide enough
information to base a
decision on?

15. Infographics: Demo 2
15
I wouldn’t believe your radio: developing tools to critically appraise atypical materials and improve information and
visualization literacy.
https://debategraph.org/Handler.ashx?path=ROOT%252fu23%252fAdult_Physical_Acti
vity_Levels.jpg
Are there any
references to
back up where
the content and
data has come
from?

16. Infographics: Demo 3
16
I wouldn’t believe your radio: developing tools to critically appraise atypical materials and improve information and
visualization literacy.
https://static1.squarespace.com/static/53cd915ce4b0216897eda42c/t/55af0414e4b0bc5e7b2403fc/
1437533208415/infographic-6.jpg?format=750w
Are the graphics
necessary, do they
add value?

17. Infographics: Demo 4
17
I wouldn’t believe your radio: developing tools to critically appraise atypical materials and improve information and
visualization literacy.
https://www.nutsforlife.com.au/media/infographics/#
Is it free from bias?
Who produced this?
And for what
reason/purpose?

18. Infographics: Demo 5
18
I wouldn’t believe your radio: developing tools to critically appraise atypical materials and improve information and
visualization literacy.
https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/6
77546/Infographic5.png
Can you understand
this infographic
without any prior
knowledge on the
subject?

19. Infographics: Considerations 1
19
I wouldn’t believe your radio: developing tools to critically appraise atypical materials and improve information and
visualization literacy.
• This is still a new area which needs more research.
• Visualisation theory, how humans perceive colours and
visual patterns can have an impact on the message being
delivered.
• Visualisation literacy should be considered. Evidence
shows that a person’s education can impact on their ability
to understand an infographic. In addition to the ‘skill’
whereby an individual can ‘read’ an image, people with a
greater spatial awareness can read graphs better than
others.

20. Infographics: Considerations 2
20
I wouldn’t believe your radio: developing tools to critically appraise atypical materials and improve information and
visualization literacy.
• Semiotics: the interpretation of the pictures by the end user,
where we apply our own culture and meaning.
• There are different types of infographics and these may
need to be evaluated differently:
• statistical infographic
• informational infographic
• timeline infographic
• process infographic
• geographic infographic
• comparison infographic
• hierarchical infographic

21. Grey Literature: How it’s used at PHE
21
I wouldn’t believe your radio: developing tools to critically appraise atypical materials and improve information and
visualization literacy.
We gather:
Practice Examples
https://phelibrary.koha-ptfs.co.uk/practice-examples/
Impact Stories
https://phelibrary.koha-ptfs.co.uk/stories/
We use as evidence:
Items we find as a result of a
literature search
https://www.local.gov.uk/sites/default/files/documents/22.35%2
0Supporting%20young%20parents_05%20-%2027.03.pdf

22. Grey Literature: Searching
22
I wouldn’t believe your radio: developing tools to critically appraise atypical materials and improve information and
visualization literacy.
What we found:
Good base – not starting from scratch.
Didn’t meet our needs; we wanted more flexibility to
account for multiple formats of grey literature.
Jess Tyndall,
Flinders University
JBI CILIP / ILG

23. Grey Literature: Design
23
I wouldn’t believe your radio: developing tools to critically appraise atypical materials and improve information and
visualization literacy.
Ongoing! Virtual Journal Club and
PHE KLS consultation
Checklist Flowchart

24. Grey Literature: Why a flowchart?
24
I wouldn’t believe your radio: developing tools to critically appraise atypical materials and improve information and
visualization literacy.
• Too many options!
• Not all questions are [as] appropriate to all types of grey
literature
• Including all possible questions would lead to an unwieldy
checklist
• Needed something dynamic

25. Grey Literature: Future Plans
25
I wouldn’t believe your radio: developing tools to critically appraise atypical materials and improve information and
visualization literacy.
Testing and quality control
Launch to stakeholders
Upload and make publicly
available

26. Grey Literature: Demo 1
26
I wouldn’t believe your radio: developing tools to critically appraise atypical materials and improve information and
visualization literacy.
Website Blog
Annual
Report
Dataset Video Thesis

27. Grey Literature: Demo 2
27
I wouldn’t believe your radio: developing tools to critically appraise atypical materials and improve information and
visualization literacy.
Annual
Report
Date Authority
Does this item support other evidence?
What value does this add?
Objectivity Clear Aim

28. Lessons learnt
28
I wouldn’t believe your radio: developing tools to critically appraise atypical materials and improve information and
visualization literacy.
If the tool you want doesn’t exist – invent it
Have a support network of helpful critics
Have a plan and realistic timetable
Once completed, share – for the benefit of you and
others

29. References
CASP Checklists
https://casp-uk.net/casp-tools-checklists/
CEBM Checklists
https://www.cebm.net/2014/06/critical-appraisal/
Joanna Briggs Institute Checklists
http://joannabriggs.org/research/critical-appraisal-tools.html
AACODS Checklist
https://dspace.flinders.edu.au/xmlui/bitstream/handle/2328/3326/AACODS_Ch
ecklist.pdf
CILIP / ILG CRAAP Test
https://infolit.org.uk/wp-content/uploads/ResearchSmarter/Docs/CRAAP.pdf
29
I wouldn’t believe your radio: developing tools to critically appraise atypical materials and improve information and
visualization literacy.

30. Knowledge and Library Services, Public Health England
Thank you for listening!
Any Questions?

31. Steph Grey
Learning & Research
Support Librarian
Public Health England
Email:
stephanie.grey@phe.gov.uk
Twitter: @StephGrey84
Trish Lacey
Knowledge & Evidence
Specialist. East Midlands
and West Midlands
Public Health England
Email:
patricia.lacey@phe.gov.uk
Phone: 0121 214 9144