Presentation by Alison Brettle and Anne Webb on behalf of NW Clinical Libarian Systematic Review Group - International Congress of Medical Librarianship 2009

1. Approaches to evaluating
Clinical Librarian Services:
a systematic review
Presented by:
Alison Brettle, University of Salford, Salford, UK
Anne Webb, The Christie NHS Foundation Trust, Manchester, UK
On behalf of NW England Clinical Librarians Systematic Review Group
Partly funded by North West Health Care Libraries Unit
Library and Information Health Network North West

2. NHS and the North West
Nationwide employs:
• around 90,000 hospital doctors
• 35,000 general practitioners (GPs)
• 400,000 nurses
• 16,000 ambulance staff
(www.nhs.uk) Department of Health
NW SHA 10 Strategic Health Authorities (SHA)
Acute Trusts, Specialist Trusts, Mental Health Trusts, Primary Care Trusts (PCT) Ambulance Trusts
The North West covers the 5 areas of Greater Manchester, Merseyside, Cheshire,
Cumbria and Lancashire with a population of 6.8 million.
Approximately 8% of people work in the healthcare sector – including a team of librarians
interested in clinical librarianship!

3. UK North West Clinical Librarian
Systematic Review Group:
Lucy Anderson, NHS Bury
Alison Brettle, University of Salford
Michelle Maden-Jenkins, Edge Hill University
Maden-
Rosalind McNally, National Primary Care Research and
Development Centre, University of Manchester
Tracey Pratchett, University Hospitals of
Pratchett,
Morecambe Bay NHS Trust
Jenny Tancock, University Hospitals of
Tancock,
Morecambe Bay NHS Trust
Debra Thornton, Blackpool, Fylde and Wyre
NHS Foundation Trust
Anne Webb, The Christie NHS Foundation
Trust

4. Drivers
The Hill Report (2008) – includes
recommendations that librarians evaluate their
services
Measuring impact of services is a challenge
Previous research demonstrated variability in
models and limited effectiveness of Clinical
Librarian Services

5. What’s Known to Date?
3 other reviews
Winning & Beverley (2003)
(2003)
Cimpl Wagner & Byrd (2004)
Weightman & Williamson (2005)
Identified Weaknesses
Small sample size
Low response rate
Response bias
Poor reporting/inadequate information on methods
Researcher bias/desirability bias
Non-
Non-specific patient care outcomes
Non-
Non-use of reliable/valid methods
Subjective reporting of results
Results less likely to be quantified

6. Overall Aim
To update previous reviews
Provide guidance for future evaluations.
Clarification of the models of clinical
librarianship and how best to measure
impact of services in relation to these?
(Hill Report)

7. Objectives of Our Study
1. To determine which models of Clinical Librarian
services have been evaluated
2. To determine whose perspective has been
evaluated
3. To determine what outcome measures have
been used
4. To determine the quality of the methods used

8. Methods 1- Searching
1-
Searched 20 databases from 2001 onwards
Scanned references and hand searched 2 journals
HILJ, JMLA
Google search
Grey literature – mail lists, known contacts, CILIP
Update etc.
Bibliographies

9. Methods 2 – Filtering and
Extraction
Filtering and article selection
Initial filter of irrelevant articles undertaken by 2
people.
Scanned titles and abstracts (in pairs)
Obtained full papers (checked in pairs)
Developed and evaluated critical appraisal/data
extraction tool
Extracted relevant papers (in pairs)
Data management tools: Refworks and Excel

10. Inclusion Criteria
Studies that meet the Hill definition
Studies which are described as outreach but the focus is
to support patient care
Studies which describe services providing patient
information – evaluation outcome relating to patient care
Published post 2001
Reports evaluation methodology
English language

11. Results 1
Potentially relevant citations identified
n=2040
Excluded
n = 857
Initial assessment of titles and abstracts
n = 456
Full text assessment –
potentially relevant items
n = 91
Excluded
n = 62
Papers going forward to appraisal/ data extraction
n= 29
Final Inclusion
n = 21

12. Breakdown by Country
Country No.
UK 15
US 5
Holland 1

13. Breakdown by Study Design
Study design No. of papers
Survey 16
Qualitative 10
Experimental (e.g. RCT) 1
Quasi-experimental (e.g. Pre-post test) 1
Service evaluation 13
Action Research 2
Case Study 7
Other 5
Unclear 0
Not stated 0

14. Breakdown of Service Models
Information at the point of Information at the point of
need need plus critical appraisal
and synthesis
Question and Answer Question and Answer
service service plus critical appraisal
1 and synthesis
3
Outreach Outreach plus critical
14 appraisal and synthesis
= informationist
4

15. Breakdown of Outcomes
Outcome criteria
Usage statistics 62%
Use of information 52%
Relevance of results 48%
Usefulness of results 48%
Time saving 48%
General impact on patient care 38%
Time to respond 33%
Other 24%
Improvement in information literacy 23%
Improvement in confidence 4%
Cost 0%
Not stated 0%

16. Breakdown of Perspective
No. of
Perspective papers
User 16
Librarian 7
Library Service 11
Organisation 3
Unclear 1

17. Breakdown of Quality
Quality measure (Weightman et al) no. %
Appoint researchers who are independent of the library service 5 23
Ensure that all respondents are anonymous and that they are 4 19
aware of this.
Survey all members of chosen user group(s) or a random 12 57
sample
Agree a set of questions that are objective, well used in previous 9 42
research, and developed with input from library users.
Use the critical incident technique. 6 29
Combine a questionnaire survey with a smaller, but also random 8 38
sample of follow-up interviews.

18. Conclusion
1. Models of Clinical Librarian Service – 4
models evaluated
2. Evaluation has been mainly from a
service and service user perspective
3. A wide range of outcome measures have
been used
4. Quality has improved, but needs to
improve further

19. Thank You!
Thank you for listening
Thank you to the NW Clinical Librarian Systematic
Review Group
Thank you to North West Health Care Libraries Unit
for funding
Any questions?
Contacts:
a.brettle@salford.ac.uk
anne.webb@christie.nhs.uk

20. References
Cimpl Wagner, K. and Byrd, G.D. Evaluating the effectiveness of clinical medical
librarian programs: a systematic review of the literature. Journal of the Medical
Library Association 92 (1):14-33, 2004.
(1):14-
Hill, P. Report of a national review of NHS library services in England: from
knowledge to health in the 21st Century, National Library for Health, 2008. Available
from: http://www.library.nhs.uk/aboutnlh/review (accessed 5th May 2008)
Rankin, J. A. et al .The emerging Informationist Specialty: a systematic review of the
literature JMLA 96 (3): 194 - 206, 2008
Weightman, AL and Williamson, J. The value and impact of information provided
through library services for patient care: a systematic review, Health Information and
Libraries Journal, 22: 4-25, 2005
Journal, 4-
Weightman AL, Urqhuart C, Spink S, Thomas R. The value and impact of information
provided through library services for patient care: developing guidance for best
practice. Health Information and Libraries Journal 2008;26:63-71.
2008;26:63-
Winning, M.A. and Beverley, C.A. Clinical librarianship: a systematic review of the
literature. Health Information & Libraries Journal 20: Suppl-21, 2003.
Suppl-

21. Glossary
CILIP - Charted Institute of Library and Information
Professionals
DH - Department of Health
HCLU - North West Health Care Libraries Unit
HILJ - Health information and Libraries Journal
JMLA - Journal of the Medical Library Association
LIHNN - Library and Information Health Network
North West (England)
NHS - National Health Service
PCT - Primary Care Trust
SHA - Strategic Health Authority