This presentation describes the librarian's role in providing evidence based resources to improve public health.

1. “Modules of Workforce Development: Pragmatic
Approaches to Knowledge Acquisition,
Application, and Information Dissemination”
Enterprise Access to Evidence-Based Literature:
A Dynamic Digital Library for Public Health

2. Enterprise Access to Evidence-Based Literature:
A Dynamic Digital Library for Public Health
Karen Dahlen, Project Consultant
Elaine Martin, Director
New England Region NN/LM Lamar Soutter Library
University of Massachusetts Medical School
This project is currently funded by the NLM via contract through the
NN/LM, New England Region

3. Enterprise Library Approach to Knowledge Acquisition,
Application & Information Dissemination
Measure Importance of Trusted Information Access
Ensure Direct Access from Intranet-Based Site
Access (Without Name/Password Required)
Support Core Competencies/Training
Use Survey Instruments Adapted to the Location
Relate Cost Efficiencies to Enterprise Licensing/Creative Partnerships
Document Institutional Change in a Dynamic Environment
Evaluate Over Time (Illustrate Efficiencies)

4. Shifting the Culture
to Evidence-Based Practice
• Culture of evidence-based practice was affected when funding for public
health libraries was eliminated.
• State Libraries (in many states) were mandated to support state
agencies, but had no collections to support public health.
• Streamlined access to resources through the Digital Library Platform--
along with trust in new relationships--prompted resurgence in use of
resources and has facilitated this cultural shift.
• Training stimulated access to resources reviving evidence-based
interest and relationship to practice.
• Project has the ability to leverage government resources (NLM, CDC,
NAL) to improve evidence-based practice.

5. ENTERPRISE ACCESS TO EVIDENCE-BASED LITERATURE: A
DYNAMIC DIGITAL LIBRARY FOR PUBLIC HEALTH
Background (Why this Project)
Project Rationale
Partnerships

6. Council on Linkages:
Academia & Public Health Practice
Public Health Competencies
• Analytic/Assessment Skills
– References sources of public health data and information.
– Information technology to collect, store, retrieve data.
– Utilizes data to address scientific, political, ethical, and
social public health issues.
• Policy Development/Program Planning Skills
– Analyzes information relevant to specific public health
policy issues
• Public Health Sciences Skills
– Conducts a comprehensive review of the scientific
evidence related to : a public health issue, concern, or
intervention.
– Retrieves scientific evidence from a variety of text and
electronic sources.
• Knowledge Informatics
– Access trusted resources directly from the
desktop.
– Describe relationships and functionality of e-
resources to core public health interests.
– Introduce specific types of resources: coding
and classification tools.
• Informatics Skills
– Train on interoperable features that support
public health work, including connectivity to
trusted sites; advanced search filters, use of
topical alerts and information mgt software.
Competencies: Council on Linkages Project Tied to Competencies

7. Obstacles to Getting Information

9. Gathering and Analyzing EB Information
Rapid Detection of Epidemics
Detecting an anthrax
epidemic one day earlier
would save $1-7 billion
CDC Presentation by
C. Safran citing:
http://www.cdc.gov/ncidod/eid/vol3no2/kaufman.htm
Kaufmann AF, EID, V3, N2

10. ENTERPRISE ACCESS TO EVIDENCE-BASED LITERATURE: A
DYNAMIC DIGITAL LIBRARY FOR PUBLIC HEALTH
Library Without Signature Building
Goal of the Project
Who Participates
Objectives & Data Sets
Development of Digital Library

11. PHIA: Innovative Business Model
Not Tied to Traditional Building
Madison, WI Atlanta, GA

12. Goal of PHIA
Public Health Information Access
To determine what
resources are core,
useful, evidence-based to
advance public health
practice.
Full text
directly
available
through DL
Full text
available
through LP
Core to
public
health

13. Unique Aspects of Digital Library
• Resides on PHD Intranet (outside a signature building)
without need for id and password.
• Relies on collaboration, interoperability & funding.
• Move toward analysis; less time on access
• Skills & training support personalized, interactive
styles tied to continuum of learning and institutional-
specific priorities and projects.
• Collection of evidence occurs in many aspects of
Brownson wheel not just at the literature review stage.
• Fundamental change in the way resources are
distributed and utilized.

14. Public Health Departments
& Library Partners
State Public Heath Department Library Partners
AK Alaska Division of Public Health (AKDH) Univ of Alaska (Anchorage) Medical Library
AR Arkansas Department of Health (ADH) Univ of Alabama Medical Sciences Library
CO Colorado Dept of Public Health & Environment Poudre Valley Health System Library
CT Connecticut Public Health Department University of Conn Maynard Stowe Library
HI Hawaii Department of Health University of Hawaii Medical Library
IN Indiana State Department of Health Ruth Lily Medical Library, Indiana University
KY Kentucky Department for Health University of Kentucky Medical Library
ME Maine CDC Maine Medical Center Library
MA Boston Public Health Commission* University of Massachusetts Soutter Library
MA Massachusetts Public Health Department Lemuel Shattuck Hospital Library
MD Maryland Dept of Health & Mental Hygiene University of Maryland HS Library
NH New Hampshire DHHS New Hampshire DHHS Library
RI Rhode Island Department of Health (RIDH) Rhode Island Hospital/Lifespan
VT Vermont Department of Health (VDH) UVM Dana Medical Library
WI Wisconsin Department of Health Services Ebling Library of the Health Sciences
WV West Virginia Bureau of Public Health West Virginia University Libraries
WA Washington State Department of Health University of Washington HS Library

15. PHIA Evaluation Process
Tied to Data Model
To advance access to evidence-based
resources in support of improved
public health practice.
What resources are high use?
What technical issues arise?
What resources are missing? How can training be
improved?
What project components lead to improved
efficiencies?
What cost efficiencies are involved with an enterprise
approach to digital library implementation, and
training.
• Logic Model
– Resources, activities, outputs, outcomes
(minutes, visits, trainings, people lists, IP
management, levels of troubleshooting,
RML reports.
• Data Tools
– Overarching questions
– Journal SurveyMonkey
– Pre-Post Training Surveys
– Just-in-time Information Checks
– Formal Evaluation (interviews & focus
group sessions

16. Data Sets & Sources
• Resource Metrics Collected
• Journal Survey Monkey Identifies
Perceived Need & Library Use.
• Document IT technical issues.
• Licensing Issues Related to
Journals and Databases.
• Training Surveys Collected
• Preliminary Meetings/Updates Held.
• Trainings Held
• Year End Evaluation
--Interviews & Focus Groups

17. View of PHIA Digital Libraries
Connecticut Digital Library Arkansas Digital Library

18. ENTERPRISE ACCESS TO EVIDENCE-BASED LITERATURE: A
DYNAMIC DIGITAL LIBRARY FOR PUBLIC HEALTH
Resources on Digital Library
Use of Resources
Article Delivery Use & Costs

19. Databases & Major
Publishers of Resources
PubMed , Other NLM Databases
NAL, CDC & Other Gov’t Resources
CLSI (Clinical Laboratory Standards)
Global Health
Cochrane Library
– Systematic Reviews
Stat!Ref—E Books (35)
-- Coding & Classification
ASABE –Health Linked to Agriculture

20. Identifying Core, Useful, &
Evidence-Based Journals
Publisher Total Use Journal Title Use
Mass Medical Society (1) 14,136 New England Journal of Medicine 14,136
AAP(1) 9,888 Pediatrics 9888
Oxford University Press (20) 23,369 Clinical Infectious Diseases 3147
Springer (18) 2, 485 Maternal & Child Health 474
ASM Journals (12) 3,518 Journal of Clinical Microbiology 1729
Annual Reviews (9) 697 Annual Review Public Health 394
Sage (11) 939 The Diabetes Educator 135
BMJ (9) 2932 British Medical Journal 1832
Mary Ann Liebert (11) 1798 Breast Feeding Medicine 355
University of Chicago Press (6) 955 Infection Control & Hospital Epidemiology 749
Lippincott/Ovid (6) 122 J Public Health Mgt Practice 55

21. NEJM Use 2010-2013
0
1000
2000
3000
4000
5000
6000
7000
4864
1772
982
554 314 527 337 311
890
308
1183 1178
494 422
2010=2087 2011=11,194 2012=14,022 2013=14,136

22. 298 287
233
175
135
94 90 89 82 78 71
42 32 18 5
ASM: Journal of Clinical Microbiology
Total Use 2013=1729

23. Oxford University Press Use
Jan-Dec 2013
Total Use by Each PHD
N=13,827 High Use Journal Titles
Journal Title Total Use
Clinical Infectious Disease 3147
American Journal of Epidemiology 941
Schizophrenia Bulletin 771
The Journal of Infectious Diseases 627
ICES: Journal of Marine Science 474
JNCI: Journal of the National Cancer
Institute 434
Journal of Antimicrobial Chemotherapy 297
International Journal of Epidemiology 288
Alcohol and Alcoholism 274
Journal of Analytical Toxicology 262
Health Promotion International 252
Rheumatology 246
Age and Ageing 240
PHD Total
Alaska DPH 1245
Arkansas DH 1483
BPHC 182
CDPHE 608
CT DPH 1234
HI SDH 638
ISDH 700
KYDPH 1134
Maine CDC 757
MDHMH 1427
MDPH 1817
NH DHHS 157
RIDH 620
VDH 709
WI DHS 1116

24. STAT!Ref E-Books/Tools Provide Basic
Understanding of Topic
Filters & Tools
– Codes[5]
• CPT with RVUs Data File, INGENIX®
(2011)[1]
• ICD-9+CM - Volumes 1, 2 & 3 (2012)
– Titles By Discipline[344]
Searchable Alerts (by topic)
– Functionality with NLM
– Chapters link to PubMed
– TOXNET linked to Search
– Related Concepts/Meta…
Functionality with CDC
– MMWR, Community Guide;

25. STAT!Ref E- Books
for Public Health
Titles Containing PH Content
E-Book Use
ACP PIER, Journal Club & AHFS DI Essentials (2013) 482
Oxford Textbook of Public Health - 5th Ed. (2009) 407
Manual of Clinical Microbiology - 10th Ed. (2011) 403
AHFS Drug Information (2013) 305
Red Book: 2012 Report of the Committee on Infectious
Diseases - 29th Ed. (2012) 241
Emerging Infections Series (2008 - 2010) 216
Textbook of Modern Toxicology, A - 4th Ed. (2010) 209
Coding & Classification Books
E-Book Use
ICD-9-CM - Volumes 1, 2 & 3 (2013) 881
CPT with RVUs Data File, INGENIX (2013) 474
ICD-10-PCS: Procedure Coding System (2014) 12
ICD-10-CM: Clinical Modification (2014) 15
Diagnostic and Statistical Manual of Mental Disorders
4th Ed. (DSM-IV-TR, 2000) 107

26. CLSI (Clinical & Laboratory Standards)
20 Documents Used 790 Times
April 21, 2014
Doc # Document Name #
M100
-S24
M100-S24: Performance Standards for
Antimicrobial Susceptibility Testing; Twenty-
Fourth Informational Supplement
248
EP09-
A3
Measurement Procedure Comparison and
Bias Estimation Using Patient Samples;
Approved Guideline—Third Ed.
102
EP12-
A2
User Protocol for Evaluation of Qualitative
Test Performance
55
MM0
9-A2
Nucleic Acid Sequencing Methods in
Diagnostic Laboratory Medicine; Approved
Guideline—Second Edition
54

27. Calculations Related
to Enterprise Licensing
State PHD FTE 10% Rule
ADH 2781/1390 139
AKDPH 525 52
BPHC* 1100/550 55
CDPHE 1200/600 60
CT DPH 780 78
Hawaii DPH 3000/1500
150
ISDH 780 78
Kentucky BPH 400 40
MDHMH 8000/4000 400
Maine CDC 395 39
MDPH 3000/1500 150
NH DHHS 279 30
RIDH 400 40
VDH 300 35
WDPHS 400 40
VW DHS 780 78
Washington
1600/800 80
Total PHIA FTE 10,999 1546
Enterprise Licensing is
based on 10% of the Total
FTE level to capture
baseline information and
measure use/interest of
resources.

28. Article Delivery Costs Via Library Partnerships
May 2011 – April 2012; Cost=$18,384 May 2012 --- April 2013; Cost=$12,241
STATE ILL DD Total
Connecticut 0 83 83
Maine 560 364 924
Massachusetts
(BPHC) 8 3 11
Massachusetts
(MDPH) 1 1 2
New
Hampshire 0 0 0
Rhode Island 47 36 83
Vermont 21 191 212
Colorado 323 156 479
Total 960 834 1794
STATE ILL DD Total
Connecticut 2 20 22
Maine 154 90 244
Massachusetts
(BPHC) 9 10 19
Massachusetts
(MDPH) 7 6 13
New
Hampshire 0 0 0
Rhode Island 82 79 161
Vermont 54 243 297
Colorado 112 40 152
Arkansas 32 73 105
Kentucky 8 3 11
Wisconsin 0 0 0
Total 462 593 1055

29. Training Topics Embrace
National and State Interests
Maryland DHMH Health Initiatives
• Health Disparities(sexual minorities)
• Preventive Services ACA* HIV Screening
• Immigrant Health
Undocumented individuals under ACA
• Drug Resistant Disease which impacts
Impact on TB, STI (plus others)
• Health Care Reform and Interpersonal
Violence/Domestic Violence
• Infectious Diseases; Hepatitis C Virus
HIV Infections --HIV and HCV Co-Infection

30. Number of Trainings/Diversity of Workforce
STATE PHD Trainings Attendance
RI RIDH 2 30
VT VDH 3 46
ME
MAINE
CDC* 3
52
MA BPHC* 4 58
MA MPHD* 3 32
NH NH DHHS 2 24
CO CDPHE* 5 100
CT CT DPH* 5 101
KY KDPH 2 46
AR ADH 2 31
WI WI DPH 2 33
AK AK DPH 2 50
HI HI DPH 2 68
IN ISDH 2 60
MD MDHMH 2 38
TOTAL 15 41 769
Maryland Indiana Hawaii
Nutrition Specialist (2) State Epidemiologist Education Coordinator
(2)
Research Statistician Supervisor – Labs + (3) Asthma Coordinators(2
Policy Analyst (2) Injury Prevention
Epidemiologist
Legislative Liaison
Program Mgr (2) Staff Attorney Epidemiologist
Epidemiologist Microbiologist ( 2) Research Analyst (2)
Medicare
Specialist/Advisor (2)
Regional Program
Director
Program Coordinator,
SAPB
Policy Advisor (2) Director of Prog
Evaluation
Epidemiologist (3)
Psychiatric VT Director of Prog
Development
Genetic Counselor
Executive Ass’t Field Epi Director Tobacco Prevent Mgr
Deputy
Secretary
Access Services Mgr Informatics Analyst
Director VSA IT Supervisor Nutritionist

31. ENTERPRISE ACCESS TO EVIDENCE-BASED LITERATURE: A
DYNAMIC DIGITAL LIBRARY FOR PUBLIC HEALTH
Enterprise Efficiencies
Unique Aspects of the Business Model
Evaluation Processes

32. Cost Efficiencies:
Article Analysis
Direct Access to Articles Alternative Article Delivery
Contract Year Total # Amount $
May 2013-March 2014 1038 10,102
May 2012-Apr 2013 1055 12,241
May 2011-Apr 2012 1794 18,385
May 2010-Apr 2011 672 8,263
Contract Year Total # Amount $
May 2013 –April 2014 17+ 285,409
May 2012-Apr 2013 15 200,352
May 2011-Apr 2012 11 156,280
May 2010-Apr 2011 9 96,252

33. Enterprise Licensing Efficiencies: Single Titles
Contract Year May 2012-April 2013
Journal Title # Uses Cost
Per Use
Replacement
Cost
Cost of
License
Am J Tropical Med & Hygiene 410 $1.58 $4,510 $ 650
Am J Respiratory & Critical Care Med 189 $14.28 $2,079 $2700
Health Affairs 2038 1.81 $22,418 $3690
Infection Control & Hospital
Epidemiology
749 1.50 $8,239 $1125
Public Health Reports 658 4.69 $7,238 $3085
Intern’l J of Tuberculosis & Lung
Diseases
192 5.83 $2112 $1120
Annual Review Series 595 12.60 $6545 $7500
Pediatrics 7846 .68 $86,306 $5300

34. PHIA: Innovative Business Model
• Enterprise licensing saves time, money, and effort.
• Central management of IP addresses ensures quick turn-around.
• Ability to understand change (System, personnel, environment).
• Resources are identified via benchmarking,metrics of direct use through
Digital Library, and requests from PHDs.
• Journals are identified via “Article Delivery” on Library Partner side.
• Alternative delivery of resources strengthens state relationships with
immediate access (within 4-24 hours).

35. EVALUATION PROCESSES
Levels of Data Collection
• Journal SurveyMonkey (baseline data for PHIA and PHD)
• Enterprise Licensing (cost effectiveness over time)
• Vendor statistical reports (validates “use of resources”)
• Monthly reports from “library partners” (measures use and identify
resources to develop Digital Libraries).
• Ongoing capture of suggested e-books, journals, and databases to
enhance collection
• Pre-Post training links
– (data related to knowledge of resources)
• Feedback from hands-on training
• Interviews with leaders and workforce
• Focus group sessions (after one year; subsequently)
• Quarterly/Annual Reports (submitted to NLM)

36. Results
• Digital libraries will be in place in 17 PHDs by July.
• Full-text access to more than 150 e-journals, 5
databases, reports and more.
• Introductory sessions have introduced PHD
leadership to the project and explained obligations.
• More resources have been added to “digital library.”
• Structured training has been held in 15 PHDs.
• More than 600 people have been trained with
backup models currently in test stage.

37. NACCHO Award Brings
Recognition to Project
• March 2012 and 2013, the PHIA PHD Digital Library Project received
a “promising practice award” from NACCHO.
• Presentations have been held at national and local meetings, e.g.,
APHA in Washington, DC in 2011 and Boston in November 2013.
• Article was published in AJPH in January 2014. Project was cited in
August 2012 EID (CDPHE).
• Presentations have been made at national public health meetings,
national and regional library meetings.

38. Testimonial
“Great recognition for this very valuable
initiative. Kudos and thank you.”
Jewel Mullen, MD, MPH, MPA, Commissioner,
Connecticut Department of Public Health

39. Discussion & Outcome
How Do We?
• Expand PHIA to all 50 states.
• Enhance partnerships & relationships.
• Continuously train PH Workforce to improve
competencies (knowledge and skills) given
staff turnover.
• Build a sustainable model including cost
structure and project management.