Dr. Blackford Middleton keynote address at MIE2009 in Sarajevo, Bosnia-Herzogovina, Sept. 1, 2009.

1. Blackford Middleton, MD, MPH, MSc, FACP, FACMI, FHIMSS
Chairman, Center for Information Technology Leadership
Corporate Director, Clinical Informatics Research & Development
Partners Healthcare System
Harvard Medical School
Harvard School of Public Health

2.  What is Clinical Decision Support?
 The Evidence For and Against CDS
 Current examples and R&D Projects from Partners
 The Clinical Decision Support Consortium

3.  “What information consumes
is rather obvious: it
consumes the attention of its
recipients.
 Hence a wealth of information
creates a poverty of attention,
and a need to allocate that
attention efCiciently among the
overabundance of information
sources that might consume it.”
 Changing clinician roles:
 From Omniscient Oracle… to
Knowledge Broker.

4. acted upon
analyzed
compiled
After B Blum, 1984

5.  Medical literature doubling every 19 years
 Doubles every 22 months for AIDS care
 2 Million facts needed to practice
 Covell study of LA Internists:
 2 unanswered clinical questions for every 3 pts
• 40% were described as questions of fact,
• 44% were questions of medical opinion,
• 16% were questions of non‐medical information.
Covell DG, Uman GC, Manning PR.
Ann Intern Med. 1985 Oct;103(4):596-9

6.  Generally, with direct observation, or interview
immediately after clinical encounters, physicians have
approximately one question for every 1‐2 patients
 Independent estimates: 0.6, and 0.62 Q/pt
 Holds across PCP and specialty care
 Holds across urban and rural
Gorman, 1995
Gorman and Helfand 1995

7. An objective measure of the amount of literature generated by medical
scientists annually

8. Original research
Negative
18%
variable
results Dickersin, 1987
Submission
46% 0.5 year Kumar, 1992
17 years Acceptance 14% of
to apply
Koren, 1989
Negative
results 0.6 year
research knowledge
Kumar, 1992
Publication 17:14
35% to patient care! 0.3 year Poyer, 1982
Balas, 1995
Lack of
numbers Bibliographic databases
50% 6. 0 ­ 13.0 years Antman, 1992
Poynard, 1985
Reviews, guidelines, textbook
Inconsistent 9.3 years
indexing Patient Care
Balas Yearbook Medical Informatics 2000gtre4, courtesy M Overhage

9. "...The curse of medical education is the
excessive number of schools. The
situation can improve only as weaker
and superfluous schools are
extinguished."
“Society reaps at this moment
but a small fraction of the
advantage which current
knowledge has the power to
confer.”
Abraham Flexner,
Medical Education in the United States and Canada.
Boston: Merrymount Press, 1910

10.  “Instead of teaching
doctors to be intelligent
map readers, we have
tried to teach every one
to be a cartographer.”
 “We practice healthcare
as if we never wrote
anything down. It is a
spectacle of fragmented
intention.”  Larry Weed, M.D.
 (father of “S.O.A.P.” note)

11.  Prone to error
 Lots of information but no data
 Limited decision support, or quality measurement
 Does not integrate with eHealthcare
 Will not transform healthcare

12.  Medical error, patient safety, and quality issues
 98,000 deaths related to medical error
 40% of outpatient prescriptions unnecessary
 Patients receive only 54.9% of recommended care
 Fractured healthcare delivery system
 Medicare benegiciaries see 1.3 – 13.8 unique providers
annually, on average 6.4 different providers/yr
 Patient’s multiple records do not interoperate
 An ‘unwired’ system
 90% of the 30B healthcare transactions in the US every year
are conducted via mail, fax, or phone

13. “…driven primarily by local
norms that tend towards heavier
TEXAS
use of discretionary services –
El Paso
such as diagnostic testing and
surgical versus less invasive
interventions – for which there
are no clear clinical guidelines.”
790 mi.,
Peter Orszag, OMB Blog
1271 km
http://www.whitehouse.gov/omb/
blog/
McAllen
http://tr.im/sVLA

15.  “A knowledge‐based system is an AI program whose
performance depends more on the explicit presence of
a large body of knowledge than on the presence of
ingenious computational procedures…”
Duda RO, Shortliffe EH. Expert systems research.
Science. 1983 Apr 15;220(4594):261-8.

16.  Algorithmic
Inference Engine
 Statistical
 Pattern Matching Knowledge
Base
 Rule‐based (Heuristic)
 Meta‐heuristic
 Fuzzy sets
 Neural nets
 Bayesian

17. A B
Blois MS. Clinical judgment and computers.
N Engl J Med. 1980 Jul 24;303(4):192‐7.

19.  Formatting
 Results review, “pocket rounds” reports
 Interpreting
 EKG, PFTs, Pap, ABG
 Consulting
 QMR, DxPlain, Iliad, Meditel, Abd Pain, MI risk
 Monitoring
 Alerts: Critical labs, ABx/Surgery, ADEs
 Critiquing
 Vent mgmt, anesthesia mgmt, HTN Rx, Radiology test
selection, Blood products ordering
Kuperman GJ et al. J Hlth Info Mgmt (13)2, pg 81-96

20.  CDS yields increased adherence to guideline‐based care, enhanced
surveillance and monitoring, and decreased medication errors
 (Chaudhry et al., 2006)
 CDS, at the time of order entry in a computerized provider order entry
system can help eliminate overuse, underuse, and misuse.
 (Bates et al., 2003; Austin et al., 1994; Linder, Bates and Lee, 2005; Tierney
et al., 2003)
 For expensive radiologic tests and procedures this guidance at the point of
ordering can guide physicians toward ordering the most appropriate and
cost effective, radiologic tests.
 (Bates et al., 2003; Khorasani et al., 2003)
 Showing the cumulative charge display for all tests ordered, reminding
about redundant tests ordered, providing counter‐detailing during order
entry, and reminding about consequent or corollary orders may also impact
resource utilization
 (Bates and Gawande, 2003; Bates, 2004; McDonald et al., 2004).

21.  Savings potential: $44 billion
 reduced medication, radiology, laboratory, and
ADE‐related expenses
 Advanced CDS systems
 Savings potential only with advanced CDS
 cost give times as much as basic CDS
 generate 12 times greater ginancial return
 A potential reduction of more than 2 million adverse
drug events (ADEs) annually
http://www.citl.org
Johnston et al., 2003

22.  Han YY (Pediatrics 116:6, Dec 2005)
 Analyzed data 13 prior, and 5 months post, implementation
of CPOE in critical care
 Pre CPOE mortality rate 2.8%, Post 6.57%
 3.28 Odds ratio after multivariate analysis adjusting for
covariates
 Conclusion
 Order delay due to lack of pre‐register
 Up front time cost to enter orders
 Nurses away from bedside, at computer
 Altered interactions between ICU team members
 Delayed pharmacy administration
 Problems with order timing (subsequent doses)

23.  Information Errors  HCI/Workglow Errors
 Assumed dose  Patient selection
 Med d/c failure  Med selection
 Procedure‐linked med error  Unclear log on/off
 Give now, and prn d/c error  Meds after surgery
 Antibiotic renewal  Post surgery suspended meds
 Diluent option error  Time/data loss when CPOE
 Allergy display down
 Conglict or duplicate med  Med delivery error
 Timing errors
 Delayed nursing
documentation
 Rigid system design
Koppel R et al. JAMA 293:10, Mar 2005

24. Record Review Proactive Templates/ Warnings/
Scheduling Alerts Guidelines
& Update Reminders Order Sets Feedback
Before the Encounter During the Clinical Encounter After the Encounter
Patient Prepares History and
End of Visit Results Arrive
for the Visit Physical
Patient Health Relevant Consequent Time-Based
Communication
Reminders Information Info Display Actions Checks
Adapted from Osherorff JA, Pifer EA, Sittig DF, Jenders RA, and Teich JM. Clinical
Decision Support Implementers' Workbook. 2004.

28. Bates et. al. JAMA 1998.

29. Secure Messaging
Patient Lists
Schedule
Clinical Alerts
Knowledge
Links
Population Task Management
Management

30. Information Access  Knowledge Linking

31. KnowledgeLink in the Workflow

32. Patient
Disease
Management

33. Smart View: Smart Smart
Data Display Documentation Assessment,
Orders, and Plan
Assessment and
recommendations generated from
rules engine
• Lipids
• Anti‐platelet therapy
• Blood pressure
• Glucose control
• Microalbuminuria
• Immunizations
• Smoking
• Weight
• Eye and foot examinations

34. Medication Orders
Lab Orders
Referrals
Handouts/Education

35. Smart Form Used Control
0% 10% 20% 30% 40% 50% 60% 70% 80%
Up­to­date BP result <0.001
Change in BP therapy if above goal 0.05
Up­to­date height and weight 0.004
Change in therapy if A1C above goal
0.006
Up­to­date foot exam documented <0.001
Up­to­date eye exam documented <0.001
# of deHiciencies addressed <0.001

36. Targets are 90th percentile for
Red, yellow, and green indicators show HEDIS or for Partners providers
adherence with targets
Zero defect care:
• Aspirin
• Beta‐blockers
• Blood pressure
• Lipids

38. Discrepancy
Details

39. More medication changes in visits after diabetes journal submission:
Grant RW et al. Practice-linked Online Personal Health Records for Type 2
Diabetes: A Randomized Controlled Trial. Arch Intern Med. 2008 Sep 8;168(16):
1776-82. .

40.  New appreciation for potential unintended
consequences of CDS
 Knowledge “hardwired” into applications
 Knowledge‐engineering tools assume authors know
what to put into them
 Proprietary knowledge representation standards: not
re‐usable, not easily shared
 Lack of healthcare leadership or resource investment
in processes for knowledge acquisition and
management

41. A Roadmap for Na,onal Ac,on on Clinical Decision
Support
“to ensure that op-mal, usable and effec-ve clinical
decision support is widely available to providers,
pa-ents, and individuals where and when they need it
to make health care decisions.”!
Osheroff JA, Teich JM, Middleton B, Steen EB, Wright A, Detmer DE. J. Am. Med. Inform.
Assoc. 2007;14(2):141-145.

42. To assess, deSine, demonstrate, and evaluate best
practices for knowledge management and clinical
decision support in healthcare information technology
at scale – across multiple ambulatory care settings and
EHR technology platforms.
www.partners.org/cird/cdsc

43.  How do we improve the translation of knowledge in clinical practice guidelines into
actionable CDS in healthcare information technology?
 How do we optimally represent knowledge and data required to make actionable
CDS content in both human and machine readable form?
 How do we collate, aggregate, and curate knowledge content for CDS in a
knowledge portal used by members of the CDS Consortium? How may we use such a
tool to support knowledge management and collaborative knowledge engineering
for clinical decision support at scale, across multiple healthcare delivery
organizations, and multiple domains of medicine?
 How do we demonstrate broad adoption of evidence‐based CDS at scale in a wide
array of HIT products used in disparate ambulatory care delivery settings?
 Further, how do we deploy clinical decision support services in healthcare
information technology in a manner that improves CDS impact?
 How do we take the learnings garnered through the course of these investigations
and broadly disseminate them broadly to key stakeholders?

44. Decision Tables GEM
Arden
GEODE-CM
ONCOCIN EON(T-Helper) GLIF2 GLIF3
MBTA
EON2
Asbru
PRODIGY PRODIGY3
Oxford System
DILEMMA PROforma
of Medicine
PRESTIGE
1980 1990 2000
P. L. Elkin, M. Peleg, R. Lacson, E. Bernstam, S. Tu, A. Boxwala, R. Greenes, & E. H. Shortliffe.
Toward Standardization of Electronic Guidelines. MD Computing 17(6):39-44, 2000

45. Shahar Y, et al. JBI 2004

46.  Knowledge management lifecycle
 Knowledge specigication
 Knowledge Portal and Repository
 CDS Knowledge Content and Public Web Services
 Evaluation
 Dissemination
1. Knowledge Management Life Cycle
2. Knowledge 3. Knowledge Portal and 4. CDS Public Services
Specification Repository and Dashboard
5. Evaluation Process for each CDS Assessment and Research Area
6. Dissemination Process for each Assessment and Research Area

47. Machine Execu$on
Abstract Representa$on
Semistructured Recommenda$on
Narra$ve Guideline
Narra,ve Recommenda,on layer
Semi‐Structured Recommenda,on layer
Narra$ve text of the recommenda$on from the published guideline.
Abstract Representa,on layer
Breaks down the text into various slots such as those for applicable
Machine Executable layer
clinical scenario, the recommended interven$on, and evidence
Structures the recommenda$on for use in par$cular kinds of CDS tools
• basis for the recommenda$on
Knowledge encoded in a format that can be rapidly integrated into a
Reminder and alert rules
CDS tool on a specific HIT plaLorm
Standard vocabulary codes for data and more precise criteria
• Order sets
(pseudocode)
E.g., rule could be encoded in Arden Syntax
A recommenda$on could have several different ar$facts created in this layer,
one for each kind of CDS tool
A recommenda$on could have several different ar$facts created in this
layer, one for each of the different HIT plaLorms

48.  For each knowledge representation layer in CDS stack:
 Data standard (controlled medical terminology, concept
deginitions, allowable values)
 Logic speciSication (statement of rule logic)
 Functional requirement (specigication of IT feature
requirements for expression of rule, etc.)
 Report speciSication (description of method for CDS impact
measurement and assessment)

50. Collaboration
eRoom for Adult
Primary Care

51. 1 Oct 08 9:55pm
• How does everyone feel about this?
• Should we turn the reminder off for a
shorter period of time if
“Done Elsewhere” is chosen?
51

52. Personal Health Rule builder
Science
Informa,on Network
Knowledge
respository
Clin. Inf. System
Pa,ent
Rule engine
Medical Professional Community (”Crowd”)
PeKer K. Risøe
HSPH HPM512 2009

53. “I conclude that
though the individual
physician is not
perfectible, the
system of care is,
and that the
computer will play a
major part in the
perfection of
future care systems.”
Clem McDonald, MD NEJM 1976
Thank you!
Blackford Middleton, MD
bmiddleton1@partners.org
www.partners.org/cird
www.citl.org