The slides from the keynote given by Dr. Dan Malone RPh, PhD at the First International Drug-Drug Interaction Knowledge Representation Workshop on October 6th 2014 (http://icbo14.com/sessions/drug-drug-interaction-knowledge-representation-workshop/). Posted with his permission.

1. Jumping the crevasse between
assertions of drug interactions and
clinical relevance
Daniel C. Malone, RPh, PhD
Professor
The University of Arizona

2. The Problem!
You wouldn’t believe how many BIG BAD drug‐drug interactions
there are. Just ask your doctor about all the DDI alerts she gets!

3. Prescriber’s Knowledge
Computer Screening
Pharmacist’s Knowledge
Latent Failures
Patient Risk Factors
Drug Administration
Patient Education
Monitoring
ADR
A + B
“When the Holes Line Up”
Defenses
Hansten PD, Horn JR. Modified from: James Reason, Human Error, 1990

4. Market Removals Due to
Drug‐Drug Interactions
• Terfenadine (Seldane®) – 1998
• Mibefradil (Posicor®)‐ 1998
• Astemizole (Hismanal®) – 1999
• Cisapride (Propulsid®) – 2000
• Cerivastatin (Baycol®) – 2001

5. 5
VA practitioner knowledge of
drug-drug interactions
• 168 responses to a postal survey
• 135 physicians
• 22 nurse practitioners
• 11 physician assistants
• Clinicians correctly categorized 53% of drug-drug
interactions
• But, only
• 64% correctly answered sildenafil-isosorbide (28% not sure)
• 58% correctly answered cisapride-erythromycin (27% not
sure)
• 43% correctly answered phenelzine-sertraline (46% not sure)
Source: Glassman PA et al. Medical Care 2002; 40:1161-1171

6. 6
National Survey of Prescribers’
Knowledge of DDIs - Methods
• Postal survey of prescribers (12,500)
• Sample: Identified via pharmacy claims to a pharmacy benefit
manager
• Cases – history of 1 or more DDI’s
• Controls – match on prescribing either objective or precipitant
medication
• Practice characteristics
• Respondents asked to classify 14 drug pairs
• Contraindicated
• May be used together but with monitoring
• No interaction
• Not sure
• Usual source of drug-drug interaction information
Ko et al. Drug Saf. 2008;31(6):525‐536.

7. 7
Summary of Prescriber DDI
Knowledge
• Correct classification of drug-pairs
• Mean (SD) = 6.0 (3.1)
• Overall – 42.7% of drug pairs correctly
identified
• 30% or more of respondents answered “unsure”
for 8 of the 16 the drug pairs
• 2 combinations are contraindicated

8. Background
• Drug‐drug interaction alerts can be useful
• Prescriber knowledge of DDIs is lacking1,2
• 42.7% of drug pairs correctly identified1
• VA practitioners generally agree that DDI alerts
are useful3
1) Ko et al. Drug Saf. 2008;31(6):525‐536. 2) Glassman. Med Care. 2002;40(12):1161‐
1171. 3) Ko et al. JAMIA 2007;14:56‐64

9. Sensitivity of Computer Software to Detect Drug
Interactions in Arizona Pharmacies (N=64)
75%
80%
83%
87%
88%
86%
89%
45%
81%
90%
75%
84%
70%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Warfarin + sulfamethoxazole/trimethoprim
Warfarin + naproxen
Warfarin + gemfibrozil
Warfarin + fluconazole
Warfarin + amiodarone
Simvastatin + gemfibrozil
Simvastatin + amiodarone
Simvastain + itraconazole
Nitroglycerin + sildenafil
Digoxin + itraconazole
Digoxin + clarithromycin
Digoxin + amiodarone
Carbamazepine + clarithromycin
Saverno et al. JAMIA; 2011:18:32-37

10. DDI Prevalence in Elderly
• Elderly veterans with new
DDI at ED discharge:1
13%
• Older adults exposed to a
“major” DDI:2 4%
• Medicare Part D enrollees
exposed to certain DDIs:
7.3%
1) J Am Geriatr Soc. 2008;56:875‐80. 2) JAMA. 2008;300:2867‐78.

11. Who Prescribes Drug‐Drug
Interactions?
70
60
50
40
30
20
10
0
Potential Drug‐Drug Interactions
by the Same Prescriber

12. Concordance Among DDI Compendia

13. Methods
• Review of print versions of compendia for
“major” interactions
• Inclusion Criteria:
• DDI listed in at least 3 compendia
• Available in US for human use
• Medications likely to be dispensed in community
pharmacy
• Medications likely captured in electronic database
• Interacting medications not used for therapeutic
benefit
Abarca J et al. J Am Pharmacist Assn 2003: 44:136-141.

14. Rating Systems & Selection
Criteria
• Evaluation of Drug Interactions
• Uses 4-item summary measure based on:
• Potential harm to the patient
• Frequency and predictability of occurrence
• Degree and quality of documentation
• Code 1: highly clinically significant
• Code 2: moderately clinically significant
• Code 3: minimally clinically significant
• Code 4: not clinically significant
Abarca J et al. J Am Pharmacist Assn 2003: 44:136-141.

15. Rating Systems & Selection
Criteria
• Drug Interaction Facts
• Uses 5-item summary measure based on:
• Severity (i.e., major, moderate, minor)
• Documentation (i.e., established, probable,
suspected, possible, unlikely)
• 1: major/established, probable, suspected
• 2: moderate/established, probable, suspected
• 3: minor/established, probable, suspected
• 4: major,moderate/possible
• 5: minor/possible or any/unlikely
Abarca J et al. J Am Pharmacist Assn 2003: 44:136-141.

16. Rating Systems & Selection
Criteria
• Drug Interactions: Analysis and Management
• Used 5-item summary measure based on:
• Severity
• Corresponding documentation
• Availability of alternatives are considered
• 1: Avoid combination
• 2: Usually avoid combination
• 3: Minimize risk
• 4: No action needed
• 5: No interaction
Abarca J et al. J Am Pharmacist Assn 2003: 44:136-141.

17. Rating Systems
• Drug-REAX (MicroMedex)
• Used 5-item severity scale
• Major
• Moderate
• Minor
• None
• Not specified
• No summary measure
Abarca J et al. J Am Pharmacist Assn 2003: 44:136-141.

18. Results
• 62 ‘major’ DDIs identified
• Additional criteria:
• 18 ‘major’ DDIs excluded :
• 8 DDIs - not available in U.S. (e.g., terfenadine, mibefradil)
• 4 DDIs – not dispensed from a community pharmacy
• 4 DDIs – not likely to be captured in electronic database (e.g,
ethanol, tyramine-containing foods)
• 1 DDI – occurs upon discontinuation (clonidine-β blockers)
• 1 DDI – used for therapeutic benefit (phenothiazine-SSRI)
Abarca J et al. J Am Pharmacist Assn 2003: 44:136-141.

19. Agreement Among Four Drug
Interaction Compendia
 DDIs in 4 of 4: 2.2% (9/406)
 DDIs in 3 of 4: 8.6% (35/406)
 DDIs in 2 of 4: 17.4% (71/406)
 DDIs in 1 of 4: 71.7% (291/406)
 Intra-class Correlation Coefficient: -0.092
Abarca J et al. J Am Pharmacist Assn 2003: 44:136-141.

20. Clinical Pharmacology and Therapeutics 2011: 87:48‐51

21. “Only 13.6% of “critical interactions” listed in both compendia

22. Quantity and Quality of DDI Evidence
– Interactions with Macrolides
Harper, Jackson, and Malone – unpublished data

23. Reasons for Differences in Drug‐Drug
Interaction Compendia
• Rating systems are different
• Lack of high‐quality studies
• Case reports
• Small pharmacokinetic studies
• Rating is subjective
• Different editors/contributors
• Few “tools” for evaluating “poor quality” evidence

24. Hierarchy of Evidence
Systematic Reviews
RCTs
Controlled Clinical Trials
and Observational Studies
Uncontrolled Observational Studies
Case reports and case series
Expert Opinions
Lowest risk of bias
Hypothesis
Testing
Hypothesis
Generating

25. Theophylline – Allopurinol:
A Major Drug Interaction?
Theophylline
1A2 16%
35% 1-methylxanthine
1A2 40% 3A, 2E1
3-Methylxanthine
1,3 dimethyluric acid
Xanthine Oxidase
1-methylurinc acid

26. Theophylline – Allopurinol:
A Major Drug Interaction?
• Listed as major in several CDS databases
• Two studies found no change in
theophylline PK with allopurinol 300 mg
daily x 7 days.1
• One study found ~25% increase in AUC
and half-life after 2 weeks of concurrent
allopurinol 300 mg BID.2
1. Vozeh S et al. CPT. 1980;27:194‐7. 2. Manfredi RL et al. CPT 1981;29:224‐9

27. Theophylline Label
Table II. Clinically significant drug interactions with theophylline*.
Drug Type of Interaction Effect
Alcohol A single large dose of alcohol (3
mL/kg of whiskey) decreases
theophylline clearance for up to 24
hours.
30% increase
Allopurinol Decreases theophylline clearance at
allopurinol doses ≥600 mg/day.
25% increase
Source: http://dailymed.nlm.nih.gov/dailymed

28. Azithromycin
Product Label
• “The following drug interactions have not been reported in clinical
trials with azithromycin; however, no specific drug interaction
studies have been performed to evaluate potential drug‐drug
interaction. Nonetheless, they have been observed with macrolide
products. Until further data are developed regarding drug
interactions when azithromycin and these drugs are used
concomitantly, careful monitoring of patients is advised:
• Digoxin–elevated digoxin levels.
• Ergotamine or dihydroergotamine–acute ergot toxicity
characterized by severe peripheral vasospasm and dysesthesia.
• Triazolam–decrease the clearance of triazolam and thus may
increase the pharmacologic effect of triazolam.
• Drugs metabolized by the cytochrome P450 system–elevations
of serum carbamazepine, cyclosporine, hexobarbital, and
phenytoin levels.”
Source: http://dailymed.nlm.nih.gov

29. Health Systems Approach to DDIs
• Evidence for DDIs is lacking
• Very few well-controlled studies
• Lack of concordance among DDI compendia1
• Differing severity rating systems, terminology,
methodologies
• Limitations of DDI clinical decision support2-4
• “Alert fatigue”
• High rates of alert override
1) Abarca et al. J Am Pharm Assoc (2003). 2004;44(2):136-141. 2) Grizzle et al. Am J Manag Care.
2007;13(10):573-578. 3) Murphy et al. Am J Health Syst Pharm. 2004;61(14):1484-1487. 4) Abarca et
al. J Manag Care Pharm. 2006;12(5):383-389.

30. Problems With DDIs in CDS Systems
• Classification systems that are based on rules of
questionable relevance
• Reliance on label without informed review or
evaluation
• Assumption of ‘class–based’ interactions
• Conclusion: “the current DDI alert system is broken”1
1Hatton RC, et al. Ann Pharmacother. Mar 2011;45(3):297‐308.

31. Model for DDI alerting in CPOE
Knowledge‐base (rules)
Integration
software
Patient database
(meds, etc.)
Knowledge
engineering
Literature,
domain
expertise
Potential DDI
report
MD
Pharmacist
Nurse

32. Too many alerts!
• Organizational alert override rates are high
• 96% in Netherlands1
• US medical center2
• 461 different physicians, 18,354 medication
orders
• 2,455 alerts
• DDI override rate: 95.1%
1van der Sijs H, Aarts J, Vulto A, Berg M. Overriding of drug safety alerts in computerized physician order entry. J Am Med Inform
Assoc. 2006 2006 Mar‐Apr;13(2):138‐47. 2Bryant AD, Fletcher GS, Payne TH. Drug interaction override rates in the Meaningful use
era: no evidence of progress. Applied Clinical Informatics 2014; 5:802‐813

33. Case Study ‐ Sentara Healthcare
• Amiodarone + warfarin DDI fires 1000 times
• 960 – alert overridden/bypassed
• 40 – warfarin dose adjusted
• 10 – INR or CBCs ordered
• 30 – INR or CBC frequency adjusted
• 50 – pt. education or anticoagulation clinic scheduled
• 5 – alert overridden, INR > 5

34. Practitioners’ Views on DDI Alerts
• Methods
• Random sample of 100 to 125 practitioners
in outpatient clinics at 6 VAMCs
• Results (N = 258) (Response rate: 36%)
• Internal medicine – 31%
• Primary care – 29%
• Years in practice (mean) - 14.8 (10.6)
• Rx’s written per week – 97.7 (155.5)
Ko et al. JAMIA 2007; 14: 56-6434

35. Practitioners’ Views of Alerts
Prescribers’ Views on DDI Alerts Mean
Response
I am satisfied with the accuracy of DDI
alerting system
3.1
DDI alerts provide me with information I
already know
3.4
The DDI system provides alerts that seems
to be just about exactly what I need
2.7
DDI alerts change my initial prescribing
decisions
1.9
1= strongly disagree; 2 = disagree; 3 = neither disagree or agree;
4 = agree; 5 = strongly agree
35

36. Attitudes Toward Improving DDI
Alerts
Attribute Mean
Response
DDI alerts should be accompanied by management
alternatives
3.8
DDI alerts should be accompanied by more detailed
information about the interaction
3.7
DDI alerts should only appear once in the order entry process 3.6
DDI alerts are presented in a useful format 3.2
1= strongly disagree; 2 = disagree; 3 = neither disagree or agree;
4 = agree; 5 = strongly agree
36

37. Top Ranked Changes to VA’s
Alerting System
Change Weighted
Preference Score
Provide management options for DDIs 1.53
Show DDI alerts one at a time 1.24
Make it more difficult to override lethal
interactions
0.95
Eliminate all DDI alerts 0.05
Rank 1 = 3 points, Rank 2 = 2 points, Rank 3 = 1 point
37

38. “Asthma sufferer wins $28.6 million award” –
Seattle Time (9/3/94)
• 24 year old man on theophylline went to
ER with an infection, prescribed
ciprofloxicin by ER physician
• Theophylline levels doubled, patient had
permanent brain damage
• Patient awarded $22.5 million in pain
and suffering

39. Tranylcypromine (Parnate®) +
Phenylpropanolamine (Tavist‐D®)
• Patient on Parnate went to ER for URI:
received Rx for Tavist-D®
• Rx filled at his regular pharmacy
(technician ignored computer warning)
• Patient developed an acute hypertensive
episode resulting in a stroke
• Patient committed suicide, leaving a note
that the disabilities induced by the stroke
were intolerable
Williams KG. Am J Health-Syst Pharm 1996;53:1709.

40. Current “Lists” of Important DDIs
Source Purpose Systematic
Review
Expert
Panel
Consensus
Process
Classen 2011 /
Leapfrog List
Verify whether an inpatient COPE
system has the potential to intercept
critical DDIs
Unknown Unknown Unknown
CMS FTag 329‐
Unecessary
Medications
Medications requiring increased
involvement from consultant
pharmacists.
Unknown Yes Unknown
Malone 2004
To develop a list of clinically important
DDIs in outpatient setting
Yes; primary
literature and
tertiary
references
Yes
Modified Delphi
process
Phansalkar 2012 /
ONC List
To reduce alert fatigue and describe
the most clinically significant high‐priority
DDIs
Unclear Yes Mixed approach
Pharmacy Quality
Alliance
Evaluate prescription drug plans and
ambulatory/community pharmacists
No No Yes
van Roon 2005 /
G‐standard
Dutch database for CDS in Netherlands
Yes Yes Yes

41. Rationale for Changing the Approach
• Drug interaction clinical decision support should
improve patient safety
• Instead….
• Evidence lacking to support effectiveness
• Excessive irrelevant alerts
• Wide variation across health systems for DDI CDS
• Pandemic clinician annoyance
• What’s needed
• Guiding principles for evidence‐based, clinically relevant,
consistent alerts with improved usability
• Evidence of effectiveness

42. Rationale for Changing the Approach
• Concerns about process used to generate
current “lists”
• No well‐defined, broadly accepted, uniform
standard for rating:
• Strength (quality) of DDI evidence
• Strength of recommendations for patient
risk management
• Concerns about liability

43. AHRQ Conference Series
• DDI CDS conference series to:
• Develop guidelines for systematic appraisal of DDI
evidence (Evidence Workgroup)
• Recommend principles for including DDIs in CDS
(Content Workgroup)
• Establish preferred strategies for presenting DDI
CDS (Usability Workgroup)
• Consensus recommendations by international
experts
https://sites.google.com/site/ddiconferenceseriessite

44. Project At A Glance

45. DDI CDS Conference Website
https://sites.google.com/site/ddiconferenceseriessite/home

46. DRug Interaction eVidence Evaluation
(DRIVE) Instrument
Category Evidence
Sufficient
Sufficient
evidence to
evaluate a
clinically
relevant
drug
interaction
One or more of the following:
• Well‐designed and executed prospective controlled studies
• Well‐designed and executed retrospective controlled studies
• Case reports or series demonstrating probable or highly
probable causality of an interaction (DIPS score of 5‐10)*
• Reasonable extrapolation on the basis of:
• Studies of drugs with similar pharmacologic properties
• Studies with in vitro substrate data
• Human genetic polymorphism studies
* Drug Interaction Probability Scale; Horn et al. Ann Pharmacother 2007;41:674‐80.

47. DRug Interaction eVidence Evaluation
(DRIVE) Instrument
Category Evidence
Insufficient
Insufficient
evidence to
evaluate a
clinically
relevant drug
interaction
One or more of the following, without supporting
evidence from the “sufficient” category:
• Extrapolation on the basis of studies with in vitro
inhibitor or inducer data
• Case reports or series demonstrating only possible or
doubtful causality of an interaction (DIPS score of <5)
• Studies of poor design or execution
• Hypotheses‐generating research methods
• Unsupported data on file or unsupported
recommendations from product manufacturer
• Animal data
* Drug Interaction Probability Scale; Horn et al. Ann Pharmacother 2007;41:674‐80.

48. AHRQ Conference Series
Workgroup Recommendations
• Establish national expert consensus panel
• Oversight by national organization
• Develop and maintain standard set of DDIs for CDS
• Employ transparent, systematic, evidence‐driven
process
• Grade quality of evidence
• Incorporate expert and clinical advice
• Provide graded recommendations for risk management
• Collect and incorporate user feedback
• Ongoing reevaluation and updates

49. Transparent, Systematic Process for a
Standard Set of DDIs
1. Evidence
Synthesis &
Grading
2. Expert
Advice
National DDI Expert Panel
Oversight by Central
Organization
3. Consensus
Graded
Recommendations
6. Reevaluation
& Updates
5. Stakeholder
Feedback
4. CDS
Implementation

51. Support
Agency for Healthcare Research and Quality
• Grant #1R13HS021826‐01 (PI –Malone)
• Grant #1R13HS018307‐01 (PI –Malone)
National Library of Medicine
Grant #R01LM011838‐01 (PI – Boyce)
Additional Support
• Cerner
• Epocrates
• First Databank
• Truven Health Analytics
• Wolters Kluwer Health
• Elsevier

52. Acknowledgements
• Lisa Hines, PharmD, University of Arizona
• Richard T. Scheife, PharmD, FCCP, Tufts University
• Darrell R. Abernethy, MD, PhD, Food and Drug Administration
• Richard Boyce, PhD, University of Pittsburgh
• Clarissa Borst, PharmD, Elsevier
• Sophie Chung, PharmD, Epocrates, athenahealth, Inc.
• Susan Comes, PharmD, Epocrates, athenahealth, Inc.
• John Horn, PharmD, University of Washington
• Gretchen Jones, PharmD, (formerly) Epocrates, athenahealth, Inc.
• Jeremiah Momper, PharmD, PhD, University of California, San Diego
• Alissa Rich, PharmD, MBA, (formerly) Cancer Treatment Centers of America
• Stephen J Sklar, PharmD, Wolters Kluwer Health
• Christine D Sommer, PharmD, FDB (First Databank, Inc.)
• Tricia Lee Wilkins, PharmD, MS, Office of the National Coordinator for Health Information Technology
• Michael A Wittie, MPH, Office of the Chief Medical Officer, Office of the National Coordinator for Health
Information Technology
• Samantha K Wong, BSPharm, RPh, Cerner

53. Acknowledgements
• Lisa Hines, PharmD, University of Arizona
• Hugh Tilson, MD, DrPH, University of North Carolina
• David W Bates, MD, Harvard Medical School
• Joseph T Hanlon, PharmD, MS, University of Pittsburgh
• Philip Hansten, PharmD, University of Washington
• Amy L Helwig, MD, MS, Office of the National Coordinator for HIT
• Stefanie Higby‐Baker, RPh, MHA, CPHIT, Cerner Multum
• Shiew‐Mei Huang, PhD, Food and Drug Administration
• David R Hunt, MD, FACS, Office of the National Coordinator for HIT
• Marianne le Comte, PharmD, Royal Dutch Association for the Advancement of Pharmacy
• Karl Matuszewski, MS, PharmD, FDB (First Databank, Inc.)
• Gerald McEvoy, PharmD, ASHP
• Anthony Perre, MD, Cancer Treatment Centers of America
• Lynn Pezzullo, RPh, CPEHR, Pharmacy Quality Alliance
• John Poikonen, PharmD, MedVentive
• Kathy Vieson, PharmD, Elsevier Clinical Solutions
• David M Weinstein, RPh, PhD, Lexicomp, Wolters Kluwer Health
• Michael A Wittie, MPH, Office of the National Coordinator for HIT

54. Acknowledgements
• Thomas H Payne, MD, FACP, University of Washington
• Bruce W Chaffee, PharmD, University of Michigan Health System
• Raymond C Chan, PharmD, Sentara
• Peter A Glassman, MBBS, VA, Greater Los Angeles Healthcare System
• Brian Galbreth, PharmD, PeaceHealth Southwest Medical Center
• Christian Hartman, PharmD, MBA, FSMSO, Pharmacy OneSource, Wolters Kluwer Health
• Seth Hartman, PharmD, Oregon Health & Science University
• Joan Kapusnik‐Uner, PharmD, FDB (First Databank, Inc.)
• Gilad J Kuperman, MD, PhD, New York‐Presbyterian Hospital
• Gordon Mann, RPh, Clinical Informatics, Epic
• Shobha Phansalkar, RPh, PhD, Wolters Kluwer Health (formerly with Partners Healthcare System)
• Alissa Russ, PhD, Roudebush VA Medical Center
• Hugh Ryan, MD, Cerner Corporation
• Howard Strasberg, MD, MS, Wolters Kluwer Health
• Amanda Sullins, PharmD, Cerner
• Vicki Tamis, PharmD, BCPS, PeaceHealth Southwest Medical Center
• Heleen van der Sijs, PharmD, PhD, Erasmus Medical Center

55. Thanks!
Questions and Comments?
Answers (Even Better)?!