Más contenido relacionado
Similar a Text Mining the MAUDE Database (20)
Text Mining the MAUDE Database
- 1. Signal Detection of
Adverse Medical Device
Events in the
FDA MAUDE Database
Eric Brinsfield, MS
Presenter & Research Collaborator
David Olaleye, MSCE, PhD
Author & Primary Research Statistician
SAS Institute Inc.
Cary, NC
Company Confidential - For Internal Use Only
Copyright © 2010, SAS Institute Inc. All rights reserved.
- 2. Disclosure Statement
Both presenters are employees of SAS Institute
We have no conflicts of interest
Disclaimer
The views and opinions expressed in the following
PowerPoint slides are those of the individual presenters
and should not be attributed to ISPE or to SAS Institute.
2
Company Confidential - For Internal Use Only
Copyright © 2010, SAS Institute Inc. All rights reserved.
- 3. Study Objectives
Determine if text mining can be used to:
Detect signals of adverse events in
spontaneous reporting data
Better understand or triage signals
generated by traditional disproportionality
methods
Phase 1:
Evaluate unsupervised text mining
Using FDA MAUDE database
Focused on stents
3
Company Confidential - For Internal Use Only
3
Copyright © 2010, SAS Institute Inc. All rights reserved.
- 4. Definitions
Safety Signal
“A report or reports of an event with an unknown
causal relationship to treatment that is recognized as
worthy of further exploration and continued surveillance.”
» Council for International Organizations of Medical
Sciences (CIOMS)
“Recognition” is often the results of
Analytical or automatic signal detection methods that look
for unexpected patterns in data sources such as:
» Spontaneously reported data
» Observational healthcare data
» Insurance claims data
4
Company Confidential - For Internal Use Only
Copyright © 2010, SAS Institute Inc. All rights reserved.
- 5. Methods
Data Source = MAUDE
FDA Spontaneous Reporting System Database for Med. Devices
MAUDE - Manufacturer and User Facility Device Experience
Contains the narrative entered by the reporter
Target Devices
endovascular graft system and coronary stent devices
devices classified as a stent in the “product_category_code”:
» MAF, MIH, NIN, NIO, NIP, NIM, and NIQ
» http://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfPCD/classif
ication.cfm?ID=896
5
Company Confidential - For Internal Use Only
5
Copyright © 2010, SAS Institute Inc. All rights reserved.
- 6. MAUDE (2001-2008)
Spontaneous “safety” reports on medical devices
Strengths:
Only surveillance system which covers devices
marketed in the entire US
Largest number of case reports on adverse outcomes
and malfunctions for medical devices
Provides opportunity to detect signals of new, rare
and unusual adverse clinical outcomes
» Usually warrants further investigation
Includes narrative description of events
» Better than AERS which does not include narrative
6
Company Confidential - For Internal Use Only
Copyright © 2010, SAS Institute Inc. All rights reserved.
- 7. Characteristics of Spontaneous Report
Suitable for hypotheses generation; not for confirmation or
rates computation
Exhibit under-reporting and other reporting biases; lack of
control group, etc.
Data quality, ascertainment, accuracy and completeness of
information are usually poor
Includes events and incidents not causally related to
medical device exposure
Does not distinguish between label versus off-label uses of
approved products
Contains minimal or no patient history and other potential
causal factors
Do not provide estimates of exposure (worse in drugs) 7
Company Confidential - For Internal Use Only
Copyright © 2010, SAS Institute Inc. All rights reserved.
- 8. Study Design: Case-Series
Source Population
All MAUDE reports received between 2006 – 2010
(N=35954)
Study Device Cohort Study Events
MAUDE reports for stents • Death (D)
with product codes: • Injury (I)
MAF, MIH, NIN, NIO, • Malfunctions (M)
NIP, NIM, and NIQ • Other (O)
Device-Adverse Outcomes Pairs
(N=28)
8
Company Confidential - For Internal Use Only
Copyright © 2010, SAS Institute Inc. All rights reserved.
- 9. Analysis Steps
1. Attempt analysis using text mining alone
Ignore device
Evaluate if general text mining provides any insights
2. Perform standard disproportionality analysis on
structured data
PRR, EBGM, Adj. Residual
3. Identify device-AE pairs that have:
High scores
Especially in the “Other” classification
4. Investigate the device with text mining
Include all outcome classifications
9
Company Confidential - For Internal Use Only
Copyright © 2010, SAS Institute Inc. All rights reserved.
- 10. Text Mining Process
Parse terms to create documents-terms frequency matrix
Use singular value decomposition (SVD) to measure
association and perform hierarchical clustering
Use entropy method to cluster SVDs for documents
classification
10
Company Confidential - For Internal Use Only
Copyright © 2010, SAS Institute Inc. All rights reserved.
- 11. Sample Narrative - Injury
Example of Manufacturer Report - Injury
the product labeling for a p154 states that this product is indicated for
use in pts with obstruction of major biliary ducts. the product labeling
also states that the stent may be increased post-placement by
expanding with a larger diameter balloon. the following was obtained
through conversation with the user facility on 1/15/98. after deciding that
a ptca procedure in a renal artery did not yield adequate results, the md
attempted to place a medium biliary stent in the artery. the physician
reported to co. that he had difficulty visualizing the stent and that it was
difficult to place. he deployed the stent, but was not satisfied with the
outcome. in response, he decided to place a second stent inside of the
first. however, the stents interlocked and the physician decided to
remove both stents, he was able to withdraw the stent up to sheath tip in
the femoral artery, but needed a vascular surgeon to completely remove
them. info regarding the type of removal procedure has not been
provided to co. the physician further stated that he believes that the first
stent had not fully opened. it was mounted on a meditech glidex balloon.
11
Company Confidential - For Internal Use Only
Copyright © 2010, SAS Institute Inc. All rights reserved.
- 12. Concept Linking and Exploration
12
Company Confidential - For Internal Use Only
Copyright © 2010, SAS Institute Inc. All rights reserved.
- 13. Text mining clusters over all reports
13
Company Confidential - For Internal Use Only
Copyright © 2010, SAS Institute Inc. All rights reserved.
- 14. Text Mining Over All Reports
Results are interesting
But inconclusive
Lose track of the device
But may detect new trends for all stent devices in study
Cannot really do comparisons due to lack of denominator.
(Same problems as always with spontaneous reports.)
Next, run disproportionality to narrow the focus…
14
Company Confidential - For Internal Use Only
Copyright © 2010, SAS Institute Inc. All rights reserved.
- 15. Disproportionality Results for
Carotid Stent (NIM)
Adjusted MGPS
Adverse Event Freq Residual (EBGM)
Death 276 0.8 0.86728
Injury 2043 1.1 1.130264
Malfuntion 271 0.4 0.424803
Other 24 1.6 3.648191
* Adjusted Residual: Flagged at values over 1.5
EBGM: Flagged at values over 2.0
15
Company Confidential - For Internal Use Only
Copyright © 2010, SAS Institute Inc. All rights reserved.
- 16. Target for Text Mining Evaluation
NIM showed high proportion of “Other”
Based on relative percentage of reports
Based on signal scoring algorithms
All methods suggested a flag
Although only 24 cases, the method could show promise
Run text mining against all NIM reports
Include all outcomes to fully understand reports
Look for possible explanations or hypotheses
16
Company Confidential - For Internal Use Only
Copyright © 2010, SAS Institute Inc. All rights reserved.
- 17. N
Text Mining Clusters for NIM
+percutaneous +unk dissection performance +unstable repaired 376
+malfunction 'device remains implanted'
reactions collapsing +'premature deployment' cracks +'product quality 41
issue' replaced performance collapsed
malfunction fractures +fracture +'inaccurate delivery' fractured 846
malfunctions drift +'premature deployment'
+normal dissection +unk 'no information' +na collapsing unk fractured 69
'device issue' broke 'no known device problem' +break broken +'shaft 227
break' reaction reactions
abnormal +fracture +continuous +bent cracks unk +break fractured 65
filter +na 'difficult to advance' breaks reaction +bent +malfunction 150
replaced
'device remains implanted' collapsed 'no flow' performance +crack 44
+break +collapse filter
fractured repair +crack breaks +collapse +'product quality issue' 'no 31
flow' reaction 17
Company Confidential - For Internal Use Only
Copyright © 2010, SAS Institute Inc. All rights reserved.
- 18. 18
Company Confidential - For Internal Use Only
Copyright © 2010, SAS Institute Inc. All rights reserved.
- 19. Manual Review of Text for “Other”
Most were not adverse events that persisted
Some seemed like “FYI” reports.
Two included notification of a formal study
Most patients still had the stent in place (assumed)
Some cases of installation problems:
Potential installer error
Most did not involve an adverse event
19
Company Confidential - For Internal Use Only
Copyright © 2010, SAS Institute Inc. All rights reserved.
- 20. Conclusion
Text mining shows promise for recognizing
primary words and patterns
Hard to form hypotheses from bulk text mining on
spontaneous database
Combination with disproportionality analysis
creates signals that can be further analyzed with
text mining
Terms in the “Other” category overlap with other
categories
20
Company Confidential - For Internal Use Only
20
Copyright © 2010, SAS Institute Inc. All rights reserved.
- 21. Next Steps
Need further analysis that includes:
Large target group for further triage
“Other” was too small in this case
Preferred term matching and encoding
To clean up fuzziness and reduce clusters
Content categorization
Look for more structure and combine with ontologies
Sentiment analysis
Determine if overall sentiment was good or bad
21
Company Confidential - For Internal Use Only
21
Copyright © 2010, SAS Institute Inc. All rights reserved.
- 22. Final Thoughts
“What the future portends is more and more
information — Everests of it. There won’t be
anything we won’t know. But there will be no
one thinking about it.”
From:
New York Times - August 13, 2011
The Elusive Big Idea
By NEAL GABLER
Neal Gabler is a senior fellow at the Annenberg Norman Lear Center at the University of
Southern California and the author of “Walt Disney: The Triumph of the American Imagination.”
We need to help make time for thinking.
22
Company Confidential - For Internal Use Only
Copyright © 2010, SAS Institute Inc. All rights reserved.
- 23. Thank You
Contacts:
Eric Brinsfield eric.brinsfield@sas.com
David Olaleye david.olaleye@sas.com
Company Confidential - For Internal Use Only
Copyright © 2010, SAS Institute Inc. All rights reserved.