A short tutorial on sensitivity, specificity and likelihood ratios. In this presentation, I demonstrate why likelihood ratios are better parameters compared to sensitivity and specificity in real world setting.

1. Sensitivity, Specificity
and Likelihood Ratios
K.S. Chew
Faculty of Medicine and Health Sciences
Universiti Malaysia Sarawak
Email: kschew@unimas.my1/25/2016 1

2. Sensitivity
• Proportion of patients with disease who are tested positive
with a test
• A 100% sensitive test will not have any false negative results
(although it may have a high rate of false positive results)
• Therefore, a negative result of a highly sensitive test means
it is likely to be a true negative (it rules out the disease)
“SN-OUT”
1/25/2016 2

3. Specificity
• Proportion of patients without disease who are tested
negative with a test
• A 100% specific test will not have false positive results
(although it may have high rate of false negative results)
• Therefore, a positive result of a highly specific test means it
is likely to be true positive (it rules in the disease)
“SP-IN”
1/25/2016 3

4. Sensitivity
Disease +ve Disease -ve
Test +ve a (TP) b (FP)
Test –ve c (FN) d (TN)
1/25/2016 4
TP = True positive
FN = False negative
FP = False positive
TN = True negative
Sensitivity = (a)/(a+c)

5. Positive Predictive Value
Disease +ve Disease -ve
Test +ve a (TP) b (FP)
Test –ve c (FN) d (TN)
1/25/2016 5
TP = True positive
FN = False negative
FP = False positive
TN = True negative
Positive PV = (a)/(a+b)

6. Specificity
Disease +ve Disease -ve
Test +ve a (TP) b (FP)
Test –ve c (FN) d (TN)
1/25/2016 6
TP = True positive
FN = False negative
FP = False positive
TN = True negative
Specificity = (d)/(b+d)

7. Negative Predictive Value
Disease +ve Disease -ve
Test +ve a (TP) b (FP)
Test –ve c (FN) d (TN)
1/25/2016 7
TP = True positive
FN = False negative
FP = False positive
TN = True negative
Negative PV = (d)/(c+d)

8. Sensitivity and Specificity
Image taken from: http://library.med.utah.edu/WebPath/TUTORIAL/BIOSTATS/BIOSTATS.html
1/25/2016 8
To increase sensitivity, shift to the left
(purple line)
But by shifting to the left, it
increases proportion of
false positive, which means
reduced specificity

9. Sensitivity and Specificity
Image taken from: http://library.med.utah.edu/WebPath/TUTORIAL/BIOSTATS/BIOSTATS.html
1/25/2016 9
To increase specificity, shift to the right
(purple line)
But by shifting to the right,
it increases proportion of
false negative, which
means reduced sensitivity

10. Example: Troponin assays
• First generation assay: cut-off 0.5 microgm/l
• 3rd generation assay: 0.05 – 0.10 microgm/l
• High-sensitive troponin (hsTn): 0.0030 microgm/l
• High-sensitive Roche Elecsys: 0.0014 microgm/l
• The diagnostic sensitivity of hsTn assays (ability to rule-out
MI) are of the order of 90–95% when tested at the point of
admission (still misses 5 - 10% of cases)
1/25/2016 10
Ref: Gamble et al, Br J Cardiol. 2013;20(4)

11. Causes of elevated troponins
• Myocardial ischemic conditions
• ACS
• Myocardial ischemic conditions other than ACS
• Systemic conditions
• Myocardial injury without ischemic insults
• Systemic conditions – renal failure, sepsis
• Specific identifiable precipitants – cardiac contusion, burns
>30% BSA
1/25/2016 11

12. Sensitivity and Specificity
• A trade-off
• When sensitivity increases, specificity decreases
• When specificity increases, sensitivity decreases
1/25/2016 12
Image taken from: http://groups.csail.mit.edu/cb/struct2net/webserver/about.html

13. Receiver Operating Characteristics Curve
• When sensitivity increases, specificity decreases
• Therefore, when sensitivity increases, (1 – specificity)
increases
• AUC – represents how good a test is
1/25/2016 13

14. Area under curve (AUC)
• Specificity is a measure of true negative; therefore (1 –
specificity) is a measure of false positive
• While AUC of 1 represents a perfect test; AUC of 0.5 is a
worthless test (a.k.a for every one true positive, there is an
equal chance of getting one false positive)
• Interpretation:
• 0.90 -1 = excellent
• 0.80 - 0.90 = good
• 0.70 - 0.80 = fair
• 0.60 - 0.70 = poor
• 0.50 - 0.60 = fail
1/25/2016 14

15. Example:
1/25/2016 15
Reichlin T, Hochholzer W, Bassetti S,
Steuer S, Stelzig C, Hartwiger S, et al.
Early Diagnosis of Myocardial Infarction
with Sensitive Cardiac Troponin Assays. N
Eng J Med 2009;361(9):858-67.

16. Methods
• Multi-center, n = 718, symptoms suggestive of MI
• Diagnostic accuracy of different troponin assays
• Abbott–Architect Troponin I
• Roche High-Sensitive Troponin T
• Roche Troponin I, and Siemens Troponin I Ultra)
• vs standard assay (Roche Troponin T).
• Final diagnosis determined by 2 independent cardiologists:
reviewing clinical history, physical findings, labs, ECG, echo,
angio findings, etc
1/25/2016 16
Reichlin et al 2009

17. Results
1/25/2016 17
Reichlin et al 2009

18. Results
• AUC significantly higher for:
• Abbott–Architect Troponin I, 0.96 (95% CI 0.94 to 0.98)
• Roche High-Sensitive Troponin T, 0.96 (95% CI 0.94 to 0.98)
• Roche Troponin I, 0.95 (95% CI, 0.92 to 0.97)
• Siemens Troponin I Ultra 0.96 (95% CI, 0.94 to 0.98)
• standard assay, 0.90 (95% CI, 0.86 to 0.94)
1/25/2016 18
Reichlin et al 2009

19. Results
1/25/2016 19
Reichlin et al 2009

20. Likelihood Ratios
• Positive likelihood ratio refers to the likelihood of a patient
with the disease to be tested as positive compared to a
patient without the disease
• Negative likelihood ratio refers to the likelihood of patient
with the disease to be tested negative as compared to a
patient without the disease
• Every test has both LR (+) and LR (-)
1/25/2016 20

21. Likelihood Ratios
• LR are more helpful than sensitivity and specificity because
sensitivity and specificity are derived from population where
we already know whether they have or do not have the
disease
• Whereas LRs tell us prospectively how a positive or
negative test results affect the likelihood of patient to have a
disease when we do not know whether they have it or not
• Likelihood ratios have factored in the sensitivity, specificity of
the test (the TP, TN, FP, FN)
1/25/2016 21

22. Likelihood Ratios
• Positive likelihood ratio refers to the likelihood of a patient
with the disease to be tested as positive compared to a
patient without the disease
• LR (+)
• = (True positive)/(False positive)
• = (sensitivity)/(1-specificity)
• The higher LR (+), the better the test to RULE IN the
disease
1/25/2016 22

23. Likelihood ratios
• Negative likelihood ratio refers to the likelihood of patient
with the disease to be tested negative as compared to a
patient without the disease
• LR (-) = (False Negative)/(True Negative)
• = (1 – sensitivity)/(specificity)
• The smaller the LR (-), the better the test TO RULE OUT
the disease
1/25/2016 23

24. Usefulness of LRs
• To choose a diagnostic test
• E.g. which test would be the best to RULE IN a disease?
• Which test would be the best to RULE out a disease?
• To calculate a post-test probability (use Fagan Normogram)
1/25/2016 24

25. Example:
1/25/2016 25
Collins et al, J Cardiac Failure 2015:21(1)

26. Fagan Nomogram
1/25/2016 26

27. Example:
• Why PERC score should only be used when Well’s criteria is
in the low risk category?
• LR (-) of PERC is 0.17 (95% CI: 0.11 – 0.25)
• Ref: Carpenter CR, et al (2009). Differentiating low-risk and
no-risk PE patients: the PERC score. J Emerg Med, 36 (3),
317-22
1/25/2016 27

28. PERC Score
• PERC score is a rule-out criteria for pulmonary embolism
where if none of the 8 PERC criteria are present in a patient,
PE can be ruled out clinically
• B = Blood in sputum (hemoptysis)
• R = Room air O2 Sat>95%
• E = estrogen or homonal use
• A = Age >50 years
• T = Thrombotic events (DVT, PE) or its possibility
• H = HR >/= 100/min
• S = surgery past 4 weeksl
1/25/2016 28

29. Determine your point of equipoise?
• Point of equipoise is the balance point when the risk-
benefit of investigating further for PE vs risk-benefit of
NOT investigating further for PE.
• Kline et al (2004) – point of equipoise for PE is 1.8%.
1/25/2016 29

30. Expolarating a LR (-) of 0.17
and a Post-test probability of 1.8%
Therefore, the pre-test probability
must be below 10%
1/25/2016 30
Determine
that your
post-test
probability is
no more than
1.8% (point
of equipoise)
LR (-) for
PERC

31. Wells criteria
Only in the low risk category
of Wells Criteria where the
probability of PE is below
10%. Therefore, PERC score
should be used only when the
Wells score is in the low risk
category
1/25/2016 31

32. Recommended Video Tutorials
• 6 short video series on sensitivity and specificity:
• https://www.youtube.com/watch?v=U4_3fditnWg&list=PL41c
kbAGB5S2PavLIXUETzAmi5reIod23
• On likelihood ratios:
• https://www.youtube.com/watch?v=TzPvCSFZUSQ
1/25/2016 32