1. Grading Strength of Evidence Interactive Quiz Prepared for: The Agency for Healthcare Research and Quality (AHRQ) Training Modules for Systematic Reviews Methods Guide www.ahrq.gov
2.
3.
4.
5.
6.
7.
8.
9.
Editor's Notes
Grading Strength of Evidence: Interactive Quiz
Strength of Evidence Versus Quality Is grading the strength of evidence the same as rating the quality of studies? Incorrect. Individual studies are rated for their internal validity or risk of bias. Grading pertains to entire bodies of evidence about important key questions, outcomes, or comparisons. The quality of individual studies is just one of four required domains in strength-of-evidence grading. Incorrect. Grading pertains to entire bodies of evidence about important key questions, outcomes, or comparisons. Because domains other than the quality (risk of bias) of individual studies must also be used in strength-of-evidence grading, evaluating study quality and strength of evidence at the same time is insufficient. Correct. Strength-of-evidence grading is one of the final steps in the review process. It requires judgments about four key domains. Only one of these is quality (risk of bias) of the individual studies.
Required Domains Strength-of-evidence grading specifies required domains. What are they? Correct. The four required domains are risk of bias (i.e., quality ratings for individual studies with respect to a specific outcome), consistency (i.e., degree of similarity in the effect sizes of different studies within an evidence base with respect to a specific outcome), directness (i.e., whether evidence reflects a single, direct link between the interventions of interest and the ultimate health outcome under consideration, or, for comparisons, whether the evidence base has head-to-head studies), and precision (i.e., degree of certainty for estimate of effect with respect to a specific outcome). Incorrect. The four required domains are risk of bias, consistency, directness, and precision. Publication bias is an additional domain that can and should be used only when relevant. Incorrect. The four required domains are risk of bias, consistency (which is the missing required domain in this answer), directness, and precision. Strength of association is an additional domain referring to magnitude of effect that can be used when relevant.
Strength-of-Evidence Grades Which list below reflects the correct grades for strength of evidence? Incorrect. Very low is not an option. The four correct options for scores are high, moderate, low, and insufficient. Incorrect. Very high and very low are not used. The four correct options for scores are high, moderate, low, and insufficient. Correct. Strength of evidence can have three “strength” grades relating to the confidence one has that evidence reflects the true effect and that future research might confirm or overturn the results. Insufficient implies either that one has no relevant evidence or that the evidence is so mixed and inadequate that one cannot use any of the other levels.
Discretionary Domains What discretionary domains can be used to reach a strength-of-evidence grade? Correct. Any of these four domains can be used when they are relevant and the appropriate data or statistical tests are available. They are used only after the required domains have been scored. Incorrect. The correct other domains are dose-response association, plausible confounders, strength of association, and publication bias. Applicability is not used in grading strength of evidence; it is reported separately. Funding sources are not considered in strength-of-evidence grading. Incorrect. The correct other domains are dose-response association, plausible confounders, strength of association, and publication bias. Number of studies (or articles) by itself is not considered in strength-of-evidence grading. Applicability is not used in strength-of-evidence grading; it is reported separately.
Combining Scores for Overall Grade In combining scores for domains into an overall grade for strength of evidence, which of the following approaches can be used? Incorrect. Any of these three approaches can be used. The critical element in selecting any option is careful documentation of the methods. Incorrect. Any of these three approaches can be used. The critical element in selecting any option is careful documentation of the methods. Incorrect. Any of these three approaches can be used. The critical element in selecting any option is careful documentation of the methods. Correct. Any of the three approaches can be used. The critical element in selecting any option is careful documentation of the methods.
Consensus Scoring In scoring required and additional domains and in arriving at an overall grade for strength of evidence, which of the following should be done? Incorrect. At least two reviewers with appropriate clinical and methodological expertise should participate. Incorrect. At least two reviewers with appropriate clinical and methodological expertise should participate, but differences should be mediated or adjudicated by a third party. Only the consensus grade should be reported. Correct. At least two reviewers with appropriate clinical and methodological expertise should participate, and differences should be mediated or adjudicated by a third party. Only the consensus grade should be reported.
Summary
Author This interactive quiz augments the module on grading strength of evidence. It was prepared by Kathleen N. Lohr, Ph.D., a Distinguished Fellow at RTI International, and is based on an update of chapter 11 in version 1.0 of the Methods Guide for Comparative Effectiveness Reviews (updated chapter available at: http://effectivehealthcare. ahrq.gov/ehc/products/60/318/2009_0805_ grading.pdf ).
![[object Object],[object Object],[object Object],[object Object],Strength of Evidence Versus Quality](data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7)