hanya Untuk Perkuliahan

1. CRITICAL APPRAISAL
Dr. Elsye Maria Rosa, M.Kep

2. Objectives
 Define evidence-informed decision-making and describe
its importance to nursing practice
 Identify the parts of a relevant, answerable question
 Describe the different types of questions relevant to
nursing practice
 Define the term critical appraisal and discuss its relevance
to nursing practice, policy and research
 Identify criteria appraisal criteria for health care
intervention studies
 Develop skill in applying the criteria for critical appraisal of
an intervention study to determine the quality and
applicability of the research.

3. What is evidence-informed decision-
making?
“The conscientious, explicit and judicious use of
the current best evidence in making decisions
about the care of individual patients”
(Sackett et al, 1996)

4. Health care
resources
Patient
preferences
and actions
Research
evidence
Clinical state, setting,
and circumstances
Clinical Expertise
Clinical Decision
A Model for Evidence-Based Clinical Decisions
[Haynes, R.B., Devereaux, P.J., & Guyatt, G.H. (2002). Clinical expertise in the era of evidence-based medicine and
patient choice. ACP J Club, 136, A11-14]

5. Why is evidence-based practice important?
• Demonstrate that nursing actions and decisions
are clinically appropriate and result in positive
outcomes for clients, their families and health care
system as a whole
• Demonstrate professional accountability to clients
• Provide evidence for the cost-effectiveness of
nursing care

7. Steps to Evidence-Informed Decision-
Making:
1. Define: Formulate a focused answerable question from the
practice situation based on your information needs
2. Search: Efficiently search for research
3. Appraise: Critically and efficiently appraise the research sources
4. Synthesize: Interpret/form recommendations for practice based
on the literature
5. Apply: Apply the results to the client/population
6. Implement: Decide whether (and plan how) to implement the
adapted evidence into practice
7. Evaluate: Evaluate the effectiveness of implementation efforts

8. Introduction to critical
appraisal

9. PICO

10. Bagaimanakah pencegahan terhadap kejadian
pressure ulcers dengan teknik “X” jika
dibandingkan dengan pelayanan standar yaitu
diganti posisi setiap 2 jam terhadap munculnya
gejala munculnya pressure ulcers pada pasien
yang dirawat diruang perawatan dalam jangka
panjang dimana pasien mempunyai resiko untuk
pressure ulcers?
KASUS-1 : MOHON
MASUKANNYA

11. Langkah ke-1 Pertanyaan Klinik dengan
PICO/PICOT Format
JENIS-JENIS PERTANYAAN KLINIS (Melnyk & Fineout-Overholt,
2011)
Intervention question
• Meneliti mengenai keefektivan dari
suatu treatment/intervensi
Diagnostic question
• Meneliti mengenai manfaat,
keakuratan, seleksi, atau interpretasi
dari suatu alat/instrumen
Prognostic question
• Meneliti mengenai keadaan pasien
terkait kondisi tertentu atau
mengidentifikasi faktor-faktor yang
mungkin mengubah prognosis pasien
Etiology question
• Meneliti mengenai hubungan sebab
akibat dan sesuatu yang mungkin
merugikan
Meaning question
• Meneliti mengenai makna dari
sesuatu hal

12. Formulate a Focused Answerable Question
• P opulation / Situation
• I ntervention / Exposure
• C omparison /Counter intervention
• O utcome
• T imeframe

13. Population/Situation:
• Client group or clinical scenario of interest
• Single patient or group of patients with a
particular condition or health care problem,
e.g.,:
• A person with a health condition
• People with hypertension (a group of people with a
particular condition
• Primary health care for the elderly (an aspect of health
care delivery)

14. Intervention/Exposure:
• Interventions can be:
• Therapeutic
• Preventive
• Diagnostic
• Organizational
• If exploring the meaning of a phenomena, the
question may involve a situation rather than an
intervention

15. Comparison / Counter Intervention:
• Clinical decisions involve choosing between
alternative courses of action (or no action)

16. Outcome:
• What is the outcome, or the effect we are hoping
to achieve by the using the intervention?
• May be more than one outcome that is important
to the question

17. Time Frame:
• What is the period of time over which the question
occurs?
• What is the optimal time to measure a change in
the outcome(s)?

18. Prognostic
Apakah diet karbohidrat mampu memprediksi pemeliharaan berat badan yang
sehat (BMI< 25) selama lebih dari 6 bulan pada pasien yang mempunyai
riwayat keluarga obesitas (BMI> 30)?
Diagnostic
Apakah d-dimer assay lebih akurat dalam mendiagnosa deep vein thrombosis
jika dibandingkan dengan ultrasound pada pasien suspected deep vein
thrombosis?
Etiology
Apakah wanita kulit putih yang terpapar sinar UV ray berkepanjangan dan
tidak menggunakan protection (>1 jam) meningkatkan resiko terkena
melanoma jika dibandingkan wanita kulit hitam yang tidak terpapar UV ray?
Meaning
Bagaimanakah wanita paruh baya dengan fibromialgia mempersepsikan
kehilanggan motor functions?
TENTUKAN DESIGN PENELITIANNYA

19. Step 1:Formulate a focused answerable question
• Formulating Answerable Clinical Questions (Centre for Evidence-
Based Medicine, Mount Sinai Hospital)
• The Well-Built Clinical Question (Duke University Medical Center
Library and Health Sciences Library, UNC-Chapel Hill)
• Formulating Patient Centered Questions (University Library,
University of Illinois at Chicago)
• Asking Focused Questions (Centre for Evidence-Based Medicine,
Oxford)

20. Types of questions relevant to nursing
practice
• Therapy/Health care interventions: What is the effectiveness of different interventions?
• Causation and harm: What might be causing disease/ill health/adverse effects?
• Diagnosis or assessment: Does a diagnostic test differentiate between people with and
without a condition?
• Prognosis: What are potential future outcomes of a condition?
• Economic evaluation: What is the cost-effectiveness of different interventions?
• Meaning: Describing, exploring and explaining aspects of health and illness.

21. Nursing Graduate Seminar
200921
Different clinical questions require evidence from
different research designs
Type of Question Suggested Best Type of Study Design
Therapy/Health care
intervention
RCT>cohort>case control>case series
Diagnosis Prospective, blind comparison to a gold standard
Prognosis Cohort study>case control>case series
Harm (Causation or
etiology)
RCT>cohort>case control>case series
Economic evaluation Economic analysis
Meaning of illness Qualitative

22. Qualitative Survey
(cross sectional)
(Randomised)
crossover
(Randomised)
Parallel group
Cohort study
Cross-sectional
(analytic)
Case-control study
Q1
Q3
Q2
All studies
Experimental
Analytic
(PICO or PECO)
Descriptive
(PO)
Theoretical
(PS)
Observational
analytic
What type of research design will you use?
[Glasziou, P., & Heneghan, C. (2009). A spotter’s guide to study designs. EBN Notebook, 12, 71-72]

23. Kata kunci untuk mencari bukti-bukti = kata-kata yang ada
dalam PICO/PICOT
Cari kata-kata lain yang mempunyai makna sama seperti
kata-kata yang ada di PICO/PICOT
Setiap jenis pertanyaan mempunyai hierarchy of evidence
yang berbeda
• Database:
• Pubmed
• CINAHL
• Ovid-medline
• National Guideline Clearing house
• Chochrane Databases
Langkah ke-2 Mencari dan Mengumpulkan
Bukti-bukti

24. Pubmed: Patient center care

26. Step 2: Collect the best evidence:
Hierarchy of Pre-Processed Evidence
[Adapted from Haynes, R.B. (2007). Of studies, summaries, synopses, and systems: The “5S”
evolution of information services for evidence-based healthcare decisions. Evidence-Based Nursing,
10, 6-7]
EXAMPLES:
SYSTEMS:
Computerized decision support
SUMMARIES::
Evidence-Based Guidelines
~ Evidence-Based Texts
SYNOPSES OF SYNTHESES:
DARE ~ Evidence-Based
Abstract Journals
SYNTHESES (Systematic
Reviews): Cochrane Database
of Systematic Reviews
SYNOPSES OF SINGLE
STUDIES:
Evidence-Based Abstract
Journals
SINGLE STUDIES:
Clinical Queries

27. Step 3: Critically appraise the literature for
validity and applicability
What is critical appraisal?
• Critical appraisal is the process of assessing and
interpreting evidence by systematically considering its
validity, results and relevance to an individual's work.

28. Langkah ke-3 Melakukan Critical Appraisal
Terhadap Bukti-bukti
Critical Appraisal menyesuaikan dari jenis/level artikel
Pertanyaan utama dalam Critical Appraisal adalah
• Apakah hasil dari penelitian tersebut valid?
• Apakah penelitian tersebut menggunakan metodologi penelitian
yang baik?
• Apakah hasil dari penelitian tersebut reliable?
• Apakah intervensinya bekerja dengan baik?
• Sebesar apa efek dari intervensi tersebut?
• Apakah hasil penelitian tersebut akan membantu dalam
melakukan perawatan untuk pasien saya?
• Apakah sample penelitiannya mirip dengan pasien saya?
• Apakah keuntungannya lebih besar dari pada resikonya?
• Apakah intervensi tersebut mudah untuk di implementasikan

29. Relevance of Critical Appraisal to Nursing
Practice, Policy and Research
• Use the literature more effectively in answering clinical
questions to guide clinical practice
• Distinguish stronger evidence from weaker evidence –
identify high quality research
• Identify the methodological strengths and limitations,
results and relevance of the studies in answering a
clinical question

30. Three Steps in Using an Article from the
Health Care Literature
1. Are the results valid?
2. What are the results?
3. Are the results applicable (and useful) to
my client care population?
[DiCenso, A., Guyatt, G., & Ciliska, D. (2005). Evidence-
based nursing: A guide to clinical practice]

31. Are the results valid?
• Are the study methods sufficiently rigorous to
ensure that the study results represent an
unbiased estimate of the true effect?
OR
• Are the study methods sufficiently biased to
lead to a false conclusion?

32. Are the results valid?
• Final assessment of validity is never a yes/no
decision
• Validity as a continuum ranging from strong
studies to weak studies
• Evaluation of the validity of a study involves some
subjectivity

33. What are the results?
• Size and precision of the estimate of effect

34. Are the results applicable (and useful) to
my client care population?
• Can you apply the results to patients in your
clinical setting?
• Were all important outcomes considered?
• Are the likely intervention benefits worth the
potential harm and costs?

35. Applying appropriate criteria
• User Guides to Evidence-Based Practice
• 1993-2000: Evidence based medicine working group: “User guides
to the Medical Literature” in JAMA
• 2002: Guyatt et al. “User guides to the medical literature: A manual
for evidence based clinical practice”
• 2005: DiCenso, Guyatt, & Ciliska. “Evidence-based nursing: A
guide to clinical practice”

36. Applying appropriate criteria
• Critical Appraisal Forms:
• Critical Appraisal of Articles on CAUSATION
• Critical Appraisal of Articles on THERAPY/INTERVENTIONS
• Critical Appraisal of SYSTEMATIC REVIEWS
• Critical Appraisal of Articles on PREVALENCE AND INCIDENCE
• Critical Appraisal of QUALITATIVE RESEARCH
• Critical Appraisal of Articles on PROGNOSIS
http://www.cche.net/usersguides/prognosis.asp
• Critical Appraisal of GUIDELINES [AGREE: Appraisal of Guidelines
Research and Evaluation Instrument]
http://www.agreecollaboration.org/pdf/agreeinstrumentfinal.pdf

37. CRITICAL APPRAISAL
CHECKLIST
• JBI_Critical_Appraisal-
Checklist_for_Analytical_Cross_Sectional_Studie
s2017.pdf
• JBI_Critical_Appraisal-
Checklist_for_Cohort_Studies2017.pdf
• JBI_Critical_Appraisal-
Checklist_for_Qualitative_Research2017.pdf
• JBI_Quasi-Experimental_Appraisal_Tool2017.pdf
• JBI_RCTs_Appraisal_tool2017.pdf

38. Example: The Effectiveness of a Nurse-Led
Interprofessional Team Approach to Fall Prevention
in Older Home Care Clients at Risk of Falling
Markle-Reid, M., Browne, G., Gafni, A., Roberts, J., Weir, R., Thabane, L., Miles, M., Vaitonis, V.,
Hecimovich, C., Baxter, P., & Henderson, S. (2010). The effects and costs of a multifactorial and
interdisciplinary team approach to fall prevention for older home care clients “at risk” for falling: A
randomized controlled trial. Canadian Journal on Aging, 29(1), 139-161

39. Critical Appraisal Criteria for Health
Care Intervention Studies
Screening Questions:
1. Did the study ask a clearly focused question?
Consider if the question is ‘focused’ in terms of:
• The population studied
• The intervention given
• The outcomes considered

40. Research Question:
• `What is the effectiveness of a 6-month nurse-led interprofessional
team approach to fall prevention compared with usual home care
services in older home care clients “at risk” of falling with respect to
the number of falls and fall risk factors (slip or trip, health-related
quality of life and function, depressive symptoms, nutritional status,
gait and balance, cognitive function, fear of falling)?
• P Frail older adults using home care services at risk for falling
• I Nurse-led interprofessional team approach to fall prevention
• C Usual home care services
• O Number of falls, slip or trip, health-related quality of life
and function, depressive symptoms, nutritional status, gait and
balance, cognitive function, fear of falling
• T 6 months

41. Critical Appraisal Criteria for
Health Care Intervention
Studies
Screening Questions:
2. Was this a randomized controlled trial (RCT) and was it
appropriately so?
Consider:
 Why this study was carried out as an RCT
 If this was the right research approach for the question being
asked

42. Critical Appraisal Criteria for
Health Care Intervention
Studies
• Is it worth continuing?

43. Critical Appraisal Criteria for
Health Care Intervention
Studies
3.How were participants allocated to intervention and control
groups?
Consider:
 How participants were allocated to intervention and control
groups. Was the process truly random?
 Whether the method of allocation was described
 How the randomization schedule was generated
 Were the groups comparable at baseline on characteristics
that might explain the outcome?

44. Population
Eligible Sample
Baseline Measures
Ineligible
R n=
Treatment Control
Follow-up Measures Follow-up Measures

45. Strengths
• Random allocation of participants to groups
ensures that groups are similar in all respects
except exposure to the outcome;
• Prospective design ensures that exposure to
the intervention precedes the development of
the outcome;
• Greater likelihood that participants, health
care providers, and outcome assessors can
be blinded.

46. Limitations
• May not be ethical or feasible;
• Volunteer bias limits generalizability;
• Problems associated with a longitudinal design:
• Time
• Expense
• Dropouts
• Final results may not be available for several
years.

47. Critical Appraisal Criteria for
Health Care Intervention
Studies
4. Were participants, staff and study personnel
‘blind’ to participants’ study group?
Consider:
• The fact that blinding is not always possible
• If every effort was made to achieve blinding
• If you think this matters to the study
• The fact that we are looking for ‘observer bias’

48. Critical Appraisal Criteria for
Health Care Intervention
Studies
5. Was follow-up complete?
Consider:
• If any intervention-group participants got a control group
option or vice versa
• If all participants were followed up in each study group
• Was there loss to follow-up bias?
• If all of the participants’ outcomes were analysed by the
groups to which they were originally allocated (intention to
treat analysis)

49. Study Flow
R
Eligible Clients
n=267
Informed consent from 40.8% of
eligible applicants
n=109
Refused 141
Unable to Contact 13
Language 4
158
Group 1
n=54
Group 2
n=55
n=5 (9.3%)
n=49 (90.7%)
Interprofessional Falls
Prevention Team
Usual Home Care
Services
n=12 (21.8%)
n=43 (78.2%)
Group Size
Interventions
Dropouts @ 6 month follow-
up: 17 clients (15.6%)
Analyzed @ 6 month follow-
up: 92 clients (84.4%)

50. Critical Appraisal Criteria for
Health Care Intervention
Studies
6. Were participants in both groups followed up and
data collected in the same way?
Consider:
• Was data collected in the same way and at the
same time interval for both groups?

51. Data Collection
• Multiple sources of data:
• In-person interviews
• Data from service provider agencies
• Measurement of Clinical Outcomes: Baseline and
6-months

52. Outcomes
Effects:
Frequency and Context of Falls (Falls Surveillance Report)
Functional Health Status and Quality of Life (SF-36 Health Survey)
Cognitive Status (SMMSE)
Depressive Symptoms (CES-D)
Gait and Balance (POMA)
Nutritional Status (SCREEN II)
Environmental Safety (HOME FAST)
Perceived Self-Efficacy (MFES)
Costs: Health Services Utilization (HSSU and CCAC Utilization Data)

53. Critical Appraisal Criteria for
Health Care Intervention
Studies
7. How are the results presented and what is the main result?
Consider:
• What are the results?
• How large this size of result and how meaningful it is
• How you would sum up the bottom-line result of the trial in
a few sentences

54. Results
• At 6 months, there was no difference in the mean
number of falls between groups;
• Subgroup analyses showed that the intervention was
effective in reducing falls in men, 75-84 years of age,
with a fear of falling, or a negative history of falls;
• Greater reduction in number of slips and trips in the
intervention group;
• Greater improvement in role functioning related to
emotional health in the intervention group;
• No additional cost from a societal perspective, e.g. both
interventions cost the same.

55. Critical Appraisal Criteria for
Health Care Intervention
Studies
8. How precise are these results?
Consider:
• If the result is precise enough to make a decision
• If a confidence interval were reported. Would
your decision about whether or not to use this
intervention be the same at the upper confidence
limit as at the lower confidence limit?

56. Critical Appraisal Criteria for
Health Care Intervention
Studies
9. Were all important outcomes considered so the results
can be applied?
Consider whether:
• How the participants could be different from your
population in ways that would produce different results
• Your local setting differs from that of the trial
• Is the intervention feasible in your setting?
Consider outcomes from the point of view of the:
• Individual
• Policy maker and professional
• Family caregivers
• Wider community

57. Critical Appraisal Criteria for
Health Care Intervention
Studies
9. Were all important outcomes considered so the results
can be applied?
Consider whether:
• Any benefit reported outweighs any harm and/or cost
• Policy or practice should change as a result of the
results of this trial

58. Methodological Strengths
• Study design: randomized controlled trial
• Follow-up period was appropriate
• Follow-up was complete (<20% loss to
follow-up)
• Groups comparable at baseline

59. Methodological Limitations
• Small sample size
• Volunteer bias
• Potential for recall bias related to reporting falls
• Strategies for collecting data regarding the number of
falls

60. Other Resources
• Evidence-Informed Decision-Making Resource Modules posted on Avenue to Learn
• Bandolier Evidence-Based Health Care http://www.medicine.ox.ac.uk/bandolier/
• Tutorials:
• Introduction to Evidence-Based Medicine (Duke University Medical Center Library)
http://www.hsl.unc.edu/services/tutorials/ebm/index.htm
• Directories:
• Evidence Based Medicine (EBM) Tookit (New York University School of Medicine. The
Frederick L. Ehrman Medical Library):
• www.urmc.rochester.edu/hslt/miner/digital_library/evidence_based_resources.cfm
• Resource Guide for Evidence-Based Practice (University of Alberta Libraries)
http://www.library.ualberta.ca/subject/evidence/guide/index.cfm
• McKibbon, A. (1999). PDQ: Evidence-based principles and practice. Hamilton, Ontario: B.C.
Decker Inc.
• Evidence Based Practice Tips http://www.ebmtips.net/risk001.asp