1. QJM Advance Access published October 8, 2008
Q J Med
doi:10.1093/qjmed/hcn131
Teaching critical appraisal and statistics in anesthesia
journal club
R.S. MOHARARI1, E. RAHIMI1, A. NAJAFI1, P. KHASHAYAR2,
M.R. KHAJAVI1 and A.P. MEYSAMIE3
From the 1Department of Anesthesiology, 2Research and Development Center and 3Department of
Community Medicine, Tehran University of Medical Sciences, Sina Hospital, Tehran, Iran
Received 21 April 2008 and in revised form 9 September 2008
Summary
Background: This study was designed to assess evaluating a randomized controlled trial paper
the effectiveness of a series of journal clubs held using the CONSORT checklist before and after the
for anesthesiology residents in promoting their course.
awareness of research methods and statistical Results: Residents’ awareness in the application of
analysis, as well as their skills in critical thinking information improved (P = 0.012), as well as
and appraisal. research methodology (combined study design and
Material and methods: Twenty-four journal club application of information, P = 0.017). Their ability
sessions were held between September 2006 and in critical appraisal did also significantly rise at the
August 2007 for 16 residents of anesthesiology. end of the course (P < 0.001).
A 31 multiple-choice question (MCQ) was taken as Conclusion: Journal clubs can enable residents to
pretest and posttest to evaluate the participants’ develop the knowledge, expertise and enthusiasm
level of awareness in research methodology and needed to undertake research plans and can also
statistical analysis. Their competence in critical enhance their ability in critical thinking and
thinking and appraisal was also evaluated by scientific reading.
Introduction
Residency training represents an ideal time when Attendance was closely monitored. In each session,
clinicians may learn and practice the necessary a single article relevant to anesthesiology and
skills for searching, evaluating and applying medical critical care medicine was discussed with respect
knowledge. The journal club may play an important to statistical analyses and research methodology
role in this regard.1 The present study was designed (type of study, biases and methods). Suitable articles
to assess the effectiveness of a regular journal club were identified by Medline and EMBase English
in teaching anaesthetic trainees research methodol- searches and were given to the participants 4 days
ogy and critical appraisal skills. beforehand. At the beginning of each session, a
senior resident presented a 10-min summary of the
article including its objectives and findings. Then, a
senior clinician focused in detail on a single
Material and methods methodological issue according to a prepared
Twenty-four journal-club sessions (biweekly for 1 h) syllabus (principles of statistics and research; con-
were held between September 2006 and August cepts of risk, exposure, odds and bias; block
2007 for residents of anesthesiology (years 1–3). randomization; case–control studies, cohort studies,
Address correspondence to M.R. Khajavi, MD, Sina Hospital, Tehran-11367-46911, Iran.
email: mohammadreza.khajavi@gmail.com
! The Author 2008. Published by Oxford University Press on behalf of the Association of Physicians.
All rights reserved. For Permissions, please email: journals.permissions@oxfordjournals.org Page 1 of 3

2. R.S. Moharari et al.
Table 1 Comparison of the scores of the pretest and posttest multiple choice questions to assess the improvement of the
participants’ level of awareness
Pretesta Posttesta Mean differencea 95% CI P-value
Methodology 6.8 Æ 2.5 8.8 Æ 3.9 2.0 Æ 3.3 0.4 to 3.7 0.017
Study design 4.8 Æ 2.0 5.2 Æ 2.2 0.4 Æ 2.0 À0.6 to 1.5 0.367
Application of information 2.0 Æ 1.2 3.6 Æ 2.4 1.6 Æ 2.4 0.4 to 2.8 0.012
Statistical analysis 3.8 Æ 2.2 3.8 Æ 2.3 0.0 Æ 2.0 À0.9 to 0.9 1.000
Total 10.6 Æ 4.0 12.6 Æ 5.5 2.0 Æ 4.0 0.0 to 4.0 0.045
a
Mean Æ SD.
randomized controlled trials and use of the Table 2 The CONSORT checklist showing the mean
CONSORT checklist). score in terms of each item after the course
In the early sessions, theoretical issues were
discussed in a didactic style. As the knowledge of Item Posttest
the residents improved, the sessions became mainly
student-centered. To assess the overall improvement 1 Title and Abstract 0.78 Æ 0.9
in the participants’ awareness level, an identical 2 Background 1.56 Æ 0.6
pretest and posttest including 31 multiple-choice 3 Participants 0.89 Æ 0.7
(MCQ) questions was administered (Chronbach-a 4 Intervention 1.00 Æ 0.8
5 Objectives 1.06 Æ 0.7
score = 0.7); their mean values were compared using
6 Outcomes 1.17 Æ 0.7
paired samples test. Subgroup analyses were also
7 Sample size 1.61 Æ 0.5
performed for research methodology (22 MCQs, 11 8 Randomization—sequence generation 1.11 Æ 0.5
for study design and 11 for application of informa- 9 Randomization—allocation concealment 1.17 Æ 0.4
tion) and statistical analysis (nine MCQs), using 10 Randomization—implementation 0.67 Æ 0.9
Student’s t-test. In addition, to evaluate the partici- 11 Blinding 0.67 Æ 0.7
pants’ competence in critical thinking and appraisal, 12 Statistic analysis 1.17 Æ 0.8
they were asked to appraise a single randomized 13 Participant flow 1.06 Æ 0.8
clinical trial (RCT) paper, using the CONSORT 14 Recruitment 1.50 Æ 0.8
checklist, before and after the course. Two profes- 15 Baseline data 0.89 Æ 0.9
sors were also asked to do the same test to provide 16 Numbers analyzed 1.72 Æ 0.6
17 Outcomes and estimation 1.72 Æ 0.6
the reference answers; in case of any discrepancy, a
18 Ancillary analyses 1.50 Æ 0.9
third professor was asked to do the test. Participants
19 Adverse events 1.11 Æ 0.9
were instructed to give points to each of the 22 20 Interpretation 1.72 Æ 0.7
individual items in the CONSORT checklist 22 Generalizability 1.50 Æ 0.7
between 0 (not mentioned in the article at all) and 22 Overall evidence 1.50 Æ 0.6
1 (discussed in the article). Then, the scores of
individual items on every answer sheet were
summed to yield a total score for its respondent. declared a lack of knowledge in this area. However,
The mean values of these total scores before and their posttest appraisal total score (mean Æ SD,
after the course were compared using Paired 27.06 Æ 2.98) showed significant improvement
Samples Test. (Table 2). However, when we segregated the
total 22 items of the CONSORT checklist into 14
well-discussed items (19.56 Æ 1.82) and 8 poorly-
Results discussed items (7.50 Æ 1.58), there appeared a
There was a significant improvement in the partici- significant difference between the two compart-
pants overall awareness level (Table 1) and this was ments (mean difference = 12.06; 95% CI
clearly for ‘application of information’ but not for 10.90–13.21; P < 0.001 using unpaired t-test).
‘study design’ or ‘statistical analysis’. After combin-
ing study design and application of data (method-
ology), the study revealed a significant difference
Discussion
between the pre and posttest scores (P = 0.017). The potential use of a journal club in improving
The pretest score on critical thinking and apprai- reading habits1 and awareness of biostatistics2 has
sal was deliberately considered zero as all residents already been described. Other authors have also
Page 2 of 3

3. Critical appraisal and statistics
indicated that teaching the principles of critical References
appraisal may also be achieved by this means.3–5
1. Linzer M, Brown JT, Delong ER, Siegel WC. Impact of
Another review of six studies confirmed the journal medical journal club on house-staff reading habits, knowl-
club’s significant role in promoting critical appraisal edge and critical appraisal skills: a randomized, control trial.
skills6 but this view is not widely held.7 J Am Med Assoc 1988; 260:2537–41.
Our study similarly suggested an improvement in 2. Cheatam ML. A structured curriculum for improved resident
critical appraisal competency and in epidemiologic education in statistics. Am Surg 2000; 66:585.
awareness level of the residents. We concur with 3. Kellum JA, Rieker JP, Power M, Powner DJ. Teaching critical
Kuhn et al.8 that the unavailability of trained faculty appraisal during critical care fellowship training: A founda-
members in the very areas, lack of sufficient time tion for evidence-based critical care medicine. Crit care Med
2000; 28:3067–70.
and budget shortages are the factors which can limit
the effectiveness of the journal club. We also found 4. Seeling CB. Affecting residents’ literature reading attitudes,
that the participants were unaware of their lack of behaviors and knowledge through a journal club interven-
tion. J Gen Intern Med 1991; 6:330–4.
knowledge of epidemiology and critical appraisal
skills, a situation that improved by the end of the 5. Cramer JS, Mahoney MC. Introducing evidence based
medicine to the journal club, using a structured pre a post
sessions. test: a cohort study. BMC Med Educ 2001; 1:6.
In summary, we conclude that the journal club
6. Ebbert JO, Montori VM, Schultz HJ. The journal club in post
represents a simple but effective means of effectively graduate medical education: a systematic review. Med Teach
engaging residents in developing their research and 2001; 23:455–61.
critical appraisal skills. We recommend that further 7. Bennett KJ, Sackett DL, Haynes RB, Neufeld VR, Tugwell P,
research we control groups and larger sample sizes Roberts R. A controlled trial of teaching critical appraisal of
should be performed in order to fully exploit the the clinical literature to medical students. J Am Med Assoc
teaching potential of the journal club. 1987; 257:2451–4.
8. Kuhn GJ, Wyer PC, Cordell WH, Rowe BH. A Survey to
determine the prevalence and characteristics of training in
Acknowledgement evidence – based medicine in emergency medicine. J Emer
Med 2005; 28:353–9.
We acknowledge the support of the Research and
Development Center of Sina Hospital.
Page 3 of 3