Vicki LeBlanc PhD (University of Toronto) introduces us to her data on feedback and enhanced hybrid simulation learning on communication skills. Her expertise is in Human Factors and Decision-making particularly in stressed and non-ideal situations.

1. Does It Work?..and Why?
Research into the effectiveness of simulation as a
teaching tool
Vicki R LeBlanc, PhD
Wilson Centre; University of Toronto

2. SIMULATION
The representation or
reproduction of
something real
by imitation

3.  Physical objects (task trainers)
 Process or skill (suturing)
 People (standardized patients, confederates)
 Environments (virtual worlds, immersive simulation)
SIMULATION

4. • Struggle - resources, access, time, funding, etc…
• Position papers & advocacy (Ziv, Gaba, Reznick, etc..)
• Climate of scepticism
1980S-MID 2000S

5. • National/International Societies
• Simulation Conferences & Courses
• Accreditation Standards
2011: CLIMATE OF
LEGITIMACY

6. ….Important role in acceptance
RESEARCH

7. • Robust acceptance by trainees
• Enhanced learning
• Transfer to clinical setting
RESEARCH EVIDENCE

8. Educators and students exposed to
mannequin-based simulation
85% of students & 85% of educators rated session as
excellent
> 80% sim-based training should be required for all medical
students
Gordon et al., Acad Med 2001
ROBUST ACCEPTANCE

9. • Robust acceptance by trainees
• Enhanced learning
• Transfer to clinical setting
RESEARCH EVIDENCE

10. D Tabak, CA Moulton, A Birze,
H MacRae, D Nestel, R Kneebone, V LeBlanc
ENHANCED
LEARNING USING
HYBRID
SIMULATIONS

13. Specific Research Aims
To determine the effectiveness of the hybrid
simulation in teaching communication and
interpersonal skills to medical students and
surgical residents

14. Study Design
4th
Year
Medical
Students
N = 16
PGY1
Surgery
Residents
N = 16
Wound Closure
Drunk and Angry
Urinary
Catheterization
Uncomfortable
Patient
Feedback
Volar Cast
Irritated
Skin Lesion
Blind and Afraid
Pre-Test Post -Test
No Feedback

15. Outcome Measures
Validated communication scale (2 blinded raters)
Empathy
Response to patient’s feelings and needs
Coherence
Plan and organization of the interview
Verbal Expression
Command of language
Non Verbal Expression
Engagement with non verbal expression
Hodges B. Med Teach 2003; 25:250-254
Validated technical scales

16. 0
10
20
30
40
50
60
70
80
90
100
overall emphathy cohesion verbal non-verbal
Communication Scores: Pre Intervention
No Feedback
Feedback
%
No main effect of feedback p=.752

17. 0
10
20
30
40
50
60
70
80
90
100
overall empathy cohesion verbal non-verbal
* *
Significant effect of feedback p<0.05
Communication Scores: Post Intervention
* * *
No Feedback
Feedback
%

18. Technical Scores: GRS
No Feedback
Feedback
%
Pre Post
0
10
20
30
40
50
60
70
80
90
100
catheter-pre wound-pre skin exc-post cast-post

19. 0
10
20
30
40
50
60
70
80
90
100
Cath Wound Skin Cast
No Feedback
Feedback
Technical Scores: Checklist
%
Pre Post

20. IPPI can be used to deliver effective formative feedback
Feedback > Exposure
Improvements not at cost of non-intervention skill
Conclusions

21. The advantages of
simulation-based
education lead to
enhanced performance
and clinical care
TRANSFER TO CLINICAL
SETTING

22. ENHANCED LEARNING
20 anesthesiology residents; 2-h training session:
- High-fidelity simulation-based training
- Interactive seminar
Each trainee then weaned patients from CPB
- within 2 weeks (posttest)
- within 5 weeks (retention test)
Bruppacher et al., Anesthesiology; 2010

23. Post Test Retention
Non technical
skills
Clinical
checklist
Non technical
skills
Clinical
checklist
Seminar
11.8 75.4 11.7 77.0
Simulation
14.3 89.9 14.1 93.2

24. TRANSFER TO CLINICAL
SETTING
Simulation-based learning of central line insertions
- Greater success
- Reduced complications
- Reduced infection rates
Barsuk et al; Arch Intern Med; 2009

25.  BUT…..Some evidence of lack of improvements:
 Olympio (2003):
 no improvement in management of intubation
 Borges (2010):
 no changes in management of “cannot intubate, cannot
ventilate” scenarios
 Wenk (2009):
 simulation not better than PBL for intubation

26. Not possible to state that simulation, itself, IS or IS NOT effective
for learning
Important to understand what elements of simulation facilitate
learning

27. It’s no longer a question of
IF

28. But a question of
HOW

29. FEEDBACK
IMPORTANT CHARACTERISTICS

30. Key element to learning and skill acquisition
 without debriefing, students fail to learn
Issenberg et al. (2005) Medical Teacher; Stefanidis et a.l (2007) Surgery
Xeroulis et al. (2007) Surgery; Walsh et al. (2009) Academic Medicine
FEEDBACK

31. Medical students learning suturing and knot tying
skill
Control group: practice with no feedback
Concurrent: feedback as student performs the task
Summary: feedback after completion of the task
SUMMARY VS CONCURRENT FEEDBACK
Xeroulis et al., 2006; Surgery

32. Immediate post-test:
• Control < Concurrent = Summative
1-month retention test:
• Control = Concurrent < Summative
RESULTS

33. Continuous, intensive feedback inhibits learning:
• Over-guidance: learners overly depend on feedback
and perform poorly in it’s absence
• Inhibits intrinsic learning strategies and problem
solving activities necessary to master skills
NOT JUST ANY FEEDBACK

34. PRACTICE
IMPORTANT CHARACTERISTICS

35. Podcast Study- In Progress
 How to optimize podcasts in health professions
education
• Mental practice:
-cognitive rehearsal of the steps in a tasks without overt
physical movement
• Modeling:
-Viewing of a demonstration of what needs to be
learned
Fahad Alam, Sylvain Boet, Vicki LeBlanc

36. Pre-Test (Multiple Choice)
Randomization
Podcast
No Mental Practice
No Modeling
(n=15)
Podcast
With Mental Practice
No Modeling
(n=15)
Podcast
No Mental Practice
With Modeling
(n=15)
Podcast
With Mental Practice
With Modeling
(n=15)
Post-Test (Multiple Choice)
Basic Airway Management
Simulation Scenario

37. Preliminary results
 Data collection: mid-point, with 7-8 per group
 MCQ improvements:
Control < Mental Practice = Modeling < MP and Modeling

38. Summary
Significant scholarship in simulation over 20 years
 - Learner reactions
 - Can lead to improved performance
 - Learning transfers to clinical setting
Important focus of current research…
…understanding how to optimize simulation-based learning

39. “The field of patient simulation has gotten to
where it is now because it makes sense, but
we need more science behind it to guide its
use.”
Jim Gordon