The top articles in medical education for 2017 focused on improving feedback practices through various methods. One article described using the R2C2 model to structure feedback conversations and found it enabled meaningful and goal-oriented discussions. Another article found that an institution's culture is central to how residents perceive feedback quality and credibility. A third article identified qualitative differences in the feedback male and female residents receive, highlighting the need for awareness of potential gender bias. An additional article demonstrated that high rates of direct observation were achievable in an ambulatory setting despite initial faculty skepticism. Overall, the articles emphasized the importance of feedback and observation for trainee development and highlighted approaches to enhance current practices.
Call Girls Shimla Just Call 8617370543 Top Class Call Girl Service Available
Top Articles in Medical Education 2017
1. Top Articles in Medical
Education 2017: Applying the
Current Literature to
Educational Practice and
Scholarship
Donna D’Alessandro, MD
Steve Paik, MD, EdM
Mike Ryan, MD, MEHP
Teri Turner, MD, MPH, MEd
H. Barrett Fromme, MD, MHPE
Leora Mogilner, MD
Alix Darden, PhD, Med
7 May 2018
3. Goal
•Broadly review the medical education
literature of 2017 to select the top articles
that could impact practice
4. Objectives
By the end of the session, participants will be able to:
• List major thematic areas of investigation and publication
in medical education for the year 2017
• Discuss the outcomes of the top articles in medical
education in 2017
• Formulate approaches to incorporating medical education
innovations into their own practice.
• Identify areas of scholarly interest for themselves for
further reading, curricular application, and innovative
scholarship
5. Road Map for Session
•Who We Are
•Methods
•Topic Areas
•Key Point
•Methods
•Results
•Summary
•Question and Answers
6. The Team
• H. Barrett Fromme, MD, MHPE, University of Chicago
• Director of Faculty Development in Medical Education, Assoc Residency PD
• Donna M D’Alessandro, MD, University of Iowa
• Educator of Residents and Students
• Alix Darden, PhD, MEd, University of Oklahoma Health Sciences Center
• Director of Faculty Development, Dept of Pediatrics
• Leora Mogilner, MD, Icahn School of Medicine at Mount Sinai
• Director of Advocacy and Community Pediatrics
• Steve Paik, MD, EdM, Columbia University School of Medicine
• Pediatric Residency Program Director
• Michael S Ryan, MD, MEHP, Virginia Commonwealth University
• Assistant Dean for Clinical Medical Education
• Teri L. Turner, MD, MPH, MEd, Baylor College of Medicine
• Vice Chair of Education, Department of Pediatrics
7. Inclusion Criteria
•Relevant to academic pediatricians
•Could change practice at individual or
programmatic/institutional level
8. Journals Reviewed
• Academic Medicine
• Academic Pediatrics
• BMC Education
• Journal of the American Medical Association
• Journal of General Internal Medicine
• Journal of Graduate Medical Education
• Medical Education
• Medical Education Online
• Medical Science Educator - IAMSE
• Medical Teacher
• NEJM
• Pediatrics
• Teaching & Learning in Medicine
9. Methods, Stage 1
•Seven medical educators reviewed journals
•Two authors read all titles/abstracts in each
journal to find articles that met inclusion
criteria
10. Methods, Stage 1
•Two authors for each journal selected abstracts of
interest
•Each reviewed combined article titles and scored:
• 0 = not relevant
• 1 = may be relevant
• 2 = highly relevant
•All four point abstracts automatically advanced
•All three point abstracts discussed for inclusion
11. Methods, Stage 2
•Each abstract assigned to two reviewers to read full
article
•Each full article was scored:
• 0 = do not consider, will not change practice
• 1 = consider, could change practice
• 2 = definitely include, will change practice
•All four points articles automatically included
•All three point articles discussed for inclusion
12. Final Stage
•Final articles grouped by topics
• Summary and slides created
• Peer reviewed by two members of group
13. Selection Process
1682 abstracts
13 journals
114 abstracts
selected for
discussion
75 articles
selected for
review
22 articles for
presentation
53 not presented
39 not selected
1568 not
discussed
16. R2C2 in Action: Testing an Evidence-Based Model to
Facilitate Feedback and Coaching in Residency
Sargeant J, et al.
JGME. 2017; 9(2): 165-170
17. Key Points
•The R2C2 model enabled meaningful,
collaborative, reflective, goal-oriented
feedback discussions.
•A defined coaching phase prompted giver
and receiver to think differently and more
positively about feedback and framed
conversations as opportunities to coach for
improvement.
R=rapport building R= reactions to feedback C=content of feedback C=coaching
18. What They Did
•Purpose: To determine the R2C2 model’s utility and
acceptability for engaging residents in their feedback,
and in using it to plan for improvement
•A qualitative study of 7 IM and Peds residents and
their 5 supervisors
•Thematic analysis
19. What They Found
•Supervisors valued having a structure for
feedback conversations and appreciated the
opportunity to coach.
•Residents appreciated working collaboratively
with their supervisor.
•Supervisors needed time to learn to use the
R2C2 model.
22. “It’s Just Not the Culture”: A qualitative study exploring
residents’ perceptions of he impact of institutional
culture on feedback
Ramani S, et al.
Teaching and Learning in Medicine. 2017; 29(2): 153-161
23. Key Points
•Institutional culture is central to resident’s
perceptions of the quality, credibility, and
acceptability of feedback.
•Training on techniques for delivering
feedback alone, is unlikely to enhance its
impact on resident performance.
24. What They Did
•Purpose: To examine resident opinions on institutional
factors that affect the quality of feedback
•Qualitative study using a constructivist grounded
theory approach
•Thematic analysis
26. What They Found
• The cultural norm lacks clear expectations and messages
around feedback
• The prevailing culture of niceness does not facilitate honest
feedback
• Bidirectional feedback is not part of the culture
• Faculty-resident relationships affect credibility and
receptivity to feedback
• There is a need to establish a culture of longitudinal
professional growth
27. Why We Chose It
•Faculty development is not the only solution to
feedback problems
•Importance of feedback as a bidirectional exchange
within a social culture that encourages reflective
practice and ongoing professional development for
both parties
28. Gender Differences in Attending Physicians’ Feedback
to Residents: A qualitative analysis
Mueller AS, et al.
JGME. 2017; 9(5): 577-585
29. Key Points
•There are qualitative differences in the kind of
feedback that male and female residents
receive.
•Raise awareness of gender bias in perceptions
of residents’ capabilities.
30. What They Did
•Purpose: To examine the feedback that male and
female residents received from attending physicians
•Qualitative content analysis using a post-positivist
paradigm
•Collected several variables for analysis
32. What They Found
Discordant feedback to
female residents
generally focused on
masculine traits of:
• Autonomy
• Assertiveness
0
10
20
30
40
50
60
70
Positive and Constructive
Feedback
No negative comments
related to traits
Criticised multiple times
for lacking traits
Gender Difference in Feedback
Male Female
33. Why We Chose It
•Gender differences exist in feedback
•Raise awareness of both gender bias in perceptions of
residents’ capabilities and gender stereotypes of what
traits are valued
Author
unknown
but is
attributed
by some
to Emma
Watson
34. Successful Implementation of a Direct Observation
Program in an Ambulatory Block Rotation
Smith J, et al.
JCME. 2017; 9(1): 113-117
35. Key Points
•Despite initial faculty skepticism, the authors
achieved high rates of frequent observation
on a breadth of clinical skills for assessing
residents in an ambulatory setting.
36. What They Did
•Purpose: To evaluate the effects of a novel direct
observation evaluation system on both faculty and
trainees
•Program evaluation research
•Several variables analyzed
39. Why We Chose It
•Challenges the notion that if we asked faculty to do
more observations it will increase perceived faculty
burden
•Discrepancy between residents and faculty regarding
which clinical domain was the most useful for
observation
40. Section Summary
• The R2C2 feedback model provides a framework for bi-
directional meaningful feedback and coaching
• To make an impact on trainee performance and ultimately on
patient care, we have to change the institutional culture of
feedback
• We need to be cognizant and work to be consistent in our
feedback messages regardless of the gender of our trainees
• Frequent direct observation is feasible in the outpatient
setting
41. How Is My Trainee Progressing?
Let Me Count the Ways.
Alix Darden, PhD, MEd
42. Justify Your Answer: The Role of Written Think Aloud
in Script Concordance Testing
Power, A, et.al
Teaching and Learning in Medicine. 2017; 29(1):59-67
44. What They Did
•Purpose: Compare clinical reasoning as assessed by a
quantitative test, script concordance test (SCT) and
qualitative test, think aloud for SCT cases.
• Script Concordance Test
(SCT) of Clinical
Reasoning -24
• Written Think aloud - 3
Pediatric
Residents
N=91
• Script Concordance Test
of Clinical Reasoning -
24
• Written Think aloud - 3
Panel of
Experts
(POE) N=12
Compare Residents
Answers
1. SCT & Think Aloud
2. POE Think aloud
45. What They Found
Think Aloud illuminates:
1. Incorrect clinical reasoning despite correct SCT
response
2. Sound clinical thinking with a suboptimal SCT
response
3. Question Misinterpretation
46. Why We Choose It
•Think aloud as a method to document learner
thought processes is under utilized in medical
education, yet is a powerful tool for enriching
quantitative assessment data.
47. Promoting Responsible Electronic Documentation:
Validity Evidence for a Checklist to Assess Progress
Notes in the Electronic Health Record
Bierman, JA, et.al.
Teaching and Learning in Medicine. 2017; 29(4):420-432
48. Key Points
•The Responsible Electronic Document
checklist, can be used as a tool to
systematically review the quality of trainee
inpatient EMR progress notes.
•Trainee inpatient progress notes need
improvement.
49. What They Did
•Purpose: Create and validate a checklist, 18 closed
items, 4 open-items, to assess trainee inpatient
progress notes in EMR.
Response
Process
Validation
Internal
Structure
Validation
50. Responsible Electronic Document (RED) Checklist
Northwestern University Feinberg School of Medicine
Subjective
The note contains:
1. Current patient concerns or symptoms
No
0 1
Yes
2
N/A
Objective
The Physical exam contains:
2. Succinct vitals
3. Examination of all systems relevant to today’s positive
symptoms
4. Examination different from previous day’s exam
The data portion of the note contains:
5. Labs only if they are new
6. Reports of studies only if it is the first day they are
included
Mark on scale as defined in key: 0 1 2 N/A
7. A summary or impression of study reports
51. Assessment and Plan
The assessment and plan meets these criteria:
8. A summary statement is included.
9. The summary statement is different from previous day’s
statement.
10. Positive symptom(s) from subjective section are
included.
11. A problem-based assessment is included.
Mark on scale as defined in key: 0 1 2 N/A
12. The status of each problem is described.
13. Lab abnormalities are interpreted.
14. Interpretation of studies is included.
15. Problems are written as diagnoses or accompanied by
differentials.
16. Active problems are accompanied by clinical reasoning.
17. Problems are associated with brief, clear plans.
18. Assessment and plan is different from previous day’s
assessment and plan.
No
0 1
Yes
2
N/A
Summary – open ended
A good progress note is Truthful, Reasoned, Updated and Succinct. Please comment
on characteristics of this note that fulfill or lack these.
52. What They Found
• Trainee notes show substantial room for improvement
• Average note score – 66.9% (SD=10.6, range=33.4%-
93.3%)
• 7 minutes to complete – rater does not need to know
patient or review chart. Compare one daily note to the
previous days note
• Good Interrater reliability – Cohen’s kappa coefficient = 0.67.
C-
53. Why We Chose It
•AAMC and ACGME both emphasized the
importance of clinical documentation
training.
•The Responsible Electronic Document
Checklist provides a systematic, easy to use
assessment tool that also provides feedback
to the trainee.
54. The development of the PARENTS: a tool for parents
to assess residents' non-technical skills in pediatric
emergency departments.
Moreau KA, et.al
BMC Medical Education. 2017; 17: 210 - 220
55. Key Points
With a validated tool, PARENTS,
1. parents of pediatric patients can be
involved in the assessment of residents,
2. assessment of residents’ non technical skills
in Pediatric Emergency Departments can be
improved
3. parents perspectives can be used to
improve resident training
56. What They Did
•Purpose: Create and validate a 19 item assessment
tool of residents’ ER non-technical skills for parents of
patients to use.
Response
Process
Validation
Internal
Structure
Validation
57. PARENTS
1.Did the resident identify him/herself as a resident
2: Was the resident’s ID badge or nametag visible?
3: Did the resident wash his/her hands? Yes No NA
4: ...enter the room with some basic knowledge
of your child’s condition?
5: ...listen to you and allow you to speak
without interruption?
6: ...appear to understand what you had to say?
7: ...explain what he/she was doing for your
child and why?
8: ...interact with you comfortably?
9: ...interact with your child comfortably?
10: ...be flexible in his/her thinking and
approach depending on your needs and those
of your child?
11: ...show concern for your feelings and those
of your child?
12: ...pay attention to you and your child during
your interactions with him/her?
Very
poor
Poor Fair Good Very
good
NA
58. PARENTS
13: ...explain your child’s treatment or
prescribed medication, including possible side
effects?
14: ...determine next steps about care or
treatment with you, including any follow-up
plans?
15: ...discuss what to do if your child has any
problems or complications related to his/her
condition?
16: ...answer your question?
17: ...explain things in a way that you could
understand?
Very
poor
Poor Fair Good Very
good
NA
Open ended questions
18. What can the resident do to improve his/her interactions with
caregivers and their children?
19. Please use the space below to provide additional comments on
the resident’s skills when interacting with you and your child?
59. Why We Chose It
• Parent assessment of trainee non-technical
skills provides an important, authentic
assessment opportunity for residents.
60. Mixed Messages or Miscommunication? Investigating
the Relationship Between Assessors’ Workplace-
Based Assessment Scores and Written Comments
Sebok-Syer, SS, et al.
Academic Medicine. 2017; 92(12): 1774-1779
61. Key Point
•Narrative comments are more balanced for trainees
who are perceived to have a deficiency. Types of
comments are not consistent across trainee levels
leading to a “hidden code”.
Overall Resident Assessment –
Clinical Competency Committee
360
Comment Checklist
Residency
assessment
program
62. What They Did
•Purpose: Examine relationships between checklist, task
ratings, global ratings, and narrative comments in
resident assessment tools.
Task ratings
360o
Rating
Checklist
Comments
• McMaster Modular
Assessment Program
• 23 PGY 1 & 2 - EM
• Regression Analysis
• Content analysis
63. What They Found – Quantitative data
“Significantly associated with qualitative rating of
task strengths and weaknesses
Regression variable for assessors’ selection of the checklist option
“Done, but needs attention. (n=321)
Model Variables Beta SE Wald Probability
Intercept 4.95 1.32 3.75 <0.001
Task rating -1.06 0.29 -3.61 <0.001
Global rating 0.026 0.27 0.09 0.92
Quality rating of
task strengths
-0.41 0.21 -3.47 <0.001
Quality rating of
task weaknesses
0.36 0.14 2.42 0.016
Task comment
length
0.013 0.031 0.40 0.69
64. What They Found – Qualitative data
•~30% of assessors avoid written comments even
when required by system
• Comments associated with trainees with perceived
deficiency(s)
• Addressed both strengths and area(s) of improvement
• Focus - criterion referencing
•Comments associated with high performing trainees
• Addressed strengths in global manner
• Focus - norm referencing
65. Why We Chose It
• A novel study identifying associations with
different resident assessments (qualitative and
quantitative).
• Can lead to faculty development for members of
competency committees in how to interpret
narrative comments and various rating models.
Resident Assessment
C
B A
66. Section Summary
•Development of unique competency-based
assessment tools and methodology aid in
training future physicians.
•Think aloud – getting inside the trainee thought
process
•EMR progress note assessment
•Parent assessment of non-technical skills
•Methodology for comparing instruments within
an assessment plan
68. Medical Students’ Professional Development as
Educators Revealed Through Reflection on Their
Teaching Following a Students-as-Teachers Course
Yoon MH, Blatt BC, and Greenberg LW
Teaching and Learning in Medicine
2017; 29(4):411-419.
69. Key Point
•Medical student (MS) self-assessment
narratives provide an understanding of their
developing identities and emerging
professional self-concept as educators
70. What They Did
• Purpose: Explore MS4 reflections in a students-as-teachers
course using written self-assessment narratives
• 1 year elective at George Washington University in
Washington DC
• 6 workshops and practicum where MS4 co-teach in
standardized patient physical diagnosis encounters with
younger MS
• 2011-13, MS4 were asked to reflect on their course efforts
and assign a grade to themselves with written justification
• Inductive content analysis
71. What They Found
Educator Identity
Growth
Professional
Development
Personal Growth
• Using teaching
strategies for adult
learning
• Preparing to teach
physical diagnosis
• Incorporating clinical
correlations
• Giving and receiving
feedback
• Creating a positive
learning climate
• Growing as
educators
• Modeling
professionalism
• Exceeding course
requirements
• Peer counseling
• Many comments
related to
professionalism
• Gained confidence
• Increased comfort with
teaching
• Developed camaraderie
with other educators
72. Why We Chose It
•This explores how students translate their
experiences in a students-as-teachers course
into their identities as a professional and
educator
•These could be foundational courses for
developing a workforce of skilled medical
educators
73. A Multi-Institutional Longitudinal Faculty
Development Program in Humanism Supports the
Professional Development of Faculty Teachers
Branch WT, et al.
Academic Medicine. 2017; 92(12):1680-1686.
74. Key Point
•Participation in a longitudinal, multi-
institutional, faculty development program
facilitated the professional development in
humanism for its participants
75. What They Did
•Purpose: To describe a multi-year, longitudinal, multi-
institutional faculty development program in
humanism
• 2005-2017
• 30 institutions in US and Canada
• Local facilitator and 8-12 faculty members
• 12-18 month program of bi-monthly sessions
• Evaluations – faculty and matched-control
• Questionnaires and narratives
76. What They Found
•993 faculty participated + some residents
•Participants were highly engaged with little
drop out and consistent attendance
•Participants scored higher on overall
humanism questionnaires than matched
controls including trends for individual
questionnaire items
•Participants progressed toward more advanced
levels of humanism self-identity as measured
by Robert Kegan’s stages of adult development
77. Why We Chose It
•Describes a successful long-term, multi-
institutional, faculty development program
supporting the professional self-identity
growth of medical educators
•Used as a institutional model to strengthen
the humanistic side of medical education and
improve the learning environment
78. Creating a Medical Education Enterprise: Leveling the
Playing Fields of Medical Education vs. Medical
Science Research Within Core Missions
Thammasitboon S, Ligon BL, Singhal G, Schultze GE,
Turner TL.
Medical Education Online. 2017;22:1377038
79. Key Points
•Business organizational themes were used to
organize and develop the structure of a
medical education enterprise
•The themes framed practical strategies for
empowering and advancing the scholarly
endeavors of clinical-educators within a
pediatric department
80. What They Did
•Purpose: Create an educational enterprise within the
pediatric department at Baylor College of Medicine,
that levels the playing field for clinician-educators
relative to clinician-scientists and empowers them
•Describes the process of enterprise creation using
Bowman and Deal’s business organization model with
4 frames
81. What They Found
Framework Meaning Strategies to Support
Clinician-Educators
Structural Administrative structure, rules,
policies, & the organizational
reporting structure
• Created a Center specifically for
medical education
• Decentralized leadership
organization
• Offered grants
• Developed promotion pathways
• Defined dissemination success for
educational scholarship
Human Resource Optimizes the organization’s
personnel through recruitment,
development, empowerment &
support
• Used the Center to assist less
experienced clinician-educators
• Endowed Chair
• Offered incentives (travel, etc.)
Political Aligns power and authority to
manage resources, negotiate
conflict and form alliances
• Financially incentivized
educational scholarship
• Expanded venues for scholarly
engagement and dissemination
• Educational awards
Symbolic Objects, people, events, or
stories, used to communicate
missions and values for
supporting cultural identity &
shared vision
• Provided opportunities to
recognize all types of scholarship
• Reminded leaders about the
educational mission’s importance
• Provide consistent messaging
about educational mission’s value
82. What They Found
•12/15 college-wide educational scholarship awards
•Winners of college-wide faculty educational
excellence awards increasing
•National meeting presentations dramatically
increasing
•Strongly supports the clinician-educator
83. Why We Chose It
•Flexible structure that offers pragmatic
strategies that institutional can use to
support clinician-educators
•Strategies discussed could be implemented in
isolation or in combination, at many (all?)
institutions
84. Section Summary
• Educators start to view themselves as educators early in their
professional careers
• Increased support for medical educators advances their self-
identity and the improves the their outcomes as teachers and
scholars over their professional careers
• Supporting the development of medical educators across their
professional lifespan is makes everyone a winner!
87. How Supervisor Experience Influences Trust,
Supervision, and Trainee Learning: A Qualitative Study
Sheu L, Kogan J, and Hauer K
Academic Medicine
2017; 92(9): 1320-1327.
88. Key Point
•Supervisors’ approach to trust and
supervision varies with their level of
experience and this variation can directly affect
trainee learning
89. What They Did
•Purpose: to investigate how supervisor experience
influences trust, supervision, and trainee learning
• Two phase qualitative study:
• Phase 1: Reviewed supervisor interviews from 2 institutions
(UCSF and HUP) to develop supervisor “exemplars” (early,
developing and experienced)
• Phase 2: Trainee focus groups at a single institution to validate
accuracy of exemplars developed in Phase 1 and explore impact
on learning
90. What They Found – Phase 1
Early Supervisor Developing Supervisor Experienced Supervisor
Data Granular:
Emphasis on trainee task
completion
Individualized: Emphasis on
trainee skills and ability
Holistic:
Emphasis on trainee
qualities/behaviors
Approach Err towards more
supervision
Tailored supervision Trend towards greater
autonomy
Perspective Reflect on recent
experiences as trainee
Reflect on own early
supervisor experience
Draw on institutional
knowledge/global experience
Clinical Personal uncertainty Growing confidence Confidence in assessing trainee
skills
91. What They Found – Phase 2
Themes identified:
• Shift in trainee preference and learning needs over time
• Desire for flexibility and an individualized approach to supervision to
promote learning
92. Why We Chose It
•Demonstrates that supervisor experience has a
direct impact on trainees’ learning and satisfaction
•Faculty development can help supervisors provide
the flexible supervision appropriate for trainees’
needs
•Opportunity to help trainees learn how to “manage
up” and give constructive feedback to their
supervisors to address their learning needs
93. Entrustment of the On-call Senior Medical Resident Role: Implications
for Patient Safety and Collective Care
Huda N, Faden L and Goldszmidt M
BMC Medical Education
2017; 17 (121)
94. Key Point
•Identifies senior medical resident core on-call
supervisory tasks that can be used for training and
assessing residents prior to making entrustment decisions
95. What They Did
•Purpose: to understand the clinical activities of the
on-call senior medical resident (SMR) and provide a
model for entrustment decisions for this role
• Four-phase constructivist grounded theory approach
conducted at 2 academic medical centers in Ontario:
• Phase 1: Case study
• Phase 2: Focus groups
• Phase 3: Literature search
• Phase 4: Two return-of-findings focus groups
96. What They Found
1-Overseeing Ongoing Patient Care
2-Briefing
Ensures that the junior
trainee has needed
information and approach
for patient assessment
3-Case Review
Ensures that problems
have been identified, plan
addresses each problem
and junior MD
understands the plan
4-Documentation
Ensures that admit note,
SMR note and patient
orders are consistent and
complete
5-Preparing for Handover
Prepares the junior MD for
their case presentation to
the attending physician
• Five core on-call supervisory practices that support collective care and
patient safety were identified:
97. Why We Chose It
• Identifies a set of core on-call supervisory tasks of the senior medical
resident and their impact on patient safety and trainee learning
• These findings can be used for training and assessing trainees prior to
making entrustment decisions
98. The Educational Climate Inventory: Measuring Students’ Perceptions of
the Preclerkship and Clerkship Settings
Krupat E, Borges N, Brower R, Haidet P, Schroth S,
Fleenor Jr T, and Uijtdehaage S.
Academic Medicine
2017; 92 (12): 1757-1764.
99. Key Point
• The Educational Climate Inventory (ECI) allows
educators to assess students’ perceptions of the
medical school learning environment and
differentiates between a learning or mastery-
oriented climate vs. performance-oriented climate
100. What They Did
• Purpose: to develop and validate an instrument
to assess educational climate and examine the
relationship between ECI and other variables
• A pool of 50 items was rated by 1st, 2nd and 3rd year
students at 6 US medical schools
• Exploratory factor analysis (EFA) and confirmatory factor
analysis (CFA) were performed
• They explored the relationship between ECI and other
factors (pass-fail vs. traditional grades, satisfaction with
medical school experience/choice of medicine as a career,
and satisfaction with year in medical school)
101. What They Found
• 1441 students completed the surveys (56% response rate)
Factor Analysis:
• 3 distinct factors resulting in a 20-item scale were identified:
Factors Cronbach
alpha
1. Centrality of learning and mutual respect
Ex. “In this medical school we focus on a sense of discovery and the
excitement of inquiry”
.88
2. Competitiveness and stress
Ex. “The atmosphere here is highly competitive”
.80
3. Passive learning and memorization
Ex. “Most of what we do here is focused on the passive transfer of
knowledge”
.71
102. What They Found
Relationship of ECI scores to other variables:
• Students’ ratings of the preclerkship learning climate
were more performance-oriented in schools with grades
compared to schools with a pass-fail system (P=.04)
• Clerkship students rated their learning climate as
significantly more performance-oriented than
preclerkship students (P<.001)
• Students perceiving their environment as more
performance-oriented were less satisfied with their own
medical school and their decision to pursue medicine as a
career (P<.001)
103. Why We Chose It
•Validates the ECI, an instrument that can be used by
educators to assess the learning environment with
the goal of ensuring that a positive learning
environment exists
•Provides insight into the impact of educational
climate on student satisfaction with their school
and choice of career
104. Transition to Residency: Using Specialty-Specific
Clinical Tracks and Advanced Competencies to Prepare
Medical Students for Internship
Khan M, Splinter A, Kman N, Leung C, Rundell K, Davis J
and McCallister J
Medical Science Educator
2017; 27: 105-112
105. Key Point
• Description of a novel 4th year medical student
competency-based, specialty-specific
curriculum that prepares students with the
skills they need to be successful interns
106. What They Did
•Purpose: redesign of fourth year curriculum based on
ACGME Core Competencies and AAMC Entrustable
Professional Activities (EPAs) to ensure students are
prepared for internship
•Curriculum redesign at one medical school (Ohio State
University College of Medicine)
107. What They Found
• Acute care setting (ER and
subinternship)—8 weeks
• Outpatient ambulatory setting—8
weeks
Required
• Clinical Tracks
•Provide set of guidelines for clinical
rotations and competency-based
assessments of essential intern skills
• Advanced Competencies
•Enhanced content that maps to core
ACGME competencies;
interdisciplinary, generalizable to
multiple practice areas
Elective
Revamped4thYearCurriculum
108. Why We Chose It
•Uses ACGME milestones to evaluate performance,
ensuring a common language that can translate to
readiness assessment at the start of internship
•Has the potential to better prepare students for
internship and ease the transition from medical
school to residency (no evaluation data published
yet)
109. Section Summary
• Supervisor experience directly impacts trainees’ learning and
satisfaction
• Knowledge of core on-call supervisory tasks can be used for
training and assessing trainees prior to making entrustment
decisions
• The Educational Climate Inventory (ECI) allows schools to assess
student perceptions of the educational environment, which
directly impacts student learning and satisfaction
• Competency-based, specialty-specific curricula may help fourth
year medical students develop the skills they need to be
successful interns
111. The effect of white coats and gender on medical
students’ perception of physicians
Ladha, M. et al.
BMC Medical Education 2017; 17: 93
112. Key Points
•White coats did not change the perception of
physician’s ratings by medical students
•However gender and possibly race altered
students’ ratings of physicians in the perception
of:
• Trustworthiness
• Physician management
• Competence
• Professionalism
• Perception of medical error
113. What They Did
•Purpose: To determine if components of physical
appearance of the physician affected students ratings
of competence
•Cross-sectional study using self-administered
questionnaire with four scenarios portraying potential
physician error
•Data collected:
• 5 point Likert scale rating physician qualities,
appropriateness of actions and whether the physician
committed the error
114. What They Found
•Primary Outcome – White Coat
• No difference in altering perception in any domains
•Secondary Outcome – Gender and Race
• Gender – Male
• Trustworthiness
• Appropriateness of action
• Competence
• Medical Error (less correlated)
• Case/Caucasian
• Competence
• Medical Error (less correlated)
115. Why We Chose It
•Medical Schools put effort into evaluations
•Implicit Bias is a factor that goes beyond just
patient care
•There are a many subtleties we may not be
addressing in our evaluation system
116. Acculturation Needs of Pediatric International Medical
Graduates: A Qualitative Study
Osta AD, et al.
Teaching and Learning in Medicine. 2017; 29: 143-152
117. Key Points
•Residency Training is a time of adjustment for
all trainees and there are many acculturation
issues for IMGs
•Pediatric-specific acculturation issues were
elicited from this qualitative study which
were not previously elucidated.
118. What They Did
• Purpose: To explore if pediatric IMG residents have
specific acculturation needs that have not been previously
identified.
• 90-minute semi structured focus groups with Pediatric
non US-IMG residents at one academic medical center
• Main focus group questions
• What has been the most challenging part of living and practicing
medicine in the United States?
• What are some of the communication challenges that you have
faced while practicing medicine as a pediatric resident?
• How is it different being a pediatrician in the United States as
compared to your culture?
121. Why We Chose It
•Residency Training is an acculturation challenge during
a stressful transition
•This study elucidated acculturation issues that were
specific to IMG trainees training in Pediatrics.
•The topics would be helpful for addressing curriculum
and faculty development enhance training and
transitioning IMG trainees
122. Long-term benefits by a mind-body medicine skills
course on perceived stress and empathy among
medical and nursing students
van Vliet, M., et al.
Medical Teacher. 2017; 39 (7): 710-719
123. Key Points
•Mind Body Medicine course had short term
and long term benefits on
•Fostering Empathy
•Decreasing Perceived stress
•Preventing increased Personal Distress
124. What They Did
•Purpose: To explore short term and long term effects
of the MBM course among medical students and
nursing students
•Exploratory Controlled, quasi-experimental study to
evaluate the effects of Mind body Medicine course
using a validated questionnaire over 4 time points
•Analysis
• Descriptive statistics for baseline characteristics
• Effect analysis using linear mixed models
126. What They Found
•MBM Course:
•Significant decrease in perceived stress in
nursing students who had baseline high stress
levels
•Prevented higher levels of personal stress in
medical students who had lower levels of
baseline stress
•Improved fantasy and empathetic concern
compared to control over 12 months
127. What They Found
•MBM Course:
•Significant decrease in perceived stress in
nursing students who had baseline high stress
levels
•Prevented higher levels of personal distress in
medical students who had lower levels of
baseline stress
•Improved fantasy and empathetic concern
compared to control over 12 months
128. Why We Chose It
•Validated measures were used to show
internal and external validity of the MBM
course
•Preparation of stress may help in transition
from the academic to clinical practice
•Stress management training and wellbeing
early in the professional development
process may improve wellness in the future
129. Section Summary
• White coats did not have an effect on evaluations but gender
and race may be a factor in faculty assessment and all
assessment that we may need to better evaluate
• Acculturation issues specific to pediatric training programs are
different and important to incorporate into training and faculty
development
• Mind Body Medicine started early in medical school may have
a lasting effect on learners in transitioning to clinical years
130. Improving the Medical Student
Experience: Adding Value,
Refining Evaluations, and
Helping Them Pee Happy
Michael S. Ryan, MD MEHP
131. Numerical versus narrative: A comparison between
methods to measure medical student performance
during clinical clerkships
Bartels J, et al.
Medical Teacher. 2017; 39: 1154-1158
132. Key Points
•Narrative comments from evaluations were
at least as reliable, if not more, than
quantitative scores on final evaluations
•There was a strong correlation between
narrative comments and quantitative scores
133. What They Did
•Purpose: To determine whether narrative data would
be equal or more reliable than quantitative data from
clerkship evaluations
•Retrospective analysis of ITERs (in-training evaluation
reports, a.k.a. final evaluations) collected from the M3
Neurology clerkship at the University of Rochester
•Data collected:
• All ITERs from 50 randomly selected students
• Original numeric score (Likert-scale, 1-10)
• New narrative score (60-100) generated by 5 grading committee members
• Analyzed for correlation and inter-rater reliability
134. What They Found
•Inter-rater reliability
• Original numeric scores = ICC .62 (95% CI = .41, .83)
• New narrative scores = ICC .88 (95% CI = .83, .92)
•Original numeric scores were highly positively
correlated (r = .81) with the new narrative
scores
135. Why We Chose It
Observations Impression Score Evaluation
Score
Score
How Faculty Observations Translate into Student Evaluations
Issues
1. Halo effect
2. Poor depth
3. Inflation Elimination of the “middle man!”
Traditional systemNarrative-only system
136. Developing validity evidence for the written Pediatric
history and physical exam evaluation rubric
King MA, et al.
Acad Pediatr. 2017; 17: 68-73
137. Key Points
•P-HAPEE is 10-item instrument which can be
used to assess the quality of medical
students’ H&Ps
•The instrument has strong content, internal
structure (inter-rater reliability), and
response process validity
138. What They Did
•Purpose: To develop and gather validity evidence for a
Pediatric H&P evaluation rubric
• Development of P-HAPEE rubric
• Content developed from accreditation bodies, published instruments,
expert consensus
• > 50 educators critically reviewed rubric
•Data collected on 30 H&Ps with variable
characteristics. Analyzed for validity evidence:
• Content validity, Internal structure, response process
139. What They Did: Content
• Content: Traditional elements (Histories, PE, studies, A/P)
• Scale: 5-points (None, Some, All)
• “A medical student is expected to perform at a score of 3 or better by the end of
his/her third year.”
• Narratives and Final Overall Rating
1 2 3 4 5
Absent, unsupported, misses
many critical findings, includes
excessive irrelevant data, fails
to include physical
exam/diagnostic study findings,
and/or restates findings
without synthesis
Identifies some defining history
AND physical exam/diagnostic
study findings while omitting
most of the irrelevant data.
Uses some medical terms and
semantic qualifiers to
synthesize an assessment
Selects critical defining history
AND physical exam/diagnostic
study findings. Uses appropriate
medical terms and semantic
qualifiers to synthesize an
accurate and concise summary
statement.
Notes:
Example (Assessment Section)
140. What They Found
Category ICC (95% CI)
Information-gathering, history 0.81 (0.74-0.87)
Information-gathering, physical examination 0.83 (0.77-0.88)
Information synthesis and clinical reasoning 0.89 (0.86-0.92)
Total score 0.85 (0.83-0.88)
Overall assessment 0.89 (0.81-0.95)
• Greater IRR for high and low quality (vs. medium quality) H&Ps
• Training time: 30-60 minutes
• Response process
• Time to score each: 15.1 min/19.3 min (attending/resident, p <.001)
• Easiest: HPI, PE, Intro
• Most difficult: Assessment, problem ID, plan
Internal structure : Inter-rater reliability
141. Why We Chose It
•1 of only 2 published instrument developed to assess
the quality of the Pediatric H&P
•In comparison to other instrument, P-HAPEE had:
•Smaller number of items (10 vs. 38)
•Greater IRR
•Utility in EMR and non-EMR generated notes
•Multi-center involvement
142. Group observed structured encounter (GOSCE) for
third-year medical students improves self-assessment
of clinical communication
Ludwig AB, et al.
Medical Teacher. 2017; 39: 931-935
143. Key Points
•Group OSCEs offer a more cost-conscious
alternative to the traditional OSCE and allows
for both the development and assessment of
group/team-based skills
•While valuable for formative feedback, there
are barriers to their use for summative
assessment of individual learners
144. What They Did
•Purpose: To describe the feasibility and effectiveness
of a Group OSCE (GOSCE)
• Description of a formative educational innovation
• Groups of 4-6 M3 students +1 faculty preceptor on Internal Medicine
• 4 station SP-based GOSCE, 1 student leads each
• Focused on: behavioral change, difficult encounter, shared decision-making,
delivering bad news
•Data collected:
• Pre/post surveys to assess confidence in communication
• Survey rating cases and experience
145. What They Found
I feel confident in my ability to…
Pre-test
% agree or
strongly agree
N = 155
Post-test
% agree or
strongly agree
N = 155
Absolute
difference
p value
Give feedback to peers 60.00% 80.70% 20.70% <.0001
Respond to patients when they become angry 45.20% 63.90% 18.70% <.0001
Shared decision making 69.70% 86.50% 16.80% <.0001
Elicit all of my patient’s concerns 78.70% 91.00% 12.30% .0009
Deliver bad news 34.20% 46.50% 12.30% .008
Communication with patients within a group 72.30% 83.20% 10.90% .01
Take a smoking history/motivate to quit 64.5% 74.8% 10.3% .02
Communicate using nonverbal cues 87.70% 94.90% 7.20% .02
Clarify patient’s responses when I interview them 92.90% 98.70% 5.80% .007
3/12 were not significant: communicate in general, show respect, and verbal communication
146. Why We Chose It
•Few reports of GOSCE in the literature
•Though limited in terms of application for summative
and/or individual evaluations, GOSCEs may offer
opportunities for:
1. Training/feedback on group dynamics
2. Peer role modeling
3. Processing feedback encounters
4. Lower cost compared to single-learner OSCE
147. How can medical students add value? Identifying
roles, barriers, and strategies to advance the value of
undergraduate medical education to patient care and
the health system.
Gonzalo JD, et al.
Acad Med. 2017; 92: 1294-1301
148. Key Points
•The authors propose 7 specific roles and
several strategies which could be used to
enhance the value of medical students within
the healthcare system
149. What They Did
• Purpose: To describe the activities, tasks, and roles that could
be afforded to medical students which would add value for the
health system
• AMA Accelerating Change in Med Ed (2016) conference
• Plenary session (large/small group discussions) on medical
student value
• Qualitative analysis of:
• Field notes from AMA staff
• Written form completed by small group
150. What They Found
Proposed “new” roles for Medical Students
Patient navigator
Safety analyst
Care transition facilitator
Member of QI Team
Population Health Manager Patient care tech
Scribe
151. What They Found: Barriers,
Strategies, and Outcomes
Student engagement, skills,
assessment
Service vs. Learning
Resources, Logistics,
Supervision
Productivity and Billing
Culture
Faculty Factors
Reshape focus to value-
added roles
Focus on continuity of
learning
Increase student “touch
points” in the system
Student experiences add
value
Enhanced education in
clinical and health systems
science
Achieve quadruple aim:
1) Improve patient
experience
2) Improve population
health
3) Decrease cost
4) Improve life of health
care workers
Barriers
General Strategies
Outcomes
Short-term
Long-term
152. Why We Chose It
•Challenges the peripheral/non-essential role that
medical students often have on modern healthcare
teams
•Provides concrete guidance to increase the value
medical students have on health care systems and
outcomes
153. Section Summary
• Narrative comments are at least as reliable, if not more, than
numerical scores obtained on M3 clerkship evaluations
• P-HAPEE is an encounter note assessment instrument with
substantial inter-rater reliability which can be used to evaluate
the quality of medical student Pediatric H&Ps
• Group OSCEs provide an opportunity for learners to develop
team-based skills, learn from their peers, and obtain formative
feedback on interactions with patients and teams
• Medical students should be given new roles and responsibilities
to increase their value to the healthcare team
