Pediatric Academic Societies Meeting Invited Science Presentation May 2017

1. Top Articles in Medical
Education 2016: Applying the
Current Literature to
Educational Practice and
Scholarship
Donna D’Alessandro, MD
Steve Paik, MD, EdM
Mike Ryan, MD, MEHP
Teri Turner, MD, MPH, MEd
H. Barrett Fromme, MD, MHPE
Leora Mogilner, MD
Alix Darden, PhD, Med
8 May 2017

2. Disclosures
•None to report

3. Goal
•Broadly review the medical education
literature of 2016 to select the top articles
that could impact practice

4. Objectives
By the end of the session, participants will be able to:
• List major thematic areas of investigation and publication
in medical education for the year 2016
• Discuss the outcomes of the top articles in medical
education in 2016
• Formulate approaches to incorporating medical education
innovations into their own practice.
• Identify areas of scholarly interest for themselves for
further reading, curricular application, and innovative
scholarship

5. Road Map for Session
•Who We Are
•Methods
•Topic Areas
•Key Point
•Methods
•Results
•Summary
•Question and Answers

6. The Team
• H. Barrett Fromme, MD, MHPE, University of Chicago
• Director of Faculty Development in Medical Education, Assoc Residency PD
• Alix Darden, PhD, MEd, University of Oklahoma Health Sciences Center
• Director of Faculty Development, Dept of Pediatrics
• Donna M D’Alessandro, MD, University of Iowa
• Educator of Residents and Students
• Michael S Ryan, MD, MEHP, Virginia Commonwealth University
• Assistant Dean for Clinical Medical Education
• Leora Mogilner, MD, Icahn School of Medicine at Mount Sinai
• Director of Advocacy and Community Pediatrics
• Teri L. Turner, MD, MPH, MEd, Baylor College of Medicine
• Vice Chair of Education, Department of Pediatrics
• Steve Paik, MD, EdM, Columbia University School of Medicine
• Pediatric Residency Program Director

7. Inclusion Criteria
•Relevant to academic pediatricians
•Could change practice at individual or
programmatic/institutional level

8. Journals Reviewed
• Academic Medicine
• Academic Pediatrics
• BMC Education
• Journal of the American Medical Association
• Journal of General Internal Medicine
• Journal of Graduate Medical Education
• Medical Education
• Medical Education Online
• Medical Science Educator - IAMSE
• Medical Teacher
• NEJM
• Pediatrics
• Teaching & Learning in Medicine

9. Methods, Stage 1
•Seven medical educators reviewed journals
•Two authors read all titles/abstracts in each
journal to find articles that met inclusion
criteria

10. Methods, Stage 1
•Two authors for each journal selected abstracts of
interest
•Each reviewed combined article titles and scored:
• 0 = not relevant
• 1 = may be relevant
• 2 = highly relevant
•All four point abstracts automatically advanced
•All three point abstracts discussed for inclusion
•Articles organized into general themes

11. Methods, Stage 2
•Each abstract assigned to two reviewers to read full
article
•Each full article was scored:
• 0 = do not consider, will not change practice
• 1 = consider, could change practice
• 2 = definitely include, will change practice
•All four points articles automatically included
•All three point articles discussed for inclusion
•Occasional two point article included if
complementary to other article

12. Final Stage
•Final articles grouped by topics
• Summary and slides created
• Peer reviewed by two members of group

13. Selection Process
6339 abstracts
13 journals
251 abstracts
selected for
discussion
56 articles
selected for
review
20 articles for
presentation
36 not presented
179 not selected
6104 not
discussed

14. Final Topics
•OSCEs
•Trainee Self-Assessment Skills
•Professionalism
•Clinical Teaching
•Milestones and Training Outcomes
•Graduate Medical Education

15. Optimizing OSCE’s to Promote
Learner Assessment
Leora Mogilner, MD

16. Selecting and Simplifying: Rater
Performance and Behavior When
Considering Multiple Competencies
Tavares W. et al
Teaching and Learning in Medicine 2016; 28:1, 41-51

17. Key Point
•When raters are asked to consider multiple
competencies and attend to multiple issues
simultaneously, they may become
overwhelmed, which can negatively affect
the validity of the assessment

18. What They Did
•Purpose: To examine the impact of increasing task
demands on raters on rating quality
•85 experienced faculty members observed and rated 3
videotaped clinical performances
• Half rated 7 dimensions of performance (7D) and half rated 2 dimensions (2D)
• Half were also required to rate the actor’s performance in the simulation
(“distracting task”)
•Data collected:
• Interrater reliability
• Quantification of the number of behaviors used to inform ratings
• 7D participants were interviewed to explore the strategies they used when rating
under conditions that may have felt overwhelming

19. What They Found
•2D group IRR > 7D IRR
•Internal consistency similar
•2D group identified more behaviors of relevance
than the 7D group in common dimensions
• History Gathering
• Procedural Skills

20. What They Found
•7D raters identified various cognitive strategies they
used to complete the tasks
• Simplification
• Prioritization/selection
•Strategies were idiosyncratically applied and may
contribute to rater error

21. Why We Chose It
•Having to rate too many aspects of a
student’s behavior can diminish the reliability
of these ratings
•It is important to consider the rating
demands imposed on faculty when
developing assessment tools

22. Do Standardized Patients have
Concerns About Students Not
Captured by Traditional Assessment
Forms?
Blatt B. et al
Teaching and Learning in Medicine 2016;28:4, 395-405

23. Key Point
•A narrative assessment method may add
value to the traditional quantitative score by
identifying and characterizing problematic
student behaviors not captured by the
traditional assessment form

24. What They Did
• Purpose: To explore standardized patients (SP’s) "off the record"
concerns about students not captured via checklists
• Added a “Concerns” item to a traditional assessment form for an
end of third-year clinical skills examination
• SP’s asked to identify students about whom they had “gut level”
concerns and provide narrative comments to explain why
• Six SP cases were delivered to 551 students at three medical
schools
• “Concerns” data was analyzed quantitatively and qualitatively

25. What They Found
• 40% of students had one concern identified and 13% of
students had 2 or more concerns
• 3 major areas of concern: communication and interpersonal
skills (60%), history taking (23%) and physical exam (17%)
• In the group with 2 or more concerns (N=58), SP comments
identified potential clinical problems not identified by scores
on the traditional assessment form
Assessment Domain Students passing a domain on the
traditional form who had at
least 1 concern by SP
Communication/Interpersonal 84% (27/32)
History 42% (18/43)
Physical 48% (20/42)

26. Why We Chose It
•The "Concerns" item has the potential to
identify "at-risk" students not captured with
traditional forms
•These comments provide educators with a
unique assessment perspective and may
have implications for the teaching of
communication/interpersonal skills in clinical
skills courses

27. OSCE Circuit Performance Effects:
Does Circuit Order Influence Scores?
Monteiro S. et al
Medical Teacher 2016; 38:1, 98-100

28. Key Point
•The order of stations within an OSCE did not
have an impact on overall performance

29. What They Did
•Purpose: To see if order of stations in an OSCE
affects scores
•Analyzed almost 1000 exam scores from six different
OSCEs administered to undergraduate medical
students at McMaster University looking for potential
positive or negative effects of circuit order on
performance
•Conducted ANOVAs to evaluate performance
aftereffects following the highest scoring station and
the lowest scoring station

30. What They Found
•The negative or positive feedback a subject
received in one station did not have a
significant impact on subsequent
performance. Thus, there was no difference
between subjects’ scores based on circuit
order.

31. Why We Chose It
•This study reassures us that typical OSCE
design does not bias outcomes in favor of
certain groups of students
•We do not need to be overly concerned
about circuit order when designing OSCEs

32. Done or Almost Done? Improving
OSCE Checklists to Better Capture
Performance in Progress Tests
Pugh D. et al
Teaching and Learning in Medicine 2016; 28:4, 406-414

33. Key Point
•A non-binary checklist can be a useful tool for
scoring OSCE progress tests as it provides
flexibility to raters and allows for provision of
feedback about items that were attempted
versus done correctly

34. What They Did
• Purpose: To analyze the consequences of using nonbinary checklists
for scoring OSCE progress tests
• Non-binary checklists used for two Internal Medicine OSCE progress
tests (nonprocedural and procedural) at one institution; 119 tests
included
• Items were rated as either “done satisfactorily,” “attempted” or “not
done”
• Analyzed the impact of different scoring methods varying in
stringency (hawk, dove and hybrid) using these checklists;
determined which scoring method was best suited for a given task
• Compared difficulty, reliability, item-total correlations and pass rates
using the three scoring methods

35. What They Found
•Mean OSCE scores were highest when calculated
using the dove method and lowest using the hawk
method
•Overall score reliability did not differ significantly
between the three methods
•Discrimination, as measured by ITC, differed as a
function of the scoring method
•There was a main effect of PGY level regardless of
scoring method used, suggesting that all three
methods could differentiate between PGY levels

36. Why We Chose It
•OSCE progress tests, which are used to
measure the progression of clinical skills over
time, are relatively new
•This study provides insight into how to best
design and score OSCE progress tests to assess
the clinical skills of examinees at different
levels of training

37. Section Summary
• The quality of learner assessment obtained from OSCE's may
be affected by a variety of factors:
• Raters
-Having to rate too many aspects of a learner's performance
may adversely affect rating quality
• Standardized patients
-Narrative comments from standardized patients can
provide a unique assessment perspective
• Test format
-Non-binary checklists can enhance the information obtained
from OSCE’s
-Circuit order does not affect overall outcome

38. Developing Trainee Self-
assessment Skills Through
Feedback and Goal Setting
Alix Darden, PhD, MEd

39. Direct Observation of Clinical Skills
Feedback Scale: Development and
Validity Evidence
Halman S, Dudek N, Wood T, et al
Teaching and learning in medicine. 2016;28(4):385-394

40. Key Finding
•The Direct Observation of Clinical Skills
Feedback Scale (DOCS-FBS) is an instrument
which can be used to assess verbal feedback in
a clinical setting.
•The authors provide validity evidence for scores
from this instrument.

41. What They Did
•Purpose: To develop and obtain validity evidence for a
rating scale designed to assess the quality of verbal
feedback provided in the clinical setting
•Three phases:
1) Determined features of high-quality feedback using
nominal group technique
2) Developed the DOCS-FBS scale
3) Assessed the DOCS-FBS scale
for validity evidence

42. What They Found
•Phase 1: Identified 12 features of highly
effective feedback
•Phase 2: Generated a 9 item scale called “The
Direct Observation of Clinical Skills Feedback
Scale” (DOCS-FBS)
•Phase 3: Demonstrated validity evidence
from 3 sources (content, response process,
and internal consistency)

43. DOCS-FBS
1. Offers the learner an opportunity to reflect before feedback is provided
1. Trainee not given
opportunity to reflect on
performance
2. Trainee asked about
performance but not given
opportunity to reflect
3. Trainee allowed to reflect on
performance and to discuss
2.Feedback was provided in a respectful manner
1. threatening, judgmental or
belittling tone
2. Non-threatening tone but
perhaps judgmental or
provided in inappropriate
environment
3. Non-threatening or judgmental,
preceptor adapts to trainee reactions,
appropriate non-verbal language and
culturally sensitive
3. Appropriate communication style
1. Preceptor delivers message
in manner that is obviously not
well understood by trainee
2. Preceptor uses appropriate
communication style but some
elements lacking
3. Preceptor involves trainee in
conversation and adapts communication
style as required
4. Feedback focused on a specific behavior
1. No specific behavior was
identified only general
statements provided
2. A modifiable behavior was
identified but no or limited
feedback was provided
3. Preceptor identifies a specific
behavior and bases feedback around
this.

44. 5. Feedback was constructive
1. No suggestions geared
toward identified behavior
2. Concise issue raised but limited
suggestions provided to trainee
3. Concise issues identified and
trainee provided with information
to close a gap in knowledge
6. Ends with an action plan with goal to modify or reinforce an observed behavior
1. Feedback terminated
with no plans for follow-up
or reevaluation.
Broad action plan is suggested but
no specific to behavior or
encounter.
Clear plan to modify or reinforce a
behavior.
7.Limited to a manageable number of points (generally 2-3)
1. No points or too many
identified.
Attempted to limit to manageable
number of points but room for
improvement
Limited to a manageable number
of points that were appropriate for
training level
8. Appropriate time allotted to give feedback
1. Feedback rushed or too
lengthy.
2. Appropriate amount of time set
aside but certain issues rushed or
belabored
3. All issues addressed with
appropriate time, opportunity to
address pertinent points raised.
9. Preceptor verifies understanding of feedback
1. No verification of
understanding of points
raised during feedback
2. Preceptor verifies
understanding but does not
provide adequate clarification as
needed
3. Preceptor verifies understanding
and offers adequate clarification as
needed

45. Why We Chose It
•High quality feedback is vitally important to trainees
learning and has been shown to improve the quality of
their work
•Many faculty do not provide high quality feedback even
when provided faculty development
•This instrument provides a valid method to easily
evaluate faculty giving feedback – providing valuable
feedback on feedback!

46. Identifying educator behaviors for
high quality verbal feedback in health
professions education: literature
review and expert refinement.
Johnson CE, Keating JL, Boud DJ, et al.
BMC Med Educ. 2016;16:96.

47. Key Finding
•Identified the key elements of an educator’s
role in high quality verbal feedback in clinical
practice
•Developed a set of observable feedback
behaviors that could engage, motivate and
enable a learner to improve their clinical
performance

48. What They Did
•Purpose: To determine the components
required by educators to engage, motivate,
and enable a learner to improve his/her skills
•Two stages:
1) Conducted literature review using a “snowball
technique”
2) Created and refined educator behaviors using a Delphi
technique

49. What They Found
•Identified 18 distinct elements and 25 educator
behaviors to facilitate effective verbal feedback in
clinical practice
•Organized in 4 themes
1. The learner has to “do the learning”
2. The learner is autonomous
3. The importance of the learner-educator relationship
4. Collaboration

50. Why We Chose It
•Effective verbal feedback is critical in work-
based learning but problems with the current
practice are common
•Evidence-based tools of observable feedback
behavior provide a systematic method to
improve work-based feedback and thus
learning.

51. Can Individualized Learning Plans in
an advanced clinical experience
course for fourth year medical
students foster Self-Directed
Learning?
Chitkara MB, Satnick D, Lu W-H, Fleit H, Go RA,
Chandran L.
BMC Med Educ. 2016;16(1):232

52. Key Finding
•Individualized learning plans (ILPs) improve
self-directed learning (SDL) strategies among
medical students.
•They may serve as useful tools to help shape
future learning goals as students transition to
residency training.

53. What They Did
• Purpose: To determine the impact of ILPs on SDL for fourth
year medical students
• Mixed methods study at Stony Brook University
• Context: Advanced Clinical Experience (ACE)
• All students given time to complete ILP
• Given 1 week to pursue independent learning goals
•Data collected from convenience sample who
completed Pediatrics or Internal Medicine ACE
• Motivated Strategies for Learning Questionnaire (MSLQ) – pre/post
• Self-report surveys
• ILP’s
• End-of-course reflective essays

54. What They Found
MSLQ self
directed learning
domains
Overall
Mean (SD)
Pediatrics
Mean (SD)
Medicine
Mean (SD)
Self-Efficacy (pre
ACE)
5.1 (.86) 5.0 (.98) 5.2 (.73)
Self-Efficacy (post
ACE)
5.4 (.85)** 5.3 (.98) 5.5 (.68)
Self-Regulation
(pre ACE)
4.9 (.83) 5.0 (.88) 4.8 (.79)
Self-Regulation
(post ACE)
5.2 (.88)** 5.3 (.87) 5.1 (.89)
**p <0.01
MSLQ results of students pre and post the ACE Course

55. What They Found
Completion of ILP and Perceptions of Preparedness
Survey items Overall (n=48)
Completely achieved ILP goals
‘No’ (n=25) ‘Yes’ (n=23)
Prepared for residency (Rate 1–5), no. (%) – Pre ACE
1-3 ratings 31 (65 %) 17 (68 %) 14 (61 %)
4-5 ratings 17 (35 %) 8 (32 %) 9 (39 %)
Prepared for residency (Rate 1–5), no. (%) – Post ACE
1-3 ratings 16 (33 %) 12 (48 %) 4 (17 %)
4-5 ratings 32 (67 %) 13 (52 %) 19 (83 %)

56. What They Found
Qualitative analysis themes
1. Identifying personal strengths and weaknesses
“It forced me to highlight some of my deficits and/or knowledge
gaps, which is something I may not have done on my own”
2. Preparation for residency
“It gave me a chance to balance the clinical duties and educational
workload of a resident with same “real-life” responsibilities I will be
facing as an intern”
3. Acknowledging challenges and limitations
“realizing that I could not fully accomplish all my goals in a 1-month
period... I overcame these challenges by telling myself to just put in
a decent effort each day without getting discouraged”

57. Why We Chose It
•Activities that require students to
reflect on their learning needs and
develop learning goals are needed
throughout their medical education.
•These activities promote of the
development of life-long learning
skills.

58. Practical Suggestions for the Creation
and Use of Meaningful Learning Goals
in Graduate Medical Education
Reed S, Lockspeiser TM, Burke A, et al
Academic Pediatrics. 2016;16(1):20-24

59. Key Finding
•The development and use of learning goals by
trainees helps develop their lifelong learning
skills.
•A variety of practical strategies for implementing
learning goals for each stakeholder is provided.

60. What They Did
•Purpose: To provide practical guidance on how to
develop and use learning goals
•Perspective from the APPD, developed from:
• Collective experiences at the local level
• Prior research of the authors
• Review of the literature
• Interviews and Focus Groups

61. What They Found
Learner Faculty Environment
• Many settings
• Specific structure
• Schedule time
• Emphasize value
• Share
• Provide questions
• Create own goals
• Role play
• Address barriers
• Documentation
• Time and resources

62. Why We Chose It
•Need to promote life-long learning skills in
trainees
•Needs to be a multipronged approach
•Knowing a variety of strategies is useful
Life-long
learner

63. Section Summary
•Quality of verbal feedback can be assessed using
the DOCS-FBS instrument
•A series of observable behaviors can be used to
engage, motivate and enable a learner to
improve his/her clinical performance
•Life-long learning skills can be developed early in
training and best practices for all stakeholders
have been identified

64. Professionalism: A Complex
Environment for Learners
H. Barrett Fromme, MD, MHPE

65. A National Survey of Pediatric Residents’
Professionalism and Social Networking:
Implications for Curriculum
Development
Kesselheim JC, at al.
Acad Peds. 2016; 16:110-114.

66. Key Finding
•Disconnect between pediatric resident
attitudes and actions in social media use
•Educational interventions for social media
use are likely increasing in pediatric
residencies

67. What They Did
•Purpose: To characterize pediatric residents’
experience related to social networking and compare
that to program director data
•Survey of categorical pediatric residents from 13
programs
•Questions regarding:
• Familiarity with social networking sites (SNS)
• Perceptions of professionalism on SNS
• Educational interventions regarding SNS
•Compared to prior PD data

68. What They Found
0% 20% 40% 60% 80% 100%
Comments about peers
Comments about staff
Comments about patients
Comments about workplace
Comments about self
Thoughts/Observations
Never
1-4 times
>4 times
How often residents’ reported incidence of inappropriate SNS
posts over the past year

69. What They Found
How often residents’ reported incidence of inappropriate SNS posts over the past year
Activity
Frequency with which other
residents engage in activity
Perceived appropriateness
Never Monthly Daily Completely
Inappropriate
In-Between Completely
Appropriate
“Friending” 2% 44% 50% 0% 9% 90%
Joining social network 9% 38% 39% 1% 11% 89%
Friending colleague or peer 2% 60% 34% 0% 13% 87%
Posting thoughts/observations 34% 41% 4% 22% 61% 17%
Posting comments re:
workplace/workplace issues
37% 54% 3% 51% 48% 1%
Friending current patients/families 86% 4% 0% 89% 10% 0%
Friending former patients/families 87% 4% 0% 72% 28% 0%

70. Why We Chose It
•SNS Education needs to be expanded that is:
•Case-based, longitudinal
•Utilizes residents in the development and
teaching
•Emphasizes peer regulation
•Models exemplary behaviors as well as what not
to do

71. Speaking up: using OSTEs to
understand how medical students
address professionalism lapses
Tucker CR, at al.
Med Educ Online. 2016; 21:1-11.

72. Key Finding
• M4 students did not consistently address
professionalism lapses by their peers when
assessed in an OSTE
• Lapses related to procedural issues or
technical skills were more likely addressed
than personal issues or cultural competency

73. What They Did
•Purpose: To determine how M4 students address
professionalism lapses when they are purposefully
incorporated into an OSTE
•Mixed methods analysis of OSTE data over 2 year
period
•OSTE structure
• M4s watched video encounter of an standardized learner (M1 level)
performing an exam
• M4 provided feedback to standardized learner
• Faculty and standardized learner completed checklist
• Debriefing sessions recorded and transcribed

74. What They Found
0%
10%
20%
30%
40%
50%
60%
70%
Issues Identified by M4 Students
Percentwhoidentifiedissue

75. What They Found
Easy to provide feedback Difficult to provide feedback
“Procedural” issues (e.g.
hand-washing, draping)
Interpersonal skills
History and physical
examination performance
Personal dress/attire
Cultural competency

76. Why We Chose It
•Peer feedback on professionalism is difficult
•Certain areas pose greater challenges
•Context impacts responses to
professionalism lapses

77. Section Summary
•Professionalism is a broad and diverse topic
for trainees, that goes beyond direct patient
care
•Peer regulation may be valuable, but it is
difficult
•Effective educational strategies are essential

79. Clinical Teaching:
Faculty and Family
Donna D’Alessandro, MD

80. Clinical Teaching As Part of Continuing
Professional Development:
Does Teaching Enhance
Clinical Performance?
Jodelyn M. Lockyer, Carol S. Hodgson, Tzu Lee, et.al.
Medical Teacher. 2016;38(8):815-22.

81. Key Finding
•Physicians who have higher clinical
performance assessments spend more time
teaching
Teaching
Patient
Care

82. What They Did
• Purpose: To determine if there were differences in
physician clinical performance based on clinical
teaching when assessed through multisource
feedback
• Quantitative study of the Physician Achievement
Review (PAR) program of the College of Physicians
and Surgeons of Alberta, Canada. Surveys of clinical
performance as evaluated by colleagues, co-workers
and patients
• PAR data was analyzed by physician specialty,
academic appointment and time teaching

83. What They Found
Academi
c Appt
None
Academic
Appt
Full Time
Academic
Appt
P Value
Teaching Percent
0%
Teaching
Percent
15%
Teaching
Percent
P value
Family
Medicine
N=1831
PAR = 13, 526
4.43 4.64 0.001 4.43 4.53 0.001
Medical
Specialists
N=1510
PAR = 11,253
4.46 4.53 0.001 4.42 4.54 0.01
Surgeons
N=542
PAR= 4073
4.52 4.63 0.01 4.57 4.58 N.S.
PAR Data for Medical Colleagues

84. Why We Chose It
•This is a large data set
from multiple sources
•It’s outcome is simple
but clearly shows an
association between
improved higher clinical
performance ratings and
teaching
Teaching
Patient
Care

85. Ward Rounds With or Without an
Attending Physician: How Interns
Learn Most Successfully
L. Barry Seltz, MD, Erin Preloger, MD, Janice L. Hanson,
PhD, EdS, Lindsay Lane, BM, BCh
Acad Pediatr. 2016 Sep-Oct;16(7):638-44.

86. Key Findings
•Notable and unique contributions to learning
occurs when the attending is present and
absent
•During rounds, interns learn best when actively
engaged
Image: Tim Foley, UVA
Magazine

87. What They Did
• Purpose
1) To explore educational value of ward rounds for interns
2) To compare value with/without the attending present
• Qualitative study of 20 Pediatric interns from University of
Colorado
• Compared experiences at 2 clinical sites
• Site 1: Interns rounded with attending in FCR format
• Site 2: Interns rounded with senior resident only
• Semi-structured interviews, using grounded theory, analyzed
using constant comparative method

88. What They Found
Theme Subthemes
What is being learned 1. Learning related to patient care
2. Learning relate to leading a team
Learning environment 1. Rounds without an attending
2. Rounds with an attending
Learning and work balance
Way interns learn 1. Self-directed learning
2. Interactive learning
3. Experiential learning

89. What They Found
Attending Present Attending Absent
• Broader concepts
• EBM
• More formal, stress may promote
learning
• Less time efficient
• Practical, day-to-day management
• Less formal may increase learning
and autonomy
• Collaborative learning
• More time efficient
“Learning is optimized from experiencing ward
rounds with and without an attending physician”

90. What They Found
Interns learn best by active engagement:
1. Patient care discussions
2. Physical examination demonstrations
3. Visual learning (e.g. radiograph review)
4. Purposeful questioning in supportive environment
5. “Short and Sweet” teaching pearls

91. Why We Chose It
•Highlights value in both
types of rounds - attending
present and absent
•Illuminates what learning
occurs on the inpatient
wards
•Implications for staffing,
faculty development, and
residents as teachers
programs
Image: Cook County Hospital

92. Promoting Resident Autonomy During
Family-Centered Rounds: A
Qualitative Study of Resident,
Hospitalist, and Subspecialty
Physicians
Jimmy Beck, MD, MEd, Terry Kind, MD, MPH, Rebecca
Meyer, MD, MEd, Priti Bhansali, MD, MEd
J Grad Med Educ. 2016 Dec;8(5):731-738.

93. Key Finding
•Attending behaviors were identified which
promote resident autonomy before, during,
and after family-centered rounds
Image: University of Michigan

94. What They Did
•Purpose: To identify strategies used by attending
physicians to promote resident autonomy during
family centered rounds (FCR)
•Qualitative study of 10 attendings and 14
residents from Children’s National Health
System
•Semi-structured interviews of attendings, focus
group discussions with residents. All coded and
themed using qualitative content analysis

95. What They Found
•Residents and attendings had similar views
on ways to promote resident autonomy
•Thirteen themes identified, organized into
categories based on timing

96. What They Found
1) Set
expectations
2) Non-verbal
signals
1) “Huddle”
2) Plan rounds
1) Deliberate
positioning
2) Orient
families
3) Relinquish
control
4) Delegate
teaching
5) Flexibility
6) Probe for
rationale
7) Use silence
1) Promote
reflection
2) Facilitate
feedback
Start of
rotation
Before FCR During FCR After FCR

97. Why We Chose It
•These strategies are easy
to understand and to
apply practically
•Could be used for faculty
development programs,
or fellow and resident
training in education
Image from University of Iowa

98. Section Summary
•Faculty – there are educational benefits to rounding
both with and without attendings
•Family – faculty have a specific behaviors that
promote resident autonomy during FCRs

99. Leveraging milestones to enhance
training outcomes across the medical
education continuum
Teri Turner, MD, MPH, MEd

100. Reporting Achievement of Medical
Student Milestones to Residency
Program Directors: An Educational
Handover
Sozener C.B. et al.
Academic Medicine. 2016; 91:676-684

101. Key Finding
•It is feasible to assess medical student attainment
of specialty specific milestones prior to the start
of residency.
•A post-match milestone assessment can provide
program directors information on strengths and
weaknesses of incoming trainees.

102. What They Did
•Purpose: To assess the feasibility of an educational
handover from UME – GME within programs at U of
Michigan
•Used existing student assessment data and mapped
to the EM milestones
•Assigned milestone achievement levels for each
student
•Sent this data as a 2nd “Dean’s letter” to program
directors post-match

103. What They Found
•Handover tool was feasible
•Assessed all but 3 of the 23 EM milestones
•PDs felt they could use handover tool for residency
intervention
0
4
1
5
1
4
0
1
2
3
4
5
Used data from current
MSPE for adapting
residency training
Handover document
provided new info
Handover document would
allow for residency
intervention
Yes No
NumberofPDs
PDs thoughts on MSPEs and New
Handover Document

104. Why We Chose It
•Method to improve communication and standardize
assessment about student competency prior to
residency training
•Provide information on strengths and weaknesses
•Allow for early intervention and thus optimization of
GME training

105. The Community Health and Advocacy
Milestones Profile (CHAMP): A Novel
Tool Linking Community Pediatrics
and Advocacy Training to Assessment
of Milestones-Based Competence in
Pediatric Residency Training
Hoffman B.D. et al.
Academic Pediatrics. 2016; 16:309-313

106. Key Finding
•A tool which connects milestones to training
objectives in community health and advocacy
•A method for program evaluation and
improvement
•A model for mapping curricula to milestones

107. What They Did
•Purpose: link Community Pediatrics and Advocacy
objectives to assessment of milestones
•10 participants in randomly selected pairs matched 250
objectives to 5 competency domains (1 pair/domain)
•18 of 41 colleagues participated in a modified Delphi
process for consensus on matches
•Removed all matches with a mean of < 3.5
•Compiled data into a Community Health and Advocacy
Milestones Profile (CHAMP) Mapping Tool

108. Figure
Academic Pediatrics 2016 16, 309-313DOI: (10.1016/j.acap.2016.03.006)
Copyright © 2016 Academic Pediatric Association Terms and Conditions www2.aap.org/commpeds/cpti/CHAMP.html
What They Found

109. Why We Chose It
•No need to “reinvent the wheel”
•Assessment guide – maps
objectives in community
health/advocacy to 12 of the 21
milestones
•Helps identify gaps in training

110. Competency Evaluations in the Next
Accreditation System: Contributing to
Guidelines and Implications
Park Y.S. et al.
Teaching and Learning in Medicine. 2016; 28(2):135-145

111. Key Finding
•Assessments in GME need to be considered
within a “systems framework” to inform
competency decisions
•Ten guidelines for using end-of-rotation
evaluations for measuring and assessing
milestones

112. What They Did
•Purpose: To examine validity evidence of end-of-
rotation evaluation scores to measure competencies
and milestones
• Response process (by rater and by delays in evaluating)
• Internal structure
• Relations to other variables
• Composite score reliability in an assessment system
•Reviewed 2,701 evaluations for 142 IM residents for
21/22 IM milestones

113. What They Found
•Faculty showed the greatest ability to discriminate
differences
•Fellows and peers provide unique perspectives for
Professionalism and ICS
•Delayed evaluations - greater likelihood of raters
assigning the same scores across items
•Higher correlations between PC, MK and PBLI
•Combining rotation evaluation scores improves
reliability

114. Why We Chose It
•Provides practical and useful guidelines for workplace-
based assessment
• Obtain 10 or more evaluations for use during CCC
• Make decisions at the core-competency level
• Examine trends to project growth and identify problems
• Promote rater training
• Support data collection and monitoring

115. Section Summary
•It is feasible to handover learners from UME to GME
using specialty specific milestone based assessments
•There are tools to help us map objectives to
milestones within competency domains
•Using guidelines for end-of-rotation evaluations can
facilitate and improve decisions on resident progress

116. Graduate Medical Education
Management Strategies
Steve Paik, MD, EdM

117. Number Needed to Eat:
Pizza and Resident Conference
Attendance
Michael J. Cosimini, Liza Mackintosh, Todd P. Chang
Med Educ. 2016 Dec;50(12):1204-1207

118. Key Finding
•Providing food at conferences resulted in a
modest improvement in conference attendance
•The “Number needed to eat” was 10.6

119. What They Did
• Purpose: To determine the value of providing food at resident
educational conferences
• Retrospective study at a single university-based center
• Pizza was provided 2 Fridays/month at noon conference
• Attendance tracked electronically through badge swipe
• Compared overall attendance, arrival time, and percent arriving on time
during occasions in which food was provided vs. not provided
• Number Needed to Eat = residents provided with food
additional residents present

120. What They Found
Resident Year Food
(N=20) %
No Food
(N=16) %
PL 1 (N=32) 41.8% 40.2%
PL 2 (N=31) 29.2% 25.7%
PL 3 (N=27) 33.1% 26.9%
Total (N=90) 35.0% 31.3%
• Overall attendance increased 3.7% when food was provided
= 3.33 residents, p = 0.04
• Number Needed to Eat = 10.6
• Total cost = $46/for each additional resident to attend

121. What They Found
Resident Year Food
Mean Minutes Late
No Food
Mean Minutes Late
PL 1 (N=32) 4.1 5.86
PL 2 (N=31) 3.4 3.83
PL 3 (N=27) 3.78 3.13
Total (N=90) 3.82 4.59
• On-time arrivals (before 12:05) increased 11% when food was
provided (p <0.0001)

122. What They Found
•Food increased conference attendance but was
minimally effective, probably not cost-effective
•Authors make other points by noting
• “Costs were … increased by the consumption of pizza by
financially strapped medical students and academic
paediatricians...”
• Lead author even has a conflict of interest…

123. What They Found
•The authors may be overlooking some
important points:
• Arrival times – attendees are there for more of the
conference
• Why is the high NNE of 10 bad?
• Other benefits of food – meal sharing is one of the most
fundamental social interactions among humans across time and
space

124. Why We Chose It
•Spark discussion about
the cost-benefit of this
as a strategy for an
improved educational
environment
•Lots of people would be
interested because of
the culinary theme
Image: Glassdoor

125. National Cluster-Randomized Trial of
Duty-Hour Flexibility in Surgical
Training
Bilimoria KY, et al.
N Engl J Med 2016; 374: 713-727

126. Key Finding
•Flexible duty-hour policies were non-inferior
to current ACGME (pre 2017) policies
•Patient outcomes – non-inferior
•Well-being and educational quality – no
significant difference

127. What They Did
• Purpose:
• To compare ACGME duty hour policy to a flexible
policy to evaluate
• Primary outcome: Patient safety
• Secondary outcome: Resident perception and
satisfaction for well-being, education and patient
care
• Randomized trial
• 117 General Surgery Programs

128. What They Found
•Primary Outcome:
•Dissatisfaction for well being and education
quality showed NO difference
•Secondary Outcome: Perception of Negative
effect of duty hours went Down
• Decreased: Patient safety; continuity, clinical skills, autonomy
• Increased: Time; family; rest and health

129. Why We Chose It
•Large scope of collaboration between
residency programs
•Evidence that helped to inform ACGME
•ACGME listened

130. Factors Associated With Resident
Continuity in Ambulatory Training
Practices
Fortuna RJ, et al.
J Grad Med Educ. 2016; 8(4): 532-534

131. Key Finding
•Factors that increased resident continuity in
resident practices:
• Increased faculty time
• Having Scheduling protocols
• Absence of advanced practice providers
•Increased Continuity associated with
improved quality measures

132. What They Did
•Purpose:
• To compare continuity in resident practices to nonteaching
practices to identify factors associated with continuity and
quality metrics
•4 resident primary care practices
• 26 resident teams in internal medicine, pediatric,
family medicine and medicine-pediatric
• 117,235 visits
•30 affiliate nonteaching primary care clinics
• 207242 visits

133. What They Found
• Continuity better at nonteaching vs resident practices
• Factors associated:
• Scheduling protocol
• Absence of APP
• Increased faculty clinical time
• Policies for handoff senior to intern
• Quality
• Improved rates of diabetic control, HTN, screening
colonoscopy and mammography

134. Why We Chose It
•Evidence comparing nonteaching practices vs
resident practices used to show factors that
improve continuity
•Resident practices with highest continuity
approached nonteaching practices (it can be
done!)

135. Section Summary
•Data is vital even for the most obvious things:
• Providing food is an important motivator and educational
tool
• Large collaborative research on work hours impacted
policy
• Scheduling protocols in resident clinics impact continuity
•Educational research provided evidence that had
impact at the:
• Trainee level level
• Patient outcome level
• Policy Level

137. Special Thanks
• Michael Ryan, MD, MEHP, Virginia Commonwealth
University
• For being our equal partner in journal reviewing and slide editing
• The Academic Pediatric Association
• Especially the Education Committee
• Editors of Academic Pediatrics
• For allowing use of images from their articles
• The authors of the articles included here
• All medical education scholars for their amazing work