The Teaching Pyramid Observation Tool (TPOT) is an observation and interview tool used to assess teachers' implementation of practices from the Pyramid model, which promotes social-emotional competence and addresses challenging behaviors. The TPOT was developed to measure fidelity of Pyramid model implementation. It provides information to describe implementation quality, compare practices across teachers, and identify training needs. The TPOT includes subscales for key practices, red flags, and strategies for responding to challenging behaviors. Studies show the TPOT has good psychometric properties and its scores correlate with other measures of classroom quality. Coaching using TPOT data can help teachers improve their Pyramid model implementation over time.
2. Overview of TPOT
• Observation- and interview-informed,
judgment-based rating scale
• Developed to assess teachers’ implementation
of universal and targeted Pyramid model
practices and teachers’ capacity to implement
tertiary/individualized practices
4. Teaching Pyramid Observation Tool (TPOT)
• The TPOT was developed to measure the fidelity with which
teachers implement Pyramid Model practices
• Provides information that can be used to:
– Describe “quality” of implementation of TPOT practices
– Compare implementation within and across
teachers/classrooms
– Identify needs of teachers for training and support
5. TPOT
• Scoring booklet
– Classroom information
– Schedule
– Items/indicators
– Space for notes
• Manual
– Definitions
– Items/indicators with scoring guidance,
rules, and clarifications
6. Using the TPOT
• Observations
–Conducted for a minimum of 2 hours
• Must observe centers or free play, at least one
teacher-directed activity and transition between
activities
• Focus of observation is primarily lead teacher’s
behavior, but consider all adults
• Interviews
– For those practices that might not be observed during a
2-hour observation
7. Organization of the TPOT:
Subscales
• Key Practice Items: Multiple indicators associated
with each item
– Each indicator rated yes, no, or N/0 (only when noted)
• Red Flags
– Each item rated yes or no
• Using Effective Strategies for Responding to
Challenging Behavior
– Includes three indicators that represent “essential” strategies for responding
to challenging behavior and three additional strategies that might be used
– Each item rated as yes or no
– Only scored when challenging behavior is observed during the observation
period
8. Key Practice Item Subscale
Observation items
1. Schedules, routines, & activities (SR)
2. Transitions between activities (TR)
3. Supportive conversations (SC)
4. Promoting engagement (ENG)
5. Providing directions (PD)
6. Collaborative teaming (CT)
7. Teaching behavior expectations (TBE)
8. Teaching social skills and emotional
competencies (TSC)
Observation and interview items
9. Teaching friendship skills (FR)
10. Teaching children to express
emotions (TEE)
11. Teaching problem-solving
(TPS)
Interview items
12. Interventions for children with
persistent challenging behavior
(PCB)
13. Connecting with families (COM)
14. Supporting Families in using
Pyramid Model practices (INF)
15. Challenging Behavior Subscale:
Using Effective Strategies to Respond to Challenging Behavior (SCB)
• Only scored when challenging behavior occurs. If
challenging behavior occurs during the observation,
score for each incident of challenging behavior.
• If one or more incidences of challenging behavior occur that
meet the definition for challenging behavior, record a short-
hand note in the ‘incident’ box to describe the type of
challenging behavior you observed
• For each incident that occurred, mark ‘yes’ if you observed
the teacher using a practice related to the SCB indicator or
mark ‘no’ if you did not observe the teacher using the practice
to address the challenging behavior
19. Studies to Examine the Psychometric Integrity of
Teaching Pyramid Observation Tool (TPOT) Scores
• Primarily Head Start Classrooms
• TPOT observations
– 50 classrooms
– 2 raters
– 3 occasions
• 300 total TPOTs
• Generalizability study to look at
dependability of scores across
items, raters, occasions of
measurement
• CLASS
(Pianta, LaParo, & Hamre, 2008)
– 50 classrooms
– Administered between 2nd and 3rd
TPOT observation
– Inter-observer agreement for 34% (n
= 17)
– Convergent score validity
20. Generalizability Study
• Examine potential sources of
error in TPOT scores
• Teachers X Raters X Indicators
X Occasion
• Less than .01% of variance on
key practice indicators due to
raters
• .22% of variance on key
practice indicators due to
occasion
• 5.1% of variance on key
practice indicators due to
classroom/teacher
• Phi coefficient for key practice
indicators = .89
• G coefficient for key practice
indicators = .95
• Good news!
• Dependability in rank
ordering of classrooms
and dependability in
scores across raters,
occasions
• Phi coefficient = important
because want teachers to
reach criterion with
respect to implementation
of TPOT indicators
Snyder, P., Hemmeter, M.L., Fox, L., Bishop, C., &
Miller, M.D. (2013). Developing and gathering
psychometric evidence for a fidelity instrument.
Journal of Early Intervention, 35, 150-172.
21. TPOT and CLASS Correlations
N = 50
TPOT Key
Practices
Emotional Support
(ES)
.70
Classroom
Organization (CO)
.73
Instructional Support
(IS)
.76
Snyder, P., Hemmeter, M.L., Fox, L., Bishop, C., & Miller, M.D. (2013).
Developing and gathering psychometric evidence for a fidelity instrument.
Journal of Early Intervention, 35, 150-172.
22. Noteworthy Correlations:
TPOT and CLASS*
• TPOT Key Practices subscale and every CLASS dimension
and domain
• TPOT Red Flags subscale and every CLASS dimension
and domain (negative relationships)
• General teaching items on TPOT (SR, TR, SC, ENG, PD)
and each CLASS dimension and domain
• Most targeted teaching items on TPOT (TBE, TSC, TEE,
TPS, FR) and Instructional Support CLASS domain
• TPOT Connecting with Families with each dimension and
domain on CLASS
Additional detail in Chapter 7 in TPOT manual
23. Noteworthy Correlations
TPOT and ECERS-R
• TPOT Key Practices subscale and overall ECERS-R
• 10 of the 14 TPOT key practice items and overall
ECERS-R
• TPOT Red Flags subscale and overall ECERS-R
(negative relationships)
• TPOT Supporting Family Use of Pyramid Model
Practices and 5 of 7 ECERS-R subscales
• TPOT Strategies to Promote Engagement and 6
of 7 ECERS-R subscales
Additional detail in Chapter 7 in TPOT manual
24. Pre-SET and Select TPOT-Pilot Version Key Practice Items
(N = 31 Classrooms)
Note. Adapted from Steed and Pomerleau (2012). N = 31 classrooms.
a = Seven environmental items included on pilot version of TPOT.
* = p < .05 ** = p < .01
Additional detail in Chapter 7 in TPOT manual
25. Figure 1. Mean Teaching Pyramid Observation Tool implementation scores across
waves by experimental condition. Total number of TPOT indicators = 108.
Statistically significant and noteworthy differences at Wave 4 [t(40.03)=6.80, p<.001,
Cohen’s d=2.6)
Results: Implementation of Teaching Pyramid Practices
Hemmeter, M.L., Fox, L., Snyder, P., & Algina, J. (2011, April). Efficacy of a classroom-wide model for promoting social-
emotional development and preventing challenging behavior. Paper presented at the annual meeting of the American
Educational Research Association, New Orleans, LA.
26. Results: SSIS Child Outcomes
Adjusted Means Cohen’s d
Effect SizeIntervention Control
Target Children
Social 88.6 84 .41
Problem Behavior 108.7 115.5 -.52*
Non Target
Children
Social 103.8 96.4 .46*
Problem Behavior 95.2 99 -.29
Hemmeter, M.L., Fox, L., Snyder, P., & Algina, J. (2011, April). Efficacy of a classroom-wide
model for promoting social-emotional development and preventing challenging behavior. Paper
presented at the annual meeting of the American Educational Research Association, New
Orleans, LA.
27. Figure 2. Mean frequency of positive social interactions during 60 min observation session across
waves for Cohort 1 target children whose teachers were in the intervention or control condition.
An average of the frequency of positive social interactions for the 2 to 3 target children in each
classroom was used to derive the means reported for each group at each wave.
Results: Target Child Social Interactions
Hemmeter, M.L., Fox, L., Snyder, P., & Algina, J. (2011, April). Efficacy of a classroom-wide model for promoting
social-emotional development and preventing challenging behavior. Paper presented at the annual meeting of the
American Educational Research Association, New Orleans, LA.
28. Using the TPOT to Inform Decision
Making
• Using the TPOT in coaching
– Running TPOT
– Formal TPOT
– Goal setting/action planning
• Using the TPOT program wide
– Monitor implementation of PW implementation
– Plan professional development
• Using the TPOT in monitoring/evaluation
Arrows animated in this order:ItemIndicatorsNo OpportunityScore ColumnsItem refers to the title or heading of several indicators.Indicators are the individual skills or practices you are looking for during the observation.Some of the indicators have an “N/O” in the notes column – in these cases, it is because there might be no opportunity to observe this in the classroom. Rather than scoring “no” you can score that you did not get the chance to see it.Most of the indicators do not have an n/o option; you will just score yes or no in the score columns.
Here is item 1 – schedules, routines, and activities. This is one of the observation only items.
Here is item 9 – teaching friendship skills. This is one of the observation and interview items.
Here is a snapshot of the red flag item page. Remember, these are practices that are important to have in place. If they are not in place, you will want to work towards implementing them first.
Here is a snapshot of the challenging behavior item.