Effects of Scaffolding HOT Questions on Critical Thinking and Self-Efficacy
1. EFFECTS OF SCAFFOLDING HIGHER ORDER THINKING
QUESTIONS ON READER SELF-EFFICACY AND CRITICAL
THINKING OF SIXTH GRADE STUDENTS
Jason L. McKinnon
Western Connecticut State University
March 31, 2012
1
2. Dissertation Committee
Primary Advisor
Frank LaBanca, EdD
Secondary Advisors
Marcia A.B. Delcourt, PhD
Jennifer Mitchell, EdD
Reader
Michael Hibbard, PhD
2
3. Rationale
1. The promotion of higher order thinking (HOT)
questions in the classroom is inconsistent.
2. Teachers spend considerable time preparing
students for standardized tests in grades 3-12.
3. Clear scaffolding steps are not readily available for
classroom teachers.
4. Link between self-efficacy and higher order thinking
is not evident in the literature.
5. Common Core State Standards encourage higher
order thinking.
3
6. Definition of Key Terms
Critical Thinking is defined as a process of purposeful, self-
regulatory judgment.
Bloom’s Taxonomy (revised) provides a way to organize
thinking skills into six levels, from the most basic to the higher
order levels.
Higher order thinking is a systematic way of using the mind
to confirm existing information using various degrees of
abstraction.
6
7. Definition of Key Terms
Scaffolding is defined as a course of action that enables a child
or novice to solve a problem, carry out a task or achieve a goal
which would be beyond his unassisted efforts.
Self-Efficacy is defined as a person’s belief in their ability to
acquire new information or complete a task or activity to a
prescribed level of performance.
Zone of Proximal Development (ZPD) describes a relationship
where a learner can acquire greater independence, skills and
knowledge with skilled help.
7
8. Review of Literature
The learning principles underpinning this research can be drawn
from the following theoretical constructs:
• Sociocultural Theory
– Vygotsky (1978) Social interactions play a significant role in a
child’s development.
• Social Cognitive Theory
– Bandura (1986) One’s social cognition frames behavior
within an interactive context of personal thoughts and
beliefs regarding one’s own ability or performance.
8
9. Review of Literature
• Zone of Proximal Development: Vygotsky (1978,
1986)
Learning Zone
9
10. Review of Literature (scaffolding)
Researchers Sample Study Findings
Size
Liang (2011) n = 85 Impact of the Scaffolding student
middle Scaffolded Reading responses to short
school Experience (SRE) on stories produced a
students student learning in significant result
reading. compared to a
traditional basal
program.
(p < .01)
Oliveira (2009) n = 15 Teachers were Student-centered
offered training on a questions were
typology of questions found to establish
framework exploring longer student
the degree of responses and
student- promote higher
centeredness of levels of thinking as
teacher’s oral measured by Bloom’s
questions. taxonomy. 10
11. Review of Literature (self-efficacy)
• Bandura defines self-efficacy as a person’s judgments of his or
her ability to perform a task (1996).
• Self-efficacy exerts its influence through four major
processes: cognitive, motivational, affective, and selective
processes (Bandura, 1993).
Researchers Sample Size Study Findings
Barkley (2006) n = 400 Students with high Results indicated that
middle self-efficacy scores when grades six, seven
school predicted reading and eight grade
students comprehension students demonstrated
achievement on the efficacy beliefs, there
Stanford Achievement was a positive
Test. correlation to reading
achievement. (p < .01)
11
12. Review of Literature (HOT & CT)
Researchers Sample Size Study Findings
Moseley, et al. Review of Thinking Skills Bloom’s revised
(2005) 35 Frameworks for Use in taxonomy was
theoretical Education and identified as one of
frameworks Training. three frameworks to
and be of greater value in
taxonomies. promoting higher level
thinking.
Miri, David, & Uri n = 177 Purposeful teaching Students who received
(2007) high school for the promotion of instruction that
students higher order thinking fostered higher order
skills: A case for critical thinking skills
thinking. improved critical
thinking in science.
(p < .01)
12
13. Research Question 1
Is there a statistically significant difference in the critical thinking
skills of students who have participated in an instructional
scaffolding intervention focused on higher-order thinking
questions and those who have not?
13
14. Research Question 2
Is there a statistically significant difference in students’ self-
perceptions of themselves as readers who have participated in
an instructional scaffolding intervention focused on higher-order
thinking questions and those who have not?
14
15. Research Question 3
Is there a statistically significant difference in the frequency of
higher-order thinking questions asked by teachers and students
participating in an instructional scaffolding intervention focused
on higher-order thinking questions and those who have not?
15
16. Research Question 4
Is there a statistically significant correlation between critical
thinking skills (Mental Focus, Learning Orientation, Creative
Problem-solving, Cognitive Inquiry and Scholarly Rigor) and
reading self-efficacy (Progress, Observational Comparison, Social
Feedback, and Physiological States)?
16
18. Methodology:
Description of Setting
• Suburban community of approximately 24,000
people
• Homogeneous population: 93.4 % white
• Median household income: $105,000
• Two middle schools in the same district serving
approximately 1,300 students
18
19. Methodology:
Description of Sample
• n = 286 students
• Sample of convenience
• Intact classroom groups of grade 6 students
• Four teachers participated in this study
– 2 teachers from treatment (6 classes per teacher)
– 2 teachers from comparison (6 classes per teacher)
• 8-week study
19
20. Methodology: Description of Sample
Sample Sample Sample Sample
Population
Group n n n n
N
RQ 1 RQ 2 RQ 3 RQ 4
Treatment 129 121 124 129 116
Comparison 157 154 157 157 135
Total 286 275 281 286 251
20
21. Methodology: Description of Sample
Teacher Group Masters Degree Experience
Teacher A Treatment Yes 11-years
Teacher B Treatment Yes 20-years
Teacher C Comparison Yes 16-years
Teacher D Comparison Yes 16-years
21
23. Methodology:
Treatment
Treatment teachers received training in:
•Explicit scaffolding of strategies and steps through a
scaffolding map
•Decision making tree to determine if questions are
higher order thinking questions
•Scholarly description of questions using Bloom’s
revised taxonomy (2001)
•Unit of study that supported the development of HOT
questions
23
25. B
Figure 6. Decision making tree to assist teachers in determine level of
thinking. Adapted with permission from Critical thinking and formative
assessments, by N. Moore and T. Stanley (2010).
25
28. Instrumentation
The California Measure of Mental Motivation (CM3)
(Giancarlo, 2010)
• Instrument that measures critical thinking according to
five scales:
– Mental Focus, Learning Orientation, Creative Problem-
Solving, Cognitive Inquiry, and Scholarly Rigor
• 4-point Likert-type Scale
• Confirmed validity and reliability
• Test administered pre- and post treatment
28
29. Instrumentation
The Reader Self-Perception Scale (RSPS)
(Henk & Melnick, 1995)
• Instrument measures student perceptions of reading
self-efficacy
-Progress, Observational Comparison, Social Feedback, and
Physiological States
• 5-point Likert-type scale
• Measures four areas related to Bandura’s model of
self-efficacy
• Confirmed reliability and validity
• Test administered pre- and post treatment 29
30. Instrumentation
The Classroom Practice Record (CPR)
(Westberg, Archambault, Dobyns, & Salvin,1993)
• Observational tool designed to collect descriptive
information by coding specific information and
interactions
• Inter-rater agreement established at 82%
• Calculation of Cohen’s Kappa = .77 (Curdy, 2009).
30
31. Research Design
Group Pretest Treatment Posttest
Treatment group
(Scaffolding HOT O X O
questions)
Comparison group
(Traditional O O
instruction)
31
32. Research Design
• Level of Significance:
– Bonferroni correction set at p<.017 for research
questions one, two, and four (.05/3 = .017)
– Research question three set at alpha level = .05
• Type of Data:
– Interval-level data in the form of subscale means
(CM3 & RSPS)
– Categorical level data in the form of the CPR
32
33. Research Design-RQ 1
Code and Value Cleaning Pretest and Posttest Effects
• Data screening • Verification of normality
• Visual Inspection for (Skewness & Kurtosis)
missing values • Shapiro Wilkes utilized (no
• Pretest descriptive statistics significance)
• Homoscedasticity (Box’s M)
• Groups did not differ
33
34. Findings- RQ 1
• CM3 administered upon completion of the 8-week
treatment
• Subscales reflect SD ranging from 6.62 to 8.93
• Means ranged from 27.92 to 34.39
• MANOVA test revealed no significance differences
between the posttest means with Wilks’ Lambda = .
97 F(5,269) = 1.43, p = .21
34
35. Research Design-RQ 2
Code and Value Cleaning Pretest and Posttest Effects
• Data screening • Verification of normality
• Visual Inspection for (Skewness & Kurtosis)
missing values • Shapiro Wilkes utilized (no
• Pretest descriptive statistics significance)
• Homoscedasticity (Box’s M)
• Groups did not differ
35
36. Findings- RQ 2
• RSPS administered upon completion of the 8-week
treatment
• Subscales reflect SD ranging from .51 to .96
• Means ranged from 3.50 to 34.50
• MANOVA test revealed no significance differences
between the posttest means with Wilks’ Lambda = .
98 F(5,269) = 1.16, p = .37
36
37. Research Design-RQ 3
• Research question 3 utilized the CPR to record
the frequency of HOT & U/R questions
• Chi-squares require that certain criteria be
satisfied (Huck, 2008)
• 2x4 Independent Chi-square was utilized with
a four category response variable (Huck, 2008)
37
38. Findings- RQ 3
• The chi-square value* (x2= 940.16, df = 3, p<.05)
demonstrates that there is a significant difference
between the observed (posttest) and expected
(pretest) data.
• Standardized residual values above the absolute
value of two were identified as major contributors to
the chi-square.
• Key Finding: teachers & students in the treatment
group asked significantly more HOT questions than
teachers and students from comparison group.
* p < .01
38
39. Findings- RQ 3 (continued)
Treatment Comparison
Observed Expected a Standard Residual b Observed Expected a Standard Residual b
Teacher
102 45 8.50 30 49 -2.71
HOT
Teacher U/R 26 116 -8.36 131 158 -2.15
Student
81 7 27.97 10 7 1.13
HOT
Student U/R 12 19 -1.61 17 17 0.00
39
40. Research Design-RQ 4
• A correlation conducted posttest only (n =
251).
• A student’s CM3 score was compared to their
RSPS score
• Scatterplots were created to visually inspect
for outliers (Huck, 2008).
• A bivariate two-tailed Pearson correlation was
conducted to measure the relationship.
40
42. Findings- RQ 4
• Progress subscale from RSPS:
– When I read, I don’t have to try as hard as I used to.
– I am getting better at reading.
– When I read, I need less help than I used to.
– Reading is easier for me than it used to be.
• Progress subscale Mean= 4.44; SD = .54 (little
variability)
42
43. Limitations of Study
1. Quasi-experimental design
2. Length of treatment
3. Study generalized to similar population
4. Adherence to treatment protocols
43
44. Implications for future research
Research Findings Implications for Relation to
question future research the literature
HOT question use Examine if HOT question Broadening the study
alone did not impact use beyond a treatment across disciplines and
student critical period of 20-weeks could content-specific
1 thinking. impact critical thinking subjects would
skills of students. enhance students’
abilities to apply skills
(Halpern, 1998).
HOT question use Examine how teacher Lengthen treatment
alone did not impact feedback concerning beyond 20-weeks for
reader self-efficacy. student question use students to
2 improves student self- appreciate their
efficacy. growth (Graham &
Weiner, 1996).
44
45. Implications for future research
Research Findings Implications for Relation to
question future research the literature
-Explicit scaffolding - The use of HOT -Broadening the study
techniques in the area questions and Bloom’s across disciplines and
of HOT questions taxonomy is one tool that content-specific
3 were found to be a teacher can use in the subjects would
successful to promote classroom. A search for enhance students’
HOT question use. related tools to promote abilities to apply skills
question use is needed. (Halpern, 1998).
-Certain dimensions -Future research could be -The lack of literature
of reader self-efficacy aimed at widening this in this area prompted
correlate with certain relationship to include this exploration.
4 dimensions of critical academic achievement in
thinking. other core subjects.
45
Observations as a school administrator…According to Liang (2008), the promotion of HOT questions in the classroom is less frequent and deliberate than is needed. According to (Linda Darling-Hammond 2010) this mechanistic form of teaching often accompanies traditional test preparation and can diminish a teacher’s focus on the effective instruction 3. According to Fisher and Frey (2010), teachers inconsistently apply scaffolding techniques in the classroom. This study aims to consolidate this understanding 4. According to an ERIC and EBSCO search, literature exploring the link b/t reader self-efficacy and critical thinking is not available. 5. Common Core State standards emphasis depth rather then breadth of knowledge and target HOT.
Student Progress: According to the National Assessment of Educational Progress (2008) NAEP achievement in fourth and eighth grades have been stagnant during the past ten-years. While American schools seem proficient at teaching basic reading skills, where there has been marginal growth, students are increasingly challenged to broaden these reading skills to higher levels. Scaffolding: According to Wood and Wood (1996), the teacher is not provided with concrete direction on the “nature of the guidance and collaboration needed that promotes development” (p. 5) 3. Higher Level Thinking: According to Geertsen (2003), the “indiscriminate use of terms such as critical thinking, reflective thinking, and high-level thinking has created unnecessary confusion” (p.1) Therefore, HOT remains as an elusive concept to grasp. 4. HOT Questions: The need to identify a structure to counteract this disparity is therefore important. As a result, Bloom’s revised taxonomy (Anderson, et al., 2001) is an important tool in the treatment to operationalize higher order questions. This pragmatic structure will be more easily understood and implemented in the classroom. A ccording to an EBSCO and ERIC search of more than 1.3 million records and 320,000 full-text articles, a link between higher order thinking and reader self-efficacy was not found.
The potential benefits of this research will help determine if scaffolding the development of HOT questions will significantly improve reader self-efficacy and critical thinking. In addition, the study could potentially provide teachers with clear scaffolding steps and questioning strategies that will improve instruction while increasing understanding of student learning. Finally, I hoped to explore the link b/t reader self-efficacy and critical thinking.
For the purposes of my study, this is how I defined the following terms: 1. Giancarlo’s definition, (CM3) 2. Organized thinking into six levels (remembering, understanding, applying, analyzing, evaluating, and creating) and represented a pragmatic way to design effective instructional tasks. 3. According to Resnick (2003) higher order thinking resists precise forms of definition…in this study, it was operationalized using Bloom’s revised taxonomy
Scaffolding: Vygotsky, (Wood, Bruner and Ross) . In this study, scaffolding will consist of questions, prompts, cues, modeling and dialogue. Clark and Graves (2004) linked scaffolding to the Gradual Release of Responsibility model Self-Efficacy: Bandura
The learning principles underpinning this research can be drawn from the following theoretical constructs:
Vygotsky (1978, 1986) described the ZPD with two developmental levels: the actual developmental level, which is what the learner can achieve by himself or herself and, the potential level of development, which is established when a learner is assisted by a more expert other. The ZPD is operationalized through the support of a scaffold where both teacher and student are actively engaged in the learning process
The review of literature in the area of instructional scaffolding notes that: The term scaffolding was introduced by Wood, Bruner and Ross (1976) in an attempt to operationalize the concept of teaching in the ZPD. They described scaffolding as a “process that enables a child or novice to solve a problem, carry out a task or achieve a goal which would be beyond his unassisted efforts” (p. 90). Just to pick out a couple of studies, Liang and Oliveira Liang (2011) This study was a pretest-posttest, quasi-experimental design with the student as a unit of study. Students in the comparison group were taught short stories using a traditional basal approach and assessed. Students in the treatment group were taught short stories using both the SRE approach. Comprehension and reader responses were significant at the P=.01 level for reading comprehension and reader responses. Oliveira (2009) Qualitative study focused on 15 k-6 teachers who received 1-day of training on questioning. Post treatment, the teachers were video taped during six lessons and their questions coded. The level of student thinking in their responses to teacher questioning improved according to Bloom’s taxonomy…showing that scaffolding student questions increased HOT.
As for the review of literature in the area of self-efficacy: Barkley= in his non-experimental study of 400 middle students, found that students with high self-efficacy correlated positively (P<.01) to reading achievement on the Stanford Achievement Test (Correlational analysis was conducted to determine statistical significance between scales of the Teacher and Student Efficacy Beliefs Survey (Barkey, 2006)
For the purposes of this study, Higher Order Thinking is defined as a hierarchy of learning that corresponds with Bloom’s revised taxonomy of overlapping levels above Understanding (Anderson, et al., 2001). Critical Thinking is defined as a process of purposeful, self-regulatory judgment. (Defined by the developers of the CM3 (Giacarlo, 2010) In Mosseley, et al, (2005) did descriptive review of 35 theoretical frameworks and taxonomies, Bloom’s revised taxonomy was identified as one of three frameworks to be of greater value due to its value in promoting higher level thinking. Miri, David and Uri (2007), in their longitudinal case study of grade 10 and grade 12 students (N=177), found that purposeful teaching for the promotion of higher order thinking skills enhanced critical thinking of students compared to traditional approaches
intact groups will be utilized
As discussed later in the chapter, this sample of convenience (n=286) varied marginally depending on the code and value cleaning required for each research questions Different colors here…
Change colors
Prior to the 8-week treatment, teachers in the treatment group will receive professional development training by this researcher for 3 hours on the topic of teacher questioning and scaffolding. This training focused on 4 areas that I will discuss later. Following initial training, PD will be on-going. As a result, an additional 2.0 hours of professional development followed through out the treatment focusing on a unit of study and HOT question tree
One of the goals of this study was to make scaffolding more explicit in the classroom.
Chin (2007) noted that teacher questioning can serve to scaffold and advance student thinking Adapted with permission from the work of Fisher and Frey (2010), a scaffolding map identified explicit scaffolding techniques through the use of Questioning: asking clarifying questions, referential questions, heuristic questions and HOT questions; followed by prompts and cues students when misconceptions arose. When prompts and cues did not lead to deeper understanding, then teachers moved to more direct explanations and modeling.
Teachers found they needed another layer of support to determine the difference b/t lower order vs. higher order thinking questions. This flowchart scaffolded their thinking and helped teachers understand the dichotomy more explicitly.
Higher order thinking in this study was operationalized using Bloom’s revised taxonomy (Anderson, et al., 2001). Bloom’s Revised Taxonomy identifies Applying, Analyzing, Evaluating and Creating as higher order thinking strands. In Bloom’s earlier taxonomy, Knowledge and Comprehension were considered lower levels on the taxonomy; according (Anderson, et al ., 2001), Understanding and Remembering (U/R) correspond to this hierarchy. While Understanding and Remembering represent lower levels in the hierarchy; mastery of these foundational levels were prerequisite to mastery of the next levels (Anderson, et al., 2001).
This slide represents the unit of study that assisted teachers to focus on embedding HOT questions into their daily language arts lessons. This unit of study and accompanying activities can be found in your packet and was part of the professional training teachers received prior and during the treatment phase. Important to note: The comparison group followed the grade 6 curriculum derived from the State Standards which did not focus on HOT questions or levels of thinking. The unit of study served as a focus for the treatment and a way to ensure treatment fidelity Emphasize 22 observations…treatment followed… Explain that teachers did 4 weeks of short stories and lt circles
The instrument utilized to measure research question one was the California Measure of Mental Motivation (CM3) (Giancarlo, Blohm & Urdan, 2004). The CM3 is a 72-item, self-report instrument designed to measure student critical thinking The CM3 has five scales that include: Mental Focus, Learning Orientation, Creative Problem- solving, Cognitive Integrity and Scholarly Rigor.
The instrument utilized to measure RS question 2 is the RSPS, which consists of 32 items that represent 4 scales (Progress, Observational Comparison, Social Feedback, and Physiological States). These four factors were based on the work of Bandura (1977, 1982) and Schunk (1983) related to their basic model of self-efficacy. I used terms self-perceptions and self-efficacy interchangeably
In this case, I recorded verbal interactions between the teacher and student, or vice versa, particularly when in the form of a Remembering/Understanding (R/U) or HOT question. At the beginning of this study, this researcher and his advisor observed four 41-minute lessons whereby we coded questions individually. Both the researcher and advisor are certified by the state of Connecticut (certification 092) for Intermediate Supervision and Evaluation and both regularly observe classroom teachers as part of their professional responsibilities.
This study is a quasi-experimental research design with a pretest-posttest comparison group. Intact classroom groups were utilized and as a result there was no random selection of participants to comparison and treatment groups. This design was selected to investigate the impact that scaffolding the development of HOT questions in the classroom had on students’ critical thinking, readers’ self-efficacy and the frequency of HOT questions
Because three of the four tests conducted in the study used the same participants’ pretest and posttest scores as sources of data, a more stringent alpha level was utilized to guard against a Type 1 error. Using a Bonferroni technique, the alpha value of p <=.05 was selected and subsequently divided by the number of comparisons (.05÷3), thus establishing an alpha level of p <=.017. The alpha level for research question three is established at p <.05 on the basis of the data being collected to assess the relationship between expected and observed frequencies. OR The three research questions all have the commonality of utilizing the same individuals as sources of data. Because of this similarity a Bonferroni adjustment technique was used.
Research question one consisted of one independent variable with two levels (treatment and comparison groups); the focus on grade six students was a constant; and five dependent variables (Mental Focus, Learning Orientation, Creative Problem-solving, Cognitive Inquiry and Scholarly Rigor) as measured by the CM3 used to examine critical thinking. A Multivariate Analysis of Variance (MANOVA) was applied to determine if there was a significant difference between the dependent variables based on the independent variable of treatment and comparison groups. After code and value cleaning, the data were examined to review normality and homescedasticity.
Research question one consisted of one independent variable with two levels (treatment and comparison groups); the focus on grade six students was a constant. There were four dependent variables (Progress, Observational Comparison, Social Feedback, and Physiological States) as measured by the RSPS used to examine reader self efficacy. A Multivariate Analysis of Variance (MANOVA) was applied to determine if there was a significant difference between the dependent variables based on the independent variable of treatment and comparison groups. After code and value cleaning, the data were examined to review normality and homescedasticity.
Research question one consisted of one independent variable with two levels (treatment and comparison groups); the focus on grade six students was a constant. Data were collected from 22 observations in treatment and comparison classrooms on the frequencies of HOT and U/R questions for both teachers and students. A chi-square was applied to measure if the scaffolding of HOT questions will be significantly different between expected (pretreatment) and observed (post treatment) frequencies. Criteria: First, sample size and “expected frequencies need to be sufficiently large for the chi-square to function as intended. Second, expected values of less than five cannot be evident in more than 20% of the cells and values need to be greater than zero. Third, data must be reported in raw frequencies and be independent. A two-group (treatment/comparison) independent chi-square test was utilized with a four category response (HOT/UR) variable (Huck, 2008).
* actually significant to p<.01 level
After receiving the eight-week treatment focusing on scaffolding the development of HOT questions, teachers and students in the treatment group asked significantly more higher order thinking questions than teachers and students from comparison group. Conversely, teachers from the treatment group asked significantly less U/R level questions. Students in the treatment asked less U/R level questions, but this difference was not a major contributor to the Chi square (R=-1.61).
Research question four examined the relationship between critical thinking (Mental Focus, Learning Orientation, Creative Problem-solving, Cognitive Inquiry and Scholarly Rigor) and reader self-efficacy (Progress, Observational Comparison, Social Feedback, and Physiological States). A bivariate Pearson correlation was utilized to measure the strength of this relationship. This researcher decided to utilize a Pearson product-moment correlation coefficient r reported as a decimal between -1.00 and +1.00 (Huck, 2008).
This slide demonstrates that five subscales of an instrument correlate positively to three-out-of-four subscales of another instrument, at the alpha level of .001 Hence, certain dimensions of critical thinking were correlated with certain dimensions of reader self-efficacy at the p <.001 level. The strength of this relationship varied according to the scale; for example Physiological states correlated with Mental Focus in the high positive range .744; while Observation Comparison correlated with Mental focus a low positive level .320 All levels significant, except PROGRESS!!!
Possible reasons: -Progress defined as present level of performance compared to past level of performance -Small SD of .54 indicates very little variability from the mean possibly due to homogeneous group of high achieving students -The lack of information available in the literature might be an area of future study; certainly a reason for this exploration in this study
Quasi-experimental design---could not control for class variability. Length of treatment-only 8 weeks…students could be familiar with instruments. I tried to minimize this by NOT providing student feedback. Study can only be generalized to a similar community comprising of similar demographics. Consider studies with similar homogeneous population Adherence to Treatment protocols-internal threat to validity controlled by classroom observations and the unit of study.