Writing good peer instruction questions. Presented at the CSULA STEM Summer Institute on Active Learning in the STEM classroom. Peter Newbury September 2013

1. Writing good peer instruction questions
1
(Image:stoolIIbytilanesevenonflickrCC)
constructivist
Peer Instruction
Writing Good
Questions
Peter Newbury, Ph.D.
Center for Teaching Development,
University of California, San Diego
pnewbury@ucsd.edu
@polarisdotca
ctd.ucsd.edu
September 11, 2013 CSULA Unless otherwise noted, content is licensed under
a Creative Commons Attribution-NonCommericial 3.0 License.

2. Writing good peer instruction questions2
peer instruction with clickers
interactive demonstrations
surveys of opinions
reading quizzes
worksheets
discussions
videos
student-centered instruction

3. Clicker Question (Economics)
Writing good peer instruction questions3
For which of the following professionals is driving an
expensive car a credible signal of their relative abilities
(that is, compared to others in the same profession)?
A) a carpenter
B) a realtor
C) a politician
D) a major league baseball player
(Steve Morris, UCSD)

4. Typical episode of peer instruction
Writing good peer instruction questions4
Alternating with 10-15 minute mini-lectures,
1. Instructor poses a conceptually-challenging,
multiple-choice question.
2. Students think about question on their own.
3. Students vote for an answer using clickers,
smart phones, colored/ABCD voting cards,
Poll Everywhere,…
4. The instructor reacts, based on the
distribution of votes.

5. In effective peer instruction
Writing good peer instruction questions5
 students teach each other while
they may still hold or remember
their novice preconceptions
 students discuss the concepts in their
own (novice) language
 the instructor finds out what the students know (and
don’t know) and reacts, building on their initial
understanding and preconceptions.
students learn
and practice
how to think,
communicate
like experts

6. Effective peer instruction requires
Writing good peer instruction questions6
1. identifying key concepts, misconceptions
2. creating multiple-choice questions that
require deeper thinking and learning
3. facilitating peer instruction episodes that
spark student discussion
4. resolving the misconceptions
before
class
during
class

7. t h e l e a r n i n g c y c l e
Peer instruction helps students learn...
Writing good peer instruction questions7
BEFORE DURING AFTER
setting up
instruction
developing
knowledge
assessing
learning
Adapted from Rosie Piller, Ian Beatty, Stephanie Chasteen

8. What makes a good clicker question?
Writing good peer instruction questions8
clarity Students should waste no effort trying to figure
out what’s being asked.
context Is this topic currently being covered
in class?
connection to
learning goals
Does the question make students do the right
thing to demonstrate they grasp the concept.
distractors What do the “wrong” answers tell you about
students’ thinking?
difficulty Is the question too trivial? too hard?
stimulates
thoughtful
discussion
Will the question engage the students and
spark thoughtful discussions?
Is there potential for you to be “agile”?
(Adapted from Stephanie Chasteen, CU Boulder)

9. t h e l e a r n i n g c y c l e
Peer instruction helps students learn...
Writing good peer instruction questions9
BEFORE DURING AFTER
setting up
instruction
developing
knowledge
assessing
learning
Adapted from Rosie Piller, Ian Beatty, Stephanie Chasteen

10. Clicker question
Writing good peer instruction questions10
Melt chocolate over low heat. Remove the chocolate
from the heat. What will happen to the chocolate?
A) It will condense.
B) It will evaporate.
C) It will freeze.
(Question: Sujatha Raghu from Braincandy via LearningCatalytics)
(Image: CIM9926 by number657 on flickr CC)
assess prior knowledge

11. Clicker question
Writing good peer instruction questions11
Which had the most positive impact on the modern world?
A) coffee
B) tea
C) chocolate
D) spice
E) sugar
(Herbst, UCSD)
provoke thinking
clarity
context
learning goals
distractors
difficulty
discussion

12. Clicker question
Writing good peer instruction questions12
In your opinion, which had the most positive impact on
the modern world?
A) coffee
B) tea
C) chocolate
D) spice
E) sugar
(Herbst, UCSD)
provoke thinking

13. Clicker question
Writing good peer instruction questions13
A ball is rolling around
the inside of a circular
track. The ball
leaves the track
at point P.
Which path
does the ball
follow?
P
A
B
C
E
D
(adapted from Mazur)
predict

14. t h e l e a r n i n g c y c l e
Peer instruction helps students learn...
Writing good peer instruction questions14
BEFORE DURING AFTER
setting up
instruction
developing
knowledge
assessing
learning
Adapted from Rosie Piller, Ian Beatty, Stephanie Chasteen
The students have not
(re)solved concept X.
But they’re know X exists
and why X is interesting.

15. t h e l e a r n i n g c y c l e
Peer instruction helps students learn...
Writing good peer instruction questions15
BEFORE DURING AFTER
setting up
instruction
developing
knowledge
assessing
learning
Adapted from Rosie Piller, Ian Beatty, Stephanie Chasteen

16. Clicker question
Writing good peer instruction questions16
Which of these are reasons for the seasons?
i. the height of the Sun in the sky during the day
ii. Earth’s distance from the Sun
iii. how many hours the Sun is up each day
A) ii only
B) iii only
C) i and ii
D) i and iii
E) i, ii and iii
clarity
context
learning goals
distractors
difficulty
discussion
probe misconception

17. Clicker question
Writing good peer instruction questions17
How many of these are reasons for the seasons?
height: the height of the Sun in the sky during the day
distance: Earth’s distance from the Sun
hours: how many hours the Sun is up each day
A) none of them
B) one
C) two
D) all three
probe misconception

18. Clicker question
Writing good peer instruction questions18
Select the line that
you feel has the
strongest imagery in
“Fast rode the
knight” by Stephen
Crane (1905).
analysis
Fast rode the knight
With spurs, hot and reeking,
Ever waving an eager sword,
"To save my lady!"
Fast rode the knight,
And leaped from saddle to war.
Men of steel flickered and gleamed
Like riot of silver lights,
And the gold of the knight's good banner
Still waved on a castle wall.
. . . . .
A horse,
Blowing, staggering, bloody thing,
Forgotten at foot of castle wall.
A horse
Dead at foot of castle wall.
A
B
C
D
E
F
G
H
I
J
K
L
M
N
O
P
(David Kurtz, via LearningCatalytics)

19. Clicker question
Writing good peer instruction questions19
Select the line that
you feel has the
strongest imagery in
“Fast rode the
knight” by Stephen
Crane (1905).
analysis
Fast rode the knight
With spurs, hot and reeking,
Ever waving an eager sword,
"To save my lady!"
Fast rode the knight,
And leaped from saddle to war.
Men of steel flickered and gleamed
Like riot of silver lights,
And the gold of the knight's good banner
Still waved on a castle wall.
. . . . .
A horse,
Blowing, staggering, bloody thing,
Forgotten at foot of castle wall.
A horse
Dead at foot of castle wall.
A
B
C
D
E
(David Kurtz, via LearningCatalytics)

20. Clicker question
Writing good peer instruction questions20
Which of the following is an incorrect step when using
the substitution method to evaluate the definite integral
A)
B)
 
4
0
32
1 dxxx
3
1 xu 
dxx
du 2
3

C.
D. none of the above

4
03
1
duu
(adapted from Bruff (2009))
evaluation

21. Clicker question
Writing good peer instruction questions21
Evaluate:
A)
B)
(adapted from Bruff (2009))
 
4
0
32
1 dxxx
23
)65(16
9
16
C.
D.
)165(
9
2 23

3
1022
clarity
context
learning goals
distractors
difficulty
discussion
exercise skill

22. Clicker question
Writing good peer instruction questions22
Susan throws a ball straight up into the air. It goes up
and then falls back into her hand 2 seconds later.
Draw a graph showing the velocity of the ball from the
moment it leaves her hand until she catches it again.
time
velocity
2 sec0
exercise skill
(CWSEI UBC)

23. time
velocity
2 sec0
A
time
velocity
2 sec0
B
time
velocity
2 sec0
C
time
velocity
2 sec0
D
E) some other graph
Which one is the closest match to your graph? exercise skill
(CWSEI UBC)Writing good peer instruction questions23

24. t h e l e a r n i n g c y c l e
Peer instruction helps students learn...
Writing good peer instruction questions24
BEFORE DURING AFTER
setting up
instruction
developing
knowledge
assessing
learning
Adapted from Rosie Piller, Ian Beatty, Stephanie Chasteen
Students have had
opportunities to
try, fail, receive feedback
and try again without facing
a summative evaluation. [3]

25. t h e l e a r n i n g c y c l e
Peer instruction helps students learn...
Writing good peer instruction questions25
BEFORE DURING AFTER
setting up
instruction
developing
knowledge
assessing
learning
Adapted from Rosie Piller, Ian Beatty, Stephanie Chasteen

26. Writing good peer instruction questions26
Clicker question
Are features X and Y
ridges or valleys?
A) X=ridge, Y=valley
B) X=valley, Y=ridge
C) both are ridges
D) both are valleys
X
Y
(EOSC / CWSEI, UBC)
demonstrate success

27. Clicker question
Writing good peer instruction questions27
For the data given below, which is larger, the mean or
the median?
74, 32, 35, 87, 28, 36, 11, 26, 93, 56, 34, 52, 8
A) mean
B) median
(Peck, mathquest.carroll.edu/resources.html)
review / recap
clarity
context
learning goals
distractors
difficulty
discussion

28. Clicker question
Writing good peer instruction questions28
For the data set displayed in the following histogram,
which would be larger, the mean or the median?
A) mean
B) median
C) can’t tell from the given histogram
(Peck, mathquest.carroll.edu/resources.html)
review / recap

29. Clicker question
Writing good peer instruction questions29
In your opinion, which had the most positive impact on
the modern world?
A) coffee
B) tea
C) chocolate
D) spice
E) sugar
“big picture”
(Herbst, UCSD)

30. Your turn…
Writing good peer instruction questions30

31. Your turn…
Writing good peer instruction questions31
Big Idea/Concept/Skill/
Learning Outcome
Why do you need a peer
instruction question here in
the lesson?

32. Your turn…
Writing good peer instruction questions32
Question: (and choices)
Think about
clarity context
learning outcome
distractors difficulty
discussion

33. Your turn…
Writing good peer instruction questions33
What should students say
to explain why this choice
is correct/incorrect?
It’s not just about correct
or incorrect.
Direct the conversation!

34. Your turn…
Writing good peer instruction questions34
Are there really 5
meaningful conversations?
(Are there even 4?)

35. Your turn…
Writing good peer instruction questions35

36. Peer instruction helps teachers teach
Writing good peer instruction questions36
BEFORE DURING AFTER
setting up
instruction
developing
knowledge
assessing
learning
t h e l e a r n i n g c y c l e

37. t h e l e a r n i n g c y c l e
Peer instruction helps teachers teach
Writing good peer instruction questions37
BEFORE DURING AFTER
setting up
instruction
developing
knowledge
assessing
learning
Do they care about this?
Are they ready for the next topic?
What DO they care about, anyway?
What do they already know?

38. t h e l e a r n i n g c y c l eDid they notice key idea X?
Where are they in the activity?
Peer instruction helps teachers teach
Writing good peer instruction questions38
BEFORE DURING AFTER
setting up
instruction
developing
knowledge
assessing
learning
Are they getting it?
Do I need to intervene?

39. t h e l e a r n i n g c y c l eHow did I do?
Did they get it?
Peer instruction helps teachers teach
Writing good peer instruction questions39
BEFORE DURING AFTER
setting up
instruction
developing
knowledge
assessing
learning
Can I move to the next topic?
Did that activity work?

40. Writing good PI questions
How (you can help) People Learn (using peer instruction)40
It’s critical to have
 content knowledge (the concepts)
 pedagogical content knowledge (how people learn
the concepts in your discipline and how to teach them)

41. Running effective PI
How (you can help) People Learn (using peer instruction)41
It’s critical to
 teach the students how to engage in peer instruction
 choreograph each episode so students waste no
precious cognitive load wondering what to do
(call us for another workshop!)
You might not write the perfect question the first time so
 listen to the students’ conversations
 write your self some notes immediately after class
 revise and try it again next year

42. References
Writing good peer instruction questions42
1. National Research Council (2000). How People Learn: Brain,
Mind, Experience, and School: Expanded Edition. J.D.
Bransford, A.L Brown & R.R. Cocking (Eds.),Washington, DC:
The National Academies Press.
2. Bruff, D. (2009). Teaching with Classroom Response Systems.
San Francisco: Jossey-Bass.
3. Bain, K. (2004). What the best college teachers do.
Cambridge, MA: Harvard University Press.

43. Writing good peer instruction questions
43
(Image:stoolIIbytilanesevenonflickrCC)
constructivist
Peer Instruction
Writing Good
Questions
Peter Newbury, Ph.D.
Center for Teaching Development,
University of California, San Diego
pnewbury@ucsd.edu
@polarisdotca
ctd.ucsd.edu
September 11, 2013 CSULA Unless otherwise noted, content is licensed under
a Creative Commons Attribution-NonCommericial 3.0 License.
Slides and resources: tinyurl.com/PI-CSULA

44. What makes a good clicker question?
Writing good peer instruction questions
Peter Newbury, Center for Teaching Development, UCSD
clarity Students should waste no effort trying to figure
out what’s being asked.
context Is this topic currently being covered
in class?
connection to
learning goals
Does the question make students do the right
thing to demonstrate they grasp the concept.
distractors What do the “wrong” answers tell you about
students’ thinking?
difficulty Is the question too trivial? too hard?
stimulates
thoughtful
discussion
Will the question engage the students and
spark thoughtful discussions?
Is there potential for you to be “agile”?
(Adapted from Stephanie Chasteen, CU Boulder)