How does a teacher use questioning effectively? This workshop will focus on writing those questions that engage students, spark their curiosity, help recap material, give you insight into their thinking, or help them learn critical ideas in your discipline. We will focus on the use of clickers with "peer instruction" -- a research-tested method of requiring students to discuss challenging questions with one another. We will discuss how clickers can help facilitate this teaching strategy, investigate the surprising power of multiple-choice questions to achieve critical thinking skills, plus spend time discussing the elements of effective questions and practicing writing and improving questions for our classes.

1. Make Clickers Work for You
WRITING GREAT CLICKER
QUESTIONS
Dr. Stephanie V. Chasteen
Physics Department
&
Science Education Initiative
Univ. of Colorado at Boulder
http://colorado.edu/sei
Web and blog: http://sciencegeekgirl.com
Email: stephanie.chasteen@colorado.edu

2. What do you teach?
Show of hands
A. Science
B. Engineering or Math
C. Social sciences
D. Humanities
E. Administration / faculty support
F. Other

3. Have you used response systems (clickers) in
your teaching?
Take a clicker & turn it on
If the green light flashes, your
vote has been counted
A. Not at all, and I haven’t seen them used
B. Not at all, but I’ve observed their use somewhat
C. I’ve used them a little
D. I’ve used them a lot
E. I could be (should be?) giving this workshop

4. How familiar are you with Mazur’s “Peer
Instruction”
Colored cards
A. Fairly familiar, and I like it
B. Fairly familiar, but I’m not sure that I like it
C. I’ve heard of it but only have a vague idea what it is
D. Not familiar at all
E. Not sure

5. Introducing Me
5
Science Education Initiative
http://colorado.edu/SEI
Applying scientific principles to improve science
education – What are students learning, and which
instructional approaches improve learning?
Physics Education Research Group
http://PER.colorado.edu
One of largest PER groups in nation, studying technology,
attitudes, classroom practice, & institutional change.
Blogger
http://blog.sciencegeekgirl.com

7. Why question?
8
 How many times have you given a lecture and found
that students hadn’t followed you?
 Can you rely on students to ask questions if they
don’t understand something?
 Can you rely on students to know if they don’t
understand something?
 What are the benefits of questioning?
Credit: Rosie Piller whiteboard

9. Agenda
10
1. When and how we can ask questions
2. About clickers as a way to ask questions,
including some common challenges
3. Writing good questions. Example questions,
writing our own.
4. Action plan
Learning goals: Participants will be able to….
A. Explain several benefits of questioning and of using clickers to question
B. Defend the use of best practices in questioning to overcome common
challenges
C. Formulate and revise clicker questions to target student learning goals

10. Exercise #1: Question brainstorm
11
 What questions could you ask to help students
achieve your assigned learning goal -- to test mastery
and stimulate learning?
 Brainstorm as a group
5 minutes
whiteboard

11. When can we ask questions?
12
BEFORE
Setting up instruction DURING
Motivate Developing
knowledge
Discover
Predict outcome Check knowledge
Provoke thinking Application
Assess prior knowledge Analysis
Evaluation
Synthesis
AFTER Relate to big picture Exercise skill
Assessing Demonstrate success Elicit misconception
learning
Review or recap
Exit poll
Credit: Rosie Piller and Ian Beatty.

12. Some methods of asking questions
13
 Ask rhetorically
 Target the class (how?)
 Target someone in particular (in what order?)
 Wait and then… (call on whom?)
 Answer your own question
 Leave the question unanswered
Or ask out of class
 Blogs
 Discussion boards
 Homework…
Credit: Rosie Piller

13. Why use clickers to target the class?
An outline of Peer Instruction.
14

14. Anatomy of Peer Instruction
15
Ask Question
…Lecture… (Maybe vote)
Class Discussion Peer Discussion
Vote
* See also: Peer Instruction, A User’s Manual. E. Mazur.

15. Note: Grading for Formative Assessment
Motivate students to participate, without stressing over the right
answer
We recommend extra credit for:
•Mostly participation (eg., 2 points)
•Some for correctness (eg., 1 point)
A new research study (James & Willoughby, 2011) shows:
Giving points for correctness creates less productive classroom
conversations! See http://theactiveclass.com
16

16. Note: Timing / Groups
17
◦ 2-5 questions spaced
through an hour
◦ Discussion with peers
(usually nearest neighbors)

17. Questions about this process?
18
Ask Question
…Lecture… (Maybe vote)
Class Discussion Peer Discussion
Vote
* See also: Peer Instruction, A User’s Manual. E. Mazur.

18. Clickers are a tool for questioning
19
But not a magic bullet!

20. Peer instruction helps students learn
21
Research shows that:
 Students can better answer a similar question after
talking to their peers
 Peer discussion + instructor explanation works
better than either one alone
Do you want to see the
 Students like peer instruction, from intro to the
details of some of this
junior level research?
A. Yes, cut back on other stuff
 Students in courses using peer instruction to the
B. No, let’s just get
outperform those in traditional lecturewriting the Q&A a
question
courses on
C. I’d like to see it in
common test portion afterwards
See http://STEMclickers.colorado.edu for various references

21. How is a clicker question the same or different?*
26
* From other types of in-class questions
 Similar in terms of goals
 Multiple choice
 Anonymous (to peers)
 Every student has a voice – the
loud ones and the shy ones
 Forced wait time
 You can withhold the answer
until everyone has had time to
think (choose when to show the
histogram)
What does this tool help
us to do?

22. U. Colorado clicker resources…
27
Videos of effective use of clickers 2-5 mins long
http://STEMvideos.colorado.edu
Clicker resource page
http://STEMclickers.colorado.edu
• Instructor’s Guide
• Question banks
• Workshops
• Literature / Articles

23. Which of these could be clicker questions?
28
BEFORE
Setting up instruction DURING
Motivate Developing
knowledge
Discover
Predict outcome Check knowledge
Provoke thinking Application
Assess prior knowledge Analysis
Evaluation
Synthesis
AFTER Relate to big picture Exercise skill
Assessing Demonstrate success Elicit misconception
learning
Review or recap
Exit poll
Credit: Rosie Piller and Ian Beatty.

24. Let’s try it.
I think that the toughest thing about using clickers and
peer instruction must be:
A. Writing good questions
B. Getting students to engage with the questions
C. Getting students to share their answers with the
whole class / the same students always share
D. It takes too long for me to learn to do this
E. I have a lot of content to cover, it takes too much
class time

25. A science-related example…
30
Which superpower would you
rather have? The ability to…
A. Change the mass of things
B. Change the charge of things
C. Change the magnetization of things
D. Change the boiling point of things
30
Question: Ian Beatty, UNC Greensboro Image: Thibaultfr on Wikimedia

26. Example question: Math
Your sister in law calls to say that she’s having twins.
Which of the following is the most likely? (Assume she’s
having fraternal, not identical, twins)
A. Twin boys
B. Twin girls
C. One girl and one boy
D. All are equally likely
31
Derek Bruff, Vanderbilt

27. Example Question: Survey
32
Which of the following are you least comfortable using to
solve problems?
A. Kinematics
B. Newton’s Laws
C. Work-Energy Theorem
D. Momentum-Impulse Theorem
E. Angular Momentum-Angular Impulse Theorem
Ian Beatty, UMass Amherst

28. Two things to pay attention to in your questions
 What is the goal of my question? What am I trying
to accomplish?
 Is my question at the right level / variety of depth?

29. Question goals
34
BEFORE
Setting up instruction DURING
Motivate Developing
knowledge
Discover
Predict outcome Check knowledge
Provoke thinking Application
Assess prior knowledge Analysis
Evaluation
Synthesis
AFTER Relate to big picture Exercise skill
Assessing Demonstrate success Elicit misconception
learning
Review or recap
Exit poll
Credit: Rosie Piller and Ian Beatty.

30. Question Writing Depth
Very useful 91%
N=4
courses, 35%
66 Useful 36%
students 18%
Somewhat
useful
Types of clicker questions:
Mostly Challenging conceptual
useless Recalling a previous fact
Completely useless Recalling a recent fact
Plugging numbers into equation
% of students 0% 10% 20% 30% 40% 50% 60%
COLTT CU 2009

31. But how do we increase the depth of questioning?
36

32. Question Writing Depth: Bloom’s Taxonomy
37
Handout:
 Handout with handy verbs and Bloom’s Taxonomy
question stems for different
levels, e.g.:
 UNDERSTAND: match,
paraphrase, restate
 APPLY: choose, explain, show
 ANALYZE: compare, classify,
categorize
 EVALUATE: judge, criticize,
defend
 SYNTHESIS: combine, develop,
design

33. Preparing to Write Questions
Read briefly over the “tips for writing clicker
questions” handout.
Which is going to be most challenging for you?
Which would you tell a colleague about?
3 minutes

34. Preparing to Write Questions
 In groups of 2-3, choose one of the questions that
you brainstormed at the beginning of the workshop.
You will write a multiple choice version of this
question.
3 minutes

35. Gallery Walk
 As a table, look at the “example questions” trio that I
have given you. What’s a common theme(s)?
 Write the themes you find down on the sheet so that
other groups will be able to read it.
After 5 minutes, circulate to see the themes of
questions on other tables.
Shop for ideas for your own questions!
See handouts for a place to jot your notes.
10 minutes

36. Gallery Walk: Report Out
What was the theme of your question trio?
When would you use such a type of question?

37. Exercise #3: Writing Questions
42
 Using ideas you’ve learned, write a multiple choice
version of your question in groups of 2-3.
 Show your question to another group (and to me) for
suggestions on revising it.
 If you have time, write another question from
another part of the questioning cycle.
10 minutes

38. Share-Out about Question Writing
 What was challenging?
 What worked well for
you?
 What questions or
concerns do you have
about writing questions?
 How might you write
questions that integrate
with your lectures?

39. This workshop can’t do it all
44
 There are great books to read
 Pair up with other instructors
 I give free webinars (see iclicker.com)
 Next workshop, 4:30-6:00, Weds Feb 1st. (4:00-4:30
refresher course for new folks)
Making Clickers Work for You: Facilitation
I.e., taking off the rose-colored glasses. What goes
wrong? How can this technique work best?
BRING YOUR HANDOUTS!

40. Action Plan
45
 Take a few minutes to write down your action plan to
implement ideas you heard about in the workshop

41. Thanks!
Resource Page: http://STEMclickers.colorado.edu
Web and blog: http://sciencegeekgirl.com
Email: stephanie@sciencegeekgirl.com
 Many materials in this workshop (particularly the questioning cycle and the participant exercises)
were adapted fromRosie Piller, Making Students Think: The Art of Questioning. Short papers
published in: Computer Training & Support Conference, 1995; ISPI International Conferences, 1991
and 1996; ASTD National Conference on Technical & Skills Training, 1990. Related workshop
description at http://www.educationexperts.net/nstworkshop.html. Other materials (particularly
sample clicker questions and goals of clicker questions) adapted from Ian Beatty’s Technology
Enhanced Formative Assessment (TEFA) program. http://ianbeatty.com/crs
NOW: Q&A, continued work on questions and revision,
individual consultations.
NEXT WEEK, 4:30-6pm – Facilitation Tips & Techniques

42. Learning Goals
 Biology: Recognize the components of a cell and
describe why each is necessary for the function of a cell
 Physics: Identify the different ways that light can
interact with an object (i.e., transmitted, absorbed,
reflected).
 Chemistry: Explain trends in boiling points in terms of
intermolecular interactions
 Earth science: Understand the formation of the three
major types of rocks (igneous, sedimentary, and
metamorphic) and the processes by which they form,
relating them by the rock cycle.
 Math: Solve a system of linear equations in two
variables using algebra or graphing.

43. What Do I do if…?
48
What can you do if you ask questions and..
 There is no response
 The same people keep raising their hands
 The answers are called out before everyone has a
chance to think
We’ll discuss in
 The answers take too long Workshop #2.
For now: Many of
 Someone gives a wrong answer these challenges are
 Only some students are prepared addressed by clickers
?