1. Prepare for a talk
about teaching and
learning …
… by a physicist.
Liberal-arts majors may be annoying sometimes,
but there’s nothing more obnoxious than a
physicists first encountering a new subject.

2. Just in Time Teaching
A 21st Century Teaching Technique
Jeff Loats
Metropolitan State University of Denver
Department of Physics
“Learning technologies should be designed to increase,
and not to reduce, the amount of personal contact
between students and faculty on intellectual issues.”
Study Group on the Conditions of Excellence
in American Higher Education, 1984
LMU – Sept. 2012

3. The Evidence Standard
3
Teachers can feel a bit bombarded… hybrid courses,
brain-based learning, technology in the classroom,
learner-centered teaching, etc.
For me, striving to be a scholarly teacher means
following the evidence, regardless of the source.
Some common themes, supported by evidence:
• Focus and attention
• Using emotions appropriately
• Repetition and practice
• Feedback

4. Narrowing Our Focus
4
• Focus and attention
–No such thing as multitasking, etc.
• Using emotions appropriately
–A little anxiety is good, a bit more is bad, etc.
• Repetition and practice
• Feedback
Just in Time Teaching
has something unique to offer here

5. 5
In (roughly) what area do you teach?
A) Humanities
B) Natural sciences & mathematics
C) Professions & applied sciences
D) Social sciences
E) Teacher education
(there is no surer way for me to offend a good chunk of you)

6. 6
In your teaching do you have a method for holding
students accountable for preparing for class?
A) I don’t, but I ask/threaten really well.
B) I use a paper method (quiz, journal, others?)
C) I use a digital method (clickers, others?)
D) I use Just in Time Teaching.
E) I have some other method.

7. Overview 7
Talk:
1. Motivation for change
2. Basics of Just in Time Teaching
3. Mock example
4. Evidence for effectiveness
5. Summaries
Workshop:
• Try JiTT from both sides (students/instructors)
• Feedback from students
• Roadblocks and bottlenecks

8. Feedback That Works
8
“Improvement of performance is actually a function of
two perceptual processes. The individual’s perception
of the standards of performance, and her/his
perception of his/her own performance.”
The Feedback Fallacy – Steve Falkenberg
(via Linda Nilson)

9. 9
The Physics Education Revolution
• About ~20 years ago, physics teachers began
treating education as a research topic!
• Their findings were grim:
From Mazur, New Faculty
Workshop presentation, 2004
• Eric Mazur
(at Harvard!)
found this:
"But the students do
fine on my exams!"

10. How Do Others Do?
10
Conclusion: Traditional physics lectures are all similarly
(in)effective for improving conceptual understanding.

11. 11
Enter Physics Education Research (PER):
An effort to find empirically tested ways to
improve the situation.
Many top university physics departments now
have a PER research group.
University of Illinois
at Urbana-Champaign
University of Colorado
University of Washington

12. 12
Technique & Technology
Technique:
Just in Time Teaching (JiTT)
Technology:
Web based question & response tools

13. Just in Time Teaching
13
“Just-in-Time Teaching (JiTT for short) is a teaching
and learning strategy based on the interaction between
web-based study assignments and an active learner
classroom.” ~ Gregor Novak, Co-Author of the JiTT book
• Online assignments (“WarmUps”), completed
before class to promote preparation and thought.
• Responses are read “just in time”.
• Instructor modifies that day’s plan accordingly.
• Aggregate and individual (anonymous) responses
are displayed in class.

14. What JiTT is Not
14
JiTT techniques rely heavily on web-based tools.
JiTT is not about
… online courses or distance learning.
… computer-graded homework.
… delivering content via the web.
The goals of JiTT:
• Student preparation.
• Obvious communication loop.
• Improve student ownership and buy-in of class.
• Establish a community effort towards learning.

15. 15
For an average day in your class, what fraction of your
students do their preparatory work before your class.
A) 0% - 20%
B) 20% - 40%
C) 40% - 60%
D) 60% - 80%
E) 80% - 100%

16. Questions – Practice & Repetition 16
• WarmUp questions about (new) material:
– Every-day language.
– Some simple comprehension questions .
– Mostly higher level questions (a la Bloom).
– Perhaps any question is better than none.
• Connections to evidence:
– Pre-class work reduces the working memory load
during class, a factor experts easily neglect.
– Multimodal practice (not learning styles):
Reading, writing and discussion are modes of
practice that JiTT brings to the classroom daily.

17. Questions – Deliberate Practice 17
Two questions end every WarmUp:
“What aspect of the material did you find the most
difficult or interesting. Give this a bit of thought and
be specific in your answer.”
“How much time did you spend on the pre-class work
for tomorrow?”
[Results: A pretty steady average of ~40 minutes
across many courses/levels/cohorts.]
• Connections to evidence:
– Forced practice at metacognition:
Students regularly evaluate their own interaction
with the material.

18. Closing The Loop 18
• Student responses are:
– Graded on thoughtful effort that demonstrates
they did the reading.
– Sampled and categorized by the instructor in
some way to create a class response profile.
– Quoted anonymously in class to demonstrate a
variety of aspects: Misconceptions, good efforts,
superior explanations, metacognition, etc.
• Closing the loop:
– Instructor responds to some students digitally.
– Class time emphasizes interactive engagement.

19. Effective Feedback 19
Recall Faulkenberg’s criteria for feedback:
– Feedback doesn’t work if students don’t correctly
perceive the performance standards.
– Feedback doesn’t work if students cannot
correctly evaluate their own performance.
• JiTT offers chances to clarify standards in low-
stakes situations. Allows us to show model
responses that are not teacher-generated.
• Closing the loop allows students to judge whether
they have correctly evaluated their own
performance.

20. JiTT Web Tools
20
More in workshop
• JiTT can be done using many different online tools:
– Course management systems (Blackboard).
(Easy to set up and tools are good, but not perfect)
– Free service from JiTTDL.org.
(Designed just for JiTT but extra login, and the
site has not been improved in ~4 years)
– Email (No setup, very direct, but tends to be
overwhelming and discouraging)
– Blogging tools (WordPress).
(Very possible for a tech-savvy user, but you
would need to have a “hackerish” mentality).

21. 21
Here is an example WarmUp question of mine.
Answer with your clicker for this simulation.
(A full demonstration is in the workshop following).
A bucket of water can be whirled in a vertical circle
without the water falling out, even at the top of the
circle when the bucket is upside down. Explain why.
A) Nothing holds it in
B) An outward force holds the water in the bucket
C) The water’s acceleration and velocity are in
different directions
D) An inward force holds the water in the bucket

22. Example From My Class
22
A bucket of water can be whirled in a vertical circle
without the water falling out, even at the top of the
circle when the bucket is upside down. Explain…
~15% → An outward force holds it in
~30% → An inward force holds it in
~20% → Talked (correctly!) about acceleration &
velocity… but didn't really answer.
~10% → Nailed it! (or close enough)

23. Class Response: Water Bucket
23
“The water doesn't come out because you twirling the
bucket is applying the force of spinning, and the water
just kind of counteracts that motion.”
“Because the water naturally wants to keep traveling
in the same direction its being whirled around in the
water attempts to continue going up in a straight line
but the bottom of the bucket forces it to stay in the
bucket, like when you are pushed by the door of a car
while making a turn.”

24. Does It Work?
24
Mazur: After 1 year using active engagement methods:
• Big jump in
conceptual
understanding.
• Is this just new
energy into an
old course?

25. Outside Harvard
25
• Mazur’s methods
were applied
all over the country,
with impressive
results:
• A conceptually focused, learner-centered class can
create dramatically improved conceptual gains.
• The evidence indicates that problem solving skills
are improved when time is devoted to concepts.

26. Effectiveness of JiTT
26
• Back in 2004 JiTT was used by hundreds of faculty,
in more than 25 disciplines at over 100 institutions.
• Dozens of publications in different fields indicate
concrete differences in JiTT driven classes:
– Improved student preparation for class
– Improved use of out-of-class time and…
– Increased attendance & engagement in class
– Improvement in affective measures
– Increase in content knowledge

27. Does It Work In Biology?
27
• Marrs, K., 2005,
Assessment of JiTT on Student Learning
In several biology courses the author documents:
Improved study habits & preparation before class
From “References for Just in Time Teaching”:
http://serc.carleton.edu/sp/pkal/justintime/references.html#pedagogy

28. Does It Work In Biology?
28
• Marrs, K., 2005,
Assessment of JiTT on Student Learning
‘Crammed’ in ‘Crammed’ in
Biology N100 other courses
A students 16% 44%
B students 34% 63%
C students 41% 65%
D students 64% 71%
F students 68% 69%
From “References for Just in Time Teaching”:
http://serc.carleton.edu/sp/pkal/justintime/references.html#pedagogy

29. Does It Work In Biology?
29
• Marrs, K., 2005,
Assessment of JiTT on Student Learning
In several biology courses the author documents:
Improved study habits & preparation before class
Increased retention rates (DFW from 29% to 21%)
Increased cognitive gains on conceptual knowledge
… AND increased content knowledge!
From “References for Just in Time Teaching”:
http://serc.carleton.edu/sp/pkal/justintime/references.html#pedagogy

30. Does It Work In Art History?
30
• Cookman, 2009
Using JiTT to Foster Active Learning
in a Humanities Course
This chapter (available online) describes Thinking
About Reading questions (TARs). His methods,
student reactions and lessons learned are all
included.
The course is History of 20th Century Photography.
Part of “Just in Time Teaching Across the Disciplines and Across the Academy” Book
Google this title.

31. Does It Work In Art History?
31
• Cookman, 2009
Using JiTT to Foster Active Learning
in a Humanities Course
Sample:
“Summarize in your own words Knightley’s
argument that Capa staged this photograph.
Summarize in your own words Whelan’s argument
that the photograph is authentic. Whose argument do
you find more convincing? Why?”
Part of “Just in Time Teaching Across the Disciplines and Across the Academy” Book
Google this title.

32. What Might Stop You
32
• In terms of the technique:
• Time, coverage, not doing your part,
student/colleague pushback.
• In terms of the technology:
• Learning curve, technology failures, perfectionism
• In generally reforming your teaching:
• Inventing the wheel, not finding support, doing too
much at once.

33. My Summary
33
• In terms of content and changes to the classroom,
Just in Time Teaching may be among the easiest
research-based instructional strategies that you can
consistently integrate into your teaching.
• From an evidence-based perspective, JiTT addresses
areas that are often neglected by traditional
techniques.
• Student report that WarmUps make them better
prepared for class and help them learn the material.
• As with all reforms, we should be prepared to find
that students know less than we might hope.

34. Your Summary!
34
For yourself or to share with me (if you’re willing):
What part of JiTT concept/process is the fuzziest for
you after this talk?
What is the biggest reason you might not give JiTT a
try in one course next term?

35. 35
Works Cited
• Falkenberg, S. (1996). “The Fedback Fallacy”. Retrieved March, 2012
from http://people.eku.edu/falkenbergs/feedback.htm
• Nilson, L. (20011) ”The Mind Has a Mind of Its Own”, teleseminar
recorded by Emphasis on Excellence, Jun 2011
• Mazur, E. 2004 ”Introduction to Peer Instruction” talk presented at
New Physics & Astronomy Faculty Workshop, 2004, UMD.
• Hake, R.R. 1998a. “Interactive-engagement vs traditional methods: A
six thousand-student survey of mechanics test data for introductory
physics courses,” Am. J. Phys. 66(1): 64-74; online at
<http://www.physics.indiana.edu/~sdi/ajpv3i.pdf> (84 kB).
Jeff Loats
Jeff.Loats@gmail.com
303-900-2175
Take a card and visit slideshare.net/jeffloats