SGTS Theory Stream, Week 3 May 9, 2013 Learning Outcomes Assessment that supports learning Peter Newbury collegeclassroom.ucsd.edu

1. Summer Graduate Teaching Scholars
May 9, 2013
Theory 3:
Learning Outcomes and Assessment

2. Scholarly approach to teaching:
SGTS Theory Stream - collegeclassroom.ucsd.edu
2
Carl Wieman
Science Education Initiative
What should
students
learn?
What are
students
learning?
What instructional
approaches
help students
learn?
learning
outcomes
(goals, objectives)
assessment

3. (Image: NASA)
SGTS Theory Stream - collegeclassroom.ucsd.edu
3

4. Introductory “Astro 101”
This course covers Chapters
1. Mercury
2. Venus
…
8. Neptune
9. other objects
10. Formation of the Solar
System
 deduce from patterns in the
properties of the planets,
moons, asteroids and other
bodies that the Solar
System had single
formation event.
 provide notable examples
of how comets influenced
history, art and science
4
SGTS Theory Stream - collegeclassroom.ucsd.edu
Traditional Course Syllabus Course with Learning Outcomes

5. Learning outcomes
SGTS Theory Stream - collegeclassroom.ucsd.edu
5
 completes the sentence, “By this end of this
lesson/unit/course, you will be able to…”
 begins with an action verb (more below…)
 clarifies what it means to “understand” the concept
 deduce from patterns in the
properties of the planets,
moons, asteroids and other
bodies that the Solar
System had single
formation event.

6. What is the Value of Course-Specific
Learning Goals
SGTS Theory Stream - collegeclassroom.ucsd.edu
6
What was the most interesting thing you learned from
Simon & Taylor? [1]
 this
Outcomes

7. SGTS Theory Stream - collegeclassroom.ucsd.edu
7
Simon & Taylor (2009)

8. Learning outcomes are valuable to the
instructor, too.
SGTS Theory Stream - collegeclassroom.ucsd.edu
8
 crystallizes what prof actually cares about
 helped prof
 choose clicker questions for class
 write the final exam

9. …choose clicker questions for class
SGTS Theory Stream - collegeclassroom.ucsd.edu
9
ClassAction http://astro.unl.edu/classaction/

10. …write the final exam
SGTS Theory Stream - collegeclassroom.ucsd.edu
10
(10 marks) List 3 patterns of the Solar System as a
whole. Then, outline in some detail the current model for
the formation of the Solar System. In particular, make
sure you explain how the observed patterns and
regularities are related to this theory of formation.

11. Writing LOs – The Easy Way
SGTS Theory Stream - collegeclassroom.ucsd.edu
11
Astro 101 Learning Outcomes
 learn about Saturn
Assess your LOs:
 “learn”? Learn what?
 how does a student demonstrate to you s/he has
“learned about Saturn?”
 how does a student check that s/he has mastered the
Saturn part of the course?

12. Writing LOs – More Effective Way
SGTS Theory Stream - collegeclassroom.ucsd.edu
12
Astro 101 Learning Outcomes
 describe in detail the size and structure of Saturn’s
rings
 step through the gravitational feedback cycle that
keeps Saturn’s rings so thin
Assess your LOs:
1. Is this really what I want them to know?
2. Are they capable of that?
3. How do I assess their ability to demonstrate this LO?

13. Deciding on the level of a LO
SGTS Theory Stream - collegeclassroom.ucsd.edu
13
Writing learning outcomes is hard because you have to
 recognize
 declare
 (admit)
what you want your students to be capable of doing.
A good start is picking the verb describing the action
the students will perform to demonstrate their mastery
of the concept:

14. Bloom’s Taxonomy [2,3]
SGTS Theory Stream - collegeclassroom.ucsd.edu
14
Remember
Understand
Apply
Analyze
Evaluate
Create
think critically about and defend a position
transform or combine ideas to create
something new
break down concepts into parts
apply comprehension to unfamiliar situations
demonstrate understanding of ideas and
concepts
remember and recall factual information

15. Bloom’s Taxonomy [2,3]
SGTS Theory Stream - collegeclassroom.ucsd.edu
15
Remember
Understand
Apply
Analyze
Evaluate
Create
higher order thinking
lower order thinking

16. Bloom’s Taxonomy [2,3]
SGTS Theory Stream - collegeclassroom.ucsd.edu
16
Remember
Understand
Apply
Analyze
Evaluate
Create
develop, create, propose, formulate,
design, invent
judge, appraise, recommend, justify,
defend, criticize, evaluate
compare, contrast, categorize, distinguish,
identify, infer
apply, demonstrate, use, compute, solve,
predict, construct, modify
describe, explain, summarize, interpret,
illustrate
define, list, state, label, name, describe

17. Driver’s Ed 101: How to Drive in CA
SGTS Theory Stream - collegeclassroom.ucsd.edu
17
The whiteboards on the tables are numbered. Your
group will concentrate on the question on the California
DMV Test [4] matching your board’s number.
Task: Write a learning outcome
that your group’s question assesses.
(Refer to Wieman handout
for Bloom’s levels and corresponding verbs)

18. Revised Bloom’s Taxonomy [2]
18
SGTS Theory Stream - collegeclassroom.ucsd.edu

19. Vocabulary check: assessment
is that which gives a final
judgment of evaluation of
proficiency, such as grades or
scores.
(How Learning Works, p. 139)
explicitly communicates to
students about some specific
aspects of their performance
relative to specific target
criteria, and … provides
information that helps students
progress toward meeting those
criteria…[It] informs students’
subsequent learning.
(How Learning Works, p. 139)
19
SGTS Theory Stream - collegeclassroom.ucsd.edu
summative assessment formative assessment

20. Feedback and Practice that Enhance
Learning (How Learning Works [5])20
Writing – public policy course
Presentations on research – medical anthropology
SGTS Theory Stream - collegeclassroom.ucsd.edu
Instructors’ expertise
and bias not clear to
students (or themselves)

21. Feedback and Practice that Enhance
Learning (How Learning Works [5])21
Solution: Goal-directed practice coupled with targeted
feedback are critical to learning.
SGTS Theory Stream - collegeclassroom.ucsd.edu
Images:
Excellent Shot by Varsity Life on flickr CC
Music by Piulet on flickr CC

22. Feedback and Practice that Enhance
Learning (How Learning Works [5])22
Solution: Goal-directed practice coupled with targeted
feedback are critical to learning.
[G]oals can direct the nature of focused practice, provide
the basis for evaluating observed performance, and shape
the targeted feedback that guides students’ future efforts.
(p. 127)
[T]argeted feedback gives students prioritized information
about how their performance does or does not meet the
criteria so they can understand how to improve their future
performance.
(p. 141)SGTS Theory Stream - collegeclassroom.ucsd.edu

23. Scenarios
SGTS Theory Stream - collegeclassroom.ucsd.edu
23
feedback at
appropriate level
feedback not at
appropriate level
productive practice unproductive practice
practice is goal-directed practice not goal-directed
timely feedback untimely feedback
Find the person with the same
colored sheet as you. Fill out
the sheet together.

24. Feedback at Appropriate Level Feedback not at Appropriate Levelsport/hobby________________education__________________

25. Productive Practice Unproductive Practicesport/hobby________________education__________________

26. Practice Goal-directed Practice not Goal-directedsport/hobby________________education__________________

27. Timely Feedback Untimely Feedbacksport/hobby________________education__________________

28. Instructional Scaffolding
28
 Needs to be given BEFORE and BUILT INTO assignment
 Outlines what it takes to improve
 Supports Zone of Proximal Development [8]
SGTS Theory Stream - collegeclassroom.ucsd.edu

29. Appropriate Level of Challenge
29
Glued to Games [6]: Psychological
exploration (and comparison with
educational practices) of characteristics
of games that drive people to spend time
and succeed.
James Paul Gee [7] “What video games
have to teach us about learning and
literacy”
SGTS Theory Stream - collegeclassroom.ucsd.edu
angrybirds.com

30. SGTS Theory Stream - collegeclassroom.ucsd.edu
30

31. Clicker question
31
Does this rubric foster a
A) fixed/performance mindset
B) growth/mastery mindset
C) neither
D) both
SGTS Theory Stream - collegeclassroom.ucsd.edu

32. SGTS Theory Stream - collegeclassroom.ucsd.edu
Robert Talbert
tinyurl.com/RobertTalbertRubric

33. Instructional Scaffolding
33
 Rubrics
 support growth mindsets
 path to improvement
 goal-directed
[G]oals can direct the nature of focused practice, provide
the basis for evaluating observed performance, and shape
the targeted feedback that guides students’ future efforts.
 targeted feedback
[T]argeted feedback gives students prioritized information
about how their performance does or does not meet the
criteria so they can understand how to improve their future
performance.
SGTS Theory Stream - collegeclassroom.ucsd.edu

34. Assessment Strategies…
SGTS Theory Stream - collegeclassroom.ucsd.edu
34
 addressing the need for goal-directed practice
 addressing the need for targeted feedback
Work on the hand-out,
thinking about what you’ve
experienced or what you
aspire to do in your field.

35. Executive Summary:
SGTS Theory Stream - collegeclassroom.ucsd.edu
35
Plan your course
by synchronizing and
aligning your learning
outcomes, activities and
assessments.
What should
students
learn?
What are
students
learning?
What instructional
approaches
help students
learn?

36. References
SGTS Theory Stream - collegeclassroom.ucsd.edu
36
1. Simon, B., & Taylor, J. (2009). What is the Value of Course-Specific Learning
Goals? Journal of College Science Teaching, 39, 2, 52-57. PDF available at
www.cwsei.ubc.ca/SEI_research/files/LifeSci/Simon_Taylor_ValueOfCourseSpecif
icLG.pdf
2. Bloom B. S. (1956). Taxonomy of Educational Objectives, Handbook I: The
Cognitive Domain. New York: David McKay Co Inc.
Adapted from Anderson, L. W., & Krathwohl, D. R. (2001). A Taxonomy for
Learning. Teaching, and assessing: A revision of bloom's taxonomy of educational
objectives.
Revised Bloom’s Taxonomy www.celt.iastate.edu/teaching/RevisedBlooms1.html
3. Excerpt from Wieman, C. (2007). Slides from the Wieman Learning Goals
Workshop. www.cwsei.ubc.ca/resources/learn_goals.htm
4. California DMV Sample Class C Written Test 5
www.dmv.ca.gov/pubs/interactive/tdrive/clc6written.htm

37. References, continued
SGTS Theory Stream - collegeclassroom.ucsd.edu
37
5. Ambrose, S.A., Bridges, M.W., DiPietro, M., Lovett, M.C., & Norman, M.K. (2010).
How Learning Works. San Fransisco: Jossey-Bass.
6. Glued to Games: How Video Games Draw Us In and Hold Us Spellbound
http://www.gluedtogames.com/
7. Gee, J.P. (2005). Learning by Design: good video games as learning machines. E-
Learning 2, 1, 5-16.
8. Wertsch, J.V. (1984). The zone of proximal development: Some conceptual issues.
New Directions for Child and Adolescent Development, 1984, 23, 7–18.

38. Bloom’s Taxonomy of the Cognitive Domain
(Levels of Learning) [Wieman, 2007]
38
6. Synthesis: transform and combine ideas to create something new
develop, create, propose, formulate, design, invent
5. Evaluation: think critically about and defend a position
judge, appraise, recommend, justify, defend, criticize, evaluate
4. Analysis: break down concepts into parts
compare, contrast, categorize, distinguish, identify, infer
3. Application: apply comprehension to unfamiliar situations
apply, demonstrate, use, compute, solve, predict, construct, modify
2. Comprehension: demonstrate understanding of ideas, concepts
describe, explain, summarize, interpret, illustrate
1. Factual Knowledge: remember and recall factual knowledge
define, list, state, label, name, describe
SGTS Theory Stream - collegeclassroom.ucsd.edu