CAS COP Measurement session slides

1. Finding Coherence
Focus, Measure, & Connect
The Measure Sessions!
November, 2014
http://digitallearningforallnow.com
http://www.slideshare.net/jpcostasr
costa@educationconnection.org
Jonathan P. Costa

2. Student Data
Driving
Instructional
Practices &
Decision
Making
Instruction
Driving
Improved
Measures
Professional
Practice
Driving
Improved
Measures
Systems
Driving
Improved
Measures
Adult Data
Driving
Professional
Learning &
Decision
Making
Organizational
Data
Driving
Systems
Decision
Making

3. Coherence Pathways – 5,000 Foot View Leadership
Domain One
Student Learning
S-G Goals for Student Learning
• Connecticut Core/21st
Century Skills &
Content
A-G Goals for Professional Learning
• Evaluation & support
goals, SLOs, focus
goals & other
O-G Goals for Building and District
• Improvement targets
related to DPI, SPI or
other goals
Focus
Measure
Connect
O-M
Building &
District
Measures
DPI
SPI
5
10
Other…
A-M
Professional
Measures
45
40
(5/10)
S-M
Student
Measures
Smarter
Balanced &
other valued
summative,
formative,
standardized
and non-standardized
measures
To prepare every
student for
learning, life, and
work in the 21st
O-P
Mission
century.
Building &
District Practices
District or
building
level
plans or
strategies
A-P
Professional
Learning
Practices
Job
focused
& aligned
with
45
5
40
10
S-P
Instructional
Practices
CC/21CS
goal
aligned
teaching
methods
&
strategies
Domain Two
Professional Learning
Domain Three
Organizational Learning
Goals
Mission
Leadership
Practices Focus Measures
G = Goals P = Practices M= Measures Jonathan P. Costa S= Students A = Adults O = District/Building

4. Three Principles of Coherence
Measure
what you
value
Value
what you
measure
Priority
Student
Learning
Priority
Adult
Learning
Priority
Systems
Learning
Student
Learning
Adult
Learning
Systems
Learning
Focus Measure Connections

5. Coherence Pathways – Essential Connections
S-G Goals for Student Learning
• What are your most important goals for learning?
S-M
Student Measures
Leadership
Focus
Measure
Connect
How will you know if you
are improving (Measure)?
Using the identified skills, create an
assessment plan that includes; a. existing
standardized data (CAPT/CMT), b. identifies
potential Smarter Balanced/Common Core
assessments, and c. creates local
assessments that can be used for formative
measurement of student performance of
these skills.
Mission
To prepare every
student for
learning, life, and
work in the 21st
century.
Goals
Focus
Practices Measures
G = Goals P = Practices M= Measures Jonathan P. Costa S= Students A = Adults O = District/Building

6. Coherence Is Supported
by Assessment
Jonathan P. Costa
Value
Reliability

7. What Student Data Do You Have and Use?
Skill Set
Critical Attributes Assessments & Data Points
(Current)
Assessments & Data Points
(Needed)
Critical Stance
Based on Evidence
M3/E4
Construct and engage in
viable arguments based
on evidence and critique
reasoning of others.
 SMP 3 – Distinguish correct logic in
reasoning from that which is flawed.
 SMP3 –Build a logical progression of
statements to explore the truth about
conjectures.
 SMP 3 -Use stated assumptions,
definitions, and previously established
results in constructing arguments.
 SMP 3 - Recognize and use counter
examples.
 SMP 3 – Contextualize and
decontextualize quantitative
relationships
 E4 - Question an author’s or speaker’s
assumptions and premise
 21 - Effectively analyze and evaluate
evidence, arguments, claims and
beliefs
 21 - Synthesize and make connections
between information and arguments
 21 - Interpret information and draw
conclusions based on the best analysis

8. Reflections on Assessment…
• To be worth anything, all assessment must do one of these two things….
1. Improve student learning (formative)
2. Create both student and teacher accountability (summative)
• How would you define/characterize these two categories of assessment?
– Formative:
• Why –
• How -
– Summative:
• Why –
• How -

9. Reflections on Assessment…
- Finding the Balance Point -
- Given what we have discussed about the purpose and process of
formative and summative assessments… what do you think is the most
appropriate and productive % time split between these two assessment
purposes (of the total time devoted to assessment – how much for
formative and how much for summative)?
-Form / Sum -
- Given what we have discussed about the purpose and process of
formative and summative assessments… what do you think is the
CURRENT PRACTICE between these two assessment purposes in your
district (of the total time devoted to assessment – how much for
formative and how much for summative)?
-Form / Sum -

10. Reflections on Assessment…
• To effective, formative or summative assessments should be:
1. Generally Reliable (as opposed to statistically so)
2. Valid (reflects what it is intended to measure)
3. Helps the learner improve his/her performance (formative mostly)
4. Can be tracked and analyzed over time (summative mostly)
• What is your general sense of the state of quality of these elements in your school/district?
Formative Summative
Warm Cool Warm Cool

11. Reflections on Assessment…
Other data points: Required, by grade, department, standards aligned, type of administration, time required, testing window, vendor, ROI

12. 1. Philosophy – the role that assessment plays in your system – differences of
formative and summative – thinking about the differences between growth
and accountability and how to balance them in practice.
2. Technical Capacity
• Constructing effective rubrics – measuring what you value
• Designing rubric aligned tasks – connecting it to practice
• Scoring protocols – build capacity to judge student work
• Data collection and analysis tools – how to use the results
3. Systems effectiveness
• Organization of the weight of your assessments – inventory
4. Other?
Building Assessment Capacity

13. The Biggest Challenge
Getting together and building
the structures from which to
give students the feedback they
need to ensure success and
accountability in these priority
skill areas is the hardest part of
this work.
Please write 5 words that
describe last winter’s weather in
Connecticut.

14. Keep this shift in mind…
- From -
View Judge Explain
- To -
Establish a reliable standard; then…
View Describe Analyze

15. It’s our natural instinct…
Think back to the five words I asked you to
write to DESCRIBE the winter weather in
Connecticut…

16. To help this shift…
Pay close attention to the language.
Judgment - Descriptive
“reliable” vs. “corroborated by at least
one other source”
“effective” “1st person narrative”

17. Something to
Think About
• There is no shortcut to the amount of time
required to use standards and assessment
effectively.
• You either have to define the standard from
scratch through descriptive language or you
will have to define it through working with
your peers to develop anchor sets/models of
student work.
• The best strategy is probably a combination
of both approaches.

18. Align Assessments With Goals for Learning
Appropriate
Assessment
Instruments
Recall & recognition
based quizzes, tests,
and activities. Multiple
choice, matching, fill
In the blanks, etc.
(SAT/AP/Exams)
Checklists,
analytic rubrics,
or other agreed upon
skill standards
(AP/Smarter B/Exams)
Holistic and,
analytic rubrics,
or other agreed upon
skill standards
(Portfolios, Demonstrations,
Exhibitions, Etc)
Content
(Declarative)
Facts
“Define this term”
Content Skills
(Procedural)
Discrete Skills
“Factor this equation”
Meta Skills
(Contextual)
Applied Understandings
“Skilled Information Consumer”
Type of
Knowledge
Desired
Appropriate
Assessment
Standards/Scales
Take this quiz, test,
or other appropriate
content focused
assessment instrument.
Answering/responding to
classroom or text based
problems, experiments, and
writing assignments.
Complex projects,
real time explorations,
authentic problem based
skill applications.
Appropriate
Assessment
Tasks
Percentages of
right and wrong
answers on an agreed
upon numeric or letter
grade scale.
1-100/A-B-C-D
Specific criteria tied
to an agreed upon
numeric or
descriptive scale.
1-100/Pass-Fail
Insufficient-Excellent
Specific criteria tied
to an agreed upon
descriptive scale.
Insufficient-Sufficient
Proficient-Excellent
Overlap between columns can and does occur.

19. Assessing Reading Comprehension
Analytic Rubric (Matrix Format)
Clearly defined Tasks and Skills, Some Higher Order Skills
Execute, Complete, Apply, Analyze, Evaluate
Item Insufficient Sufficient Proficient Excellent
Purpose
Cannot identify the
purpose for most works
Identifies the purpose
for most works
Always identifies the
purpose for the work
Always identifies the
purpose for the work
without hesitation
Inferences
Cannot provide an
inference related to the
piece
Provides an inference
with at least one piece of
evidence to justify
his/her choice
Provides an inference
with more than one
piece of evidence to
justify his/her choice
Provides an inference
with multiple pieces of
evidence to justify
his/her choice
Point of View
Cannot identify the
point of view
Identifies the point of
view and, with
assistance, can give at
least one example
Identifies the point of
view and independently
gives at least one
example with details
Identifies the point of
view and independently
gives more than one
example with details
Opinion
Cannot state agreement
or disagreement with the
point of view or provide
even one non-disputable
fact as evidence to
defend their position
States either agreement
or disagreement with
the point of view and
provides at least one
non-disputable fact as
evidence to defend their
position
Definitively states either
agreement or
disagreement with the
point of view and
provides more than one
non-disputable fact as
evidence to defend their
position
Definitively states either
agreement or
disagreement with the
point of view and
provides several non-disputable
facts as
evidence to defend their
position

20. Traditional Content Assessments
Factual Recall and Recognition
Activities – Usually based on a
% of correct answers.
Multiple Choice, True/False,
Matching, Fill in the Blank,
Short Answer, Etc.

21. PowerPoint Skills Checklist
PowerPoint Skills What? I think
I can.
Got it!
1. Use Help – program based
2. Use the Auto-Content Wizard
3. Apply a presentation template
4. Use all 5 slide views and functions
5. Use a variety of slide layouts
6. Change toolbar views
7. Insert clip art, pictures, and/or objects
8. Create and insert a Word Art picture
9. Change your color scheme
10. Change font styles, size, and color
11. Change bullet and line format and color
12. Use an Action Button or HyperLink
13. Run a slide show
14. Change slide transitions
15. Use custom animations
16. Use pre-set animations
Checklist
Content and Simple Tasks
Identify, Recognize,
Recall, Locate

22. Project Assessment
Project
Combinations of
All Three
Item Points
Available
Points
Earned
Comments
Brainstorming 5
Contributions from all 2 (g)
Respect for other ideas 3
Discuss and Decide 15
Support each others ideas 10
Use of a voting tool 5 (g)
Draft 25(g)
Shared draft responsibility 5
Neatness and Legibility 5
Draft outline or Web 5
Evidence of feedback 5
Evidence of changes 5
Performance 15
Equality of participation 10
Time limits 5 (g)
Skills Assessment (Rubrics) 40
Social Responsibilities 20
Respond and reflect 20 (g)

23. Affective Self-Reflection Framework
Skill/Indicator Examples Observations/Evidence Comments/Reflections
Respect:
1.respects self
2.respects and accepts all
members of the community
3.respects property and the school
environment
4.respects others personal space
5.demonstrates empathy
Confident in their own abilities and
appearance
No insults or prejudicial remarks
Use polite greetings and phrases
Never abuses or damages property
Keeping equipment, books, school
related items in good condition
Respects others wishes regarding
personal space and physical contact
Does not engage in abusive behavior
toward self or others
Cares and supports others in need
Expresses concern for others
Responsibility:
1.follows through on personal
commitments or obligations
2.accepts responsibility for
personal actions and behavior
3.demonstrates social and
academic responsibility
Shows up on time for assigned duties
Completes work or assignments in
timely fashion
Does not blame others for personal
performances
Accepts responsibility and restitution
Admits mistakes
Dependable member of a team or
group

24. © JPC Sr.
2007
A Process for Building Performance Assessments
1. Identify what you are assessing for.
2. Determine critical attributes.
3. Select an appropriate assessment instrument/scale to
judge the work.
4. Describe each attribute across the selected scale.
5. Provide models of work across the scale.
6. Assess, and/or have the student assess, the work by
applying the ratings and providing explanations.

25. Editorial Guidelines
1. Pick a voice and stick with it. The learner will, the student is able, there is no, etc.
2. Pick a style and stick with it. Lists, narratives, scales, etc.
3. Remember the scale and stick with it. A “1” should be unacceptable, no credit - a “2” should be something
that everyone should be able to attain (95%), a “3” should be above average, and a “4” should be very high
performing.
4. Commit to the Pareto Principle. Focus on what is most important at every level. Do not be afraid to do a few
things well. Remember, if you can’t use these in a classroom environment, they don’t help anyone.
5. Follow your strands through the scale. If one of your critical attributes relates to a frequency issue (number
of errors) describe that all the way through the scale. Likewise, if the attribute is a quality issue (clarity of
speech) describe that through the scale as well - do not mix and drop.
6. Whenever possible, use descriptive words over judgement words. “Makes multiple spelling errors in every
sentence” vs. “Does not use proper English.”
7. Remember the big picture. These tools need to provide us with reliable feedback on student performance. It
should be assumed that all students will be asked to participate (with the normal appropriate
modifications) and that in order to be considered a “3” a student must satisfy all of the attributes of a “3.”
If I have one attribute of a “3” and the rest are at the level of a “2,” I am still a “2.”
© JPC Sr. 2007