Se ha denunciado esta presentación.
Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.
Multiple measures done right: The purpose-
driven assessment system
John Cronin, Ph.D.
VP – Education Research
Northwest E...
The pursuit of compliance is
exhausting because it is always a
moving target. Governors move
on, the party in power gets
r...
Seven standards that define the
purpose driven assessment system
The purposes of all assessments are
defined and the assessments are valid and
useful for their purposes.
DEFINE YOUR PURPO...
Northwest Evaluation
Association and Gallup - 2016
https://www.nwea.org/content/uploads
/2016/05/Make_Assessment_Work_for_...
TEACHERS, PRINCIPALS AND
SUPERINTENDENTS say students spend
too much time taking assessments.
Percent of students who say the results
from state accountability tests are useful
29
Gallup (2016, May). Make Assessment ...
Teacher and Administrator perceptions on the
purposes of assessments
0%
10%
20%
30%
40%
50%
60%
70%
80%
State
Policymakers...
Students find many assessment types useful when
the assessment is tied to a specific purpose
Classroom tests and quizzes d...
One action to take
First, document and prioritize your
purposes for assessment then…
inventory the assessments in use and
...
Teachers are educated in the proper
administration and application of the
assessments used in their classrooms.
EDUCATE TE...
What types of extra help, if any, does your teacher
or school give you if you do not do well on a…
CLASSROOM TEST STATE ST...
Teachers need to …
Understand the purpose
Follow the procedures
Properly interpret and act
One action to take
Develop internal structures and policies to provide
training for all teachers (and administrators) in y...
Assessment results are aligned to the
needs of their audiences.
ALIGN RESULTS
Assessment
audiences
• School Board
• Students
• Teachers
• Parents
• Principals, school
administrators, and teacher
leade...
Percentage of parents who say teachers
rarely or never discuss their child’s
assessment results with them
61
Gallup (2016,...
Questions parents
want answered
from assessment
• Core question - Do you know
and care for my child?
• What are my child’s...
What kind of data do parents want?
95%
95%
93%
92%
91%
90%
88%
84%
79%
77%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Mon...
Questions teachers
want answered
from assessment
• What does each student
know and not know?
• What does this student need...
One action to take
Train teachers in the skills needed to have a
positive and productive parent conversation.
Many teacher...
Redundant, mis-aligned, or unused
assessments are eliminated.
STREAMLINE MEASURES
Your current assessment tools
•What purposes do each assessment meet well?
• Comes from assessment design and capabilities...
One action to take
Identify the places where you have multiple
assessments in-use validly performing the same
purpose. Det...
Assessment results are delivered in a
timely and useful manner.
DELIVER RESULTS
Percent of students
who say they do
not receive their
state accountability
test results.
37%
For parents, assessment results
begin losing their relevance
within…
1month
after the assessments are
administered
Source ...
Timely results are related to use
If needed for instruction, results need to be faster than
school improvement conversatio...
One action to take
Review your assessment calendar, with a focus
on when data is being delivered to your
audiences. Adjust...
The metrics and incentives used encourage
a focus on all learners.
SUPPORT ALL LEARNERS
Grade Students Athletes % Fine Arts %
9 3276 1487 50% 2256 76%
10 3190 1215 41% 1656 56%
11 2967 932 31% 1233 42%
12 2795 ...
What gets measured and attended to really does matter
0
5
10
15
20
25
30
35
40
165
167
173
176
181
185
187
189
191
194
196...
Changing from Proficiency to Growth means all kids matter
0
5
10
15
20
25
30
166
169
175
178
183
186
188
190
193
195
197
1...
One action to take
Ask the question – “What 3 to 5 metrics drive
educational decision making in our school or
district? No...
The assessment program contributes to a
climate of transparency and objectivity
with a long-term focus
INVITE COLLABORATION
“Everyone is entitled to their opinion, but
not to their own facts.”
Daniel Patrick Moynihan
What is accountability?
Accountability is a dialogue between the
stakeholders and the leaders of their schools. Part of
th...
The four principles that guide your
communications to your audiences.
• Clearly identify who your audience is and address
...
Florida
District
Highly
Effective
Effective Needs
Improvement
Developing Unsatisfactory VA Score Florida
Ranking
1 44.4% 5...
Source: Aviv, R (2014, July 21). Wrong Answer. The New Yorker. Retrieved on June
16, 2016 from http://www.newyorker.com/ma...
Differences in fall-spring test durations
An illustration of gaming
15%
25%
60%
Mathematics
Spring < Fall Spring = Fall Sp...
One action to take
Review the tone of a few of your recent
communications to stakeholders about student
learning. As you r...
A purposeful assessment program
assessment system ensures you
have direction, that your direction
stays true, and that you...
Thank you for joining us!
http://info.nwea.org/MMDR-Guide-LP.html?utm_medium=press-
release&utm_source=rss&utm_campaign=mu...
Seven purposes presentation
Próxima SlideShare
Cargando en…5
×

Seven purposes presentation

John Cronin's presentation: Multiple Measures Done Right - The Purpose Driven Assessment System at the January, 2017 COSA conference

  • Inicia sesión para ver los comentarios

Seven purposes presentation

  1. 1. Multiple measures done right: The purpose- driven assessment system John Cronin, Ph.D. VP – Education Research Northwest Evaluation Association
  2. 2. The pursuit of compliance is exhausting because it is always a moving target. Governors move on, the party in power gets replaced, a new president is elected, and all want to put their own stamp on education. It is saner and less exhausting to define your own course and align compliance requirements to that. The problem of compliance-based assessment systems
  3. 3. Seven standards that define the purpose driven assessment system
  4. 4. The purposes of all assessments are defined and the assessments are valid and useful for their purposes. DEFINE YOUR PURPOSES
  5. 5. Northwest Evaluation Association and Gallup - 2016 https://www.nwea.org/content/uploads /2016/05/Make_Assessment_Work_for_ All_Students_2016.pdf Make Assessment Work for All Students: Multiple Measures Matter
  6. 6. TEACHERS, PRINCIPALS AND SUPERINTENDENTS say students spend too much time taking assessments.
  7. 7. Percent of students who say the results from state accountability tests are useful 29 Gallup (2016, May). Make Assessment Work for All Students: Multiple Assessments Matter. https://www.nwea.org/content/uploads/2016/05/Make_Assessment_Work_for_All_Students_2016.pdf
  8. 8. Teacher and Administrator perceptions on the purposes of assessments 0% 10% 20% 30% 40% 50% 60% 70% 80% State Policymakers Parents Students Teachers Administrators % State accountability assessments % Classroom tests and quizzes Principals: Do you believe the purpose of data from (state accountability assessments, classroom assessments and quizzes) in your district is well understood by each of the following:
  9. 9. Students find many assessment types useful when the assessment is tied to a specific purpose Classroom tests and quizzes developed by teachers to evaluate their students’ learning process Interim assessments that are given to students two to three times over the year so teachers can see where students are growing and where they need to learn more Performance tasks that require students to apply things they have learned to a real-world problem or situation Formative assessments 78% 76% 75% 74%
  10. 10. One action to take First, document and prioritize your purposes for assessment then… inventory the assessments in use and evaluate their alignment with your purposes. It’s a big job but worth the time!
  11. 11. Teachers are educated in the proper administration and application of the assessments used in their classrooms. EDUCATE TEACHERS
  12. 12. What types of extra help, if any, does your teacher or school give you if you do not do well on a… CLASSROOM TEST STATE STANDARDIZED TEST Before and/or after school support 43% 24% One to one support 32% 19% Access to added resources 29% 22% Small group support 24% 17% No extra support provided 20% 38%
  13. 13. Teachers need to … Understand the purpose Follow the procedures Properly interpret and act
  14. 14. One action to take Develop internal structures and policies to provide training for all teachers (and administrators) in your assessments and promote proper use. This training should help educators understand what the results can and can’t tell them, and how best to use it to make a difference for students.
  15. 15. Assessment results are aligned to the needs of their audiences. ALIGN RESULTS
  16. 16. Assessment audiences • School Board • Students • Teachers • Parents • Principals, school administrators, and teacher leaders • District Administrators • Community members • State and federal officials
  17. 17. Percentage of parents who say teachers rarely or never discuss their child’s assessment results with them 61 Gallup (2016, May). Make Assessment Work for All Students: Multiple Assessments Matter. https://www.nwea.org/content/uploads/2016/05/Make_Assessment_Work_for_All_Students_2016.pdf
  18. 18. Questions parents want answered from assessment • Core question - Do you know and care for my child? • What are my child’s strengths and weaknesses? • Is my child on track for the next grade level? • Is my child on track for college? • Is my child showing improvement? • Should I be concerned?
  19. 19. What kind of data do parents want? 95% 95% 93% 92% 91% 90% 88% 84% 79% 77% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Monitoring general progress Knowing when to be concerned Determining readiness for next grade Knowing if I need to seek extra help Monitoring standards Communicating with teacher/admin Measuring high quality teaching Helping my child with homework Adjust content to student needs Providing activities for home Source – Northwest Evaluation Association (2012) – NWEA Assessment Perceptions Study. Survey conducted by Grunwald Associates LLC
  20. 20. Questions teachers want answered from assessment • What does each student know and not know? • What does this student need to learn next? • What resources will help this student? • How can I group these students for instruction?
  21. 21. One action to take Train teachers in the skills needed to have a positive and productive parent conversation. Many teachers find these conversations intimidating. When educators are well prepared they gain confidence and develop higher levels of trust and support from most parents.
  22. 22. Redundant, mis-aligned, or unused assessments are eliminated. STREAMLINE MEASURES
  23. 23. Your current assessment tools •What purposes do each assessment meet well? • Comes from assessment design and capabilities – not your current use • Is there alignment between the assessment and the information needed? •How are you using the assessments? • Reduce use to within the design and capability • Use assessments more fully to meet other purposes
  24. 24. One action to take Identify the places where you have multiple assessments in-use validly performing the same purpose. Determine whether you can leverage other assessments to meet these needs.
  25. 25. Assessment results are delivered in a timely and useful manner. DELIVER RESULTS
  26. 26. Percent of students who say they do not receive their state accountability test results. 37%
  27. 27. For parents, assessment results begin losing their relevance within… 1month after the assessments are administered Source – Northwest Evaluation Association (2012) – NWEA Assessment Perceptions Study. Survey conducted by Grunwald Associates LLC https://www.nwea.org/content/uploads/2014/07/NWEAGRUNWALD_Assessment_Perceptions_b.pdf
  28. 28. Timely results are related to use If needed for instruction, results need to be faster than school improvement conversations Reports designed to fulfill purpose • School administrators want reports that quickly identify students who need help. • Teachers want reports that help them make instructional decisions. • Parents want to know whether there student is on track and to be warned when trouble is ahead.
  29. 29. One action to take Review your assessment calendar, with a focus on when data is being delivered to your audiences. Adjust your testing calendar or practices to ensure that data gets to educators when they most need it. Make sure assessment results are timed to fit your schedule for professional days to encourage dialogue.
  30. 30. The metrics and incentives used encourage a focus on all learners. SUPPORT ALL LEARNERS
  31. 31. Grade Students Athletes % Fine Arts % 9 3276 1487 50% 2256 76% 10 3190 1215 41% 1656 56% 11 2967 932 31% 1233 42% 12 2795 703 24% 1028 35% Percent of students participating in athletics and fine arts programs
  32. 32. What gets measured and attended to really does matter 0 5 10 15 20 25 30 35 40 165 167 173 176 181 185 187 189 191 194 196 198 200 202 204 206 208 210 212 214 216 218 220 222 224 226 228 230 232 234 236 238 240 242 244 246 248 250 252 254 257 262 NumberofStudents Fall RIT Mathematics No Change Down Up Proficiency College Readiness One district’s change in 5th grade mathematics performance relative to the KY proficiency cut scores
  33. 33. Changing from Proficiency to Growth means all kids matter 0 5 10 15 20 25 30 166 169 175 178 183 186 188 190 193 195 197 199 201 203 205 207 209 211 213 215 217 219 221 223 225 227 229 231 233 235 237 239 241 243 245 247 249 251 253 255 258 266 NumberofStudents Student’s score in fall Mathematics Below projected growth Met or above projected growth Number of 5th grade students meeting projected mathematics growth in the same district
  34. 34. One action to take Ask the question – “What 3 to 5 metrics drive educational decision making in our school or district? Now ask – “What behavior do those metrics incentivize and are all students encompassed in those metrics? Make changes as needed to be sure your metrics encourage focus on all students.
  35. 35. The assessment program contributes to a climate of transparency and objectivity with a long-term focus INVITE COLLABORATION
  36. 36. “Everyone is entitled to their opinion, but not to their own facts.” Daniel Patrick Moynihan
  37. 37. What is accountability? Accountability is a dialogue between the stakeholders and the leaders of their schools. Part of that dialogue is understanding and aligning the goals and objectives of the parents, the schools, and the larger community. Another part of that dialogue is discussing how your schools are doing in reaching those goals. The most important part of the dialogue is what you’re doing to improve performance based on this information, which is leading.
  38. 38. The four principles that guide your communications to your audiences. • Clearly identify who your audience is and address them as your partner. • Understand what that audience wants to know about your schools and their performance. • Be transparent about how you are performing on these objectives. • Communicate your strategy to improve performance, and corrective action when strategy fails.
  39. 39. Florida District Highly Effective Effective Needs Improvement Developing Unsatisfactory VA Score Florida Ranking 1 44.4% 55.6% 0.0% 0.0% 0.0% 0.39 109 2 25.0% 75.0% 0.0% 0.0% 0.0% 0.37 121 3 90.9% 9.1% 0.0% 0.0% 0.0% -0.14 2802 4 60.7% 39.3% 0.0% 0.0% 0.0% -0.14 2797 5 81.2% 18.8% 0.0% 0.0% 0.0% -0.16 2831 6 37.3% 54.2% 1.7% 0.0% 6.8% 0.12 880 7 81.3% 18.8% 0.0% 0.0% 0.0% 0.22 402 8 41.7% 55.6% 1.4% 1.4% 0.0% -0.34 3274 9 52.2% 47.8% 0.0% 0.0% 0.0% 0.16 664 10 27.0% 66.2% 1.4% 0.0% 5.4% 0 1764 11 7.1% 72.6% 9.5% 10.7% 0.0% -0.08 2445 Teacher Evaluation Ratings in Eleven Florida Schools 2013
  40. 40. Source: Aviv, R (2014, July 21). Wrong Answer. The New Yorker. Retrieved on June 16, 2016 from http://www.newyorker.com/magazine/2014/07/21/wrong-answer “After more than two thousand interviews, the investigators concluded that forty-four schools had cheated and that a “culture of fear, intimidation and retaliation has infested the district, allowing cheating—at all levels—to go unchecked for years.” They wrote that data had been “used as an abusive and cruel weapon to embarrass and punish.” Several teachers had been told that they had a choice: either make targets or be placed on a Performance Development Plan, which was often a precursor to termination. At one elementary school, during a faculty meeting, a principal forced a teacher whose students had tested poorly to crawl under the table.”
  41. 41. Differences in fall-spring test durations An illustration of gaming 15% 25% 60% Mathematics Spring < Fall Spring = Fall Spring > Fall 0.0 1.0 2.0 3.0 4.0 5.0 6.0 Spring < Fall Spring = Fall Spring > Fall GrowthIndex Mathematics Differences in growth index score based on fall-spring test durations
  42. 42. One action to take Review the tone of a few of your recent communications to stakeholders about student learning. As you read these does it feel like your touting, or are you truly educating your partners? Try writing an op-ed about your educational program that would surprise readers because of its honesty and frankness.
  43. 43. A purposeful assessment program assessment system ensures you have direction, that your direction stays true, and that you and your students eventually reach their intended destination
  44. 44. Thank you for joining us! http://info.nwea.org/MMDR-Guide-LP.html?utm_medium=press- release&utm_source=rss&utm_campaign=multiple- measures&utm_content=text-link Multiple Measures Done Right – can be downloaded from

×