State Assessment Data Meeting for Admin

August 2012
Preliminary Data
Learning Meeting

August 2012
Orting School District
Marci Shepard

“Be Present”

Orting School District  Teaching, Learning and Assessment August 2012  M. Shepard

 I can lead analysis of data using
a data-driven dialog protocol.
• Summarize best practices for data analysis
• Predict what we may see
• Make literal observations
• Draw inferences and ask questions
• Identify possible next steps

 I can explain AMOs to my staff.
• Describe what AMOs are and how they are
calculated
• Interpret AMO calculations

 I can lead analysis of data using
a data-driven dialog protocol.
• Summarize best practices for data analysis
• Predict what we may see
• Make literal observations
• Draw inferences and ask questions
• Identify possible next steps


Update from the State
Online versus Paper and Pencil


More technology difficulties
this year
• Technology glitches are reported as irregularities and may
have had a negative impact on scores.
o We have been analyzing differences between students with
reported irregularities and those in schools that did not have
irregularities, and cannot detect that this had a negative effect.
o When students were unable to enter an answer or had another
technology failure that precluded measuring their skill, a modified
scoring table will be applied.

• There are already many plans in place to fix the technology
difficulties.
o Districts will have more time with the test engine so students are not
unfamiliar with the tools and functionality.
o Test vendor (DRC) will develop a mechanism for verifying that each
district has proper set-up for online testing a month prior to testing.


Online Testing –
Mode Comparability
• Equating, which compares performance on items
common to last year’s test, shows the raw score
needed to be at Level 2, Level 3, and Level 4 are
the same in each mode – but when we apply those
cut scores, the percent of students meeting
standard on the paper tests is higher than the
percent meeting standard on the online tests in
nearly all grades and all content areas.


Online Testing – Mode
Comparability
• Districts had concerns in the past two years about online
being harder, but the psychometrics showed no
difference.

• If we gave identical paper tests, or identical online tests
to two groups of people one group might do better than
the other, and we would conclude that the groups had
different abilities (maybe one had more high performing
students). That is what we have attributed the minor
mode differences to in previous years.

• But this year brought larger differences, all in favor of
paper/pencil tests…

Online Testing –
Mode Comparability
Table below shows 2012 percent meeting standard (based only on equating
samples):
Grade % Testing Math Reading Science
Online
Online Paper Online Paper Online Paper
3 ~15% 64.5 65.2 57.8 67.8
4 ~25% 57.9 58.7 64.7 71.8
5 ~35% 62.8 63.6 67.6 71.7 59.2 67.0
6 ~50% 61.6 62.3 63.0 70.6
7 ~50% 55.0 58.4 64.9 71.0
8 ~50% 52.9 58.7 64.6 68.9 61.5 70.9


Online Testing –
Mode Comparability
• This year the differences between modes are
bigger than in first two years of online testing.

• The biggest differences are in the text-based
subjects, where student read passages online
(reading and science).

• The differences tend to be smaller in the upper
grades, but not always.

• Technology irregularities did not explain the
differences.

Online Testing –
Mode Comparability
• In consultation with our assessment vendors,
psychometrics experts and national technical
advisory committee, we made an adjustment to
online scores as part of our equating process.


Online Testing –
Mode Comparability
Table below shows 2012 adjusted percent meeting standard (based only on
equating samples):

Grade % Testing Math Reading Science
Online
Online Paper Online Paper Online Paper
3 ~15% 69.0 65.2 71.8 67.8
4 ~25% 65.6 58.7 69.2 71.8
5 ~35% 66.2 63.6 72.0 71.7 68.6 67.0
6 ~50% 61.6 62.3 67.3 70.6
7 ~50% 61.5 58.4 72.7 71.0
8 ~50% 56.8 58.7 68.7 68.9 65.4 70.9


This means……
• Any systematic differences in difficulty between
modes have already been adjusted in scores
reported to districts.

• OSPI will continue to examine mode effects during
equating to determine if an adjustment is
warranted in future years.


 I can explain AMOs to
my staff.
• Describe what AMOs are and how they are
calculated
• Interpret AMO calculations


nnual Measurable bjectives


Testing in ESEA Flexibility Waiver
• AYP rules and procedures are replaced by
Annual Measureable Objectives.
• Lowest performing schools in reading and
math need to revise their school
improvement plan using up to 20% of district
Title I monies.
• Participation in assessments and
performance of sub-groups (including
English language learners, special
education, poverty) still key.


State-developed differentiated recognition

accountability and support
o Reward Schools
• Highest performing schools
• High-progress schools

o Priority Schools
• 5% lowest performing Title I and Title 1-eligible schools with less than
60% graduation rate

o Focus Schools
• 10% of Title I schools with highest proficiency gaps

o Emerging Schools
• The next lowest 10% of schools on the Focus list, and the next 5% of
schools on the Priority list


Accountability Evolution with ESEA Waiver
Up to 2011-12 2012-13 and 2013-14 2014-15 and beyond

AYP Determinations AMO Calculations AMO Calculations
Sanctions No Sanctions (letters, transportation, No Sanctions (letters, transportation,
Set-asides etc.) etc.)

Up to 20% Set-asides for Priority, Focus, Up to 20% Set-asides for Priority, Focus,
and Emerging Schools and Emerging Schools
School Improvement
Uses AYP calculations to identify
schools and districts in a step of
improvement (Title I)

Uses AYP calculations to generate
list of Persistently Lowest Achieving
Schools ESEA Waiver Application ESEA New Accountability
Accountability System System
Used to identify Reward, Focus and Used to identify Reward, Focus
SBE/OSPI Achievement Priority and Emerging schools and Priority and Emerging
Index schools

Used to identify Award Schools


Accountability Evolution with ESEA Waiver
Up to 2011-12 2012-13 and 2013-14 2014-15 and beyond

AYP Determinations AMO Calculations AMO Calculations
Determinations based on Annual targets to close proficiency gaps Annual targets to close proficiency gaps
current status of % meeting by ½ by 2017; uses 2011 as baseline and by ½ by 2017; uses 2011 as baseline and
standard compared to Uniform adds equal annual increments (1/6 of adds equal annual increments (1/6 of
Bar (100% by 2014) proficiency gap) to get to 2017 target; proficiency gap) to get to 2017 target;
each subgroup, school, district, and each subgroup, school, district, and
AYP determinations reported state, have unique annual targets. state, have unique annual targets.
on Report Card
Calculations reported on Report Card Calculations reported on Report Card
Not making AYP results in
sanctions for Title 1 schools No sanctions No sanctions

$$$ set-asides Up to 20% Set-asides for Priority, Focus, Up to 20% Set-asides for Priority, Focus,
and Emerging Schools and Emerging Schools


State-developed differentiated recognition

accountability and support

o Annual Measurable Objectives
• Using 2011 as a baseline, OSPI set benchmarks that will cut
proficiency gaps in half by 2017 for every WA school.
• No sanctions required, but the expectation is that SIPs would
include strategies to close gaps.
• N size = 20


Annual Measurable
Objectives (AMOs)
WA has opted to establish AMOs as equal increments set toward the goal of
reducing by half the percent of students who are not proficient in all AYP sub
categories by fall 2017 (within six years)


2012–13 Waiver Tasks

• Office of Student and School Success will work with
Priority, Focus and Emerging schools to address gaps.
• The State Board of Education (SBE) and OSPI are
required to submit a revised accountability system
request, which is likely to include growth data.
• Legislature must pass a law to require ‘focused
evaluations’ to use student growth as a significant
factor.
• State must establish rules regarding use of student
growth as a significant factor in teacher and principal
evaluation and support systems.


CDs with Resources to
Use with your Staff
• This PowerPoint
• Preliminary data from the state
• OSD data comparisons
• How to calculate AMOs
• OSD AMO calculations
• Data driven dialog protocol


Questions?


State Assessment Data Meeting for Admin

Recomendados

Recomendados

Más contenido relacionado

Similar a State Assessment Data Meeting for Admin

Similar a State Assessment Data Meeting for Admin (20)

Más de Dr. Marci Shepard

Más de Dr. Marci Shepard (20)

Último

Último (20)

State Assessment Data Meeting for Admin