أ. آرلين جاليكسون بروفيسور فخري في جامعة ميتشجن الغربية الولايات المتحدة الأمريكية

1. Student
Assessment
in
21st
Century
Educa4on
Arlen
Gullickson
IEFE
Forum
2013
February,
2013
Riyadh,
Saudi
Arabia
1

2. A
note
of
AppreciaAon
• With
special
thanks
– Saleh
Alshumrani,
– Edward
Abankwa,
and
– Sahar
Abugharbieh
– Mohammed
Alyami
2

3. My
ObjecAves
• Make
the
case
for
focusing
on
formaAve
assessment
(Assessment
for
Learning)
• Focus
on
Feedback
a
key
aLribute
of
FormaAve
Assessment
• Offer
Suggested
Next
Steps
3

4. Personal
Context
50
years—teacher
and
educator
30
years—of
direct
interest
in
improving
classroom
assessment
pracAces
10
years—chair
of
the
Joint
CommiLee
on
Standards
for
EducaAonal
EvaluaAon
20+
years
in
program
evaluaAon
at
The
EvaluaAon
Center
WMU
4

5. Key
References
Knowing
what
students
Know
Na4onal
Research
Council
(2001)
Visible
Learning:
A
synthesis
of
Over
800
Meta-­‐Analyses
Rela;ng
to
Achievement
John
Ha[e
(2009)
Inside
the
Black
Box
Black
&
Wiliam
(1998)
The
Student
Evalua;on
Standards
(Now
called
the
Classroom
Assessment
Standards)
Joint
CommiLee
on
Standards
for
EducaAonal
EvaluaAon
(2003)
5

6. Reference
• Improving
Forma;ve
Assessment
Prac;ce
to
Empower
Student
Learning
Caroline
Wylie,
et
al.
(2012)
– Four
step
approach
– Key
areas
for
assessment
focus
6

7. An
Opening
Ques4on
• We
know
that
every
school,
regardless
of
where
it
is
located,
has
a
finite
amount
of
resources
available
to
serve
the
educaAonal
needs
of
its
students.
• Suppose
you
have
just
one
child.
That
child
is
nearing
school
age
and
you
have
a
major
say
in
how
your
child’s
school
will
use
its
resources
for
assessment
purposes.
• How
will
you
allocate
those
assessment
resources?
Will
you
spend
most
on
summaAve,
interim,
or
formaAve
assessments?
– ALendant
to
that,
what
are
the
reasons
for
your
choice?
7

8.
Student
Learning
EducaAon
focuses
directly
on
moving
each
and
every
student
forward
on
a
“novice
to
expert”
conAnuum.
8
Knowing
What
Students
Know
(NRC,
2001)

9. Expert
CharacterisAcs
Experts
in
a
subject
domain
– Have
extensive
factual
and
procedural
knowledge
– Organize
knowledge
into
schemas
that
support
paLerns
of
recogniAon
and
the
rapid
retrieval
and
applicaAon
of
knowledge
(scaffolding)
We
expect
teachers
to
be
experts
in
the
content
domains
they
teach
9
Knowing
What
Students
Know
(NRC,
2001)

10. Development
of
ExperAse
1) PredisposiAon
to
Learning
2) MulAple
paths
to
learning
3) Role
of
prior
knowledge
4) PracAce
and
feedback
5) Transfer
of
knowledge
6) Role
of
social
context
7) Impact
of
cultural
norms
and
learner
beliefs
8) Assessment
10
Knowing
What
Students
Know
(NRC,
2001)

11. PredisposiAon
to
Learning
• Children
are
naturally
curious
and
natural
problem
solvers
• Some
things
are
learned
naturally
and
viewed
as
a
part
of
normal
growth
and
development
• Children
can
reason
adeptly
• They
will
try
to
solve
problems
and
persist
in
trying
• Children
are
not
predisposed
to
learn
all
things
• They
can
be
deliberate,
self-­‐directed,
and
strategic
but
need
adult
guidance
when
they
are
not
predisposed.
• Oken
a
major
factor
is
moAvaAng
the
student
11

12. MulAple
Paths
to
Learning
• We
do
not
expect
students
to
move
simply
and
directly
from
subopAmal
to
opAmal
strategies
for
learning.
• Nor
do
we
expect
that
we
will
always
choose
paths
that
are
opAmal
for
learning
in
specific
situaAon.
• The
teacher’s
challenge
is
to
move/use
structures
in
ways
that
serve
students
effecAvely.
• Learning
is
consAtuted
within
parAcular
contexts
and
situaAons.
12

13. Role
of
Prior
Knowledge
• When
exposed
to
new
knowledge
we
aLempt
to
reconcile
it
with
currently
held
knowledge/
beliefs.
• Naive
concepAons
can
provide
a
good
foundaAon
for
future
learning.
• Drawing
out
and
working
with
exisAng
understanding
is
important
to
learning
13

14. PracAce
and
Feedback
• Developing
deep
knowledge
and
skill
requires
opportuniAes
for
pracAce
with
feedback
• Laws
of
skill
acquisiAon
– power
law
of
pracAce.
– Knowledge
of
results
14

15. Transfer
of
Knowledge
• Learners
must
develop
an
understanding
of
when
(under
what
condiAons)
it
is
appropriate
to
apply
what
they
have
learned.
• Transfer
is
more
likely
to
occur
when
the
person
understands
the
underlying
principles
of
what
was
learned.
• Learners
need
to
understand
how
one
problem
is
both
similar
to
and
different
from
other
problems.
15

16. Role
of
Social
Context
• Much
of
what
humans
learn
is
acquired
through
discourse
and
interacAons
with
others.
• Much
knowledge
is
embedded
within
systems
of
representaAon,
discourse,
and
physical
acAvity.
• Moreover,
communiAes
of
pracAce
are
sites
for
developing
idenAty–one
is
what
one
pracAces
to
some
extent.
The
rewards
and
meaning
people
derive
from
becoming
deeply
involved
in
a
community
can
provide
a
strong
moAve
to
learn.
• Studies
of
the
social
context
of
learning
show
that
in
a
responsive
social
se[ng,
learners
can
adopt
the
criteria
for
competence
they
see
in
others
and
then
use
this
informaAon
to
judge
and
perfect
the
adequacy
of
their
own
performance.
16

17. Impact
of
Cultural
Norms
and
Learner
Beliefs
• Cultural
a[tudes
about
cooperaAon,
as
opposed
to
independent
work,
can
affect
the
degree
of
support
learners
provide
for
each
other’s
learning.
• Personal
beliefs
about
learning
itself
significantly
affect
learning
and
performance.
17

18. ImplicaAons
for
Assessment
• Assessment
pracAces
should
focus
on
making
learners’
thinking
visible
to
themselves
and
others
• Assessment
feedback
should
be
Amely
and
informaAve.
18

19. This
Thing
Called
Assessment
• DefiniAon
is
evolving.
• Assessment
and
evaluaAon
oken
are
used
interchangeably
• DisAncAons
are
not
the
same
across
people
and
se[ngs.
• Language
differences
increase
misunderstandings
19

20. Arabic
Terms
for
Assessment
and
EvaluaAon
‫ﱘ‬‫ اﻟﺘﻘﻮ‬ •
‫ﻢ‬‫
اﻟﺘﻘﻴﻴ‬ •
20

21.
Defining
And
Describing
Assessment
Types
• Assessment
is
derived
from
assidere
to
sit
with
or
beside.
– It
is
something
we
do
with
and
for
a
student,
not
something
we
do
to
them.
(Wiggins,
cited
in
Green,
1998)
• SummaAve
• FormaAve
• Interim
21

22. SummaAve
Assessment
• Assessment
to
determine
what
students
have
learned
and
accomplished.
– Uniformly
focuses
on
achievement
in
a
course
or
curriculum
– A
current
synonym
is
Assessment
of
Learning,
– It
summarizes
the
development
of
learners
at
a
parAcular
point
in
Ame.
22

23. FormaAve
Assessment
• Assessment
used
to
serve
student
learning
Synonymous
with
Assessment
for
Learning
– a
range
of
formal
and
informal
assessment
procedures
– employed
by
teachers
during
the
learning
process
– Its
purpose
is
to
modify
teaching
and
learning
acAviAes
to
improve
student
aLainment
23

24. Interim
Assessment
• Interim
assessments
are
considered
medium-­‐
scale,
medium-­‐cycle
assessments,
falling
between
summaAve
and
formaAve
assessments
and
usually
administered
at
the
school
or
district
level.
Typically
given
several
Ames
a
year.
24

25. Some
DisAnguishing
Features
• SummaAve
assessments
– Tend
not
to
be
replicated
– ApplicaAons
tend
to
be
used
in
high
stakes
situaAons—
high
stress
– Are
widely
used
across
the
U.S.
to
make
policy
recommendaAons
– High
technical
difficulty
to
develop
– Cost
(Ame
and
money)
is
high—developed
and
sold
for
school
use
by
publishers
– Sample
broadly
across
course
and
curricular
content
– Student
collaboraAon
is
forbidden
– CheaAng
is
a
major
problem
– Have
zero
or
near
zero
effect
on
learning
25

26. Some
DisAnguishing
Features
FormaAve
Assessments
• Are
replicated
regularly,
oken
several
Ames
within
an
instrucAonal
period
• Are
narrowly
focused
• Regularly
address
facets
other
than
course
content
• Student
involvement
&
collaboraAon
is
encouraged
• Cost
per
assessment
miniscule
• Low
stress,
liLle
cheaAng
• Average
effect
on
student
learning
is
high
26

27. Serving
Development
of
ExperAse
27
Summa4ve
Forma4ve
Learning
Characteris4c
□
□
PredisposiAon
to
Learning
□
□
MulAple
paths
to
learning
□
□
Role
of
prior
knowledge
□
□
PracAce
and
feedback
□
□
Transfer
of
knowledge
□
□
Role
of
social
context
□
□
Impact
of
cultural
norms
and
learner
beliefs

28. Status
• SummaAve
is
simpler
than
formaAve
• SummaAve
Assessment
sAll
captures
the
most
aLenAon
• Enormous
interest
in
formaAve
assessment
– Spurred
by
Black
and
Wiliam
(1998)
– Large
effect
sizes
28

29. Assessment
Effects
• What
is
effect
size?
• How
big
should
effect
size
be
before
we
consider
it
important?
29

30. FormaAve
Assessment
Effect
Size
• Typical
effect
sizes
of
between
0.4
and
0.7;.
• PracAcal
consequences
of
such
large
gains:
1. An
effect
size
of
0.4
a. would
mean
that
the
average
pupil
involved
in
an
innovaAon
would
record
the
same
achievement
as
a
pupil
just
in
the
top
35
per
cent
of
those
not
so
involved.
b. would
improve
performance
of
pupils
in
GCSE
by
between
one
and
two
grades.
2. A
gain
of
effect
size
0.7,
if
realized
in
the
recent
internaAonal
comparaAve
studies
in
mathemaAcs
(TIMSS-­‐
Beaton
et
a,
1996),
would
raise
England
from
the
middle
of
the
41countries
involved
to
being
one
of
the
top
5.
30
Inside
the
Black
Box
(Black
&
Wiliam,
1998,
pp3-­‐4)

31. Feedback
• InformaAon
gathered
from
or
provided
to
students
– A
key
element
in
the
development
of
ExperAse
– Known
to
have
a
LARGE
effect
size
(0.73
on
average)
– A
determining
characterisAc
of
formaAve
assessment
Visible
Learning
(Ha[e,
2009)
31

32.
EffecAve
Feedback
Answers
Three
QuesAons
1. Feed
Up:
Where
am
I
going?
(goals)
2.
Feedback:
How
am
I
going?
3.
Feed
Forward:
Where
to
next?
32

33. Feedback
• Main
purpose
of
feedback
is
to
reduce
discrepancies
between
current
understandings
and
performance
and
a
learning
intenAon
or
goal.
• Feedback
is
a
consequence
of
performance
• Feedback
is
one
of
the
most
powerful
drivers
of
improving
performance
(d=0.73)
(Ha[e,
2009,
p.
172)
33

34. More
EffecAve
Feedback
• Provides
informaAon
on
correct
rather
than
incorrect
responses
and
• Builds
on
changes
from
previous
trails
• Occurs
when
– learning
goals
are
specific
and
challenging
but
task
complexity
is
low
– perceived
threats
to
self-­‐esteem
are
low
rather
than
high
34

35. Moving
Forward
With
Feedback
• Relate
feedback
to
learning
goals.
• Provide
cues
or
reinforcement
to
the
learner.
• Make
sure
that
students
see
and
appreciate,
and
can
act
appropriately
based
on
the
feedback—
Based
on:
– sound
evidence,
– clear
criteria
and
– transparent
• Where
possible
provide
in
the
form
of
video,
audio
or
computer-­‐assisted
instrucAon
feedback
35

36. Important
Caveats
• Roughly
a
third
of
studies
on
feedback
show
a
negaAve
effect
• Judgments
problem-­‐-­‐Giving
praise
for
compleAng
a
task
is
ineffecAve
• Teachers
do
not
do
well
at
determining
the
“next
step”
• Feedback
actually
is
bidirecAonal.
Teachers
provide
feedback
to
students;
more
importantly,
students
provide
feedback
to
teachers
and
to
each
other.
• In
most
classrooms
students
provide
most
feedback
• The
key
is
feedback
that
is
received
and
acted
upon
by
students—UlAmately,
it
is
not
what
the
teacher
says,
it
is
what
the
students
do.
36

37. Next
Steps
• Decide
on
a
Focus
(e.g.,
disposiAon
to
learn)
– What
will
make
teachers
want
to
focus
on
formaAve
assessment?
– Encourage
teacher
self-­‐assessment
– Remove
or
reduce
barriers-­‐-­‐What
would
stop
teachers?
– Simplify
the
task
– Find
ways
to
engage
and
work
together
on
common
tasks
37

38. A
General
Strategy
38

39. The
BoLom
Line
• “If
you
can
both
listen
to
children
and
accept
their
answers
not
as
things
to
just
be
judged
right
or
wrong
but
as
pieces
of
informaAon
which
may
reveal
what
the
child
is
thinking,
you
will
have
taken
a
giant
step
toward
becoming
a
master
teacher,
rather
than
merely
a
disseminator
of
informaAon.”
(Easley
and
Zwoyer
1975:
p
25)
Accessed
from
hLp://www.tki.org.nz/r/
assessment/one/formaAve_e.php
39

40. • Thank
you
for
your
paAence
and
interest.
40

41. Postscript
• AddiAonal
thoughts
on
next
steps.
41

42. ImplemenAng
formaAve
Assessment
Changes
• Narrow
focus
into
manageable
chunks
• Work
on
one
small
chunk
at
a
Ame
• Embed
assessment
in
instrucAon
• Engage
the
students
in
all
aspects
of
the
formaAve
assessment
process
• ALend
to
what
happens
when
feedback
is
provided
42

43. Deciding
what
to
Try
• Make
the
chunk
meaningful
• Make
it
small
• ALend
to
it
–
pracAce
it
in
a
variety
of
ways
over
a
substanAal
period
of
Ame
43

44. Example
• Learning
Target
Chart
• IdenAfies
concepts
to
be
covered
during
6
week
math
unit
• Students
self-­‐chart
progress
– I’ve
heard
of
this
– I
can
do
this
with
some
help
– I
can
do
this
on
my
own
– I
can
teach
someone
else
(Bearden,
2002
in
SES
Facilitator’s
Guide
by
Wingate,
2003)
44

45.
ReflecAng
on
Your
FormaAve
Assessment
• Clarity
of
Learning
and
Assessment
IntenAons
– Intended
outcomes
of
learning
and
assessment
are
clearly
stated
and
shared
with
students
• Sound
Assessment
Design
• EffecAve
Feedback
Provided
to
Students
• Strategic
Student
Engagement
45

46. Look
At
What
You
Are
Trying
To
Affect
• Knowledge:
Conceptual
understanding
of
informaAon,
theories,
principles,
and
research
• A[tude:
Beliefs
about
the
value
of
parAcular
informaAon
or
strategies
• Skill:
The
abiliAes
to
use
strategies
and
processes
to
apply
knowledge
• AspiraAon:
Desires
or
internal
moAvaAons
to
engage
in
a
parAcular
pracAce
• Behavior:
Consistent
applicaAon
of
knowledge
and
skills
46
NaAonal
Staff
Development
Council
(Killion,,
J.,
2008)

47. Simplify—Make
a
Large
Task
Small
Clarity
Design
Feedback
Engagement
Knowledge
Let’s
do
this
one
A[tude
Skill
AspiraAon
Behavior
47

48. Now
Ask
a
QuesAon
Clarity
of
Inten4ons:
Intended
outcomes
of
learning
and
assessment
are
clearly
stated
and
shared
with
students
Knowledge:
Conceptual
understanding
of
informa4on,
theories,
principles,
and
research
Simple:
What
percentage
of
your
students
can
demonstrate
that
they
know
the
intended
learning
outcomes?
More
Complex:
Do
Students
see
how
this
new
objecAve
builds
on
previous
learning?
48

49. Some
Closing
Thoughts
• Choose
a
trail
and
begin
• Remember
that
it
is
the
knowledge
and
acAon
of
students
that
tells
the
real
story
• Most
of
all
developing
strong
formaAve
assessment
skills
takes
Ame,
paAence
and
effort.
– There
is
that
first
characterisAc
of
developing
experAse
again—predisposiAon
to
learn
manifests
as
persistence.
• I
hope
that
you
will
both
begin
and
persist
in
your
efforts
to
use
formaAve
assessment
I
think
it
will
pay
you
back
many
fold.
49