ESRC Code Acts in Education seminar serieshttp://codeactsineducation.wordpress.comUniversity of Edinburgh, 9th May 2014

1. How do Learning Analytics
“act” in Education?
Simon Buckingham Shum
Knowledge Media Institute, Open University UK
simon.buckinghamshum.net
twitter @sbskmi
ESRC
Code
Acts
in
Educa/on
seminar
series
h.p://codeactsineduca7on.wordpress.com
University
of
Edinburgh,
9th
May
2014
h.ps://twi.er.com/Wiswijzer2/status/414055472451575808

2. Siri is smart…
“Find code acts in
education”

3. Siri is smart…
“Find code acts in
education”

4. h.ps://twi.er.com/Wiswijzer2/status/414055472451575808
“Note:
check
the
huge
difference
between
knowing
and
measuring…”

5. A word of
introduction…
What does
this tell you?

6. Learning
Technology
KMi,
OU
AI
&
Argumenta7on
Learning
Disposi7ons
Human-­‐Centred
Informa7cs
Learning
Analy7cs
Seman7c
Scien7fic
Publishing
Dialogue
/
Issue
/
Argument
Mapping
A word of
introduction…
What does
this tell you?

7. Critical scepticism to economic modelling

8. Critical scepticism to economic modelling

9. The conversation you don’t want to have
with your learning analytics wonk
LAK13 Panel: Educational Data Scientists: A Scarce Breed
http://people.kmi.open.ac.uk/sbs/2013/03/lak13-edu-data-scientists-scarce-breed
John Behrens
(Pearson)

10. Data ethics and societal readiness
ny7.ms/1k5rY72

11. Data ethics and societal readiness
ny7.ms/1k5rY72

12. In
what
senses
do
learning
analy1cs
“act”
in
educa1on?

13. make
the
invisible
visible
make
solids
permeable
make
the
ephemeral
persistent
Educa7onal
(indeed
all)
Researchers
In
what
senses
do
learning
analy7cs
“act”
in
educa7on?

14. make
the
invisible
visible
make
solids
permeable
make
the
ephemeral
persistent
Learning
Analy7cs
In
what
senses
do
learning
analy7cs
“act”
in
educa7on?
Educa/onal
researchers
have
always
done
this
—
what’s
new
is
that
these
tools
are
now
in
the
hands
of
those
who
actually
produce
the
ac/vity

15. outcome
In
what
senses
do
learning
analy7cs
“act”
in
educa7on?
By
changing
the
system
dynamics
researchers
/
educators
/
instruc7onal
designers
administrators
/
leaders
/
policymakers
intent

16. outcome
In
what
senses
do
learning
analy7cs
“act”
in
educa7on?
By
changing
the
system
dynamics
Faster
feedback
loops
could
enable
more
rapid
adapta7on:
of
agents’
behaviour,
and
of
learning
resources
and
designs
researchers
/
educators
/
instruc7onal
designers
administrators
/
leaders
/
policymakers
intent

17. Delega7on
of
authority
to
define
goals,
analy7cs,
and
implica7ons
In
what
senses
do
learning
analy7cs
“act”
in
educa7on?
Distribution of power
between educators,
learners, leaders,
community…?
?
?

18. In
what
senses
do
learning
analy7cs
“act”
in
educa7on?
What data, concepts
and relationships do
the analytics
designers seek to
model?
Ontologically

19. Bowker, G. C. and Star, L. S. (1999). Sorting Things Out: Classification and Its Consequences. MIT Press, Cambridge, MA, pp. 277, 278, 281
“Classification systems provide both a
warrant and a tool for forgetting [...]
what to forget and how to forget it [...]
The argument comes down to asking not
only what gets coded in but what gets
coded out of a given scheme.”

20. Buckingham Shum, S. and Deakin Crick, R. (2012). Learning Dispositions and Transferable Competencies: Pedagogy, Modelling and Learning Analytics.
Proc. 2nd Int. Conf. Learning Analytics & Knowledge. (29 Apr-2 May, 2012, Vancouver, BC). ACM. Eprint: http://oro.open.ac.uk/32823
“A marker of the health of the
learning analytics field will be the
quality of debate around what the
technology renders visible and
leaves invisible.”

21. In
what
senses
do
learning
analy7cs
“act”
in
educa7on?
What thresholds, samples,
relationships, patterns, etc. do
the algorithms encode and
seek?
On what basis is a
recommendation engine
proposing interventions?
Algorithmically

22. governingalgorithms.org
Learning
Analy7cs
In
an
increasingly
algorithmic
world
[…]
What,
then,
do
we
talk
about
when
we
talk
about
“governing
algorithms”?

23. governingalgorithms.org
A
technology
or
an
epistemology?
Barocas,
S.,
Hood,
S.
and
Ziewitz,
M.
(2013).
Governing
Algorithms:
A
Provoca1on
Piece.
Social
Science
Research
Network
Paper
2245322.
DOI:
h.p://dx.doi.org/10.2139/ssrn.2245322
Secrecy,
obscurity,
inscrutability
Agency,
automa7on,
accountabili7es
A
typology
of
algorithms
by
genre?
The
inscrutability
of
algorithms
Norma7vity,
bias,
values

24. In
what
senses
do
learning
analy7cs
“act”
in
educa7on?
What meaning-making does
the representation and
interaction design
encourage?
Semio7cally

25. In
what
senses
do
learning
analy7cs
“act”
in
educa7on?
epistemology
pedagogyassessment
Knight, S., Buckingham Shum, S. and Littleton, K. (In Press, 2014). Epistemology, Assessment, Pedagogy: Where Learning
Meets Analytics in the Middle Space. Journal of Learning Analytics. Open Access Eprint: http://oro.open.ac.uk/39226
the
‘middle
space’ of
learning analytics
in the context of computation...
What epistemological
assumptions are shaping
the assessment regime,
and hence the
pedagogy? What
questions are analytics
used to help answer?

26. In
what
senses
do
learning
analy7cs
“act”
in
educa7on?
Analytics reports at the
organisational and national
levels come with
consequences at different
scales — sometimes punitive,
often impacting millions of
people.
Poli7cally

27. Figure
from
Doug
Clow:
h.p://www.slideshare.net/dougclow/the-­‐learning-­‐analy7cs-­‐cycle-­‐closing-­‐the-­‐loop-­‐effec7vely
(slide
5)
In
what
senses
do
learning
analy7cs
“act”
in
educa7on?
All
of
the
above
are
encapsulated
in
any
learning
analy7cs
deployment

28. 28
What
kinds
of
learners?
What
kinds
of
learning?
What
data
could
be
generated
digitally
from
the
use
context?
How
is
it
‘cleaned’?
Does
your
theory
predict
pa.erns
signifying
learning?
What
human
+/or
sooware
interven7ons
/
recommenda7ons?
How
to
render
the
analy7cs,
for
whom,
and
will
they
understand
them?
What
analy7cal
tools
could
be
used
to
find
such
pa.erns?
In
what
senses
do
learning
analy7cs
“act”
in
educa7on?

29. Learning
Analy7cs
approaches
act
in
educa7on…
Ontologically
Algorithmically
Semio7cally
Systemically
Poli7cally
Authority?

30. >>>
Learning
Analy7cs
cri7cal
template…
<Learning Analytics Approach>
Ontology …
Algorithms …
Semiotics …
Epistemology …
Pedagogy …
Assessment regime …
System Dynamics …
Authority …
Politics …

31. examples

32. English
primary
school
data
RAISEonline
sta7s7cs
for
a
school,
on
‘core’
subjects
(maths,
literacy),
a.endance
and
demographics

33. Visualising “attainment” and “progress”

34. Which analytics could reflect the progress that ‘Joe’
has made on so many other fronts other than his SATS?
34

35. student
modelling
by
an
adap1ve
tutor

36. Intelligent tutoring for skills mastery
http://oli.cmu.edu

37. 37
https://grockit.com/research
Adaptive platforms generate fine-grained
analytics on curriculum mastery

38. analy1cs
for
learning
disposi1ons

39. Qualities of effective learners
Factor analysis of the literature plus expert interviews: identified seven
dimensions of effective “learning power”, since validated empirically
with learners at many levels. (Deakin Crick, Broadfoot and Claxton, 2004)

40. Qualities of effective learners
40

41. ELLI: Effective Lifelong Learning Inventory

42. Quantifying learning dispositions
agency; identity; motivation; responsibility
Buckingham Shum, S. and Deakin Crick, R. (2012). Learning Dispositions and Transferable Competencies: Pedagogy, Modelling and Learning Analytics.
Proc. 2nd Int. Conf. Learning Analytics & Knowledge. (29 Apr-2 May, Vancouver). Eprint: http://oro.open.ac.uk/32823
http://learningemergence.net/2012/04/30/learning-powered-learning-analytics
A
wholis7c
visual,
intended
to
build
intrinsic
mo7va7on,
invi7ng
stretch,
providing
a
new
language,
provoking
conversa7on
that
7es
to
the
learner’s
iden7ty

43. Quantifying learning dispositions
agency; identity; motivation; responsibility
http://learningemergence.net/2014/03/01/assessing-learning-dispositions-academic-mindsets

44. http://learningemergence.net/2014/03/01/assessing-learning-dispositions-academic-mindsets
2020? personal data cloud generates my
dispositional profile for reflection
>>> help me take responsibility for my own learning
Shaofu Huang: Prototyping Learning Power Modelling in SocialLearn
http://www.open.ac.uk/blogs/SocialLearnResearch/2012/06/20/social-learning-analytics-symposium
Simon Knight: http://people.kmi.open.ac.uk/knight/2014/02/knowledge-in-search
Social network patterns,
teamwork effectiveness
and initiation of
relationships
Questioning, arguing
and search behaviours
reveal intrinsic curiosity
and epistemic
commitments
Tagging/sharing/
blogging/social patterns
reveal how you see
connections between
ideas
Perseverance, grit,
tenacity; overcoming
panic/stress when
challenged — through
behavioural and somatic
traces

45. Open source approach to analytics:
towards true transparency and emergence for complex systems?
http://www.solaresearch.org/mission/ola

46. conclusion
analy1cs
act
in
educa1on
—
on
mul1ple
dimensions
an
analy1cs
approach
perpetuates
an
educa1onal
worldview
—
so
let’s
ensure
this
is
inten1onal
–
not
accidental...