1. 1
Learning from Meaningful,
Purposive Interaction
Fridolin Wild · Medieninformatik · Universität Regensburg ·
Knowledge Media Institute · The Open University
Representing and analysing competence
development with network analysis and
natural language processing
2. 2
Outline
Introduction and overview
Theoretical foundation
Precursor algorithms (SNA + LSA)
Algorithm: Meaningful, Purposive Interaction Analysis
• Mathematical foundation
• Visual analytics using vector maps as projection surfaces
• Implementation
Application examples for Learning Analytics
Evaluation: verification and validation
Summary and Outlook
4. 4
Introduction
Fascination with LSA and Matrix Algebra originated
in Information Retrieval (UR), then shifted to
Technology Enhanced Learning (WU+OU)
Research on Technology Enhanced Learning has
its place in the canon of Media & Computing (and
Knowledge Media)
It’s a big and growing global Software Market:
• Adkins (2011, p.6): 9.2% annual growth till 2015
• Docebo (2014,p.8): 7.9% annual growth till 2016
Drivers of Innovation: open Grand Challenges
to Research and Development in TEL
5. 5
Bridging informal
and formal
Create a unified, seamless
learning landscape with the
help of mobile devices.
learning
analytics
automated feedback using
interaction data to predict
performance.
#6
fostering
engage-
ment
Increasing student motivation to
learn and engaging the
disengaged – using technology.
How can we detect (de-)
motivation? How can make
use intrinsic/extrinsic reward
systems?
#4
New devices for
young children’s
expression of
scientific ideas
Mouse and keyboard are a
blocker to natural mapping and
new modalities of interaction
(touch, gestures) can foster a
more tactile learning.
#1
Learning to
read at home
with digital
technologies
#2
CSCL in teacher training
and professional
development
#3
e-assessment
New forms of assessment of
learning in social TEL
environments
#5
Understanding how toddler apps
can support learning.
early years
technology
dataTEL
Utilising real-time data to
improve teaching and
learning.
#7
#8
networked
learning
ecologies
Interest-driven lifelong learning
in personal learning networks
#9
#10
Fischer,Wild,
Sutherland,Zirn
(2014)
#1
6. 6
Objectives for this work (from GC #5,#6,#8)
Represent: to automatically represent
conceptual development evident from
interaction of learners with more
knowledgeable others and resourceful content
artefacts;
Analyse: to provide the instruments required
for further analysis;
Visualise: to re-represent this back to the
users in order to provide guidance and support
decision- making about and during learning.
12. 12
The Foundational Example
Particular unit of company with 9 employees
All went through trainings recently
Offered by universities (UR, OU), MOOCs, informal
learning (FaceBook, LinkedIn)
Now: Christina is off sick
HR manager to identify worthy replacement
• SNA
• LSA
• MPIA
(Wild, 2014, p.60)
17. 17
Shortcomings
Social Network Analysis (SNA)
• Blindness to content
• Relationship discovery restricted to incidences captured
• Popular for analysis, visualization, simulation, intervention
(Sie et al., 2012)
Latent Semantic Analysis (LSA)
• Blindness to purposes & structure (relations, groups, …)
• Lacking instruments for analysis
• No clear rule for number of factors to retain
• Popular for essay scoring, information retrieval, dialogue
tutoring, recommenders
19. 19
Fundamental
matrix theorem
on orthogonality
Calculating the
Nullspace Ker A:
Ax = 0 Eq.1
(Wild, 2014, p.131; redrawn
from Strang, 1988, p.140)
(Wild, 2014, p.132)
“every matrix transforms
its row space to its
column space” (Strang,
1988, p.140)
20. 20
The Eigenvalue Problem &
Singular Value Decomposition
(Wild,2014,p.143)
For every symmetric, square matrix:
(Barth et al., 1998, p.90/E):
Bx = λx
n.b.: B = AAT or ATA
Any multiplication of the matrix B with an
Eigenvector x yields a constant multiple of the
Eigenvector, scaled by the Eigenvalue λ
A = UΣVT
U = Eigenvectors(ATA)
V = Eigenvectors(AAT)
Σ = UTAV
21. 21
Base transformation (from Term-Doc space
to orthogonal Eigenspace)
(Wild, 2014, p.144)
Same dims for both
Eigenvector types (row
and column), same
Eigenvalues!
23. 23
Prediction of Threshold
Sum of Eigenvalues Σ2 = Sum of trace of matrix A
threshold = 0.8 * sum(A*A)
=> Calculate only the first k Eigendimensions, for which the
sum of Eigenvalues Σ2 does not yet pass the threshold
24. 24
Updating using ex post projection
v' = aT Uk Σ k
-1
a' = Uk Σk v' T
(Wild, 2014, p. 149f; see also Berry
et al., 1994, equation 7 and page 16
26. 26
Introducing Network Analysis Techniques
Still: result is high-volume,
sometimes even big data
Visualisation techniques
from (Social) Network Analysis
can help!
36. 36
MPIA foundational example
(path of Peter)
interface
socialweb
access
review
system
timeusage
html
management
trees
clustering
intersection
agglomerative
knowledge
learning
organisational
system
html
usage
c3
learning
knowledge
p2
system
html
social
c4
clustering
trees
agglomerative
m2
41. 41
Evaluations
The role of verification and validation
(Schlesinger, 1979, as cited in
Oberkampf & Roy, 2010, p.23)
(Wild, 2014, p.276)
42. 42
Verification Results
+ 22 examples in the documentation
(tested by the documentation checker)
[...]
* this is package ‘mpia’ version ‘0.60’
[...]
* checking examples ... OK
* checking for unstated dependencies in tests ... OK
* checking tests ...
Running ‘tests.R’
OK
[...]
43. 43
Validation Experiments
No standardised test collections
for conceptual development
Effectiveness:
• Accuracy in application (Essay Scoring)
• Convergent and divergent validity
• Annotation accuracy
• Degree of loss in the visualisation
Efficiency:
• Performance gain
45. 45
Performance Gains
Savings in calculation time through using
the threshold prediction method for SVD
calculation truncation (predicted from
original doc-term matrix)
47. 47
Innovation in TEL
Three Grand Challenges (Fischer et
al., 2014) addressed:
• “new forms of assessment for social
TEL environments” (Whitelock,
2014a)
• “assessment and automated
feedback” (Whitelock, 2014b)
• “making use and sense of data for
improving teaching and learning”
(Plesch et al., 2012)
47
learning
analytics
automated feedback using
interaction data to predict
performance.
#6
e-assessment
New forms of assessment of
learning in social TEL
environments
#5
dataTEL
Utilising real-time data to
improve teaching and
learning.
#8
PmO: When actions serve a purpose, their order cannot be reversed: The order of cooking an egg, then peeling it, cutting it into halves, and decorating it can of course be reversed – but then no longer leads to ‘eggs a la russe’ (Janich, 1998, p.137:§27).
Primacy of methodical order: human communication is the origin of information, order cannot be reversed
Makes possible: Purposive filtering, Meaning equivalence, => Vector Space, Proximity as a supplement, Expertise clusters, Reasoning about underlying competence
Idea of SNA dates back to the 1920ies; sociogram with Moreno’s 1934 book ‘who shall survive’; term SNA coined in the 50ies (Manchester);
Math behind SNA dates back to Euler’s Seven Bridges of Koenigsberg problem.