In this presentation, UMAN explores if there is a way to measure learning and personalise the user learning experience in an unobtrusive manner. The PhD thesis in this paper proposes using data-driven methods to measure learning by mining user interaction data to identify regularities that could be indicators of learning.

1. Inferring knowledge acquisition through
Web navigation behaviour
Yu He
Supervisor: Dr. Markel Vigo
Co-supervisor: Prof. Simon Harper
This work was supported by the EU's Horizon 2020 programme under grant
agreement H2020-693092 MOVING (http://moving-project.eu).

2. Multidisciplinary
Psychology E-learning
Learning
&
Cognition
Learning
Analytics
HCI
Pattern
mining
My
PhD
2

3. The MOVING platform (working environment)
3

4. The MOVING platform
4

5. Hypothesis
Web navigation behaviour Knowledge acquisition ?
Goal
5

6. Why this hypothesis？
• It was demonstrated in previous work that learning can be
associated with the leaners’ online interactive behaviours such as
browsing patterns and exploration strategies.
• Web navigation behaviours have also been investigated in online
learning platforms in related work.
These present the opportunity to relate the
students’ online navigation behaviour with their
learning progress.
6

7. Benefits
• Unobtrusiveness.
Instead of asking for users to complete surveys, quizzes or feedbacks,
users can have more ease using these platforms.
• Feasibility.
This research can provide cMOOC platforms a reliable way to measure
users' learning.
• Efficiency.
Monitoring knowledge acquisition using web behavior can be achieved
automatically with the user's data. With a reliable model, analysis and
feedback could be done automatically as well.
7

8. Research Questions
RQ1: Can we measure self-regulated online learning
unobtrusively with user's interactive navigational patterns?
Users'
interaction
data
A C
Infer
learning
progress
B
Web
navigation
behaviours
8

9. RQ2: What behaviours/patterns/patterns exhibited over time
are connected to learning?
Both in theory (i.e. from existing literature) and in practice(i.e.
from data collected).
mouseout - mouseover - select - ......
Time in between clicks......
Perform a search......
9

10. RQ3: How these behaviours/patterns affect the
learning outcome? I.e. are they helpful/unhelpful?
Helpful
Speed up the
learning
process
.
.
.
Unhelpful
low
completion
rate
.
.
.
10

11. Ground Truth
The MOVING platform will measure the performance of users through the data
generated by the following three sources, these measurements will serve as
the ground truth in our investigation:
• The adaptive training support widget which will display charts about the
usage of the different features of the platform.
• Self-assessment data from users' answers to questions provided by the
adaptive training support and their written feedback.
• Progress data from the curriculum progress widget which will show the
status of users' progress in the entire curriculum.
11

12. Massive Open Online Courses (MOOCs)
Features xMOOC cMOOC
Platform Specially designed
platform software
Use of social media
Content Video lectures Participant-driven
content
Assessment Peer assessment & Quiz No formal assessment
Communication A shared
comment/discussion
space
Distributed
communication
Recognition of
Completion
Badges of certificates None
• BOOCs, AdaptiveMOOCs, gMOOCs
12

13. First Stage
Scoping the problem and tools of the trade
Identify methods and tools to measure and understand
different types of learning
• Human learning & cognition
• Methods to measure learning & their application
13

14. • Trace-based partial ordering
A technique for automatically exploring the space of task progressions
• Item Response Theory (IRT)
A framework for knowledge assessment
• Knowledge Space Theory (KST)
For studying the hierarchical structure of knowledge and a
tool for knowledge assessment.
14

15. Second Stage
Relating Web navigation and knowledge acquisition
Identify navigation behaviours which are associated
with knowledge acquisition
• Data-driven
- pattern mining
• Hypothesis-driven
- look for known behaviours
15

16. Third and Fourth Stage
Evaluation
• How to design studies to evaluate models in second
stage?
• How does this scale on longitudinal analysis of
data?
Designing interventions
• Support those who are struggling
• Encourage engagement
16

17. Questions
• What are the existing ways to associate progression with actual
knowledge acquisition?
• What are the existing and effective ways for assessment using
interactive data?
• If navigational behaviours are shown to be effective in predicting
knowledge acquisition, how should this method be assessed
comparing to conventional measurements?
17

18. This work was supported by the EU's Horizon 2020 programme under grant
agreement H2020-693092 MOVING (http://moving-project.eu).
He Yu
The University of Manchester
he.yu@postgrad.manchester.ac.uk
Thank you!
Any questions?
Inferring knowledge acquisition through Web
navigation behaviour
18