VII Jornadas eMadrid "Education in exponential times". "Student-facing learning dashboards". Katrine Verbert. KU Leuven, Bélgica. 05/07/2017.

1. Katrien Verbert – Augment / HCI – KU Leuven
Sven Charleer – Augment / HCI – KU Leuven
Student-facing Learning Dashboards

2. LEARNING ANALYTICS
2!
“The measurement, collection, analysis, and reporting of data about
learners and their contexts, for purpose of understanding and optimising
learning and the environments in which it occurs”
J. L. Santos. Learning Analytics and Learning Dashboards: a Human- Computer Interaction Perspective. PhD dissertation, KU Leuven, 2015.
G. Siemens. “Learning analytics: envisioning a research discipline and a domain of practice”. Proceedings of the 2nd
International Conference on Learning Analytics and Knowledge . ACM. 2012, pp. 4–8.
Microlevel
intro

3. 3!
EMPOWER
predictions
recommendations
LEARNING ANALYTICSintro

4. LEARNING DASHBOARDS
4!
“A Learning Dashboard is a single display that aggregates different
indicators about learner(s), learning process(es) and/or learning context(s)
into one or multiple visualisations.”
B. A. Schwendimann, M. J. Rodríguez-Triana, A. Vozniuk, L. P. Prieto, M. S. Boroujeni, A. Holzer, D. Gillet, and P. Dillenbourg. Understandig
learning at a glance: An overview of learning dashboard studies. In Proceedings of the Sixth International Conference on Learning Analytics &
Knowledge, pages 532–533. ACM, 2016.
K. Verbert, E. Duval, J. Klerkx, S. Govaerts, and J. L. Santos. Learning Analytics Dashboard Applications. American Behavioral Scientist, 57(10):1500–1509, 2013.
Design-based research
Design guidelines
intro
awareness
(self) reflection
sense making
impact
data
questions
answers
Behavior change or
new meaning

5. !
!
Verbert,K.,Govaerts,S.,Duval,E.,SantosOdriozola,J.,Van
Assche,F.,ParraChico,G.,Klerkx,J.(2014).Learning
dashboards:anoverviewandfutureresearchopportunities.
PersonalandUbiquitousComputing,18(6),1499-1514.!

6. LEARNER PATHS
BLENDED LEARNING
F2F GROUP WORK
STUDENT-ADVISER
6! https://www.flickr.com/photos/lockechrisj/

7. Sten Govaerts, Katrien Verbert, Aberlardo Pardo, Erik Duval. The student activity meter for awareness and self-reflection.
CHI'12 Extended Abstracts on Human Factors in Computing Systems. ACM, 2012.
CREATING EFFECTIVE LEARNING DASHBOARDSblended learning
abundance of data - effort - outcome
www.role-project.eu

8. CREATING EFFECTIVE LEARNING DASHBOARDSblended learning
Verbert, K., Govaerts, S., Duval, E., Santos, J. L., Van Assche, F., Parra, G., & Klerkx, J. (2013). Learning dashboards: an overview
and future research opportunities. Personal and Ubiquitous Computing, 1-16.!

9. RQ1: How should we visualise learner data to support students to
explore the path from effort to outcomes?
RQ2: How can we promote students, inside and outside the
classroom, to actively explore this effort to outcomes path?
9!
CREATING EFFECTIVE LEARNING DASHBOARDSblended learning
abundance of data - effort - outcome

10. Twitter
Blogs
10!

11. Inquiry-Based Learning
11!

12. 12!

13. Charleer, S., Klerkx, J., Santos, J. L., & Duval, E. Improving awareness and reflection through collaborative, interactive
visualizations of badges. In Proceedings of the 3rd Workshop on Awareness and Reflection in Technology-Enhanced Learning,
pages 69-81. CEUR Workshop Proceedings, 2013
ARTEL 2014 . Graz, Austria
ARTEL 2013

14. 14!
EVALUATIONS

15. Abstract the LA data
Provide access to the artefacts
Augment the abstracted data
Provide access to teacher and peer feedback
15!
RESULTS
RQ1: What are relevant learning traces, and how should we visualise
these data to support students to explore the path from effort to
outcomes?

16. 16!
RESULTS
RQ2: How can we promote students, inside and outside the classroom,
to actively explore this effort to outcomes path?
Visualise the learner path
Integrate LA into the workflow
Facilitate collaborative exploration of the LA data

17. 17!
RESULTS

18. RQ1: visualise to facilitate exploring18!
Visualise the learner path

19. RQ1: visualise to facilitate exploring19!
Visualise the learner path

20. RQ1: visualise to facilitate exploring20!
Visualise the learner path

21. RQ1: visualise to facilitate exploring21!
Visualise the learner path

22. RQ1: visualise to facilitate exploring22!
Visualise the learner path

23. 23!
BALANCED DISCUSSION IN THE CLASSROOMF2F Group Work
RQ3: What are the design challenges for ambient Learning
Dashboards to promote balanced group participation in
classrooms, and how can they be met?
RQ4: Are ambient Learning Dashboards effective means for
creating balanced group participation in classroom settings?
over- and under-participation

24. 24!
K. Bachour, F. Kaplan, and P. Dillenbourg. An interactive table for supporting participation balance in
face-to-face collaborative learning. IEEE Trans. Learn. Technol., 3(3):203–213, July 2010.
Over-
participation:“free-
riders” can affect
the motivated
learner to reduce
contributions!
G. Salomon and T. Globerson. When teams do not function the way they ought to. International Journal of Educational Research, 13(1):89 – 99, 1989.

25. EVALUATION SETUP
25!

26. EVALUATION SETUP
case study 1
# participants 12 students!
deployment
1 3h session with dashboard!
1 3h session without dashboard!
evaluation
class discussion!
questionnaires (perceived distraction/
awareness/usefulness)!
activity/quality logging!
case study 2
# participants 19 students!
deployment
half 3h session without dashboard!
half 3h session with dashboard!
evaluation
questionnaires (perceived importance
feedback/motivation)!
activity/quality logging!
26!

27. EVALUATION SETUP
27!

28. Visualise balance in an abstract and neutral way
Add the qualitative dimension to the visualisation
Create a realistic picture of the classroom situation
28!
RESULTS
RQ3: What are the design challenges for ambient LDs to promote
balanced group participation in classrooms, and how can they be met?

29. Ambient dashboards as support for teacher/presenter
Ambient dashboards raise awareness of the invisible
Ambient feedback information can activate students
29!
RESULTS
RQ4: Are ambient LDs effective means for creating balanced group
participation in classroom settings?
Charleer, S., Klerkx, J., Duval, E., De Laet, T. and Verbert, K. (2017) ‘Towards balanced discussions in the classroom using ambient
information visualisations’, Int. J. Technology Enhanced Learning, Vol. 9, Nos. 2/3, pp.227–253.

30. SUPPORTING ADVISER-STUDENT DIALOGUE
RQ5: What are the design challenges for creating a Learning
Dashboard to support study advice sessions, and how can they be
met?
RQ6: How does such a Learning Dashboard contribute to the role
of the adviser, student, and dialogue?
30!
lack of data-based feedback

31. 31!

32. 32!

33. 33!

34. EVALUATION SETUP
design
# participants
17 study advisers (preliminary feedback)!
5 study advisers (iterative feedback)!
approach
brainstorms/observations!
iterative design!
evaluation
# participants 5 study advisors!
deployment
Engineering Science, Engineering Science:
Architecture!
97 sessions (15-30min per session)!
evaluation
15 sessions observed!
questionnaires perceived usefulness!
34!

35. RESULTS
S. Claes, N. Wouters, K. Slegers, and A. V. Moere. Controlling In-the-Wild Evaluation Studies of Public Displays. pages 81–84, 2015.
35!

36. S. Charleer, A. Vande Moere, J. Klerkx, K. Verbert, and T. De Laet. Learning analytics dashboards to support
adviser-student dialogue. IEEE Transaction on Learning Technologies, 14 pages

37. Data Confidence
Collaboration
Adviser’s role
37!
RESULTS
RQ6: How does such a Learning Dashboard contribute to the role of the
adviser, student, and dialogue?
RQ5: What are the design challenges for creating a Learning Dashboard
to support study advice sessions, and how can they be met?
Authorship
Visual Encoding
Ethics

38. 38!
Deployed at
Engineering Science, Engineering Science: Architecture,
Maths, Biology, Physics, Geology, Geography, Biochemistry,
Informatics, Bio-engineering, Engineering Technology (3
campuses)
Soon university wide

39. Katrien Verbert – KU Leuven
katrien.verbert@cs.kuleuven.be
@katrien_v
Thank you! Questions?
Augment
Lab