1.
Creating Effective Learning Analytics Dashboards
Lessons Learnt
Sven Charleer
@svencharleer
Human-Computer Interaction group
Computer Science department
EC-TEL 2016, Lyon
Joris Klerkx
Erik Duval †
Tinne De Laet
Katrien Verbert

2.
Verbert, K., Govaerts, S., Duval, E., Santos, J. L., Van
Assche, F., Parra, G., & Klerkx, J. (2014). Learning
dashboards: an overview and future research
opportunities. Personal and Ubiquitous Computing,
18(6), 1499-1514.
EFFORTS
2
MOTIVATE?
Santos, J. L., Govaerts, S., Verbert, K., & Duval, E. (2012,
April). Goal-oriented visualizations of activity tracking: a
case study with engineering students. In Proceedings of the
2nd international conference on learning analytics and
knowledge (pp. 143-152). ACM.
AWARENESS
OUTCOMES?

3.
AUTOMATE AUGMENT
Engelbart, D. C. (2001). Augmenting human intellect: a
conceptual framework (1962). PACKER, Randall and
JORDAN, Ken. Multimedia. From Wagner to Virtual Reality.
New York: WW Norton & Company, 64-90.
VS
3
Arnold, Kimberly E., and Matthew D. Pistilli. Course Signals at Purdue:
Using learning analytics to increase student success. Proceedings of
the 2nd International Conference on Learning Analytics and Knowledge.
ACM, 2012.

4.
LEARNER PATHS
PATH OF EFFORT
OUTCOMES
INFLUENCES
4 https://www.flickr.com/photos/lockechrisj/

5.
RQ2: How can we promote students, inside and outside the
classroom, to actively explore this effort to outcomes path?
RQ1: How should we visualise relevant data to facilitate students
exploring the path from effort to outcomes?
5

6.
Dashboards
6

7.
TwitterBlogs
7

8.
Inquiry-Based Learning
8

9.
Evaluation setup
9

10.
RQ2: How can we promote students, inside and outside the
classroom, to actively explore this effort to outcomes path?
RQ1: How should we visualise relevant data to facilitate students
exploring the path from effort to outcomes?
10

11.
Abstract the LA data
RQ1: visualise to facilitate exploring11
1

12.
Abstract the LA data
RQ1: visualise to facilitate exploring12
Arnold, Kimberly E., and Matthew D.
Pistilli. Course Signals at Purdue: Using
learning analytics to increase student
success. Proceedings of the 2nd
International Conference on Learning
Analytics and Knowledge. ACM, 2012.
PREDICTION
EARLY WARNING
ANALYSIS
INSIGHTS
Pham, M. C., Derntl, M., Cao, Y., & Klamma, R. (2012, September). Learning
analytics for learning blogospheres. In International Conference on Web-Based
Learning (pp. 258-267). Springer Berlin Heidelberg.

13.
RQ1: visualise to facilitate exploring
Abstract the LA data
Santos, J. L., Verbert, K., Govaerts, S., & Duval, E. (2013, April). Addressing learner
issues with StepUp!: an evaluation. In Proceedings of the Third International Conference
on Learning Analytics and Knowledge (pp. 14-22). ACM13
AGGREGATION

14.
RQ1: visualise to facilitate exploring
Abstract the LA data
14
ATTRACTIVE
https://www.flickr.com/photos/janitors/

15.
Abstract the LA data
RQ1: visualise to facilitate exploring15
ABSTRACTION
MOTIVATES

16.
Abstract the LA data
RQ1: visualise to facilitate exploring16
FIRST INCENTIVE

17.
RQ1: visualise to facilitate exploring
Provide access to the learner artefacts
17
2

18.
RQ1: visualise to facilitate exploring
Provide access to the learner artefacts
18
LINK EFFORT TO OUTCOME
ABSTRACTION = LOSS
MEMORY RECALL

19.
RQ1: visualise to facilitate exploring19
Provide access to the learner artefacts
OVERVIEW+ZOOM

20.
RQ1: visualise to facilitate exploring20
Provide access to the learner artefacts
ACCESS TO ARTEFACTS

21.
RQ1: visualise to facilitate exploring21
Provide access to the learner artefacts
EVALUATION
RELEVANT EXAMPLES
LEARN FROM PEERS

22.
RQ1: visualise to facilitate exploring
Augment the abstracted data
22
3

23.
RQ1: visualise to facilitate exploring
Augment the abstracted data
23
OVERLOAD
MEANING
LOW EFFORT

24.
RQ1: visualise to facilitate exploring24
Augment the abstracted data

25.
RQ1: visualise to facilitate exploring25
Augment the abstracted data
QUALITY INTERPRETATION

26.
RQ1: visualise to facilitate exploring26
Augment the abstracted data
PEER FEEDBACK
QUALITY/QUANTITY

27.
RQ1: visualise to facilitate exploring
Provide access to teacher and peer feedback
27
4

28.
RQ1: visualise to facilitate exploring
Provide access to teacher and peer feedback
28
LA-SUPPORTED FEEDBACK
TEACHER INTERVENTION
Martinez-Maldonado, R., Clayphan, A., Yacef, K., & Kay, J. (2015). MTFeedback: providing notifications to enhanc
teacher awareness of small group work in the classroom. IEEE Transactions on Learning Technologies, 8(2),
187-200.
Arnold, Kimberly E., and Matthew D. Pistilli. Course Signals at Purdue: Using learning
analytics to increase student success. Proceedings of the 2nd International Conference on
Learning Analytics and Knowledge. ACM, 2012.
Poulos, A., & Mahony, M. J. (2008). Effectiveness of feedback: the students’
perspective. Assessment & Evaluation in Higher Education, 33(2), 143–154.
TIMELY FEEDBACK

29.
RQ1: visualise to facilitate exploring29
RELEVANT EXAMPLES
REDUNDANCY/CONSISTENCY
TIMELY FEEDBACK
Provide access to teacher and peer feedback

30.
RQ1: visualise to facilitate exploring
Visualise the learner path
30
5

31.
some bg
RQ1: visualise to facilitate exploring
Visualise the learner path
31
details Activity Activity Artifact ActivityArtifact Activity ActivityArtifact
Abstraction/Aggregation/Notification/…
learner activity path
overview

32.
32 https://www.flickr.com/photos/lockechrisj/
Visualise the learner path

33.
RQ1: visualise to facilitate exploring33
Visualise the learner path

34.
RQ1: visualise to facilitate exploring34
Visualise the learner path
Geoff’s hypothesis
Geoff’s data collection
Geoff’s conclusion
Angela’s data collection
Angela activities Geoff’s activities

35.
RQ1: visualise to facilitate exploring35
Visualise the learner path
Geoff’s hypothesis
Geoff’s data collection
Geoff’s conclusion
Angela’s data collection
Angela activities Geoff’s activities

36.
RQ1: visualise to facilitate exploring36
Visualise the learner path
Geoff’s hypothesis
Geoff’s data collection
Geoff’s conclusion
Angela’s data collection
Angela activities Geoff’s activities

37.
RQ1: visualise to facilitate exploring37
Visualise the learner path
Geoff’s hypothesis
Geoff’s data collection
Geoff’s conclusion
Angela’s data collection
Angela activities Geoff’s activities

38.
RQ1: visualise to facilitate exploring38
Visualise the learner path
Geoff’s hypothesis
Geoff’s data collection
Geoff’s conclusion
Angela’s data collection
Angela activities Geoff’s activities

39.
RQ2: How can we promote students, inside and outside the
classroom, to actively explore this effort to outcomes path?
RQ1: How should we visualise relevant data to facilitate students
exploring the path from effort to outcomes?
39

40.
RQ2: promote exploration
Integrate the dashboard into the workflow
40
6

41.
RQ2: promote exploration
Integrate the dashboard into the workflow
41
GLANCE
AWARENESS
REFLECTION?

42.
RQ2: promote exploration
Integrate the dashboard into the workflow
42
Kapros, E., & Peirce, N. (2014, June). Empowering L&D Managers through Customisation of Inline Learning Analytics.
In International Conference on Learning and Collaboration Technologies (pp. 282-291). Springer International Publishing.
INTEGRATE INTO EXISTING SYSTEMS
Arnold, Kimberly E., and Matthew D. Pistilli. Course Signals at Purdue: Using learning analytics to increase student
success. Proceedings of the 2nd International Conference on Learning Analytics and Knowledge. ACM, 2012.

43.
some bg
RQ2: promote exploration43
RSS READER
INTEGRATE
Integrate the dashboard into the workflow

44.
some bg
RQ2: promote exploration44
Integrate the dashboard into the workflow

45.
RQ2: promote exploration
CLASSROOM PRESENCE
45
Integrate the dashboard into the workflow
Charleer, S., Klerkx, J., Duval, E., De Laet, T., & Verbert, K. (2016). Towards Balanced Discussions in the Classroom
Using Ambient Information Visualisations. International Journal of Technology Enhanced Learning. (Accepted)

46.
RQ2: promote exploration
Facilitate collaborative exploration of LA data
46
7

47.
RQ2: promote exploration47
SINGLE USER
CLASSROOM USAGE
TEACHER DRIVEN
NO REQUIREMENTS
Facilitate collaborative exploration of LA data

48.
some bg
RQ2: promote exploration48
WEEKLY GROUP SESSIONS
END OF YEAR AWARDS
Facilitate collaborative exploration of LA data

49.
RQ2: promote exploration49
!SINGLE USER
STUDENT DRIVEN
TEACHER DRIVEN
MORE REFLECTION
Facilitate collaborative exploration of LA data

50.
RQ2: promote exploration
Facilitate collaborative exploration of LA data
50
TEACHER INITIATES
STUDENTS SHOW PROOF
INSPIRE
PEER COLLABORATION

51.
RQ2: How can we promote students, inside and outside the
classroom, to actively explore this effort to outcomes path?
RQ1: How should we visualise relevant data to facilitate students
exploring the path from effort to outcomes?
51
Abstract the LA data
Provide access to the learner artefacts
Augment the abstracted data
Provide access to teacher and peer feedback
Visualise the learner path
Integrate the dashboard into the workflow
Facilitate collaborative exploration of LA data

52.
Creating Effective Learning Analytics Dashboards
Lessons Learnt
Sven Charleer
@svencharleer
svencharleer.com
Joris Klerkx
Erik Duval †
Tinne De Laet
Katrien Verbert
EC-TEL 2016, Lyon
http://www.slideshare.net/svencharleer