VII Jornadas eMadrid "Education in exponential times". Mesa redonda eMadrid Learning Analytics. "The contributions of Data Visualization & Learning Analytics for Online Courses". Ruth Cobos Pérez. 04/07/2017.

1. The Contributions of
Data Visualization &
Learning Analytics
for Online Courses
Ruth Cobos Pérez
Universidad Autónoma de Madrid
Ruth.Cobos@uam.es
Red eMadrid, www.emadridnet.org

2. Outline
Context of the presented Case Studies: MOOCs
What can we extract and do?
Initial Analysis
Data Visualization Approaches – Dashboards
Learning Analytics Approaches – Predictive models
Conclusions
2

3. 3
Context

4. 4
What can we extract from the
data?
Demographics data
User interaction with the system
Patterns of usage
Social interactions
Questionnaires data
Etc.

5. 5
What can we do with the data?
Initial Analysis Data Visualisation
Predictive models (Intervention)

6. MOOCs @ UAM
https://www.edx.org/school/uamx
https://www.edx.org/

7. - Length video < 5 min
- Length video < 10 min and > 5 min
- Length video < 15 min and > 10 min
- Length video > 15 min
Quijote501x
Initial Analysis:
Length of videos vs retention of attention

8. Initial Analysis:
Social Interaction - Quijote501x Discussion

9. Outline
Context of the presented Case Studies: MOOCs
What can we extract and do?
Initial Analysis
Data Visualization Approaches – Dashboards
Learning Analytics Approaches – Predictive models
Conclusions
9

10. 10
Two Case Studies for Dashboards
Open-DLAs: An Open Dashboard for Learning Analytics
 University Autónoma of Madrid
 edX and open edX
 xMOOC
UoS Dashboard
 University of Southampton (UK)
 FutureLearn
 cMOOC
(Cobos, R, Gil, Silvia, Lareo, A., Vargas, F. Open-DLAs: An Open Dashboard
for Learning Analytics. L@S: Third Annual ACM Conference on Learning at
Scale April 25-26, 2016, The University of Edinburgh )
(León, M., Cobos, R., Dickens, K., White, S., Davis, H. Visualising the MOOC
experience: a dynamic MOOC dashboard built through institutional
collaboration. eMOOCs 2016, Graz, Austria. 22-24 Feb, 2016. pp. 461-470)

11. Open-DLAs

12. 12
Information is organized in sections about:
 Participation in the discussion forums
 Navigation among the educational resources
 Interactions with the videos
 Performance results
Open-DLAs
Features:
 Different charts can assess learning analytics and data
visualization are available
 Interactive charts can be parameterized and resized
 Charts can be exported in formats such as .xls, .csv, .jpg, etc.
 Charts can show information of several courses, sum values and
average values

13. Next generation of Open-DLAs

14. 14
UoS MOOCs:
 Started October’13
 15+ courses
 25+ runs
 400K+ joiners
 200K+ learners
 30K+ course completers
UoS Dashboard

15. UoS Dashboard

16. ●Identifying most active members
●Identifying relationships between learners
(Claros, I., Cobos, R, & Collazos, C. An
Approach Based on Social Network Analysis
Applied to a Collaborative Learning
Experience. IEEE Transactions on Learning
Technologies, (1), 1-1)
UoS Dashboard

17. Outline
Context of the presented Case Studies: MOOCs
What can we extract and do?
Initial Analysis
Data Visualization Approaches – Dashboards
Learning Analytics Approaches – Predictive models
Conclusions
17

18. Initial Studies for Predictive
Models
UC3M and UAM
 eMadrid collaboration
UoS and UAM
 cMOOC vs xMOOC
(Cobos, R., Wilde, A.,and Zaluska, E. Predicting attrition from Massive
Open Online Courses in FutureLearn and ed. Comparing attrition
prediction in FutureLearn and edX MOOCs. Proceedings of the LAK
FutureLearn Worshop in the Learning Analytics and Knowledge 2017
Conference (LAK17), Canada., 13-17 Mar)
(J.L. Ruipérez Valiente, Cobos, R., Muñoz-Merino, P.J., Andujar, A.,
Delgado-Kloos, Early Prediction and Variable Importance of Certificate
Accomplishment, European MOOC Stakeholder Summit 2017
(eMOOCs 2017))

19. Case Study for Predictive models
Early prediction of students who will earn a certificate (or
not), with the purpose of enabling an intervention, such as
alerting those students in risk of losing their certificate
“The Spain of Don Quixote” (Quijote501x,
https://www.edx.org/course/la-espana-de-el-quijote-uamx-quijo
).
3530 learners enrolled
7 weeks

20. The course at edX

21. Dataset Description
The dataset is composed by several files in different
formats:

22. Methodology
11 variables Acquisition of
a certificate?
2. Implementation of these
classification models:
• random forests (RF)
• generalized boosted regression
modeling (GBM)
• Support Vector Machine (SVM)
• k-nearest neighbours (kNN)
• a logistic regression
3. Study of importance of
variables
1. Calculus of variables
regarding to:
•Learners’ progress
•Volume and amount of
learners´ activity
•Distribution of learners’
activity across the
different educational
resources and days of
the course

23. Methodology
11 variables Acquisition of
a certificate?
2. Implementation of these
classification models:
• random forests (RF)
• generalized boosted regression
modeling (GBM)
• Support Vector Machine (SVM)
• k-nearest neighbours (kNN)
• a logistic regression
3. Study of importance of
variables
1. Calculus of variables
regarding to:
•Learners’ progress
•Volume and amount of
learners´ activity
•Distribution of learners’
activity across the
different educational
resources and days of
the course
Type Variable
Learner Progress
Progress in problems (problem_progress)
Progress in videos (video_progress)
Volume and amount of learner
activity
Time invested in problems
(total_problem_time)
Time invested in videos (total_video_time)
Total time (total_time)
Amount of sessions (number_sessions)
Amount of events (number_events)
Learner activity distribution
Homogeneity solving problems
(problem_homogeneity)
Homogeneity watching videos
(video_homogeneity)
Number of days (number_days)
Constancy (constancy)

24. Methodology
11 variables Acquisition of
a certificate?
2. Implementation of these
classification models:
• random forests (RF)
• generalized boosted regression
modeling (GBM)
• Support Vector Machine (SVM)
• k-nearest neighbours (kNN)
• a logistic regression
3. Study of importance of
variables
1. Calculus of variables
regarding to:
•Learners’ progress
•Volume and amount of
learners´ activity
•Distribution of learners’
activity across the
different educational
resources and days of
the course

25. Summary of this case study
Objective: early prediction of students who will fail to accomplish
sufficient points to earn a certificate
 so that interventions (either automatic or through instructors) can
be performed before it is too late
5 machine learning algorithms, division of the data in the seven
course deadlines (one per week)
Results suggest that GBM model was the best from those selected
in terms of both performance and stability over the first weeks
After week three the most important variable regarding to
learners’ progress (progress in problems)
Next objective: implementation of a warning system

26. edX-MAS Tool:
edX Model Analizer System

27. edX-MAS Tool
Visualization of the Models
Classification models:
•Boosted Logistic Regression
•Stochastic Gradient Boosting
•Extreme Gradient Boosting
•Support Vector Machine
•K-Nearest Neighbors

28. edX-MAS Tool
Visualization of the Models

29. edX-MAS Tool
More Visualizations

30.  Context: Learners and Learning at MOOCs
 Great amount of data in different formats
 Learning Analytics levels:
 Initial Analysis
 Data Visualisation
 Predictive Models
 Learning Analytics are useful for:
 Detecting learners’ behaviours
 Improving learning environments
 Improving on-line learning approaches
 Identifying learners in risk
 Generating warning systems
Conclusions

31. 31