Presentation of the paper "Analysis of the Generalization of Students’ Success Predictive Models in a Series of Java MOOCs on edX". This paper was presented at the Ninth European MOOCs stakeholders Summit 2025 (eMOOCs 2025). Abstract: One of the key challenges in Massive Open Online Courses (MOOCs) is the high attrition rate. While researchers have worked on predictive models to detect students at risk, one problem is that these models are often trained us-ing data from one course. However, these models face difficulties when generalizing to other contexts. In this line, this work aims to analyze how predictive models can generalize to other similar courses. Particularly, anal-yses are conducted using a series of three courses about Java programming, with several editions in English and in Spanish, and in teacher-paced and learner-paced modes, and using variables at learner and course level. This way, it is possible to analyze the generalizability considering different com-binations of predictive models to forecast MOOC grades and whether or not students pass the course. Results show that it is possible to achieve proper transferability between models in this context, although the predictive power decreases considerably in specific combinations of courses. Moreover, trans-ferability improves when combining two courses. In addition, it is possible to achieve accurate results regardless of the language although significant dif-ferences are observed in some cases when transferring models to the courses in a different language. In contrast, there are barely differences when train-ing and predicting using teacher-paced and learner-paced MOOCs, and accu-rate results are obtained in both cases. These results entail that it is possible to achieve transferable models in MOOCs when using related MOOCs alt-hough a drop in the predictive power may appear depending on the course and language. Presenter: Pedro Manuel Moreno Marcos Authors: Pedro Manuel Moreno-Marcos, Miguel Rodríguez Guillén, Carlos Alario-Hoyos, Pedro J. Muñoz-Merino, Iria Estévez-Ayres and Carlos Delgado Kloos This work was supported by Universidad Carlos III de Madrid (UC3M) through the Grants for the Research Activity of Young Doctors of the UC3M’s Own Research and Transfer Program (ASESOR-IA project). Moreover, it was supported by FEDER / Ministerio de Cien-cia, Innovación y Universidades - Agencia Estatal de Investigación through the grant PID2023-146692OB-C31 (GENIE Learn project) funded by MICIU/AEI/10.13039/501100011033 and by ERDF/UE, by the UNESCO Chair of “Scala-ble Digital Education for All” at UC3M and by the grant RED2022-134284-T funded by MICIU/AEI/10.13039/501100011033.

1. Analysis of the Generalization of Students’
Success Predictive Models in a Series of Java
MOOCs on edX
Pedro Manuel Moreno Marcos
Pedro Manuel Moreno-Marcos, Miguel Rodríguez Guillén,
Carlos Alario-Hoyos, Pedro J. Muñoz-Merino, Iria Estévez-Ayres and
Carlos Delgado Kloos
Ninth European MOOCs stakeholders Summit 2025 (eMOOcs 2025)
Palaiseau, France, June 30th 2025

2. INDEX
1. Introduction
2. Methodology
3. Results
4. Conclusions
2

3. INTRODUCTION
Several approaches: transfer models, in-situ,
global models
Limitations of the models due to generalizability
High dropout rates in MOOCs → Predictive models
3

4. OBJECTIVES
• Analyze to what extent predictive models to forecast
success and grades can generalize to:
4
O1. other MOOCs on the same topic
O2. the same MOOCs in a different language
O3. the same MOOCs in a different instruction mode
(teacher paced / learner paced)

5. INDEX
1. Introduction
2. Methodology
3. Results
4. Conclusions
5

6. EDUCATIONAL CONTEXT
• Three MOOCs about Java programming on edX
• Similar grading criteria
• Different languages and instruction modes
6

7. VARIABLES
Dependent variables: final grade and pass/fail
• User level: avg. formative exercises, % formative
exercises, no. participations in the forum, etc.
• Course level: version, no. exercises, no. videos, duration,
language, instructor mode, average grade
Independent variables
7

8. ANALYTICAL METHODS
ALGORITHMS
Decision
Trees (DT)
Random
Forest (RF)
k-Nearest
Neighbors
(kNN)
Gradient
Boosting
(GB)
METRICS
8
• Root Mean Square
Error (RMSE)
Final grade
• Area Under the Curve
(AUC)
Pass/Fail

9. INDEX
1. Introduction
2. Methodology
3. Results
4. Conclusions
9

10. O1. TRANSFER MODEL BETWEEN
COURSES
• Stable results excepting when training with C3
• In some cases, transferred model is slightly better
• Combined models are slightly better
10

11. O2. TRANSFER MODELS WITH COURSES
IN DIFFERENT LANGUAGES
• Models computed with GB
• Use of English to train and predict
• Small differences when predicting Spanish Courses
• No improvement when training with both courses
11

12. O3. TRANSFER MODELS WITH COURSES
WITH DIFFERENT INSTRUCTION MODES
• Little differences are observed
• Results are slightly better for learner paced when
training with data from teacher paced courses → no
significant
12

13. INDEX
1. Introduction
2. Methodology
3. Results
4. Conclusions
13

14. CONCLUSIONS
14
It is posible to achieve a good level of generalizability in
general
Possible effect of the course context and sample size
Combination of courses might be beneficial
Slightly drop when modifying the language (higher in English)
Low differences depending on the instruction mode

15. LIMITATIONS AND FUTURE WORK
15
Limitations
Only one series of MOOCs was used
Sample size might affect the results
Only students who interacted with
exercises were considered
Future work
Analyze more different MOOCs
Analyze the effect of specific
variables on generalizability
Add new variables
Carry out experiments in live courses

16. ACKNOWLEDGEMENTS
• Universidad Carlos III de Madrid (UC3M) through the
Grants for the Research Activity of Young Doctors of
the UC3M’s Own Research and Transfer Program
(ASESOR-IA project)
• GENIE Learn project - Grants PID2023-146692OB-
C31, funded by MICIU/AEI/10.13039/501100011033
and by ERDF/EU. Website:
https://genielearn.uc3m.es/
• UNESCO Chair of “Scalable Digital Education for All”
at UC3M
• Grant RED2022-134284-T (SNOLA project) funded by
MICIU/AEI/10.13039/501100011033.
16

17. 17