Exploring predictive models that are closer to action by instructors. The talk proposes the use of hierarchical partitioning algorithms to produce decision trees that can be used to divide students into groups and simplify how feedback is provided.

1. Abelardo Pardo, Negin Mirriahi, Roberto Martínez-Maldonado,
Jelena Jovanovic, Shane Dawson, Dragan Gašević
KannanBflickr.com
Generating Actionable Predictive Models
of Academic Performance
International Conference on Learning Analytics and Knowledge
University of Edinburgh
29 April 2016

2. Generating Actionable Predictive Models of Academic PerformancePardo, Mirriahi, et al.
The problem
• Detailed data footprints collected
• Sophisticated algorithms applied
• Predictive models created
• How to derive/apply actions?
2
MichaelPereckasflickr.com

3. Generating Actionable Predictive Models of Academic PerformancePardo, Mirriahi, et al.
Retention/Attrition
3
TrevorHuxmanflickr.com
Predict student abandoning course/institution
E.g., Jayaprakash, S. M., Moody, E. W., Eitel, J. M., Regan, J. R., & Baron, J. D. (2014). Early Alert of Academically At-Risk
Students : An Open Source Analytics Initiative. Journal of Learning Analytics, 1, 6-47.

4. Generating Actionable Predictive Models of Academic PerformancePardo, Mirriahi, et al.
Sophisticated predictive
models
4
KevLewisflickr.com
Romero, C., López, M.-I., Luna, J.-M., & Ventura, S. (2013). Predicting students' final performance from participation
in on-line discussion forums. Computers & Education, 68, 458-472. doi:10.1016/j.compedu.2013.06.009
Classification
• Divide students in groups
• Useful for instructors
• Unclear how to intervene

5. Generating Actionable Predictive Models of Academic PerformancePardo, Mirriahi, et al. 5
Tanes, Z., Arnold, K. E., King, A. S., & Remnet, M. A. (2011). Using Signals for appropriate feedback: Perceptions and practices.
Computers & Education, 57(4), 2414-2422. doi:10.1016/j.compedu.2011.05.016
Course Performance
• Well
• Mediocre
• Poor
VitBrunnerFlickr.com

6. Generating Actionable Predictive Models of Academic PerformancePardo, Mirriahi, et al.
Disproportionate attention
6
FarrukhFlickr.com
Intervene
Wise, A. F. (2014). Designing pedagogical interventions to support student use of learning analytics. Paper presented at the
International Conference on Learning Analytics and Knowledge.

7. Generating Actionable Predictive Models of Academic PerformancePardo, Mirriahi, et al. 7
Gather data on the state of the student
Identify action to take
Deliver feedback
McKay, T., Miller, K., & Tritz, J. (2012). What to do with actionable intelligence: E2Coach as an intervention engine.
Paper presented at the International Conference on Learning Analytics and Knowledge, Vancouver, BC, Canada.
Paulflickr.com

8. Generating Actionable Predictive Models of Academic PerformancePardo, Mirriahi, et al.
Objective
1. Data indicators close to
learning design
2. Predictive model
3. Bridge between model and
application
4. Straightforward delivery method
8
OliverBraubachflickr.com

9. Generating Actionable Predictive Models of Academic PerformancePardo, Mirriahi, et al.
• Event counts from interactive
course material
• Midterm/final exam scores
• Recursive partitioning
• Divide cohort into performance
categories
9
LouishPixelflickr.com

10. Generating Actionable Predictive Models of Academic PerformancePardo, Mirriahi, et al.
Recursive Partitioning
• Arbitrary magnitudes in
factors
• Handle large number of
factors
• Handle heterogeneous factos
• Model with intuitive
interpretation
• Performance?
10
theilrflickr.com

11. Generating Actionable Predictive Models of Academic PerformancePardo, Mirriahi, et al. 11
WilliamMurphyflickr.com
• 13 Week first year Engineering
• Weekly activities (formative/summative)
• Videos, MCQ, Exercises, dashboard
• n = 272, Weeks 2-5 and 7-13

12. Generating Actionable Predictive Models of Academic PerformancePardo, Mirriahi, et al. 12
Data collected
• Indicators are directly connected with learning design
• Data structure shaped by the schedule (weeks)
• Data available in a per-week basis
• What is the expected midterm/final score in week n?

13. Generating Actionable Predictive Models of Academic PerformancePardo, Mirriahi, et al.
Result Example
• Week 10
• Predicted score at
leaves (out of 40)
• Conditions at nodes
• If (EXC.in >=22) and
(VID.PL < 8.5) then
score = 6
13

14. Generating Actionable Predictive Models of Academic PerformancePardo, Mirriahi, et al.
• Each leaf node represents a group of students
with their estimated score.
• Example: 6, 8.3, 8.4, 9.4, 9.9, 10, 15 (out of 40)
• Intervention: suggested work before exam
14
Result interpretation

15. Generating Actionable Predictive Models of Academic PerformancePardo, Mirriahi, et al. 15
shabnammayetFlickr.com
Performance
RMSE: Root mean square error, MAE: Mean absolute error

16. Generating Actionable Predictive Models of Academic PerformancePardo, Mirriahi, et al.
Conclusions and Future Work
Indicators closed to
learning design
Hierarchical partition
Student partition
respect to midterm/final
Acceptable performance
Immediate action
by instructors
16
HamishIrvineflickr.com

17. Abelardo Pardo, Negin Mirriahi, Roberto Martínez-Maldonado,
Jelena Jovanovic, Shane Dawson, Dragan Gašević
KannanBflickr.com
Generating Actionable Predictive Models
of Academic Performance
International Conference on Learning Analytics and Knowledge
University of Edinburgh
29 April 2016