I attended the Pittsburgh Summer LearnLab at Carnegie Mellon over the summer (2016). The work that I did over the week of the LearnLab went into this presentation. I conducted two linear regression models, two support vector classification models, a hierarchical clustering analytics, and a Latent Class Analysis. Visit BobBodily.com for more information about my research.

1. Adding predictive elements to
student and instructor dashboards
Robert Bodily
Brigham Young University

2. Context
• Class
• First year chemistry course
• Blended – class 3x per week,
• Resources
• 150 videos (avg. 2 min long, supplemental resources)
• 15 weekly quizzes (unlimited question attempts)
• Participants
• 200 students (online interactions)
• 96 students took the self-report resource use survey

3. Nature of the data
• Quiz
• Confidence in answer (just a guess, pretty sure, very sure)
• Time spent on quiz
• Correct/incorrect
• Number of attempts per question
• Leave tab (still open, but inactive), come back to tab (active again)
• Video
• Play, pause, skip forward/backward, change play rate, change volume,
• Dashboard
• Number of times students follow recommendations given in dashboard
• Number of clicks within the dashboard

4. Student Dashboard

5. Instructor dashboard

6. Goals
Our dashboards are completely descriptive, so I want to add predictive
elements to both the student and instructor dashboards
1. Understand what course elements are predictive of student
achievement (grade on final exam)
2. Develop an early course prediction of student success
3. Determine what student profiles exist based on online behavior
4. Develop a model to classify future students into groups

7. What course elements are predictive of
student success?
Variable Beta P-value
Online homework score 0.366 0.000
In-class IClicker scores 0.154 0.024
# of attempts/question -0.411 0.000
Amount of question navigation -0.206 0.040
# of online activity sessions -0.195 0.020
Variable Beta P-value
Read the textbook 2.443 0.059
Ask professor questions in class 7.363 0.000
Watch Khan Academy -2.738 0.051
Use the internet -3.199 0.010
Skip recitation -4.820 0.041
Model 1 – regressing online interaction data
on final exam score.
Model 2 – regressing self-report resource use
on final exam score.

8. Develop an early course prediction of student
achievement
Online student interaction data Online student interaction data AND exam scores
There is significant improvement in both models until week 3 or 4, so that seems to be a good time to make
predictions for instructors and students.

9. Clustering to find student profiles
Cluster 1: Higher prior knowledge, good study skills, uses email, does not use office hours
Cluster 2: Efficient, game the system, ok losing some points, office hour students, does not use email
Cluster 3: Work hard but inefficiently, use tutor/friends, low self-regulation, bad study habits
3

10. Develop a classification model to classify
future students into groups
Classes AIC BIC SSA BIC
Log
Likelihood
2 8299 8460 8305 -4101
3 7870 8086 7877 -3869
4 7502 7774 7511 -3668
5 7139 7467 7150 -3470
6 6813 7196 6826 -3290
7 6612 7051 6626 -3172
8 6523 7017 6539 -3110
Group counts
3. 2
1. 20
2. 119
5. 24
6. 8
4. 22
Group 3: efficient smart learners
Group 1: low online activity, efficiency driven, just getting by
Group 2: average students, lower effort, low online activity
Group 5: average students, high effort, high online activity
Group 6: low learning skills, low knowledge awareness, high effort
Group 4: low learning skills, low knowledge awareness, but put forth less effort than group 6

11. Student dashboard suggestions
• Provide students with recommendations on the things that good
students do to succeed in the course, as well as the potential effect of
these things.
• Give students feedback on how they will do on the final based on
historical students similar to them. Give them something to click to
act on this information (e.g. meet with TA, meet with instructor, etc.).
• Show students some examples of their online behaviors to make
them more aware of their online activity. Provide recommendations
to help them improve.

12. Instructor dashboard suggestions
• Provide a list of things successful students do along with their effect
on student final exam grade so the instructor can encourage students
to do them to improve in the course
• Provide a predicted pass/fail score for each student at week 3 or 4 in
the course so the instructor or teaching assistants can intervene with
potentially struggling students
• Provide a student profile for each student so the instructor can better
personalize feedback to students