Presentation at ascilite 2015, Perth, Australia. Full paper: https://www.academia.edu/19123100/An_enhanced_learning_analytics_plugin_for_Moodle_student_engagement_and_personalised_intervention
3. Learning analytics in Moodle
WHY?
• Benefits
• Familiarity for staff
• Single point of access
• Student activity already
in database – what data
are most valuable?
• Open source
http://analytics.jiscinvolve.org/wp/files/2015/10/Learning-analytics-framework.jpg
e
6. Learning analytics in Moodle
CURRENTLY AVAILABLE PLUGINS - ANALYSIS
SmartKlass
https://moodle.org/plugins/local_smart_klass
Engagement Analytics
https://moodle.org/plugins/browse.php?list=set&id=20
7. Learning analytics in Moodle
A FRAMEWORK FOR EVALUATION AND IMPROVEMENT
• IRAC framework (Jones, Beer, & Clark, 2013)
Jones, D., Beer, C., & Clark, D. (2013). The IRAC framework: Locating the performance zone for learning analytics. In: Proceedings of ascilite 2013, Sydney.
Only and all relevant information available
Needs to aid the task being undertaken
Providing support for context-appropriate action
Iterative development of other three factors
Information
Representation
Affordances for action
Change
8. Moodle Engagement Analytics
PLUGIN OVERVIEW
• Originally developed by Phillip Dawson, Adam Olley, and Ashley Holman
Moodle
Report
Logins Forums Assessments
Parameters
Traffic
lights
BYO
Moodle Engagement
Analytics Plugin
Indicators
Action
11. 1) IV1-Unit convenor survey around expectations for student early
alert system
2) IV2-Unit convenor survey around use of MEAP
4) User testing with use case (5 video sessions)
3) Prototyping-development of interface
5) Refinement of design based on usage and feedback
Overview of design process
12. What information woud you like to see to help
you decide what to include in your alert?
What would you find most useful about a
student alert system?
Collect information about expectations
1) IV1-Unit convenor survey around expectations for student early
alert system
2) IV2-Unit convenor survey around use of MEAP
13. Prototyping
1) IV1-Unit convenor survey around expectations for student early
alert system
2) IV2-Unit convenor survey around use of MEAP
3) Prototyping-development of interface
17. User testing
1) IV1-Unit convenor survey around expectations for student early
alert system
2) IV2-Unit convenor survey around use of MEAP
4) User testing with use case (5 video sessions)
3) Prototyping-development of interface
20. 1) IV1-Unit convenor survey around expectations for student early
alert system
2) IV2-Unit convenor survey around use of MEAP
4) User testing with use case (5 video sessions)
3) Prototyping-development of interface
5) Refinement of design based on usage and feedback
Refinement of design
26. Enhancements
26
AFFORDANCES FOR ACTION
• Customisable,
personalised
messages
directly to
students
• Suggested and
other snippets
(some from
PassNote)
• Save messages
for re-use
I
R
A
C
29. Preliminary outcomes
29
IMPACT OF MEAP AT MACQUARIE
• Who: unit
convenors and
student support
staff
• What: census,
updates, reminders
• Why: predominantly
for at-risk students
• How: logins,
assessment
submissions, grades,
attendance
I was surprised someone cared/was actually monitoring, kind
of a weird, I don't know totalitarian/'people are watching you'
feeling? But in this situation I was happy.
I'm glad I was contacted as it motivated me to complete the
work and almost "show" the unit convenor that I could do
it.
He gave me specific advice and encouraged me and it made
me feel much better.
The email basically kicked me into gear and I completed all my
assessments post-email to a high level.
Very useful. I wouldn't have been able to do such a large scale
analysis and identify so many students without MEAP. I
wouldn't have been able to send them such tailored, structured
and consistent messages.
30. Preliminary outcomes
30
THE DARK SIDE
• Issues around
• Message
composition
• Suggestions
• Student contexts
• Being labelled
I was contacted but in a manner that suggest I should drop out of
the course and not waste the convenor's time. She didn't ask
whether I was experiencing any issues outside of university,
but simply that I should transfer course if I can't handle the
workload.
Being labelled as lazy when you're doing your best and
don't have any other choice is quite sad.
Being aware and then being told of my own inadequacies is
confronting and, yes, does make me feel worse about life in
general. It's something I need to be told and is still that extra bit of
motivation.
The email was worded in a way that it the unit [convenor] was
trying to tell me I was doing horrible and should drop out and
didn't refer any help.
31. Future directions
31
NEXT STEPS FOR MEAP+
• Further work
• Measuring impact of
interventions
• Automated determination
of parameters (machine
learning)
• Investigating staff use
• Alternative forms of
representation
• Opening source
• Institutionalisation
32. Acknowledgements
IT TAKES A VILLAGE
• Students
• Unit convenors
• Student support staff – Janise Farrell
• Learning systems – James Hamilton, Jaymie Parker, Michael Mendoza
• Reference group
• Team members – Deborah Richards, Amara Atif
• 2015 Macquarie University Teaching Development Grant