LRT Talks 20161012 Jisc Student Experience Experts Group
1. Transforming the Student Experience
at Manchester Met
Birmingham | Oct 2016
Professor Mark Stubbs
Head of Learning & Research Technologies
http://twitter.com/thestubbs
http://slideshare.net/markstubbs
2. Strategy for Enhancing the Student Experience
Listen to Student Voice
Be Bold & Joined-up
Balance Consistency & Ownership
Address Dissatisfaction
Embed Student Voice
Empower Programme & Module Leads
with Actionable Learner Analytics
Empower Students & Personal Tutors
with Actionable Learning Analytics
3. A Coordinated Response to Improving the Student Experience
NSS Analysis,
Focus Groups, Surveys
4. Enhancing Quality and Assessment for Learning
• A coordinated ‘strike’ for step-change improvement
In the current climate
Diminishing unit of resource
Everything depends on everything else
We are large and risk averse
New Curriculum
• Designing new modules, …
New Admin Systems & Processes
• Personal timetabling, …
New Virtual Learning Environment
• Moodle & myMMU web/mobile, Talis Aspire…
New QA & QE Processes
• Facilitating curriculum transformation
B
5. Early 2010 Late 2010 Early 2011 Late 2011 Early 2012 Late 2012 Early 2013 Late 2013
Plan Prepare L4 Set Up L4 Live L5 Set Up L5 Live L6 Set Up L6 Live
Setting Up the New Curriculum
Design
New
Curriculum
Review
New
Curriculum
Approve
New
Curriculum
Set up
Curriculum
in SRS
Design
Teaching
Schedules
Produce
Timetables
Publish
Timetables
Design
Assessment
Schedules
Set up
Submission
System
Publish
Deadlines
Set up
Courses in
VLE
Build
Courses in
VLE
Great
Online
Experience
(Seamless,
Personalised,
Consistent)Set up
Reading
Lists
Pedagogic
Input
Learning Technology
Input
Library Input
Curriculum Design Course Admin/Setup
Learning, Teaching & Assessment Resources & Facilities
Publish
Reading
Lists
B
6. Balancing Consistency with Ownership
• Learning Technology Review (Mar 2010)
– Migrate from WebCT to “Core+ VLE” (Moodle)
for all modules & progs
• Joined-up, personalised, consistent
Recognised: watch & learn how staff use tech
Core: we set up & train staff and tell students to
expect a consistent ‘front door’
Arranged: we set up institutional accounts & train
staff to use
Recommended: staff set up & we train to integrate
B
12. Student Engagement Monitoring (SEM) Dashboard
An early warning system to improve retention, progression & success
Data
Warehouse
Student
Enrolments
Moodle
Usage
Submissions
& Marks
Attendance
13. Strategy for Enhancing the Student Experience
Listen to Student Voice
Be Bold & Joined-up
Balance Consistency & Ownership
Address Dissatisfaction
Embed Student Voice
Empower Programme & Module Leads
with Actionable Learner Analytics
Empower Students & Personal Tutors
with Actionable Learning Analytics
Wrap University Around Learner
Notas del editor
My name is Mark Stubbs and I’m Professor and Head of Learning and Research Technologies at Manchester Metropolitan University in the UK. In this short presentation, I will attempt to summarise work undertaken by colleagues across my institution over the last five years that have laid foundations for learning analytics on a university-wide scale.
In 2010 we set about pooling insights from across the institution on what was working and what wasn’t. Statistical analysis of our NSS scores revealed that course organisation was our Achilles heel. Student focus groups set up to discuss requirements for a new online learning environment came back not with scenarios of catching podcasts on the bus but of getting a definitive, up-to-date schedule for classes and assignments, not rumours about changes and extensions. This triangulated with comments from student surveys and the NSS, so we realised we had some priority work to do on so-called hygiene factors to ensure our courses felt more organised. We knew this work needed to dove-tail with and complement a major 350M GBP investment in our estate and initiatives designed to celebrate great teaching, such as our student-led teaching awards. Some soul-searching revealed that our size and complexity were apparent in an undergraduate curriculum that had grown unwieldy over time, and would benefit from a refresh so we set about doing this in concert with delivering the seamless, online experience our students were seeking. We also set up a series of projects to ensure our intended step-change was followed up with data-driven ongoing improvement.
In 2010, MMU’s Deputy Vice-Chancellor for Student Experience set up the EQAL Change Programme to Enhance Quality and Assessment for Learning with a Programme Board comprising the Pro-Vice-Chancellor Curriculum Innovation, Pro-Vice-Chancellor Students, Registrar and Director of Learning and Research Information Services, supported by the Head of MMU’s Business Improvement Team. Planning sessions with staff had highlighted complex interdependencies between, for instance, the way curriculum was represented in the student records system and the ability to deliver personalised information to students. As everything depended on everything else the EQAL Programme Board decided that four parallel strands needed to move forward in concert: a new undergraduate curriculum; new administrative systems and processes; a new personalised virtual learning environment; and new quality assurance and enhancement processes that could launch the new curriculum and ensure it was steered on a path of continuous improvement.
Change on this scale is unprecedented for UK HE – rewriting the entire undergraduate curriculum, while introducing a new VLE, personalised timetabling, library reading lists and ensuring that quality approval processes could cope with scrutinising 800+ new modules a year required some serious programme and project management. This slide shows the kind of dependencies that had to be managed to design, set up and deliver new undergraduate modules, and the role played by our Learning Technologists, the Centre for Learning and Teaching and Library colleagues in supplying specialist expertise at different stage. We had to build a custom curriculum workflow support system pull this off, but the real work was undertaken by academic colleagues in designing and delivering the new undergraduate curriculum. I am pleased to report that an incredible whole-university effort meant we delivered to the ambitious timeline from initial planning in 2010, through launch of the new first year in September 2011 to launch of the new second year in September 2012 and, finally, launch of the new final year in September 2013.
Throughout this process, my team was wrestling with the philosophical challenge of how to balance student desire for consistency with academic desire for creativity in the digital domain. Some of this audience may have seen our public blog of the Learning Technology Review we undertook and our decision to move away from an in-house WebCT VLE to a Core+ VLE built around a hosted Moodle that would deliver a joined-up, personalised and consistent experience for all modules and programmes. Central to that decision was a layered model used to set and manage expectations appropriately. At the heart of everything would be a core, integrated VLE driven by our student records system – it would be set up centrally; all staff would be trained in its use and all students would expect it to be a consistent ‘front door’ to online materials. Around that core, we would arrange tools that some, but not all, staff might choose to use. Institutional accounts would be set up for these tools and staff would be trained in their use, but their use was not mandatory – Turnitin is not well suited to dance or sculpture. Around these, we know that there are tools, such as Twitter, that staff were keen to set up for themselves, using their own accounts – we concentrated on providing guides for these so that they could be integrated into the Core+ VLE experience in the seamless way our students had requested. Finally, we recognised that some staff would be exploring new tools that might or might not be suitable to be recommended more widely. For these, we focussed on evaluation, partnering with and watching and learning how innovative staff used with technology to understand its wider potential.
This slide attempts to show how we deliver the seamless, personalised student experience by using Moodle to wrap the institution around each learner. Our Moodle reflects enrolments in our Student Records System. Areas within Moodle are set up with the codes from the Student Records System, which means these codes can be used along with the Student ID to present personalised information in the VLE. My team created a “mega-mashup” web-service to query university systems on behalf of the student and serve up the information students had asked for in those focus groups: deadlines, provisional marks, timetable, single-sign-on to email, module reading lists, recommended videos and past examination papers. Having delivered this into Moodle as promised for September 2011, we then surfaced the mega-mashup on our CampusM smartphone and tablet App, so that students could access high-level study information (such as deadlines, marks, reading and timetable) on the go and single-sign-on into the full Moodle environment if they needed more. We then followed up this work with further work around online assessment.
Each of the integrations was built in partnership with the area of the business that had responsibility for the data. Library colleagues had undertaken a great piece of student consultation to identify what makes a good reading list and had reflected best practice in a new List Policy, which distinguished a maximum of 3 items recommended for purchase from 10 essential starting points (that would ideally be digitised) from further reading. These tags were used in the creation of lists in Talis Aspire, and the open architecture of the Talis Aspire platform meant that these could be used, along with the consistent module identifiers, to surface the lists in Moodle, structured to reflect the List Policy and giving students seamless access to well-structured resources to support their studies and improve the sense of course organisation.
With so many major initiatives happening simultaneously, it is difficult to distinguish the effects of individual interventions, however the release of the seamless, personalised VLE to final year undergraduates coincided with step-change improvements in perceptions of course organisation and learning resources in the NSS, and subsequent cross-university work on assessment has seen similar improvement. These charts track changes in NSS satisfaction and response rates from 2008, comparing Manchester Met to the sector mean.
To give a sense of how the Data Warehouse is organised I have produced this visual, which uses the metaphor of a tube-map to highlight different lines of enquiry we’re opening up for academic colleagues to drive continuous improvement.
We have produced the first Dashboards for Programme and Unit Leaders powered by the Data Warehouse that enable colleagues not only to see trends and search student feedback, but to capture and track improvement actions planned in response. We are currently working on a new Dashboard for Heads to give an oversight of their Department’s Programmes and Units, and hope to refine this technology in partnership with academic colleagues.
We are also developing a Dashboard for Personal Tutors to give them a colour-coded summary of how their Tutees are progressing and a visual overview of their week-by-week engagement – red indicates no engagement, amber is partial, green is full.
These developments are at an early stage, but we hope that they will become powerful tools to enhance quality and help students realise their full potential at Manchester Met.