Recomendados
Recomendados
Más contenido relacionado
La actualidad más candente
La actualidad más candente (20)
Similar a Quality frameworks for e-learning (SIEAD 2018, Brazil)
Similar a Quality frameworks for e-learning (SIEAD 2018, Brazil) (20)
Más de Jon Rosewell
Más de Jon Rosewell (19)
Último
Último (20)
Quality frameworks for e-learning (SIEAD 2018, Brazil)
- 1. Quality frameworks for e-learning Jon Rosewell, The Open University (UK) INTERNATIONAL SEMINAR ON OPEN AND DISTANCE EDUCATION (SIEAD-BR 2018) Contributions from Open and Distance Education to Higher Education Quality: present and future 22nd October 2018
- 2. 174,000 students Formal, paid-for, degrees Supported distance learning Open access 50 years (nearly!) 60 million visitors Non-formal, free OER, open courseware Since 2006 (open2.net 1999) 8 million participants MOOC platform Many HE partners Open courseware platform CC-BY-SA-NC Broadcast TV partner OU/BBC audience: 265 million views Informal learning
- 3. Rise of e-learning? • Global demand for HE • MOOCs • Blended learning on campus • Society needs life-long learning – upskilling, reskilling • Short learning programmes? UK undergraduate numbers Universities UK, Patterns and trends in UK higher education 2018
- 4. Quality
- 5. What do we mean by ‘quality’ in HE? • Compliance & consumer protection – Accreditation – Guarantee of uniform standards • Reputation – Recruit good students, produce good graduates • Quality enhancement / Process improvement – Institutional mission – Stakeholder engagement – Measures of added value (‘learning gain’)
- 6. Approaches to QA in e-learning • Compliance or enhancement? • Process or product? • Input elements? • Pedagogical models? • Outcome measures? • Self-assessment or external review? • Scorecard? Benchmarking against others? Holistic: emphasis on process & context as well as product
- 7. Quality frameworks ICDE report on quality models 2015 • Review of quality frameworks • Ideally: • Multifaceted – many measures, holistic • Dynamic – flexible when technology changes • Mainstreamed – improvement through individuals • Representative – balance stakeholders • Multifunctional – e.g. external recognition, plan for improvement, create quality culture Ebba Ossiannilsson, Keith Williams, Anthony F. Camilleri, and Mark Brown (2015) Quality models in online and open education around the globe: State of the art and recommendations, ICDE Report http://www.icde.org/quality
- 8. A generic framework for QA in HE Ebba Ossiannilsson, Keith Williams, Anthony F. Camilleri, and Mark Brown (2015) Quality models in online and open education around the globe: State of the art and recommendations, ICDE Report http://www.icde.org/quality
- 9. European Standards & Guidelines (ESG) and e-learning 1.1 Policy for QA 1.2 Design and approval of programme 1.3 Student-centred learning, teaching & assessment 1.4 Student admission, progression, recognition & certification 1.5 Teaching staff 1.6 Learning resources and student support 1.6 Information management 1.8 Public information 1.9 Ongoing monitoring and periodic review 1.10 Cyclical external quality assurance
- 10. ENQA: Considerations for QA of e-learning • Recently published! • Supplement to ‘European Standards and Guidelines’ 2015 • Additional guidance and indicators Huertas et al (2018) ENQA Occasional Papers, No. 26 https://enqa.eu/index.php/publications/papers-reports/occasional-papers/
- 11. E-xcellence
- 13. Organisation of resources Strategic Management a high level view of how the institution plans its e- learning Curriculum Design how e-learning is used across a whole programme of study Course Design how e-learning is used in the design of individual courses Course Delivery the technical and practical aspects of e-learning delivery Staff Support the support and training provided to staff Student Support the support, information and guidance provided to students
- 14. Sample benchmark Curriculum design 8. … 9. Curricula are designed to enable participation in academic communities via social media tools. These online communities provide opportunities for collaborative learning, contact with external professionals and involvement in research and professional activities. 10.…
- 15. Sample indicators • There are institutional policies relating to the provision of online community spaces for student- student and student-teacher interactions. • Curriculum designers specify clearly the educational role that student-student interaction plays in their programmes. • Criteria for the assessment of student online collaboration exist and are applied consistently across programmes and courses. At excellence level • Teaching staff are supported by formal and informal staff development activity in the use of online tools for community building. • The institution works closely with professional bodies in the development of online professional communities. • Innovative assessment approaches, such as online collaborative work, peer assessment and self- assessment, form a part of the institution’s practice in this area.
- 16. Benchmarking as quality enhancement tool • Statement of best practice – Suggested indicators • Collecting evidence – Can be specific to each university • Identification of weaknesses & strengths • …leading to roadmap of actions for improvement
- 17. Different ways to use E-xcellence • Informal self-assessment using QuickScan – Identify ‘hot’ and ‘cold’ spots • Full internal self-assessment – Stakeholders collect evidence – Prepare roadmap of improvement actions • Integrate with institutional process – Embed selected benchmarks in internal process • EADTU E-xcellence Associates Label – Self-assessment, roadmap, external review recognition by EADTU NB: Resources such as manual and benchmarks are freely available!
- 18. Lessons learned
- 19. Participant feedback • Framework – Quickscan is valuable to structure discussion – Completeness of the framework is appreciated • Team working / stakeholders – People exchange perspectives with other departments • External perspective – Exchange of experience between the evaluators and staff was valuable – New ideas surfaced for course design • Reflection – A valued ‘moment of reflection’ on quality – People become aware of choices and implementations – Gives insight into strengths and weaknesses • Analysis – Opportunity to formulate e-learning policy – Provides foundations for decision making
- 20. Issues when introducing e-learning
- 21. Detailed issues • Workload management (of staff) • E-learning strategy • Academic communities / social media • Interactivity / e-learning tools Non-issues: • Reliability / performance (of VLE) • Student support generally – Not a ‘problem’ but much activity
- 22. MOOC quality
- 23. Why worry about MOOC quality? Students – know what they are committing to Employers – recognition of content and skills Authors – personal reputation, 'glow' of success Universities / providers – brand reputation Funders – philanthropists, government, investors Quality agencies – on behalf of all above
- 24. Are MOOCs different from e-learning? • MOOC vs Higher Education e-learning – Short, free, no entry requirements • MOOC participants – Motivations differ from degree students – Completion may not be not their goal But a MOOC is a Course so maybe it should be judged like any other HE course?
- 25. OpenupEd Quality Label • Derived from E-xcellence – Lightweight process • Self-assessment • Formal label – External review www.openuped.eu/quality-label
- 26. OpenupEd MOOC features • Openness to learners • Digital openness • Learner-centred approach • Independent learning • Media-supported interaction • Recognition options • Quality focus • Spectrum of diversity
- 27. OpenupEd MOOC benchmarks • Derived from E-xcellence benchmarks • For the institution: – To be checked every 3-5 years – 21 benchmark statements, in six groups: Strategic management, Curriculum design, Course design, Course delivery, Staff support, Student support • For the course: – To be checked for each MOOC – 11 benchmark statements
- 28. Benchmarks – course level 22 A clear statement of learning outcomes for both knowledge and skills is provided. 23 There is reasoned coherence between learning outcomes, course content, teaching and learning strategy (including use of media), and assessment methods. 24 Course activities aid participants to construct their own learning and to communicate it to others. 25 The course content is relevant, accurate, and current. 26 Staff who write and deliver the course have the skills and experience to do so successfully. 27 Course components have an open licence and are correctly attributed. Reuse of material is supported by the appropriate choice of formats and standards. 28 The course conforms to guidelines for layout, presentation and accessibility.
- 29. Quick scan
- 30. NA: Not achieved PA: Partially achieved LA: Largely achieved FA: Fully achieved Quick scan
- 31. OL: Openness to learners DO: Digital openness LC: Learner-centred approach IL: Independent learning MI: Media-supported interaction RO: Recognition options QF: Quality focus SD: Spectrum of diversity Quick scan
- 32. MOOC case study: OU + FutureLearn A representative Open University MOOC … published on FutureLearn • Evidence for OpenupEd features and benchmarks • Quality emerges from joint efforts of OU (university) & FutureLearn (platform provider) • Holistic approach: • Institutional and course level • Process as well as product • Structures and processes embed a concern for quality throughout development, delivery and evaluation Jansen, D., Rosewell, J., & Kear, K. (2017). ‘Quality Frameworks for MOOCs.’ In: M. Jemni, Kinshuk, & M. K. Khribi (Eds.), Open Education: from OERs to MOOCs, 261–281. Springer http://oro.open.ac.uk/47595/
- 33. Learning design
- 34. Learning design – module map
- 35. Learning design – activity classes Assimilative read, watch, listen, think about, observe Finding & handling info list, analyse, collate, find, select, manipulate Communicative communicate, debate, discuss, collaborate, present Productive build, write, make, design, construct, produce, draw Experiential practice, apply, experience, investigate, perform Interactive/adaptive explore, experiment, improve, model, simulate Assessment write, demonstrate, critique, peer review, self-assess, receive feedback
- 36. Learning design – activity planner
- 37. Learning design – in practice Toetenel, Lisette and Rienties, Bart (2016). Learning Design – creative design to visualise learning activities. Open Learning, 31(3) pp. 233–244.
- 38. Learning design – in practice Toetenel, Lisette and Rienties, Bart (2016). Learning Design – creative design to visualise learning activities. Open Learning, 31(3) pp. 233–244.
- 39. What students want – and what they need “Student satisfaction is “unrelated” to learning behaviour and academic performance, a study has found. […] while students dislike collaborative learning, they are more likely to pass if they take part in it” (Times Higher Education, Feb 12th 2018) From an analysis of 100,000 students on 151 modules More at Bart Rientes, OU Inaugural Lecture
- 40. How does student satisfaction relate to module performance?Satisfaction Students who successfully completed module Slide from Bart Rienties Inaugural lecture
- 41. Summary
- 42. In summary… • A quality framework should underpin e-learning provision – to help create a quality culture – that is more likely to produce quality e-learning – and quality enhancement • There is no simple recipe, but… – Work in a module team – Think about learning design – Think about student support
Notas del editor
- Hello, I am Jon Rosewell, I am a lecturer at the Open University, UK. I’m not going to give you a recipe for how to create quality e-learning. Instead, I will suggest using a framework to help you improve quality – we can always do better!
- The learning landscape is very diverse – MOOCs are the new kid on the block, but not the only way of delivering learning at scale The OU itself – open access degree courses, well-supported but relatively costly. Futurelearn is MOOC platform – OU and other universities are partners OpenLearn has had OER and open courseware for years before MOOC – nearly 20 if you include open2.net OpenLearn Create is sister site which allows anyone to create open (freely licenced) courseware And other channels for informal learning eg partnership with BBC – broadcasts are supported by material on OpenLearn.
- The amount of e-learning will increase because of all these reasons So it matters that we get quality right! Figure on the right shows UK undergraduates – not e-learning, all students Even in mature economy, demand for first degree is rising – top line. Demand for HE will be rising much faster globally – and e-learning can scale to pick up the demand But note the other lines – this includes the UK part-time sector which has collapsed as a result of introducing very high fees Loans are available but not if you already hold a degree – see the fall in orange line which represents reskilling/upskilling at university. So universities now have a problem offering life-long learning by conventional courses – MOOCS or short learning programmes may fill the gap
- I’m going to start with some general discussion of quality
- Quality a difficult term to pin down! At a minimum – is the course/qualification good enough to recognised / accepted? But different universities want to improve their reputation – good brand attracts good students. Teachers want to teach, to teach well, and to teach better, so quality enhancement My university has a particular mission for students who would not otherwise go to university -- disadvantaged backgrounds, low previous qualifications, disabilities Improving for them is very important to the university, but may not be visible in rankings.
- If we are concerned about quality, how do we check it? Can we check the quality of the course by looking in detail at all its parts? Can we predict the quality of a course by looking at the pedagogical model it uses, ie how it is taught? Can we judge by what happens? How many students completed? Passed the course? We need to take a holistic view, quality emerges when there is a good process
- There are existing quality frameworks that can be used. ICDE did a global review of a number of quality models They suggest choosing one which is multifaceted (many measures, holistic), dynamic (flexible to change in tech), mainstreamed (high-level improvement through individual practices), representative (balance stakeholders), multifunctional (instil quality culture, roadmap of actions, external recognition)
- The ICDE report found a good degree of consensus across commonly used frameworks They include broad issues: Strategic planning and development, curriculum design as well as course design and delivery, and support available, both to students and staff. So a very wide ranging view necessary to assure quality, not just scorecard of product
- That is echoed in European standards and guidelines which apply to QA across Europe These are 10 standards in ESG to do with internal QA Apply to all modes of delivery – face to face and distance/online Bold shows where additional guidance and indicators for e-learning might be needed -- they align roughly with the areas picked out in ICDE study
- Output from ENQA working group has just been published It supplements the ESG with some additional guidance for e-learning but standards themselves are not changed.
- I’m now going to focus on one specific framework, E-xcellence
- E-xcellence is a project about quality in e-learning in Higher Education that has been around for over 10 years. Provides a well-tested framework for thinking about quality in e-learning. There are resources on the website. There is set of benchmarks which set out what good e-learning looks like. These are captured in a manual which has a lot of useful background.
- There are six chapters which reflect broad areas of concern seen in ICDE report 35 benchmarks in total
- Here is a sample benchmark Benchmark = statement of best practice in most institutions Note they are very general, which allows each institution to do things their own way. The institution needs to provide evidence to show how they measure up to each benchmark.
- More detailed indicators for benchmarks. Examples of good and excellent practice Suggest the kinds of evidence that would support achieving a benchmark But each university may approach things differently, so other evidence is ok Not a scorecard!
- Benchmarking as quality enhancement tool Statements of good practice for comparison Identification of weaknesses & strengths by collecting evidence roadmap for improvement
- E-xcellence is very flexible. Full process (full self-assessment, external review, roadmap for actions) leads to a E-xcellence label But can be used informally and resources freely available, so you don’t have to commit to full process.
- I want to mention some experiences from E-xcellence
- Firstly, the process is valued by those who have done it. This is a summary of feedback from participants in E-xcellence reviews
- It is interesting to look back at reviews to see where universities have faced issues Looking at broad topic areas (chapters) Most issues in strategic management, curriculum design and staff support. Fewer in Course design and delivery, Fewest in Student support – need to come back to that because good student support is essential for student success
- Workload management (Staff support) highest E-learning strategy (Strategic management) Development of academic communities (Curriculum design) / Social media (Student support) Course design & delivery generally low but provision of good interactive tools is a concern Somethings aren’t seen as problems -- that includes Student support which isn’t seen as problematic, but lots of activity – understood to be important, but under control
- I want to move on specifically to MOOCs
- MOOCs are not part of ‘normal’ university teaching and they are free, so should we pay much attention to quality? Does it matter if a MOOC isn’t good? Yes – several stakeholders involved, all have an interest
- MOOCs are different from ‘normal’ HE and maybe from ‘normal’ e-learning. MOOCs are free, open (needing no prior qualifications), typically short – unlike degree course Recent work says that MOOC participants’ motivations are very different from ‘normal’ student. They may not be interested in completing, just dip in to find something they need or that interests them, skip the rest They may not think not completing is a failure But we design a MOOC as a ‘course’ (not a book, not a ‘resource’) It has a beginning and end, assessment, so ‘completion’ must be the teacher’s intention So maybe should judge similarly to other courses.
- OpenupEd is a European portal for MOOCs. Not a platform, but a way to gather MOOCs which offer a good quality experience OpenupEd quality label is derived from E-xcellence so it provides a framework for thinking about quality of MOOCs in an organised way. The materials are freely available for use in self-assessment
- OpenupEd expects MOOCs to support these distinctive features or values. They are felt to be important for a good educational experience
- The OpenupEd benchmarks are derived from E-xcellence. So they are well-tested. Many apply to the whole institution – they can be checked once and then just revisited every few years So for each new MOOC, a much smaller number and less effort required.
- Here are some benchmarks at the course level These are the ones that need checking for each MOOC. Mainly straightforward to judge.
- To help an initial quick self-assessment, there is a table to fill in. This is the course level – fits on to a single sheet of paper! List of benchmarks
- Scale – is the benchmark not achieved, partially achieved, largely achieved, full achieved? This is only for a quick self-assessment – will need to document evidence more fully Not a scorecard! – this is to prompt roadmap for improvement
- Mapping to OpenupEd features – evidence for a benchmark often is also evidence for an OpenupEd feature No extra work needed to check
- Something different about MOOCs is there is often a split between a university and platform provider. For example, a MOOC may be written at the Open University (university) and published on FutureLearn (platform provider) -- different people, different systems. So can OpenupEd work in that situation? Yes – quality emerges from joint efforts so evidence has to come from both partners. Again we see that a concern for quality is deeply embedded.
- I said I wouldn’t give a recipe but…
- It helps to think about the design of a course as a whole. This is a tool used as part of a learning design process at the OU – but there are other tools out there It helps you build up an overall picture of what a student will experience
- As a teacher you can construct a course from many different types of activity. It helps to see them in in broad classes
- This view lets you plan activities over time You can see at this stage the course is maybe a little out of balance. -- there is a lot of time spent doing assimilative activity, but almost nothing in communication & collaboration -- on the right the weekly workload is shown and that looks uneven
- This shows an example of how one course design changed. Blue shows the shape of the course at a very early stage of planning. Then there was a workshop where the team got together to look at the overall learning design. The orange shows what it looks like as a result of that
- You can see that the course team decided to reduce the time spent on assimilative activities And increase the time spent on finding and handling information and on communication and collaboration (other changes also) -- encourage the student to be more active in their learning
- A good reason for encouraging collaborative learning – student success is higher if courses are designed with communicative activities. Students love receiving lots of ‘stuff’ which they work through alone and they dislike collaborating with other students. So courses with high proportion of assimilation are popular, but students engage less well over time and may not succeed.
- So be careful of using surveys which ask students about satisfaction! Many modules, which vary in student success (vertical) and student satisfaction (horizontal). But satisfaction is not correlated to success There are some courses (one the left) which get high satisfaction scores but low completion And others (one the right) where students are very successful – but which they hate!
- So a summary overall I believe it really helps to have a quality framework to work with It helps to create a quality culture – and that will help to improve quality There is no single recipe for good quality courses – lots of scope for innovative ideas! Just keep these points in mind which are common practice in ODL universities -- work in a team of people with different skills -- think about learning design at an early stage -- make sure there are good mechanisms for student support
- Many thanks for your attention