Electronic management of assessment - Jisc Digital Media 2015
•Descargar como PPTX, PDF•
0 recomendaciones•4,854 vistas
This session provided an opportunity to hear the findings from a landscape review, engage with the challenges, and engage actively in the shaping of solutions.
Recomendados
Recomendados
Más contenido relacionado
La actualidad más candente
La actualidad más candente (20)
Destacado
Destacado (18)
Similar a Electronic management of assessment - Jisc Digital Media 2015
Similar a Electronic management of assessment - Jisc Digital Media 2015 (20)
Más de Jisc
Más de Jisc (20)
Último
Último (20)
Electronic management of assessment - Jisc Digital Media 2015
- 2. Electronic management of assessment Maximising the benefits technology can offer
- 3. » Background and context » Headline findings from EMA landscape review » Consulting on priorities » From challenges to solutions » Solution scoping activity Overview 10/03/2015 Jisc Digital Festival, 9-10 March 2015, ICC Birmingham 3
- 4. 10/03/2015 Jisc Digital Festival, 9-10 March 2015, ICC Birmingham 4 Background and context bit.ly/jisc-assess and http://bit.ly/jiscdsaf » Background and context › Recently completed – a 3 year programme exploring technology-enhanced assessment and feedback » Four main themes: › Feedback and feed forward › Enhancingstudentemployability › Strategies for change › Electronic management of assessment (EMA)
- 5. 10/03/2015 Jisc Digital Festival, 9-10 March 2015, ICC Birmingham 5 » EMA - benefits and drivers are clear » Challenges surfacing relating to process, practice and technology » Working in collaboration with the Heads of e-Learning Forum (HeLF) and UCISA to maximise the benefits technology can offer in the management of assessment Electronic management of assessment project
- 6. 10/03/2015 Jisc Digital Festival, 9-10 March 2015, ICC Birmingham 6 » Aims: › To gather examples of practices, policies & process as well as approaches to implementation › To identify the challenges around EMA, in terms of pedagogy/culture, process or technology and to identify existing solutions where they can be found › To provide interim guidance to institutions on the state of play and effective practice with approaches to implementation › To make recommendations to institutions, systems providers and Jisc on priorities to address The EMA study
- 7. » HeLF EMA surveys » Jisc assessment and feedback programme: 30+ institutions » Online questionnaire: 70 institutions » ThinkTank: 32 institutions » Other Jisc learning and teaching related projects » Literature review » Interviews with learning providers Information sources 10/03/2015 Jisc Digital Festival, 9-10 March 2015, ICC Birmingham 7
- 8. » Nuanced responses but broad trends visible » Localised initiatives but beginning to scale up » System integration is a key problem area » EMA exposes variability in business processes Headline findings 10/03/2015 Jisc Digital Festival, 9-10 March 2015, ICC Birmingham 8
- 9. Loose system integration 10/03/2015 Jisc Digital Festival, 9-10 March 2015, ICC Birmingham 9
- 10. Variation in business processes 10/03/2015 Jisc Digital Festival, 9-10 March 2015, ICC Birmingham 10
- 11. 10/03/2015 Jisc Digital Festival, 9-10 March 2015, ICC Birmingham 11 What are the pain points in EMA? Pain points
- 12. » Anonymity: both ends of spectrum motivated by student pressure » Resistance: many stakeholders take narrow view; little joined-up thinking » Scaling up: may expose technology issues & ‘harden’ staff attitudes » Feedback: storage& access issues; good practice can be hard to scale » Practice: may be simple but process workarounds cause frustration and resentment » Technology:limited range of core productshow hard can it really be? » Formative approaches: less well developed than summative What causes the pain? 10/03/2015 Jisc Digital Festival, 9-10 March 2015, ICC Birmingham 12
- 13. A lifecycle model 10/03/2015 Jisc Digital Festival, 9-10 March 2015, ICC Birmingham 13
- 14. Priorities 1 10/03/2015 Jisc Digital Festival, 9-10 March 2015, ICC Birmingham 14 Top 10 EMA challenges
- 15. Priorities 2 10/03/2015 Jisc Digital Festival, 9-10 March 2015, ICC Birmingham 15 Top 20 EMA challenges
- 16. 10/03/2015 Jisc Digital Festival, 9-10 March 2015, ICC Birmingham 16 » Review of the EMA landscape acrossUKHE andFE completed » Identified and prioritised the challenges around EMA » Workshops in Dec 2014 and Jan 2015 explored possible solutions to these prioritised challenges Where are we now? ema.jiscinvolve.org/
- 17. Where next? 10/03/2015 Jisc Digital Festival, 9-10 March 2015, ICC Birmingham 17 Phase 2.1 Oct – Aug 2015 Second phase of consultation, refining and prioritising challenges (speaking as a ‘single voice’) Moving from challenges to solutions (Oct – Jan) Solution development (Feb – July) Moving from challenge to solutions – 4 projects going forward Ongoing capture of resources – ‘EMA toolkit’ Landscape review – challenges prioritised
- 18. » EMA Requirements Map » Feedback hub » Reliable submissions » EMA systems integration web resource » Assessment & feedback toolkit Four projects 10/03/2015 Jisc Digital Festival, 9-10 March 2015, ICC Birmingham 18
- 19. 10/03/2015 Jisc Digital Festival, 9-10 March 2015, ICC Birmingham 19 » Project to identify, validate and specify key EMA requirements and workflows, through development of generic models that represent the diversity of practice » Providing clarity and transparency on workflows(particularlyfrom submission to return of grades). Helps systems suppliers better support good pedagogic practice and helps institutions review their own practice EMA requirements map
- 20. 10/03/2015 Jisc Digital Festival, 9-10 March 2015, ICC Birmingham 20 » Develop a visual way of presenting workflows; map current systems to workflows; and engage with suppliers and developers to fill gaps EMA requirements map
- 21. 10/03/2015 Jisc Digital Festival, 9-10 March 2015, ICC Birmingham 21 » System independent tool for aggregating, organising and presenting feedback at a programme level, for both staff and students. Also enables interaction around feedback between staff and students Feedback hub
- 22. 10/03/2015 Jisc Digital Festival, 9-10 March 2015, ICC Birmingham 22 » Studentsbenefit from aggregated view of their feedback to support self-reflection on progress; lecturers see a more holistic view of students’ progress and better understand an individual’s progression to identify where intervention or support is needed Feedback hub
- 23. 10/03/2015 Jisc Digital Festival, 9-10 March 2015, ICC Birmingham 23 » Resource to address lack of interoperability between marking systems and student records systems and subsequent need for ‘workarounds’ by staff » Helps institutions find solutions to EMA systems integrations issues via improved workflows and by sharing existing integrations (use cases and code) Systems integration
- 24. 10/03/2015 Jisc Digital Festival, 9-10 March 2015, ICC Birmingham 24 » Gaps can be surfaced and prioritised for potential development » A community of practice might support this resource Systems integration
- 25. 10/03/2015 Jisc Digital Festival, 9-10 March 2015, ICC Birmingham 25 » Web-based toolkit of resources: practices, processes, policies, technical integrations etc based around the assessment and feedback lifecycle » Examples of solutions , enabled by technology, based on pedagogy and underpinned by research, that can be repurposed for localcontexts EMA toolkit
- 26. 10/03/2015 Jisc Digital Festival, 9-10 March 2015, ICC Birmingham 26 » Addresses the question of ‘what does good assessment design look like?’ and enhances assessment literacies of staff and students » Navigation routes through the toolkit e.g. by identified workflow and stakeholder groups (academic staff, administrative staff, student) EMA toolkit
- 27. » Choose one project area to work on » The aim is to help to shape the detail of these projects and your facilitator will provide guidance. Discuss: › What would you like to see from this project and its outputs? › What form should the outputs take? › How might you use the outputs within your context? › What do you have that you can contribute? › How can we best reflect the needs of HE and FE within this project? Activity 10/03/2015 Jisc Digital Festival, 9-10 March 2015, ICC Birmingham 27
- 28. 10/03/2015 Jisc Digital Festival, 9-10 March 2015, ICC Birmingham 28 » Landscape report available from: bit.ly/jisc-ema » Join the conversation on the blog: ema.jiscinvolve.org/ and on twitter #jiscassess » Join the mailing list: www.jiscmail.ac.uk/tech- enhanced-assessment » Guides, case studies, videos and other resources on assessment and feedback: bit.ly/jisc-assess Find out more
- 29. Find out more… Contact… Lisa Gray Senior Co-Design manager lisa.gray@jisc.ac.uk Gill Ferrell, Consultant gill@aspire-edu.org
Notas del editor
- +++++ COVER: OPTION 2 ++++++ +++++ PLEASE FEEL FREE TO DELETE THESE NOTES WHILST CREATING YOUR PRESENTATION +++++++
- LG So, the benefits and drivers were becoming clear, but a number of issues were surfacing through a number of forums including the HeLF sig survey work and sig meetings. So earlier this year, the project started, in collaboration with the Heads of e-Learning Forum and UCISA. We aim to maximise the benefits technology can offer in the management of assessment, through: Refining our understanding the end to end assessment and feedback lifecycle across UK HE (and FE), refining our understanding of how technology does and could support this lifecycle, identifying and sharing approaches to adoption (including what is working well and what isn’t), and provide guidance on effective practice in this space.
- LG The study built on a number of previous initiatives, as well as exploring new research. Previous initiatives included the detailed research undertaken by the Heads of e-Learning Forum of their members 2011, 2012, 2013; research undertaken by Jisc projects, particularly through the Assessment and Feedback programme over the last 3 years; Building on that, the study undertook a literature review, as well as inviting interested parties to respond to a set of questions online. We had responses from over 70 institutions, mainly from HE but also from FE delivering HE from all four nations of the UK. We brought people together in a ‘Think Tank’ event in May this year, where 32 institutions were represented from those that had responded online. Further more detailed interviews were also undertaken with a number of specific practitioners. A survey is currently in development to further elicit from FE more broadly the state of play there, the results of which will feed into the project as it progresses.
- GF More people said they had completely separate systems than highly integrated. Only 3 univs said highly integrated – rest were HE in FE providers.
- GF 2 sides of same coin – harder to integrate systems (& design effective workflows) when business processes are very varied. Those who answered highly standardised tended on the whole to be quite small/specialist. The devil is in the detail – the free text comments give a lot more insight on this. Many people don’t realise the extent of process variability till they come to implement systems. Some institutions have ostensibly standardised processes but there are a lot of differences in local interpretation e.g. What constitutes a ‘late submission’?
- GF Anonymity Technology: SITS/Blackboard/Turnitin (20) and SITS/Moodle/Turnitin (15) accounted for half of respondents. Are any of the problems down to clarity of sector requirements – could we do more to speak with one voice? We know there are problems with reliability of Turnitin service: the more you go down the EMA route the more serious outages become.
- GF Explain pedagogical basis of model & how it has been adapted & used elsewhere. Relevant to HE and FE.
- LG Report launched in July this year. We also have a range of additional resources on wider topics with assessment and feedback, including four themed guides, case studies and videos on employability and assessment; feedback and feedforward; and change strategies. For more examples, see the Jisc Design Studio for all projects and themes.