What works for Higher Education assessment, and what do we wish we could have in Higher Education assessment Terese Bird keynote at Assessment on Tour London 2019.

1. Confidential
Terese Bird, Educational Designer, Leicester Medical School
ExamSoft - Liftupp Assessment On Tour, London
25-26 November 2019
Star Trek or Minority Report: Assessment and feedback demands, trends, and futures

2. Confidential 2
A bit about me and Leicester Medical School Context

3. Confidential 3
• Feedback has been referred to as
perhaps the single most important
factor in learning. (Rowntree, 1987)
• National Student Survey specifically
asks about feedback
• Students are confused what feedback
is, or should be (National Union
Students, 2018)
• Students don’t always like feedback
Students and Feedback: Love/Hate
Courtesy of micaeltatto on Flickr CC-By 2.0
https://www.flickr.com/photos/micaeltattoo/6144634043

4. Confidential 4
Assessment is student
learning health-check
which is…
• Authentic
• Situated
• Cross – contextual
• Little and often, low-stakes
Assessment: What’s the point?
NUS Assessment and Feedback
Benchmarking Tool
https://www.nusconnect.org.uk/resources/ass
essment-and-feedback-benchmarking-tool

5. Confidential 5
What we wish we had
Courtesy of Smithsonian Channel https://www.youtube.com/watch?v=rR8uXJmxHSE

6. Confidential 6
What we are in danger of having
Courtesy of Fandom https://www.inverse.com/article/8731-predictions-for-minority-report-season-2

7. Confidential 7
https://nypost.com/2018/05/17/
china-is-using-ai-to-keep-high-
school-students-in-line/

8. Confidential 8
Technology yes, but person-centred – a conversation (Laurillard, 2008)
Laurillard’s Conversational Framework (Laurillard, 2008)

9. Confidential 9
Self-regulated learning (Nicol & McFarlane-Dick, 2006)

10. Confidential 10
Part 1: Assessment for Theoretical Study (Bloom, et al 1956)
Courtesy of Vanderbilt CC-BY https://cft.vanderbilt.edu/guides-sub-pages/blooms-taxonomy/

11. Confidential 11
Traditional Higher Education Assessment and Feedback Cycle
(Leicester Learning Institute, 2018)
Courtesy of Leicester Learning Institute, University of Leicester https://www2.le.ac.uk/offices/lli/developing-
learning-and-teaching/assessment-and-feedback/assessment-and-feedback-lifecycle

12. Confidential 12
Traditional Higher Education Assessment and Feedback Cycle
(Leicester Learning Institute, 2018)
Courtesy of Leicester Learning Institute, University of Leicester https://www2.le.ac.uk/offices/lli/developing-
learning-and-teaching/assessment-and-feedback/assessment-and-feedback-lifecycle
• May not have enough time
to correct your direction
before end of term finals
• More often, little feedback
is better than one big
chunk of feedback

13. Confidential 13
My Tech-Enhanced Assessment & Feedback Dissertation Research

14. Confidential 14
• Do students who engage with the regular formative feedback in the GI unit change
their study strategy as a result, and if so in what way(s)?
• Do they look at other material they would not have looked at if they hadn’t used the
exams?
• What new study techniques, if any, did they employ?
• Do students who engage with regular formative exam feedback improve learning in
“thinking levels” associated with higher elements of Bloom’s Taxonomy?
• Does student engagement with regular formative exam feedback enable students’
learning benefits in ways that do not occur when the regular formative exams are not
given?
• Do students perceive learning benefits enough to engage with structured formative
assessment when it is optional?
Research Questions

15. Confidential 15
Weekly quizzes, instant answers with additional information

16. Confidential 16
Students log into portal to check progress

17. Confidential 17
• “A mini-revision before the proper revision” – forced earlier
study
• Uncovered trouble spots which caused them to seek
solutions
• Back-and-forth communication = feedback loop
• More active study than simply re-reading lecture notes
• Learning objectives and assessments should
constructively align “higher on the Bloom’s” (Biggs, 1996)
• Students engaged despite the fact these exams
contributed nothing to their official marks
• Single Best Answer led to Evaluation and Analysis
• Students reacted with Create to using exams in revision
• Created document of info on uncovered trouble spots
• Created answers when they tested themselves without
the multiple-choice
Findings: How did the formatives change students’ study?

18. Confidential 18
• Key words used by
students describing the
formative exams in module
evaluation Winter 2016 –
unsolicited!
Synthesis of key words in the 17 comments
which mentioned the ExamSoft formative exams
within Question 15 of survey of students’
evaluation of GI unit, Winter 2016. Total number
of comments: 26

19. Confidential 19
Part 2: Assessment for Practical / Clinical study (Miller, 1990)

20. Confidential 20
Clinical / Practical Assessment:
supportive, authentic, timely, longitudinal, cross-contextual, objective

21. Confidential 21

22. Confidential 22
• Mobile and offline
• Quickly understandable by busy
clinical staff
• Personal chat too, if possible.
Students find this most valuable
Clinical feedback opportunities must be:
Courtesy of Leicester Medical School YouTube
channel – All rights reserved

23. Confidential 23
The Holy Grail of Assessment & Feedback

24. Confidential
References
• Biggs, J. (1996) ‘Enhancing teaching through constructive alignment’, Higher Education, [online] Available from:
http://edukologija.vdu.lt/en/system/files/ConstrutivismAligment_Biggs_96.pdf.
• Bird, T. (2016) ‘Promoting Higher-Level Thinking: An Evaluation of Medical Students’ Responses to Structured Exam
Feedback from Formative Assessment in the Gastrointestinal Unit’ Submitted to University of Leicester School of Education
for degree of Masters in International Education
• Bloom, B. S., Englehard, M. D., Furst, E. J., Hill, W. H., Krathwohl, D. R. and Committee of College and University
Examiners (1956) ‘Taxonomy of educational objectives: The classification of educational goals: Handbook I, cognitive
domain’, New York, 16, p. 207, [online] Available from:
http://books.google.co.za/books/about/Taxonomy_of_educational_objectives.html?id=hos6AAAAIAAJ&redir_esc=ynhttp://1
18.97.161.124/perpus-fkip/Perpustakaan/Pendidikan &
Pengajaran/Taxonomy_of_Educational_Objectives__Handbook_1__Cognitive_Domain.pdf.
• Cox, B., Calder, M. and Fien, J. (2010) Teaching and Learning for a Sustainable Future, [online] Available from:
http://www.unesco.org/education/tlsf/mods/theme_d/mod23.html.
• Laurillard, D. (2008) Digital technologies and their role in achieving our educational ambitions, London, [online] Available
from: http://werelearning.com/moodle/course/view.php?id=10.
• Leicester Learning Institute (2018) Assessment and Feedback Lifecycle, Available from:
https://www2.le.ac.uk/offices/lli/developing-learning-and-teaching/assessment-and-feedback/assessment-and-feedback-
lifecycle
• Miller, G. E, ‘The Assessment of Clinical Skills/Competence/Performance’, Acad. Med. 1990; 65(9); 63-67
• Nicol, D. J. and Macfarlane‐Dick, D. (2006) ‘Formative assessment and self‐regulated learning: a model and seven
principles of good feedback practice’, Studies in Higher Education, 31(2), pp. 199–218, [online] Available from:
http://www.tandfonline.com/doi/abs/10.1080/03075070600572090.
• NUS (2015) NUS Assessment and Feedback Benchmarking Tool, Available from:
https://www.nusconnect.org.uk/resources/assessment-and-feedback-benchmarking-tool
• Reuters (2019) ‘China is using AI to keep high school students in line’, New York Post, Available from:
https://nypost.com/2018/05/17/china-is-using-ai-to-keep-high-school-students-in-line/
• Rowntree, D. (1987) Assessing Students: How shall we know them?, Revised Ed. London, Kogan Page.

25. Confidential
5001 LBJ Freeway Ste. 700
Dallas, TX 75244
+1 (214) 720-6119 | examsoft.com