Workshop for Warsash Maritime Academy Colleagues

1. Assessment approaches that work:
Evidence, theory, principles
@solentlearning
@tansyjtweets

2. The theory…
Assessment drives what students pay attention to,
and defines the actual curriculum
(Ramsden 1992).

4. Three questions
• Why do some/many students not collect, value
or use our feedback?
• How do students come to know and appreciate
what ‘good’ work looks like?
• Why should I bother with formative assessment
and feedback if students don’t see its value? Is it
worth the candle?

5. Chat to one another about…
• …feedback which helped
you to grow and see new
angles
• ….feedback which landed
like a lead balloon
• …why the difference?

7. In theory…
Feedback is the single most important factor
in learning
(Hattie 2009).
Formative feedback contributes to significant
learning gains
(Black and Wiliam 1998).

8. It was heavy, tons of marking for
the tutor. It was such hard work.
It was criminal.
Media Course Leader
I’m really bad at reading
feedback. I’ll look at the mark
and then be like ‘well stuff it, I
can’t do anything about it’
Student, TESTA focus group
In practice…

9. A story of disconnection

10. What students say
It’s difficult because your assignments are so detached from
the next one you do for that subject. They don’t relate to
each other.
Because it’s at the end of the module, it doesn’t feed into
our future work.
Because they have to mark so many that our essay becomes
lost in the sea that they have to mark.
It was like ‘Who’s Holly?’ It’s that relationship where you’re
just a student.

11. Structural disconnection

12. Mass HE is squeezing out dialogue
with the result that written feedback,
which is essentially a one-way communication,
has to carry almost all the burden of
student-teacher interaction
David Nicol 2010

13. Feedback as dialogue?

14. An impoverished dialogue
The many diverse
expressions of
dissatisfaction with
feedback can be taken as
symptoms of an
impoverished and
fractured dialogue
David Nicol 2010

15. A two-way dialogue?
Your essay lacked structure and
your referencing is problematic
Your classes are boring and I
don’t really like you 

16. Emotions and feedback
It’s always the negatives you remember… we
hardly ever pick out the really positive points
because once you’ve seen the negative, the
negatives can outweigh the positives.
I feel physically sick handing in an assignment. I
can’t sleep for days before because I panic that
it’s not right and it’s so pathetic.

18. Avoiding triggering emotions
They just pacify really. I went for help and they
just told me what I wanted to hear, not what I
needed to know.
Its very positive like nobody ever says ‘no you’ve
done that completely wrong’. It's always 'You've
done that very well‘. Well why have a got a low
grade then? It doesn’t really help you from
there.

19. Pigeon-holing students

20. It told you some of the problems but it doesn’t tell
you how you can manage to fix that. It was,
“Well, this is the problem.” I was like, “How do I fix
it?” They said, “Well, some people are just not
good at writing.”
(TESTA Focus group data)

21. Here are some ways
in which you can
improve…
Some people are
just not good at
writing…

22. A few questions
• How do you take criticism? What’s your
instinctive response to critical feedback?
• What helps you to take critical feedback?
• How can you use your experience to help
students use feedback to grow and develop?

23. Ways to engage students
• Conversation: who starts the dialogue?
• Cycles of reflection across modules
• Give quick generic feedback
• Motivate with quick feedback
• Get students involved – generating feedback
• Peer feedback (especially on formative)
• Technology: audio, screencast and blogging
• From feedback as ‘telling’…
• … to feedback as asking questions

24. Take student feedback seriously

25. Students to lecturers:
Critical Incident Questionnaire
Stephen Brookfield’s Critical Incident Questionnaire http://bit.ly/1loUzq0

26. How do students come to know what
‘good’ work looks like?
Go to www.menti.com & use the code 61 37 58
Vote on three options which you consider to be
most important

27. How do students come to know what good
looks like?

28. What students say
We’ve got two tutors- one marks completely differently to
the other and it’s pot luck which one you get.
They have different criteria, they build up their own criteria.
It’s such a guessing game.... You don’t know what they
expect from you.
They read the essay and then they get a general impression,
then they pluck a mark from the air.

29. What’s going wrong here?
There are criteria, but I find them really strange.
There’s “writing coherently, making sure the argument
that you present is backed up with evidence”.
Q: If you could change one thing to improve what
would it be?
A: More consistent marking, more consistency across
everything and that they would talk to each other.

30. What the papers say…
https://www.timeshighereducation.co.uk/news/examiners-give-hugely-different-
marks/2019946.article

31. Marking is important. The grades we give students
and the decisions we make about whether they pass
or fail coursework and examinations are at the heart
of our academic standards
(Bloxham, Boyd and Orr 2011).
Grades matter (Sadler 2009).

32. QAA: a paradigm of accountability?
• Learning outcomes
• Criteria-based learning
• Meticulous specification
• Written discourse
• Generic discourse (Woolf 2004)
• ‘Validating practices’ (Shay 2004)
• Transparent to staff and students
• Intended to reduce the arbitrariness of staff
decisions (Sadler 2009).

33. Having ‘an eye for a dog’

34. The Art and Science of Evaluation
Judging is both an art and a science: It is an art
because the decisions with which a judge is
constantly faced are very often based on
considerations of an intangible nature that cannot
be recognized intuitively. It is also a science because
without a sound knowledge of a dog’s points and
anatomy, a judge cannot make a proper assessment
of it whether it is standing or in motion.
Take them round please: the art of judging dogs (Horner, T
1975).

35. Implicit
Criteria
Explicit
Written
I justify
Co-creation
and
participation
Active
engagement
by students

36. Taking action: internalising goals and
standards
• Regular calibration exercises
• Discussion and dialogue
• Discipline specific criteria (no cut and paste)
Lecturers
• Rewrite/co-create criteria
• Marking exercises
• Discussing exemplars
Lecturers
and students
• Enter secret garden - peer review
• Engage in drafting processes
• Self-reflection
Students

37. Why bother with formative?
1) Low-risk opportunities for students to learn from
feedback (Sadler, 1989)
2) Helps students to fine-tune and understand
requirements and standards (Boud 2000, Nicol, 2006)
3) Feedback to lecturers from formative tasks helps to
adapt teaching (Hattie, 2009)
4) Engages students in cycles of reflection and
collaboration (Biggs 2003; Nicol & McFarlane Dick 2006)
5) Encourages and distributes student effort (Gibbs 2004).

38. What is formative assessment?
Formative assessment is concerned with how
judgements…can be used to shape and improve
students’ competence by short-circuiting the
randomness and inefficiency of trial-and-error
learning (Sadler, 1989, p.120).
Definitional fuzziness (Yorke, 2003)
TESTA – ungraded, required, eliciting feedback

39. Students say formative helps…
• It was really useful. We were assessed on it but we
weren’t officially given a grade, but they did give us
feedback on how we did.
• It didn’t actually count so that helped quite a lot
because it was just a practice and didn’t really
matter what we did and we could learn from
mistakes so that was quite useful.

40. But…
• If there weren’t loads of other assessments, I’d do it.
• If there are no actual consequences of not doing it,
most students are going to sit in the bar.
• It’s good to know you’re being graded because you
take it more seriously.
• The lecturers do formative assessment but we don’t
get any feedback on it.

41. So, how do we do it?
Five case studies of
successful formative
Identify principles that make
them work
How could you adapt them?

42. Case Study 1: Business School
• Reduction from average 2 x summative, zero
formative per module
• …to 1 x summative and 3 x formative
• Required by students in entire business school
• Systematic shift, experimentation, less risky
together

43. Case Study 2: Social Sciences
• Problem: silent seminar, students not reading
• Public platform blogging
• Current academic texts
• In-class
• Threads and live discussion
• Linked to summative

44. Case study 3: Film and TV
• Seminar
• Problem: lack of discrimination about sources
• Students bring 1 x book, 1 x chapter, 1 x
journal article, 2 x pop culture articles
• Justify choices to group
• Reach consensus about five best sources

45. Case Study 4: Media degree
• Media degree
• Presentations formative
• Students get feedback (peer and tutor)
• Refines their thinking for…
• Linked summative essay

46. Case study 5: Engineering
• Engineering
• Problem low averages
• Course requirement to complete 50 problems
• Peer assessed in six ‘lecture’ slots
• Marks do not count
• Lectures, problems, classes, exams unchanged
• Exam marks increased from 45% to 85%

47. Your task
• In groups, identify five principles for making
formative work. Write them down on flipchart
paper.
• Devise one or two adaptations for your
discipline, using the principles, and make one
poster which outlines/draws your adaptation.
You can be creative!

48. Your principles and adaptations

49. Learning-oriented summative?
https://www.youtube.com/watch?v=ZVFwQzlVFy0

50. References
Barlow, A. and Jessop, T. 2016. “You can’t write a load of rubbish”: Why blogging works as formative
assessment. Educational Developments. 17(3), 12-15. SEDA.
Bloxham, S. , P. Boyd, and Orr S. (2011) Mark my words: the role of assessment criteria in UK higher education
practices. Studies in Higher Education. 36.6. 655-670.
Boud, D. and Molloy, E. (2013) ‘Rethinking models of feedback for learning: The challenge of
design’, Assessment & Evaluation in Higher Education, 38(6), pp. 698–712.
Gibbs, G. & Simpson, C. (2004) Conditions which assessment supports students' learning. Learning and
Teaching in Higher Education. 1(1): 3-31.
Harland, T., McLean, A., Wass, R., Miller, E. and Sim, K. N. (2015) ‘An assessment arms race and its fallout:
High-stakes grading and the case for slow scholarship’, Assessment & Evaluation in Higher Education 40(4)
528-541.
Jessop, T. and Tomas, C. 2016 The implications of programme assessment on student learning. Assessment
and Evaluation in Higher Education. Published online 2 August 2016.
Jessop, T. and Maleckar, B. (2014). The Influence of disciplinary assessment patterns on student learning: a
comparative study. Studies in Higher Education. Studies in Higher Education. 41(4) 696-711.
Jessop, T. , El Hakim, Y. and Gibbs, G. (2014) The whole is greater than the sum of its parts: a large-scale
study of students’ learning in response to different assessment patterns. Assessment and Evaluation in
Higher Education. 39(1) 73-88.
Nicol, D. (2010) From monologue to dialogue: improving written feedback processes in mass higher
education, Assessment & Evaluation in Higher Education, 35: 5, 501 – 517.
O'Donovan, B , Price, M. and Rust, C. (2008) 'Developing student understanding of assessment standards: a
nested hierarchy of approaches', Teaching in Higher Education, 13: 2, 205 — 217
Sadler, D. R. (1989) ‘Formative assessment and the design of instructional systems’, Instructional Science,
18(2), pp. 119–144.
Sadler, R. (2009) Indeterminacy in the use of preset criteria for assessment and grading, Assessment &
Evaluation in Higher Education, 34:2, 159-179.
Yorke, M. (2003) Formative assessment in higher education: Moves towards theory and the enhancement of
pedagogic practice. Higher Education. 45