Workshop at University College Dublin

1. Developing assessment patterns
that work through TESTA
Professor Tansy Jessop
TESTA Workshop
University College Dublin
10 February 2017

2. The plan for this session
1. Brief overview of TESTA
2. Two case study puzzles
3. Assessment patterns which help
students to learn

4. Mixed methods approach
Programme
Team
Meeting
Assessment
Experience
Questionnaire
(AEQ)
TESTA
Programme
Audit
Student
Focus Groups

5. Growth of TESTAWidespread use

6. TESTA….
“…is a way of thinking
about assessment and
feedback”
Graham Gibbs

7. It enables you to see the whole elephant

8. Defining the terms
• Summative assessment carries a grade which
counts toward the degree classification.
• Formative assessment does not count
towards the degree (either pass/fail or a
grade), elicits comments and is required to be
done by all students.

9. TESTA Case Study X: what’s going on?
• Plenty of varieties of assessment, no exams
• Reasonable amount of formative assessment (14 x)
• 33 summative assessments
• Masses of written feedback on assignments (15,000 words)
• Learning outcomes and criteria clearly specified
….looks like a ‘model’ assessment environment
But students:
• Don’t put in a lot of effort & distribute their effort across few topics
• Don’t think there is a lot of feedback or that it very useful, and
don’t make use of it
• Don’t think it is at all clear what the goals and standards are
• …are unhappy

10. Case Study Y: what’s going on?
• 35 summative assessments
• No specified formative assessment
• Learning outcomes and criteria wordy and woolly
• Marking by connoisseurship, tacit, professional judgements
….looks like a problematic assessment environment
But students:
• Put in a lot of effort and distribute their effort across topics
• Have a very clear idea of goals and standards
• Are able to evaluate their work and have a good idea of how to
‘close the gap’

11. Assessment patterns which work
1. Part of principled whole curriculum design
2. Plenty of formative assessment
3. Students distribute effort and spend time on task
4. Challenging and high expectations
5. Feedback connects across modules
6. Students interact with and internalise standards

12. Assessment patterns (n=73 programmes)
Characteristic Low Medium High
Volume of summative
assessment
Below 33 40-48 More than 48
Volume of formative only Below 1 5-19 More than 19
% of tasks by examinations Below 11% 22-31% More than 31%
Variety of assessment
methods
Below 8 11-15 More than 15
Written feedback in words Less than 3,800 6,000-7,600 More than 7,600

13. Deep and Surface Learning (Marton and
Saljo (1976)
Deep Learning
• Meaning
• Concepts
• Active learning
• Generating knowledge
• Relationship new and
previous knowledge
• Real-world learning
Surface Learning
• External purpose
• Topics
• Passive process
• Reproducing knowledge
• Isolated and
disconnected knowledge
• Artificial learning

14. • High summative on most UK, NZ and Indian degrees
(range 12 -227 in UK)
• Pedagogies of control
• Low formative: ratio of 1:8 summative to formative
• Weakly practised and understood
1. Curriculum design question

15. Does IKEA 101 work for complex learning?

16. ..has led to over-emphasising knowing
(Barnett and Coate 2005)
• Knowing is about content
• Acting is about becoming
a historian, actor,
psychologist, or
philosopher
• Being is about
understanding yourself,
orienting yourself and
relating your knowledge
and action to the world
Knowing
Being
Acting

17. The best approach from the student’s perspective is to focus
on concepts. I’m sorry to break it to you, but your students are
not going to remember 90 per cent – possibly 99 per cent – of
what you teach them unless it’s conceptual…. when broad,
over-arching connections are made, education occurs. Most
details are only a necessary means to that end.
http://www.timeshighereducation.co.uk/features/a-students-
lecture-to-rofessors/2013238.fullarticle#.U3orx_f9xWc.twitter
A student’s lecture to her professor

18. Assessment is mainly sort of the topical knowledge and
the topics never relate. We'll never do something again
that we’ve already studied, like we learn something and
then just move on
(TESTA focus group data).
…and fragmenting content…

19. 2. Plenty of formative but…
“Innovations that include strengthening the practice of
formative assessment produce significant and often
substantial learning gains”
(Black and Wiliam, 1998, p.40).

20. Take five
• Which quote strikes you
most and why?
• What do students value
about formative
assessment?
• What are the challenges
of doing formative?

21. It was just a practice and didn’t really matter what we did and we
could learn from mistakes so that was quite useful.
He’s such a better essay writer because he’s constantly writing. And
we don’t, especially in the first year when we really don’t have
anything to do. The amount of times formative assignments could
have taken place…
It’s good to know you’re being graded because you take it more
seriously.
I would probably work for tasks, but for a lot of people, if it’s not
going to count towards your degree, why bother?
Students talking about formative

22. 3 & 4. Effort and challenging tasks
• Chickering and Gamson (1987)
• Time-on-task (Gibbs 2004)
• Arum and Roksa (2011)

23. The problem is that…
• Summative drives student effort
• ‘Pedagogies of control’
• Too much marking for staff, too little effort by
students
• Assess less, learn more
• Measure less, learn more

24. Typical student effort graph

25. Almost 50% of students in
USA leave with no
significant gains in
academic writing, complex
reasoning and critical
thinking skills…
(Arum and Roksa, 2011)

26. Significant learning gains
for students who
1) Read > 40 pages a
week of academic
writing
2) Write > 20 pages per
semester for each unit

27. From this educational paradigm…

28. Transmission Model

29. Social Constructivist Model

30. References
Arum, R and Roksa, J. (2011) Academically adrift: Limited Learning on College Campuses. University of
Chicago Press.
Boud, D. and Molloy, E. (2013) ‘Rethinking models of feedback for learning: The challenge of
design’, Assessment & Evaluation in Higher Education, 38(6), pp. 698–712. doi:
10.1080/02602938.2012.691462.
Gibbs, G. & Simpson, C. (2004) Conditions r which assessment supports students' learning. Learning and
Teaching in Higher Education. 1(1): 3-31.
Harland, T., McLean, A., Wass, R., Miller, E. and Sim, K. N. (2014) ‘An assessment arms race and its
fallout: High-stakes grading and the case for slow scholarship’, Assessment & Evaluation in Higher
Jessop, T. and Tomas, C. 2016 The implications of programme assessment on student learning.
Assessment and Evaluation in Higher Education. Published online 2 August 2016.
Jessop, T. and Maleckar, B. (2014). The Influence of disciplinary assessment patterns on student
learning: a comparative study. Studies in Higher Education. Published Online 27 August 2014
http://www.tandfonline.com/doi/abs/10.1080/03075079.2014.943170
Jessop, T. , El Hakim, Y. and Gibbs, G. (2014) The whole is greater than the sum of its parts: a large-scale
study of students’ learning in response to different assessment patterns. Assessment and Evaluation in
Higher Education. 39(1) 73-88.
Nicol, D. (2010) From monologue to dialogue: improving written feedback processes in mass higher
education, Assessment & Evaluation in Higher Education, 35: 5, 501 – 517.
O'Donovan, B , Price, M. and Rust, C. (2008) 'Developing student understanding of assessment
standards: a nested hierarchy of approaches', Teaching in Higher Education, 13: 2, 205 — 217
Sadler, D. R. (1989) ‘Formative assessment and the design of instructional systems’, Instructional
Science, 18(2), pp. 119–144. doi: 10.1007/bf00117714.