This document summarizes a presentation about taking a program-wide approach to assessment called TESTA (Transforming the Experience of Students Through Assessment). The presentation discusses issues with current assessment approaches like high summative workloads and disconnected feedback. It then describes the TESTA program, which aims to address these issues through conducting program audits, using student questionnaires, and holding focus groups. Key goals of TESTA include increasing formative assessment and improving feedback cycles. The presentation provides case studies of programs that have successfully implemented TESTA and shares lessons about facilitating educational change.

1. Pigs might fly?
Changing the assessment narrative
through TESTA
Tansy Jessop
Irish National Forum for Enhancing T&L
9 April 2019
@solentlearning
@tansyjtweets

2. This session
• Rationale for taking a programme approach
• Brief description of TESTA
• Methods and themes
• Change process
• Programme assessment: spot the flying pigs

3. Why take a programme approach?
1. A modular problem
2. A curriculum problem
3. An alienation problem
4. An engagement solution

4. A modular problem

5. A curriculum problem

6. An alienation problem
Image, "Alienation Nightmare" © 1996 by Sabu

7. An engagement solution?
More
engaging
formative
Less
measuring
Students
learning
more
Curriculum
less stuffed

8. Programme approach seems to
improve things
60%
65%
70%
75%
80%
85%
90%
95%
Q5 Q6 Q7 Q8 Q9 OS
AVERAGENSSSCORES
COMPARISON OF 32 PROGS IN 13 UNIVERSITIES WITH SECTOR SCORES
NSS 2015 SCORES TESTA SCORES

10. Research and change process
Programme
Team
Meeting
Assessment
Experience
Questionnaire
(AEQ)
TESTA
Programme
Audit
Student
Focus Groups

11. TESTA definitions
Summative:
graded assessment which counts towards the degree
Formative:
Does not count: ungraded, required task with
feedback

12. TESTA Audit: the nuts and bolts…
How do I do a
TESTA audit?
What will the data
tell me about the
assessment on the
programme?

13. Activity One: mock audit
Programme
Team
Meeting
Assessment
Experience
Questionnaire
(AEQ)
TESTA
Programme
Audit
Student
Focus Groups

14. The Audit: Caveats
1. Audit is not everything
2. Official discourse
3. Planned curriculum
4. Some better data, some weaker, some gaps

15. Mock Audit

16. • Some context
• Number of summative
• Number of formative
• Varieties of assessment
• Proportion of exams
• Written feedback
• Speed of return of
feedback
Summary of audit data

17. How does this compare with your
context?

18. Typical A&F patterns
73 programmes in 14 unis (Jessop and Tomas 2017)
Characteristic Low Medium High
Volume of summative
assessment
Below 33 40-48 More than 48
Volume of formative only Below 1 5-19 More than 19
% of tasks by examinations Below 11% 22-31% More than
31%
Variety of assessment
methods
Below 8 11-15 More than 15
Written feedback in words Less than
3,800
6,000-7,600 More than
7,600

19. Theme 1:
High summative with low formative
• Low formative to summative ratio of 1:8 (UK)
• Summative as ‘pedagogy of control’
• Formative weakly practised and understood

20. Assessment Arms Race

21. A lot of people don’t do wider
reading. You just focus on your
essay question.
In Weeks 9 to 12 there is hardly
anyone in our lectures. I'd rather
use those two hours of lectures
to get the assignment done.
It’s been non-stop
assignments, and I’m now
free of assignments until
the exams – I’ve had to
rush every piece of work
I’ve done.
CONSEQUENCES
OF HIGH
SUMMATIVE

22. It was really useful. We
were assessed on it but we
weren’t officially given a
grade, but they did give us
feedback on how we did.
It didn’t actually count so
that helped quite a lot
because it was just a
practice and didn’t really
matter what we did and we
could learn from mistakes
so that was quite useful.
The benefits
of formative

23. If there weren’t loads
of other assessments,
I’d do it.
It’s good to know you’re
being graded because
you take it more
seriously.
BUT… If there are no actual
consequences of not doing
it, most students are going
to sit in the bar.
The lecturers do formative
assessment but we don’t get
any feedback on it.

24. Formative is the hardest nut to crack…
Go to www.menti.com and use the code 97 97 66
Type in three reasons why students may be
reluctant to invest time and energy in completing
formative assessment tasks

25. 1) Low-risk way of learning from feedback (Sadler, 1989)
2) Fine-tune understanding of goals (Boud 2000, Nicol 2006)
3) Feedback to lecturers to adapt teaching (Hattie, 2009)
4) Cycles of reflection and collaboration (Biggs 2003; Nicol &
McFarlane Dick 2006)
5) Encourages and distributes student effort (Gibbs 2004).
Yet formative is vital

26. How you encourage formative
Go to www.menti.com and use the code 67 82 87
Choose your top three strategies for engaging
students in formative assessment
…Or talk to each other about successful strategies

27. Case Study 1
• Systematic reduction of summative across
whole business school
• Systematic ramping up of formative
• All working to similar script
• Whole department shift, experimentation,
less risky together

28. Case Study 2
• Problem: silent seminar, students not reading
• Public platform blogging
• Current academic texts
• In-class
• Threads and live discussion
• Linked to summative

29. Case Study 3
• Problem: lack of discrimination about sources
• Students bring 1 x book, 1 x chapter, 1 x
journal article, 2 x pop culture articles to
seminar
• Justify choices to group
• Reach consensus about five best sources
• Add to reading list

30. Case study 4
https://www.youtube.com/watch?v=ZVFwQzlVFy0

31. Case Study 5

32. Your task
• In groups, identify five principles for making
formative work. Write them down on flipchart
paper.
• How could you use or adapt this on your
programme?

33. Principles to encourage formative
1. Rebalance summative and formative
2. Whole programme approach
3. Link formative and summative
4. Authentic, public domain tasks
5. Creative, collaborative, challenging tasks
6. Relational and conversational feedback

34. More nuts and bolts: The AEQ
How does the AEQ
work? What will it
tell me about the
programme?
What will I learn
about students’
views of assessment
from the focus
group?

35. Activity Two: AEQ
Programme
Team
Meeting
Assessment
Experience
Questionnaire
(AEQ)
TESTA
Programme
Audit
Student
Focus Groups

36. AEQ 4.0 (2018)
• Fill in the AEQ 4.0 from the vantage point of
being a student in one of your classes
• Paper or online?
• https://educ.sphinxonline.net/v4/s/ha2fbs

37. Comparing Audit and AEQ data
from one programme
In pairs or groups, explore programme
audit and AEQ data from one programme.
Does anything stack up? Are there loose
ends, questions, contradictions?

38. Theme 2: Disconnected feedback

39. The feedback is
generally focused
on the module
Because it’s at the end
of the module, it doesn’t
feed into our future
work.
If It’s difficult because your
assignments are so detached
from the next one you do for
that subject. They don’t
relate to each other.
I read it and think “Well,
that’s fine but I’ve already
handed it in now and got the
mark. It’s too late”.
STRUCTURAL

40. It was like ‘Who’s
Holly?’ It’s that
relationship where
you’re just a student.
Because they have to mark so
many that our essay becomes
lost in the sea that they have
to mark.
Here they say ‘Oh yes, I don’t
know who you are. Got too
many to remember, don’t
really care, I’ll mark you on
your assignment’.
RELATIONAL

41. A feedback dialogue

42. Irretrievable breakdown…
Your essay lacked structure and
your referencing is problematic
Your classes are boring and I
don’t really like you 

43. A way of thinking about assessment and
feedback?

44. Ways to be dialogic
• Conversation: who starts the dialogue?
• Cycles of reflection across modules
• Quick generic feedback
• Feedback synthesis tasks
• Peer feedback (especially on formative)
• Technology: audio, screencast and blogging
• From feedback as ‘telling’…
• … to feedback as asking questions

45. The key to dialogue

46. Students to lecturers:
Critical Incident Questionnaire
Stephen Brookfield’s Critical Incident Questionnaire http://bit.ly/1loUzq0

47. Activity Three: Focus groups
Programme
Team
Meeting
Assessment
Experience
Questionnaire
(AEQ)
TESTA
Programme
Audit
Student
Focus Groups

48. Have a go at triangulating data
• Read through audit, AEQ and focus group data from one
programme
• Quick abstract/bullet points of what seems to be going on
• Discuss with your group/flipchart
a) What are the stand out themes?
b) What jigsaw pieces fit together?
c) What unresolved issues remain?

49. Role play (5 minutes)

50. The nuts and bolts of change…
How to move
from ‘proving’ to
‘improving’?
What are the
principles of
educational
change here?
How do I avoid
the bear traps?

51. Four propositions
1. Change is difficult. It often involves loss.
2. Change is about people: listen, respect &
involve them, and act together.
3. Change is about taking risks.
4. Sustainable change is about systems, but
systems are not forever

52. A team approach – all pulling in same direction

53. Everybody has brought
in more formative. The
idea was to consolidate
the summative
assessment and bring in
more formative.
Do we want to continue
offering twenty different
types of assessment or
do we bite the bullet and
say “We want the
students to be able to
master five of them”?
There has been more of a
spacing of assessments.
Evidence with practical impacts…
There is a lot more feed
forward, which is what
came out of the TESTA.

54. …which are solution-oriented
Already today I have seen some of yesterday’s
feedback being put into action across the team
and we are feeling excited about the changes
we are making.
Clearly some things will take a little longer but
yesterday’s meeting has bought about clarity
and given us an insight over the direction we
now want to take the programme.
Email Correspondence, Programme Leader

55. All about trust and ownership
• Post-it predictions beforehand
• Trust and confidentiality
• Admitting gaps, listening
• Respect for disciplines
• Team ownership
• Follow-up contact and team
workshops

56. Models of programme assessment:
Spot the flying pigs
1.
2.
3.
4.
5.

57. Model 1: The whole hog

58. Model 2: Half the hog

59. Model 3: The other half of the hog

60. Model 4: Both the hogs together

61. Model 5: The warthog

62. Shifting paradigms from this…

63. …to the adult equivalent of this

64. References
Barlow, A. and Jessop, T. 2016. “You can’t write a load of rubbish”: Why blogging works as formative
assessment. Educational Developments. 17(3), 12-15. SEDA.
Boud, D. and Molloy, E. (2013) ‘Rethinking models of feedback for learning: The challenge of
design’, Assessment & Evaluation in Higher Education, 38(6), pp. 698–712.
Gibbs, G. & Simpson, C. (2004) Conditions under which assessment supports students' learning. Learning
and Teaching in Higher Education. 1(1): 3-31.
Harland, T., McLean, A., Wass, R., Miller, E. and Sim, K. N. (2014) ‘An assessment arms race and its fallout:
High-stakes grading and the case for slow scholarship’, Assessment & Evaluation in Higher Education.
Jessop, T. and Tomas, C. 2017. The implications of programme assessment on student learning. Assessment
and Evaluation in Higher Education.
Jessop, T. and Maleckar, B. 2016. The Influence of disciplinary assessment patterns on student learning: a
comparative study. Studies in Higher Education. Published Online 27 August 2014
Jessop, T. , El Hakim, Y. and Gibbs, G. (2014) The whole is greater than the sum of its parts: a large-scale
study of students’ learning in response to different assessment patterns. Assessment and Evaluation in
Higher Education. 39(1) 73-88.
Nicol, D. 2010. From monologue to dialogue: improving written feedback processes in mass higher
education, Assessment & Evaluation in Higher Education, 35: 5, 501 – 517.
O'Donovan, B , Price, M. and Rust, C. (2008) 'Developing student understanding of assessment standards: a
nested hierarchy of approaches', Teaching in Higher Education, 13: 2, 205 -217.
Sadler, D. R. 1989. ‘Formative assessment and the design of instructional systems’, Instructional Science,
18(2), pp. 119–144.
Tomas, C and Jessop, T. 2018. Struggling and juggling: A comparison of student assessment loads across
research and teaching-intensive universities. Assessment and Evaluation in Higher Education. 18 April.
Wu, Q. and Jessop, T. 2018. Formative assessment: missing in action in both research-intensive and teaching-
focused universities. Assessment and Evaluation in Higher Education. Published online 15 January.