Keynote Against the Motion: Technology is a vital ingredient for the enhancement of learning

1. Against the motion:
Technology is a vital ingredient for the
enhancement of learning
Tansy Jessop
@tansyjtweets @TESTAwin
20 June 2016

2. Two projects about A&F
Five principles

3. P 1: The quality of teaching matters
more than technology

4. Lessons from Stand and Deliver
• Know your subject matter
• Select, simplify, structure and organise
• Connect with prior knowledge
• Use metaphor, analogy, story, example,
demonstration
• Challenge and demand

5. P 2: Assessment and feedback are vital
to student learning
Feedback is the single most influential factor
in student learning. (Hattie 2009)
Assessment drives what students pay
attention to, and defines the actual
curriculum (Ramsden 1992).

6. It’s the vital ingredient
Assessment is the
senior partner in
learning and
teaching. Get it
wrong, and the rest
collapses.
(Biggs & Tang 2011)

7. P 3: Assessment is more relational than
technical
In mass higher education, students are looking for
a relationship with tutors in feedback (Nicol 2010).
Dissatisfaction with feedback in the NSS is a
symptom of impoverished dialogue (Nicol 2010).

9. Paradigm What it looks like
Technical rational Focus on data and tools
Relational Focus on people
Emancipatory Focus on systems and structures

10. P 4: Effects of technology on learning
are not yet clear

11. From big to smaller…

12. And smaller and smaller…

13. With more and more information..
• 1000 Bytes = 1 Kilobyte
• 1000 Kilobytes = 1 Megabyte
• 1000 Megabytes = 1 Gigabyte
• 1000 Gigabytes = 1 Terabyte
• 1000 Terabytes = 1 Petabyte
• 1000 Petabytes = 1 Exabyte
• 1000 Exabytes = 1 Zettabyte
• 1000 Zettabytes = 1 Yottabyte
• 1000 Yottabytes = 1 Brontobyte
• 1000 Brontobytes = 1 Geopbyte

14. To what effect?

15. Wisdom from the poets
In the age of technology there is constant access
to vast amounts of information. The basket
overflows; people get overwhelmed; the eye of
the storm is not so much what goes on in the
world, it is the confusion of how to think, feel,
digest, and react to what goes on.
Criss Jami, Venus in Arms

16. My few stabs
• Fragmentation
• High speed
• What happened to ‘deep learning’?
• Why an upsurge in the ‘slow’ movement?
• Pressure, work intensification
• The rise in mental health issues
• Knowledge not wisdom
• Soundbites not substance

17. FP 5: Besides the military invented
most technology

18. Technology is neutral.
It can be used for good or ill.
Vital or deadly ingredient?

19. Assessment and feedback are the vital
victuals

20. Five approaches to enhancing learning
from A&F
1. Programme-level assessment design
2. More learning, less measuring
3. Making formative meaningful
4. Connecting feedback
5. Building shared understanding with students

22. The TESTA Methodology
75 PROGRAMME
AUDITS
Programme
Team
Meeting

23. Based on assessment principles
• ‘Time-on-task’ (Gibbs 2004)
• Challenging and high expectations (Chickering and
Gamson 1987)
• Internalising goals and standards (Sadler 1989; Nicol
and McFarlane-Dick 2006)
• Prompt, detailed, specific, developmental, dialogic
feedback (Gibbs 2004; Nicol 2010)
• Deep learning (Marton and Saljo 1976).

24. TESTA….
“…is a way of thinking
about assessment and
feedback”
Graham Gibbs

25. 1. Programme level assessment design

26. Does IKEA 101 work for complex learning?

27. Content-driven curriculum

28. Huge variations
• What is striking for
you about this data?
• How does it compare
with your context?
• Does variation
matter?

29. Assessment features across a 3 year UG degree (n=75)
Characteristic Range
Summative 12 -227
Formative 0 - 116
Varieties of assessment 5 - 21
Proportion of examinations 0% - 87%
Time to return marks & feedback 10 - 42 days
Volume of oral feedback 37 -1800 minutes
Volume of written feedback 936 - 22,000 words

30. And some patterns…
Characteristic Low Medium High
Volume of summative
assessment
Below 33 40-48 More than 48
Volume of formative only Below 1 5-19 More than 19
% of tasks by examinations Below 11% 22-31% More than 31%
Variety of assessment
methods
Below 8 11-15 More than 15
Written feedback in words Less than 3,800 6,000-7,600 More than 7,600

31. Actions based on evidence
a) Reduction in summative
b) Increase in formative
c) Streamlined varieties
d) More or less feedback depending…
e) Quantifiable
f) Every time a coconut with each feature

32. 2. More learning, less measuring
• Summative ‘pedagogies of control’
• Circa 2 per module in UK
• Ratio of 1:8 of formative to summative
• Formative weakly understood and practised

33. What students say…
• A lot of people don’t do wider reading. You just focus
on your essay question.
• In Weeks 9 to 12 there is hardly anyone in our
lectures. I'd rather use those two hours of lectures to
get the assignment done.
• It’s been non-stop assignments, and I’m now free of
assignments until the exams – I’ve had to rush every
piece of work I’ve done.

34. What students say about formative
• If there weren’t loads of other assessments, I’d do
it.
• If there are no actual consequences of not doing it,
most students are going to sit in the bar.
• It’s good to know you’re being graded because you
take it more seriously.
• The lecturers do formative assessment but we
don’t get any feedback on it.

35. Assessment Arms Race

36. 3. Make formative meaningful
1. Reduce summative, increase formative
2. Programme decision
3. Formative in the public domain
4. Link formative and summative
5. Risky, creative, challenging tasks
6. Students read and produce more
7. Deepens understanding of value of formative

37. Theme 3: Connecting feedback

38. Take five
• Choose a quote that
strikes you.
• What is the key issue?
• What strategies might
address this issue?

39. What students say…
The feedback is generally focused on the module.
It’s difficult because your assignments are so detached
from the next one you do for that subject. They don’t
relate to each other.
Because it’s at the end of the module, it doesn’t feed
into our future work.
I read it and think “Well, that’s fine but I’ve already
handed it in now and got the mark. It’s too late”.

40. Students say the feedback relationship is
broken…
Because they have to mark so many that our
essay becomes lost in the sea that they have to
mark.
It was like ‘Who’s Holly?’ It’s that relationship
where you’re just a student.
Here they say ‘Oh yes, I don’t know who you are.
Got too many to remember, don’t really care, I’ll
mark you on your assignment’.

41. How to connect feedback
• Conversation: who starts the dialogue?
• Iterative cycles of reflection across modules
• Quick generic feedback: the ‘Sherlock’ factor
• Feedback synthesis tasks
• Technology: audio, screencast and blogging
• From feedback as ‘telling’…
• … to feedback as asking questions

42. 5. Building shared understanding with
students
• Consistently low scores on the AEQ for clear
goals and standards
• Alienation from the tools, especially criteria
and guidelines
• Symptoms: perceptions of marker variation,
unfair standards and inconsistencies in practice

43. What students say…
We’ve got two tutors- one marks completely differently
to the other and it’s pot luck which one you get.
They have different criteria, they build up their own
criteria.
It’s such a guessing game.... You don’t know what they
expect from you.

44. What students say…
There are criteria, but I find them really strange.
There’s “writing coherently, making sure the
argument that you present is backed up with
evidence”.
I get the impression that they don't even look at the
marking criteria. They read the essay and then they
get a general impression, then they pluck a mark
from the air.
I don’t have any idea of why it got that mark

45. Taking action: internalising goals and
standards
• Regular calibration exercises
• Discussion and dialogue
• Discipline specific criteria (no cut and paste)
Staff Team
• Rewrite/co-create criteria
• Marking exercises (ASKE CETL)
• Design and value formative
Staff and
students
• Enter secret garden - peer review
• Engage in drafting processes
• Self-reflection
Students

46. Regardless of technology….
Enhancing learning is about
educational paradigms

47. Transmission Model

48. Social Constructivist Model

49. References
Boud, D. and Molloy, E. (2013) ‘Rethinking models of feedback for learning: The challenge of
design’, Assessment & Evaluation in Higher Education, 38(6), pp. 698–712. doi:
10.1080/02602938.2012.691462.
Gibbs, G. & Simpson, C. (2004) Conditions r which assessment supports students' learning. Learning
and Teaching in Higher Education. 1(1): 3-31.
Harland, T., McLean, A., Wass, R., Miller, E. and Sim, K. N. (2014) ‘An assessment arms race and its
fallout: High-stakes grading and the case for slow scholarship’, Assessment & Evaluation in Higher
Education, 40(4), pp. 528–541. doi: 10.1080/02602938.2014.931927.
Hughes, G. (2014) Ipsative Assessment. Basingstoke. Palgrave MacMillan.
Jessop, T. and Maleckar, B. (2014). The Influence of disciplinary assessment patterns on student
learning: a comparative study. Studies in Higher Education. Published Online 27 August 2014
http://www.tandfonline.com/doi/abs/10.1080/03075079.2014.943170
Jessop, T. , El Hakim, Y. and Gibbs, G. (2014) The whole is greater than the sum of its parts: a large-
scale study of students’ learning in response to different assessment patterns. Assessment and
Evaluation in Higher Education. 39(1) 73-88.
Nicol, D. (2010) From monologue to dialogue: improving written feedback processes in mass higher
education, Assessment & Evaluation in Higher Education, 35: 5, 501 – 517.
O'Donovan, B , Price, M. and Rust, C. (2008) 'Developing student understanding of assessment
standards: a nested hierarchy of approaches', Teaching in Higher Education, 13: 2, 205 — 217
Sadler, D. R. (1989) ‘Formative assessment and the design of instructional systems’, Instructional
Science, 18(2), pp. 119–144. doi: 10.1007/bf00117714.
Shulman, L. (2004) Teaching as Community Property. San Francisco. Jossey Bass.