3. Overview
• How accountable are we?
• What do students need for success?
• Feedback and research – pre-sessional
• Trinity – a response to contemporary
assessment
• Q&A
Alex Thorp AULC 7th – 8th January 2016
4. 4
Accountability and Assessment
As a University Language Department / University, in what
ways are you accountable for your students’ achievement?
5. 5
Accountability and Assessment
What are we accountable for?
• Poor achievement levels
• Dropouts
• Failing the course / year
• High levels of absenteeism
• Student stress levels / anxiety
• The ‘International Student Experience’
• Student identity
•
•
Do these issues stem from language skills or something more?
Dissatisfied fee-paying
students have avenues for
recourse:
• Legal action
• Igrad (feedback)
• Reputation
• PR
• Social media
• International agreements
8. 8
What do students need to succeed?
Language Skills
Bachman & Palmer (2010): communicative competence model
Communicative
Competence
Linguistic
competence
Socio-linguistic
competence
Discourse
competence
Strategic
competence
10. 10
What do students need to succeed?
Transferable Skills
1. Critical thinking & problem solving
2. Collaboration and leading by influence
3. Agility and adaptability
4. Initiative and entrepreneurship
5. Effective oral and written communication
6. Accessing and analyzing information
7. Curiosity and imagination
Tony Wagner - Expert In Residence - Innovation Lab of Harvard
12. 12
• Note taking
• Summarising
• Synthesising
• Sourcing relevant data across texts
• Planning
• Self-checking and self-evaluation
• Deducing and inferring
•
What do students need to succeed?
Study Strategies
2500+ published
academic papers
16. 16
Pre-sessional Trainers - Feedback
Sample interviews with Pre-sessional trainers
• Note-taking
• Listening and note-taking
• Self correction and reflection
• Deduce meaning of unknown item
• Speed reading – selection
• Research skills
• Planning
Study
strategies
(what is needed to
succeed)
18. Pre-sessional Trainers - Feedback
Sample interviews with Pre-sessional trainers
What negative
expectations (skills?)
do new students
bring?
Over-reliance on
dictionaries and
mobile media
M.O. University of the Arts London
Wanting to
translate every
word
(productive and
receptive).
J.B. University of Bristol Not willing to
participate in
pair work /
communication
S.H. University of Sussex
Want to be
spoon-fed /
expectation of
teacher-centred
input
S.I. University of
Birmingham
Memorisation /
working from
scripts
J.P-S. University of Edinburgh
19. 19
What should a test tell you?
In an ideal world….
Language
Skills
Transferable
Skills
Study
Strategies
20. Trinity ISE – revised 2015
An Integrated Skills exam
21. Trinity ISE – revised 2015
An Integrated Skills exam
22. The ISE formatISE Foundation ISE I ISE II ISE III
CEFR level A2 B1 B2 C1
Topic task 4 minutes 4 minutes 4 minutes 8 minutes
Collaborative task - - 4 minutes 4 minutes
Conversation task 2 minutes 2 minutes 2 minutes 3 minutes
Independent
listening task
6 minutes 7 minutes 8 minutes 8 minutes
Trinity ISE?
Overview – Speaking & Listening
Is this anything new?
23. 23
When do you speak without listening?
When do you write without reading?
Communicative competence requires assessment
that tests skill integration.
Trinity ISE?
An integrated approach
24. MACRO - Receptive
Listening Objectively
tested
MACRO - Receptive
Reading Objectively
tested
MACRO - Receptive
Speaking Subjectively
tested
MACRO - Receptive
Writing Subjectively
tested
Interactive Listening
(Subjectively tested)
Reading into Writing
(Subjectively tested)
What do students need to succeed?
Language Skills
25. 25
Topic task
Collaborative task (at higher grades)
Conversation
The test
taker
prepares
ANY topic of
their choice
for
discussion
with the
examiner
The examiner
selects topic
from a list
relevant to the
level and asks
the test taker
questions to
start a
conversation
The examiner
reads a prompt
which creates
an information
gap.
This sets the
scene for a
test-taker led
discussion
NEW rating
scale
&
Assessment
of
Interactive
listening
Trinity ISE?
Overview – Speaking & Listening
26. 26
What do students need to succeed?
Transferable Skills
planning/organisation
practising English presentation skills
creativity
self-evaluation
self-management
critical thinking
problem-solving
Preparing for
Trinity ISE:
impact
27. The ISE formatISE Foundation ISE I ISE II ISE III
CEFR level A2 B1 B2 C1
Time 2 hours
Task 1 Long reading
300 words
15 questions
Long reading
400 words
15 questions
Long reading
500 words
15 questions
Long reading
700 words
15 questions
Task 2 Multi-text reading
3 texts
300 words
15 questions
Multi-text reading
4 texts
400 words
15 questions
Multi-text reading
4 texts
500 words
15 questions
Multi-text reading
4 texts
700 words
15 questions
Task 3 Reading into writing
70-100 words
Reading into writing
100-130 words
Reading into writing
150-180 words
Reading into writing
200-230 words
Task 4 Extended writing
70-100 words
Extended writing
100-130 words
Extended writing
150-180 words
Extended writing
200-230 words
Trinity ISE?
Overview – Reading and Writing
28. Q6-10 Selecting
true statements
Understand specific
information at
sentence level
Multiple genres:
Factual texts familiar in an educational or everyday context
Q1-5 Multiple
matching
Understand the main
idea and purpose of
each text
Graphic Organiser
representing
complex information
Contemporary digital
reading strategies
ISE Reading & Writing
Example – Task 2, Multi-text reading
29. ISE Reading & Writing
Example – Task 2, Multi-text reading
Q11-15 Completing
summary notes
Understand factual
information at word or
phrase level across texts.
Use the information in a
simple summary
30. ISE Reading & Writing
Example – Task 3, Reading into Writing
• Use same texts from
the reading task
• Refer back to the
summary notes as a
spring board for
planning
• Identify relevant
information from all
texts
• Paraphrase and
summarise key
words, phrases,
sentences
• Incorporate the
information in a
response to a prompt
32. 32
Trinity ISE and the CEFR
Implications for standardisation and predictive validity.
Trinity ISE mapped from the CEFR
… not mapped to the CEFR
33. Accountability and Assessment
Summary
We are accountable for our
students’ achievement
Need to know what
applicant students
can do
Need to know what to
teach pre-sessional
students (and how)
Language Skills Study Strategies Transferable Skills
To what degree do the tests you use equip you to do this?
34. Accountability and Assessment
Summary
Trinity Integrated Skills in English Exam (ISE)
Pre-
admission
Predictive
Validity
Relevant
Skills
Part of
Pre-sessional
study
Positive
Washback
Relevant
skills
36. 36
Factsheet leaflets for
teachers & students
Brochure for
colleges and
universities
ISE Guides for teachers*
Introductory overview &
sample exam videos
Online guidance &
ISE Best practise
guide for administrators
Sample papers and
classroom activities
including Portfolio toolkit
ISE Guides for students
Resources
Available online at www.trinitycollege.com
37. Alex Thorp
Academic Support Specialist
Trinity College London
AULC Conference – Cardiff 7th – 8th January 2016
Thank you.
alex.thorp@trinitycollege.com