1. Testing Forum task one<br />Think about a course you have taught in the past which ended with a final examination. To what extent did this course have a backwash effect on the course, and was it positive, negative or a mixture of both? Explain why, and if negative, what you did or could have done in a successive course to lessen the effect. You should refer to the reading you have done for the unit when answering.<br />The TOEFL exam : A summative proficiency test with high stakes since entry into US higher education is dependent on passing this exam. <br />Backwash 1 : The effect that a final test has on the teaching programme that leads to it – teaching to the test. (Baxter 1997 : 94)<br />Backwash 2 : The effect of testing on teaching and learning….can be harmful or beneficial (Hughes 1989 : 1)<br />To answer this forum task, I’m going to focus on a present (rather than past) exam course that I’m teaching. The reason for this is because I have had very little recent practice in teaching exam classes, although my first teaching position with YLs (teens) at Regent School, Milan was a baptism of fire as far as exam classes go as I was thrown into the deep end of PET, KET and TOEFL classes. As I’m helping a 16-year-old girl study for TOEFL right now, I thought it would be interesting to comment on my teaching experiences with this particular exam and draw parallels between the exam as it was in the early 90s and how it is now. If I had the TOEFL exam in front of me right now, I would paraphrase a 1970s American cigarette advert campaign directed at female smokers, and say ‘you’ve come a long way, baby’. The early 90s version relied on a vast number of discrete, indirect mostly MC test items (which were judged sufficient evidence of ability to use the language), and required no direct oral test or testing of ability to write texts. The TOEFL examining board maintained for a long time that it was impossible to test candidates’ writing ability through a composition task due to claims of impracticality and unreliability. The lack of testing of candidates’ ability to use the language communicatively in writing and orally meant that students weren’t being prepared for two vitally important skills in the academic setting of American universities. The effect on my own teaching experience was the negative backwash effect of trawling through stultifying reams of examples of discrete items, teaching to the test and no real examination of the future use that these aspiring US-bound university students would be making of the language. Both students (earnest, studious types) passed the exam and went on to study at American universities. I am not sure though how much the exam content prepared them for their university studies.<br />A fast leap forward and the TOEFL seems to have radically changed its approach. An initial introduction of a ‘Test of written English’ supplement (TWE) was made in 1986 due to pressure from EFL teachers which convinced those responsible for the exam that a writing task would provide beneficial backwash. TOEFL has become much more of an example of a communicative language test and does seem to measure ability to take part in acts of communication. Test content reflects and simulates academic settings: the texts used are authentic examples of lectures and non-specialized academic texts, whilst the tasks require testees to perform tasks similar to those carried out by university students (listening to lectures, note-taking, extracting key information from texts, evaluating theses, engaging in discursive and expository discourse etc).<br />“The TOEFL test measures how well students use language, not just their knowledge of the language” ETS (2009 : 1)<br />This approach means that the TOEFL exam has content validity and “constitutes a representative sample of language skills, structures etc with which it is meant to be concerned.” (Hughes 1989 : 26). It also complies with Hughes’ criteria for creating beneficial backwash by testing the abilities it wants to develop.<br />The introduction of subjective, direct test items (such as the independent and integrative writing tasks and the recorded oral responses on the computer-based iBT TOEFL exam) also creates beneficial backwash since it tests these performance skills directly. Potentially negative backwash in the oral section of the exam is the fact that the test is not interactive, since the candidate responds with a monologue (to a reading passage / lecture or dialogue in a university setting) which is then digitally recorded for later scoring. Whilst such an integrative approach has its merits, the lack of interaction with an examiner or fellow testee does not simulate most natural discourse which is interactive and requires turn-taking skills, negotiation of topic and meaning etc. As Powers suggests, “ It is neither possible nor desirable to separate speaking skills from listening skills in many tests of oral production. They are interdependent skills” (seen 29/10/10). Whilst my lessons with this learner have focused on developing her writing skills, occasional practice for the speaking section of the exam has obviously ignored the sub-skills mentioned above in favour of developing her monologue skills. I feel that the effect of this on teaching / learning is negative because it lacks content and predicative validity. Plus, this, being a response to recorded stimuli, is a semi-direct test so does not conform completely to Hughes’ criterion for direct testing of the skills to be developed. <br />In the independent writing task, candidates have to write one composition of roughly 300 words long in 30 minutes. There are three categories: <br />A discursive/expository text.<br /> An invention / technological topic essay (info report/ expository synthesis)<br /> A description /analysis essay of information presented in chart/ graph form. <br />Hughes’ admonition to ‘sample widely and unpredictably’ (1989 : 54) is perhaps not met in such a narrow range of texts. The restricted area of specifications has led to practice in only the first two text types. The third has been almost entirely ignored due to the fact that “In the past, the chart /graph-based TWE topics were criticized so widely and by so many ESL researchers that ETS has stopped administering them. However, they have not been officially removed from the TWE topic possibilities and you cannot be sure that they will not be administered again”. (Barron’s TOEFL Strategies 1998 : 137). Such uncertainty has clearly had a negative backwash effect on teaching since it encourages gambling on the likelihood that the topic type will not appear in the exam. In defence of the restricted nature of the samples used, it has to be argued that the essay types reflect the writing tasks students perform in US higher education.<br /> The integrative writing task integrates reading, listening and writing skills. It encourages students to extract key information and show how the points in the lecture relate to those in the reading passage. The fact that it is integrative is positive since, as Powers points out “attempts to assess a learner’s capacity to use many bits all at the same time” complies with the naturalness criteria and the contextual constraints of the language (seen 21/09/10). It is a direct test of communicative competence, and simulates academic discourse and tasks by encouraging note-taking, listening / reading sub-skills such as understanding speaker/writer purpose, opinion, main ideas, attitude, non-explicitly stated meaning, text coherence etc.<br />The TWE has definitely resulted in a teaching programme where objectives are to practice these particular sub-skills, and also the sub-skills necessary for writing academically. The student doing the course made an autonomous decision to focus the lessons primarily on improving her writing skills and this has led to an analysis of her weaknesses in writing academic texts and a discourse / genre approach to developing writing skills through the use of model (and not so model) texts, brainstorming, planning, drafting, proofreading, redrafting etc. TL has focused on: text structural organization; cohesive devices of substitution, ellipsis and references, lexical sets; encouraging coherence through an appropriate use of discourse markers, punctuation etc; hedging language such as modality to make statements more tentative etc.,. This rhetorical analysis of texts beyond a ‘grammar at sentence level’ to a ‘whole text grammar’ might however justifiably come in for criticism for being “a sort of discourse structuralism” (Hutchinson and Waters 1987 : 37). There is a risk of replacing one kind of grammar (sentence level) with another (text level) and turning the whole process into an atomized, product approach. As Hutchinson and Waters point out: “ is it any more likely that making learners aware of the patterns in discourse will enable them to use those discourse patterns in communication?” (ibid.) <br />Teaching approaches aside, I honestly feel that the backwash effect of the TOEFL exam in itself has been mostly beneficial. Negative aspects affecting teaching / learning have principally been due to the constraints of the teaching situation: namely time constraints (a 2 month course of 3 hours of lessons a week); plus the fact that, although highly motivated, the learner has been following English courses at school at A2 level, and our previous lessons were purely conversation lessons. Whilst she is undoubtedly above a pre-intermediate level, the need to develop writing skills so quickly and to the academic level required by the TOEFL exam has proved to be quite a tall order, both for her as a learner and me as a teacher. Plus, the exam is aimed primarily at potential undergraduates (and therefore school leavers) not 16 year olds. The differences in the educational philosophies between the formal, rationale-encyclopedic definition of knowledge in Latin countries and the more learner-centred, learner-active , humanist one of the American educational system means that the student finds it difficult to engage in argumentative discourse which requires her to give and defend her opinion through evidence, example and expansion of ideas. The very skills required for the independent writing task. Lack of cultural knowledge of what is entailed in English academic writing, a lexico-grammatical performance not quite up to the level required by the exam and the sheer number of discourse features to be acquired in a short space of time have led to a teaching situation that has been less learner-centred and more transmission-style than I would have liked. <br />What could be done in a successive course to lessen such an effect?<br />In an ideal situation, I would have like to have more time to prepare the student for the exam, both in terms of level of competence and also in exam technique. This would have created more positive backwash due to a less rushed and intense atmosphere.<br />A more pragmatic solution to time constraints might be to encourage much more student autonomy outside the classroom teaching hours, and not just in the practice of writing assignments and in exam practice, but also through the provision of self-study material in the lexico-grammatical and discourse features of academic writing. Moreover, the creation of a personalized check list for the student to evaluate her own work would do much to encourage autonomy. <br />In a successive course, I would also adopt more of a process approach than the discourse approach that I have been using in this particular course. <br />TOEFL is a high stakes exam, a borderline score on the TWE can result in the rejection of a candidate’s application to an American college. As Shohamy points out, “.. tests are very powerful tools in society, they tell us what is happening to us, what our future brings….if you’ll get in to a certain programme or won’t”. (seen 20/10/10). This is certainly the case for TOEFL. My student is sitting the exam this Friday. All I can say is good luck….a lot is riding on it.<br />Bibliography :<br />Baxter, A. (1997) Evaluating Your Students Richmond Publishing <br />Hinkel , E (1998) Barron’s TOEFL Strategies Barron’s Educational Series NY <br />Hughes, A. (1989) Testing for Language Teachers CUP.<br />Hutchinson, T and Waters, A. (1987) English for Specific Purposes CUP<br />ETS The Official Guide to the TOEFL Test 3rd Ed. (2009) McGraw-Hill<br />Powers T, English Language Learning and Assessment seen 29/10/10 at http://www.btinternet.com/_ted.power/esl <br />Shohamy, A. Impact Language Testing Videos - Video 13<br />