Evaluación de la comprensión de lectura y la escritura

1. ID7756 Evaluación del Aprendizaje de Idiomas
Enero-Marzo 2014
Prof. Yris Casart
ycasart@usb.ve

2. A framework for assessing reading
Recommendations for reading
assessment

3. • Step 1: Write test specifications
• Step 2: Select an appropriate text
• Step 3: Modify the text
• Step 4: Write test questions
• Step 5: Decide on weighting
• Step 6: Validate and re-validate
Lloyd & Davidson (2009) en Coombe, Folse & Hubley (2007)

4. • Step 7: Ensure standardized test
administration
• Step 8: Ensure reliable rating/marking
• Step 9: Rescale (if necessary)
• Step 10: Provide students with diagnostic
feedback
• Step 11: Evaluate your test
• Step 12: Recycle your test
Lloyd & Davidson (2009) en Coombe, Folse & Hubley (2007)

5. 1. Use specifications
for even coverage of curricular outcomes
to promote content validity
2. Avoid skill contamination by limiting writing
3. Choose text to fit purpose of assessment
and to match students’ level.
4. Make items less difficult than passage
level
5. Test items in the order of the passage.
Coombe, Folse & Hubley (2007)

6. 6. Sample skills with range of formats.
7. Keep a file of authentic material from
newspapers, magazines, etc.
8. Avoid texts with controversial or biased
material.
9. For objective formats, try to make all
statements positive.
10. Rephrase material using synonyms to
avoid students looking for verbatim
matches.
Coombe, Folse & Hubley (2007)

7. indirect vs. direct assessment
free writing vs. guided writing
writing assessment scales
responding to student writing
10 things to remember

8. Indirect writing assessment
• It measures correct usage in sentence level
constructions and focuses on spelling and
punctuation via objective formats like MCQs and
cloze tests.
• Measures determine the S’s knowledge of writing
sub-skills such as grammar and sentence
construction which are assumed to constitute
components of writing ability
• Measures are largely concerned with accuracy
rather than communication.
Coombe, Folse & Hubley (2007)

9. Direct writing assessment
• It measures the S’s ability to communicate
through the written mode based on the production
of written texts.
• It requires Ss to come up with the content, find a
way to organize the ideas and use appropriate
vocabulary, grammatical conventions and syntax.
• Measures integrate all elements of writing.
Coombe, Folse & Hubley (2007)

10. free writing guided writing
requires Ss to read a
prompt that poses a
situation and write a
planned response
based on a combination
of background
knowledge and
knowledge learned from
the course.
requires Ss to
manipulate content that
is provided in the
prompt, usually in the
form of a chart or
diagram. It’s a bridge
between objective and
subjective formats.
Coombe, Folse & Hubley (2007)

11. • Be clear about expected form/length of response
(1 paragraph, 250 words, a letter)
• Clearly specify what you want in the prompt
(3 causes & effects, two supporting details)
• Specify discourse patterns students are
expected to use
(i.e. compare/contrast, cause/effect, description etc.)
• Ask Ss to provide something beyond the prompt
(i.e., opinion, inference, prediction.
Coombe, Folse & Hubley (2007)

12. • Considerations for GW are useful for free writing.
• Use of multiple raters. Agree on grading criteria in
advance & calibrate before grading).
• Decide on which scale to use beforehand.
• Acquaint Ss with marking scheme in advance by
using it for teaching, grading homework and
providing feedback.
• Provide Ss with enough space for an outline, a draft
and the finished product.
• Avoid issues that might offend or disadvantage Ss.
Coombe, Folse & Hubley (2007)

13. Holistic marking Analytical marking
It is based on marker's
total impression of the
essay. It is variously
termed impressionistic,
global or integrative
marking.
It requires Ts to give
separate ratings for the
different components of
language ability. It has
been termed discrete
point marking.
Coombe, Folse & Hubley (2007)

14. Advantages Disadvantages
• Reliable if done under no
time constraints & if Ts
receive training.
• Generally perceived to be
quicker than other types of
marking
• Since overall writing ability
is assessed, Ss are not
disadvantaged by one
lower component, i.e. poor
grammar.
• Scores emphasize the
writer’s strengths
• Unreliable if marking is done
under short time constraints
& with untrained Ts.
• Longer essays often tend to
receive higher marks
• Tendency for marker to
overlook sub-skills that make
up writing.
• It does not provide a profile
of the student's writing ability
and to provide feedback.
• Difficult to interpret
composite score
Coombe, Folse & Hubley (2007)

15. Advantages Disadvantages
• Generally more effective
with inexperienced
teachers.
• provide Ts with a "profile"
of their Ss' strengths &
weaknesses
• Training raters is easier
because the scales are
more explicit and detailed.
• Perceived to be more time
consuming
• A set of specific criteria has
to be written and markers
need to be trained: attend
frequent calibration sessions.
• Because Ts look at specific
areas in a given essay, the
most common being content,
organization, grammar,
mechanics and vocabulary,
marks are often lower
Coombe, Folse & Hubley (2007)

16. • Feedback provides opportunities for Ss to learn &
make improvements
• Most common type of written teacher feedback is
handwritten comments, usually at the end of the
paper or in the margins
• Some Ts like to use correction codes to provide
formative feedback and to facilitate marking
• Electronic feedback is particularly valuable
because it can be used to give a combination of
handwritten comments and correction codes.
Coombe, Folse & Hubley (2007)

17. sp Spelling
vt Verb tense
ww Wrong word
wv Wrong verb
☺ Nice idea/content!
₪ Switch placement
¶ New paragraph
? I don’t understand
Coombe, Folse & Hubley (2007)

18. 1. Give students multiple writing assessment
opportunities.
2. Test a variety of writing skills and create
tasks of varying lengths.
3. Develop prompts that are appropriate for
the Ss.
4. Evaluate all answers to one question
before going on to the next.
5. Mark only what the student has written.
Coombe, Folse & Hubley (2007)

19. 6. Have a systematic approach for dealing
with marking discrepancies: average of
the two raters for a small discrepancy and
to utilize a third rater if there is a big
discrepancy.
7. Get students involved.
8. Provide students with diagnostic
feedback.
9. Practice blind or double blind marking.
10. Calibrate and recalibrate. The best way
to achieve inter-rater reliability is to practice.
Coombe, Folse & Hubley (2007)

20. Coombe, C., Folse. K, & Hubley, N. (2007) A Practical
Guide to Assessing English Language Learners. Ann
Arbor, MI: University of Michigan Press.