3. OUTLINE
STAGES OF TEST DEVELOPMENT
SPECIFICATIONS
WRITING AND
MODERATING
ITEMS
ITEMS ON
NATIVE
SPEAKERS
VALIDATION
CALIBRATION
OF SCALES
ANALYSIS OF
RESULTS
ON NON-NATIVE
SPEAKERS
HANDBOOKS
FOR TEST TAKERS
TRAINING STAFF
STATING THE
PROBLEM
4. • SHOULD BE A TEST DESIGNED BY A SINGLE PERSON OR A TEAM?
TEAM!
• IT IS DIFFICULT TO DESIGN A TEST BECAUSE OF…
…OBJECTIVITY
…CRITICISMS
…NATIVE COMMAND OF THE LANGUAGE
7. STATING THE PROBLEM
• ONCE THE PROBLEM IS CLEAR, IT IS ALSO IMPORTANT TO:
GATHER INFORMATION ON ALREADY EXISITING TEST DESIGNED FOR SIMILAR
SITUATIONS!
8. WRITING SPECIFICATIONS FOR THE TEST
• A) CONTENT
• B) STRUCTURE, TIMING, MEDIUM/CHANNEL AND TECHNIQUES
• C) CRITERIAL LEVELS OF PERFORMANCE
• D) SCORING PROCEDURES
9. A) CONTENT
• INCLUDE CLEAR SPECIFICATIONS REGARDING:
•
•
•
•
•
•
•
•
•
•
SKILLS (SUB-SKILLS)
TYPE OF TEXTS
ADDRESSEES
LENGTH OF TEXTS
TOPICS
(READABILITY)
STRUCTURAL RANGE
VOCABULARY RANGE
DIALECT, ACCENT AND STYLE
SPEED OF PROCESSING
10. B)
STRUCTURE, TIMING, MEDIUM/CHANNE
L AND TECHNIQUES
• TEST STRUCTURE
• NUMBER OF ITEMS
• NUMBER OF PASSAGES
• TIMING
• MEDIUM
• KIND OF TEST TECHNIQUE(S)
11. C) CRITERIAL LEVELS OF
PERFORMANCE
• ACCURACY
• APPROPRIACY
• RANGE
• FLEXIBILITY
• SIZE
12. D) SCORING PROCEDURES
• WHAT RATING SCALE WILL BE USED?
• HOW MANY PEOPLE WILL RATE EACH PEACE OF WORK?
• WHAT HAPPENS IF TWO OR MORE RATERS DISAGREE ABOUT A PIECE OF
WORK?
15. B) WRITING ITEMS
• TRY TO LOOK AT THE TEST THROUGH THE EYES OF TEST TAKERS!
• AN ITEM WITHOUT A KEY IS INCOMPLETE!
• “THE BEST WAY TO IDENTIFY ITEMS THAT HAVE TO BE IMPROVED OR
ABANDONED IS THROUGH THE PROCESS OF MODERATION”.
17. INFORMAL TRIALLING OF ITEMS ON
NATIVE SPEAKERS
• 20 OR MORE…
• THEY SHOULD BE SIMILAR TO THE GROUP BEING TESTED IN TERMS OF:
AGE
EDUCATION
GENERAL BACKGROUND
18. TRIALLING OF THE TEST ON A GROUP
OF NON-NATIVE SPEAKERS SIMILAR
TO THOSE FOR WHOM THE TEST IS
INTENDED
• PROBLEMS IN ADMINISTRATION AND SCORING CAN BE NOTED.
19. ANALYSIS OF RESULTS OF THE TRIAL;
MAKING OF ANY NECESSARY
CHANGES
• STATISTICAL AND QUALITATIVE ANALYSIS.
HOW DIFFICULT ARE THE ITEMS?
DISCOVER MISINTERPRETATIONS (TO BE MODIFIED OR DROPPED)
20. CALIBRATION OF SCALES
• IT IS IMPORTANT TO COLLECT SAMPLES OF PERFORMANCE AND ASSIGN EACH
OF THEM TO A POINT ON THE RELEVANT SCALE.
31. GAP FILLING ITEMS
• IT IS A VALUABLE TECHNIQUE AS LONG AS THE CONTEXT IS PROVIDED!
• A: WHAT WILL HE DO?
• B: I THINK HE ______ RESIGN.
BUT:
A: I WONDER WHO THAT IS.
B: IT ______ BE THE DOCTOR.
A: HOW CAN YOU BE SO CERTAIN?
WILL, MIGHT,
COULD
MAY, ETC.
32. • Now you know what a teacher should do when preparing tests, what
advices would you give?
• (Each one should only mention only 1 advice)
Names
Advices
Carolina
Think of the applicability to real situations ar real life situations.
Paola
Put in your students´ shoes.
Mauricio
Consider multiple intelligences.