2. What’s wrong?
• Too many questions
• Only assess knowledge (remembering)
• Don’t clearly relate to what was taught
• Ambiguous or unclear
• Don’t provide an adequate sample of the body of
content
3. Plan
• Relate to outcomes
• Relate to level / scope of learning
• Develop an assessment plan
• Identify content topics
• Map to Learning Outcomes
• Identify levels of learning
• Identify number of tests
• Identify when
4. Select
• Type of test
• Multiple choice, true/false, matching, OR essay
• What is being measured?
• Match test type to learning outcome / level of learning
• Delivery mode / timing
• Paper-based or online
• Short- or long- time allowed
• Open book or closed
5. Evaluate
Essays – Strengths
• Measure higher level objectives
• Less time-consuming to build
• Allow students to demonstrate understanding of
complex ideas
• Provide a more authentic assessment
6. Evaluate
Essays – Weaknesses
• More time to mark
• Challenging to score objectively
• Research indicates bias is often an issue
7. Evaluate
Multiple Choice – Strengths
• Versatile – measure various levels of learning
• Allows for wide sampling of content / objectives
• Highly reliable test scores
• Machine scoring possible
• Re-use possible
8. Evaluate
Multiple Choice – Weaknesses
• Difficult to write questions to clearly address
different levels of learning
• Time-consuming
• Students require “test literacy”
• Tendency to develop “easy” questions
• May encourage guessing
Essays allow you to measure students ability to select content, organize and integrate it and present it logically. Students can show they see relationships and patterns in course content.It’s easier to construct essay questions.
Essays allow you to measure students ability to select content, organize and integrate it and present it logically. Students can show they see relationships and patterns in course content.It’s easier to construct essay questions.It’s harder to mark essays – takes longer and difficult to score objectively. Research shows that bias is a factor:When an essay is preceded by a “good” essay, it receives a lower markLocation in stack – higher it is, the higher the scoreIf weak answers appear first, remaining answers are scored lower.Scores influenced by teacher perceptions of student’s abilitiesScores influenced by handwriting, spelling, vocabulary.
Well-designed multiple choice items can measure various levels of learning. A broader scope of course content can be sampled.Provides reliable data for ongoing evaluation of student progress.Easier to administer, store, re-use, use in different delivery modes.
Writing “good” multiple choice questions requires writing skills in the teacher and reading skills in the student. Sometimes students fail because they haven’t learned “test” literacy.Test questions may actually have more than one correct answer – most MC formats are one-answer.Some teachers write questions to “fool” the students – testing attention / critical thinking but may cause negative effects, resentment – is it fair?
How to Write Better Tests: A Handbook for Improving Test Construction SkillsLucy C. Jacobs, Ph.DPlanning the testTest FormatsEssay ItemsMutliple Choice ItemsTrue-False ItemsMatching ItemsCompletion Items