2. Developing Quality Assessment
Items
During this session we will
•define the terms associated with multiple choice items.
•discuss the advantages and disadvantages of multiple choice items.
•introduce some hints for writing multiple choice items.
Capacity Development Workshop: Test and
Item Development and Design, Laos,
September 2016
3. Parts of a multiple choice item
A load is placed on balance X and then on balance Y.
What is measured by each balance?
Stimulus
Lead sentence,
Directions line
Stem or
question
Options
A,B,D are
Distractors
Rationales
Options A,B,D
could distract
unless
student is
clear with
concept of
density, mass
and weight
Options
3
Option C is the Key
Options Balance X Balance Y
A Weight Density
B Mass Weight
C Weight Mass
D Mass Density
Test construction and item analysis workshop, November 2015
Capacity Development Workshop: Test and
Item Development and Design, Laos,
September 2016
4. Parts of a multiple choice item (Continued)
• Directions Line, Lead Sentence - an introduction which directs
a student to use the stimulus to answer the item or provides
contextual information about the stimulus.
• Stimulus - information required in order to answer the item.
Only use stimulus if a student needs the stimulus to answer the
question.
• Stem - a question or statement which poses a clearly defined
problem that is aligned to the content standard or curriculum
benchmark being measured.
Test construction and item analysis workshop, November 2015
Capacity Development Workshop: Test and
Item Development and Design, Laos,
September 2016
5. Parts of a multiple choice item (Continued)
• Options - the answer choices for students to select when answering
the question.
• Key - the correct answer.
• Distractors - the incorrect answers.
• Rationales - justifications that explain why a certain distractor is
plausible, yet incorrect, or demonstrates a common misconception.
Test construction and item analysis workshop, November 2015
Capacity Development Workshop: Test and
Item Development and Design, Laos,
September 2016
6. Quick quiz (True or False or Don’t Know
[Need More Information])
1. One function of the lead sentence is to direct students to
use the stimulus to answer the item.
2. Item rationales explain why distractors are plausible.
3. Distractors are incorrect answer options.
4. A stimulus should be included on an item even if it is not
necessary to answer the item.
5. Answer options include the key (correct answer) and the
distractors.
6. Reading passages, maps, diagrams, and tables are
examples of stimuli.
Test construction and item analysis workshop, November 2015
Capacity Development Workshop: Test and
Item Development and Design, Laos,
September 2016
8. Pollination is best defined as the
a) growth of the pollen tube in the ovule.
b) germination of pollen grains.
c) carrying of pollen by insects from flower to flower.
d) transfer of pollen from anther to stigma.
Direct question and best answer type
(Type B)
Test construction and item analysis workshop, November 2015
Capacity Development Workshop: Test and
Item Development and Design, Laos,
September 2016
9. Requirements of writing good items
As test developers, you must have
• a thorough knowledge of the subject matter - including knowledge of
popular fallacies and misconceptions;
• an understanding of the content standards of the curriculum and the
performance of individuals;
• good written communication skills;
• technical item writing skills; and,
• imagination and ingenuity.
Test construction and item analysis workshop, November 2015
Capacity Development Workshop: Test and
Item Development and Design, Laos,
September 2016
10. Sources of item ideas
Sources of item ideas include
• chance ideas and inspirations;
• the work (verbal, written) of students;
• the items and ideas of other teachers;
• understanding of what an expert in this area is expected to know/
be able to do; and,
• imagination and ingenuity.
Test construction and item analysis workshop, November 2015
Capacity Development Workshop: Test and
Item Development and Design, Laos,
September 2016
11. Advantages of multiple choice items
Advantages include
• versatility (adaptable for various levels of learning outcome including,
simple recall of knowledge, analysis of phenomena, application of
principles, interpreting cause and effect relationships, etc.) ;
• increased validity (more questions therefore greater coverage of the
syllabus);
• increased reliability because of objectivity of marking; and,
• increased efficiency (easily marked).
Test construction and item analysis workshop, November 2015
Capacity Development Workshop: Test and
Item Development and Design, Laos,
September 2016
12. Disadvantages of multiple choice items
Disadvantages include
• not adaptable to measuring certain learning objectives including
articulating an explanation, displaying thought processes, etc.;
• susceptibility to guessing;
• difficulty of construction.
Test construction and item analysis workshop, November 2015
Capacity Development Workshop: Test and
Item Development and Design, Laos,
September 2016
13. A Couple of myths associated with
multiple choice items
• Multiple choice items can only be used to measure lower-level
outcomes such as those based on knowledge, facts and principles.
• Individuals can perform well simply by guessing.
Test construction and item analysis workshop, November 2015
Capacity Development Workshop: Test and
Item Development and Design, Laos,
September 2016
14. Hints for writing multiple choice items
Align the item with the learning outcome.
Business Studies Objective: Compare and contrast different forms of
business organizations (Public Company, Private Company,
Proprietorship, Partnership )
Which of the following forms of ownership is the outcome of a
voluntary agreement?
a) Public Company
b) Partnership*
c) Proprietorship
d) Private Company
Is this assessing the learning outcome?
Test construction and item analysis workshop, November 2015
Capacity Development Workshop: Test and
Item Development and Design, Laos,
September 2016
15. Hints for writing multiple choice items
Each item should focus on an important concept.
The early development of analysis of variance (ANOVA) was
mainly due to work done by Sir Ronald A. Fischer.
What was Fischer’s middle name?
a) Alan.
b) Albert.
c) Aylmer.
d) Arthur.
Is this assessing an important learning objective?
Test construction and item analysis workshop, November 2015
Capacity Development Workshop: Test and
Item Development and Design, Laos,
September 2016
16. Hints for writing multiple choice items
The language is simple, clear and unambiguous.
Is this clear and concise language?
The local community theatre group in Jakarta is performing a play in the Local Public School
auditorium. There are 15 rows of seats. Each row contains 28 seats.
What is the largest number of tickets that the community
theatre group can sell to fill the auditorium for one performance?
a) 400 tickets.
b) 420 tickets*.
c) 450 tickets.
d) 470 tickets.
Test construction and item analysis workshop, November 2015
Capacity Development Workshop: Test and
Item Development and Design, Laos,
September 2016
17. Hints for writing multiple choice items
How has this stem improved?
A theatre has 15 rows and each row contains 28 seats. What is
the largest number of tickets that the theatre can sell to fill the
auditorium for one performance?
a) 400 tickets.
b) 420 tickets*.
c) 450 tickets.
d) 470 tickets.
Better …
Test construction and item analysis workshop, November 2015
Capacity Development Workshop: Test and
Item Development and Design, Laos,
September 2016
18. Hints for writing multiple choice items
The stem and answer options must be
grammatically consistent with one another.
What is the answer? Has the question really assessed
the learning outcome?
The type of vessel that carries blood from the heart to the
lungs is an
a)artery.
b)capillary.
c)node.
d)vein.
Test construction and item analysis workshop, November 2015
Capacity Development Workshop: Test and
Item Development and Design, Laos,
September 2016
19. Hints for writing multiple choice items
How has the item improved?
The type of vessel that carries blood from the heart to the
lungs is
a)an artery*
.
b)a capillary.
c)a node.
d)a vein.
Better …
Test construction and item analysis workshop, November 2015
Capacity Development Workshop: Test and
Item Development and Design, Laos,
September 2016
20. Hints for writing multiple choice items
The stem and distractors must not give clues to the
key.
What is the answer? Has the question really assessed
the learning outcome?
Seetha is writing a report on different types of ecosystems. In
which source would she most likely find this information?
a) Cooking with Native Plants.
b) A Guide to Rock Collecting.
c) Ecosystems of Asia* .
d) The Big Book of Animal Habits.
Test construction and item analysis workshop, November 2015
Capacity Development Workshop: Test and
Item Development and Design, Laos,
September 2016
21. Hints for writing multiple choice items
The item presents a single clearly formulated question in the stem and
is written in question or sentence completion format.
Can you cover the options and still answer to the item?
From the article, the reader can tell that
a)turtles like to hide under the rocks.
b)dogs like to play in the snow.
c)squirrels like to eat mulberries.
d)cats like to chase mice.
Test construction and item analysis workshop, November 2015
Capacity Development Workshop: Test and
Item Development and Design, Laos,
September 2016
22. Hints for writing multiple choice items
How has the item improved?
From the article, the reader can tell that turtles like to
a) hide under the rocks.
b) play in the snow.
c) eat mulberries.
d) chase mice.
Better …
Test construction and item analysis workshop, November 2015
Capacity Development Workshop: Test and
Item Development and Design, Laos,
September 2016
23. Hints for writing multiple choice items
The stem and answer options are phrased in positive terms.
Have you ever been asked what is not the answer to a question?
None of the following cities are state capitals except
a) Bellary.
b) Hyderabad*
.
c) Katni.
d) Pune.
Test construction and item analysis workshop, November 2015
Capacity Development Workshop: Test and
Item Development and Design, Laos,
September 2016
24. Hints for writing multiple choice items
How has the item improved?
Better …
Which one of the following cities is a state capital?
a) Bellary.
b) Hyderabad*.
c) Katni.
d) Pune.
Test construction and item analysis workshop, November 2015
Capacity Development Workshop: Test and
Item Development and Design, Laos,
September 2016
25. Hints for writing multiple choice items
There are no repetitions in the options.
Why do you think that repetitions in the options can lower validity?
Milk can be pasteurized at home by
a)heating it to a temperature of 33o
C for 30 minutes.
b)heating it to a temperature of 43o
C for 30 minutes.
c)heating it to a temperature of 53o
C for 30 minutes.
d)heating it to a temperature of 63o
C for 30 minutes*
.
Test construction and item analysis workshop, November 2015
Capacity Development Workshop: Test and
Item Development and Design, Laos,
September 2016
26. Hints for writing multiple choice items
Better …
The minimum temperature that can be used to pasteurise milk at home is
a) 33o
C.
b) 43o
C.
c) 53o
C.
d) 63o
C*
.
How has the item improved?
Test construction and item analysis workshop, November 2015
Capacity Development Workshop: Test and
Item Development and Design, Laos,
September 2016
27. Hints for writing multiple choice items
Ensure that there is only one correct or clearly best answer.
What is the correct answer to this item?
Which one is the odd one out?
a) Billiards.
b) Cricket.
c) Hockey.
d) Football.
Test construction and item analysis workshop, November 2015
Capacity Development Workshop: Test and
Item Development and Design, Laos,
September 2016
28. Hints for writing multiple choice items
Ensure that there are neither repetitions nor opposites in the options.
Which would be the most likely effect of this change in fiscal policy?
a)The inflation rate would decline.
b)The unemployment rate would rise.
c)Consumer spending would increase. *
d)Consumer spending would decrease.
What is the correct answer to this item?
Test construction and item analysis workshop, November 2015
Capacity Development Workshop: Test and
Item Development and Design, Laos,
September 2016
29. Hints for writing multiple choice items
Answer options are plausible and similar in context,
ideas, focus, phrasing and length.
What is the correct answer to this item?
Why has the item been made d) so long?
Epistemology is the branch of philosophy dealing with
a) the nature of science.
b) morality.
c) beauty.
d) the nature and origin of knowledge – that is, the manner in which
human beings sense and process external stimuli in the form of
knowledge.
Test construction and item analysis workshop, November 2015
Capacity Development Workshop: Test and
Item Development and Design, Laos,
September 2016
30. Hints for writing multiple choice items
Specific determiners (always, all, never, only, none)
must be used cautiously.
What is the correct answer to this item? How could the weak
students get this item correct without really knowing the answer?
Achievement tests help students to improve their learning by
a) encouraging them all to study hard.
b) informing them of their progress.
c) giving them all a feeling of success.
d) preventing any of them from neglecting their work.
Test construction and item analysis workshop, November 2015
Capacity Development Workshop: Test and
Item Development and Design, Laos,
September 2016
31. Hints for writing multiple choice items
Use “All of the above” and “None of these”
sparingly.
What is the correct answer to this item?
What is wrong with this item?
Which of the following levels are included in Bloom’s Taxonomy?
a) Comprehension.
b) Application.
c) Analysis.
d) All of the above.
Test construction and item analysis workshop, November 2015
Capacity Development Workshop: Test and
Item Development and Design, Laos,
September 2016
32. Hints for writing multiple choice items
What is the problem with using “None of the above” as the
answer?
Which one of the following is a level in Bloom’s Taxonomy for the
cognitive domain?
a)Critical Thinking.
b)Scientific Thinking.
c)Reasoning.
d)None of the above.
Another Example...
Test construction and item analysis workshop, November 2015
Capacity Development Workshop: Test and
Item Development and Design, Laos,
September 2016
33. Hints for writing multiple choice items
Ensure that answer options do not overlap with each
other.
What is the correct answer to this item?
What is wrong with this item?
If the scores on a test have a reliability of 0.75, what percentage of an
observed score is attributable to errors of measurement?
a) Over 5%.
b) Over 10%.
c) Over 20%.
d) Over 30%.
Test construction and item analysis workshop, November 2015
Capacity Development Workshop: Test and
Item Development and Design, Laos,
September 2016
34. Hints for writing multiple choice items
How has the item improved?
Better …
If the scores on a test have a reliability of 0.75, what percentage
of an observed score is attributable to errors of measurement?
a) 2.5%.
b) 5%.
c) 25%.
d) 50%.
Test construction and item analysis workshop, November 2015
Capacity Development Workshop: Test and
Item Development and Design, Laos,
September 2016
35. Writing multiple choice items from an
editorial point of view
The items must be
• free from spelling, punctuation, grammatical and other editorial
faults;
• presented with appropriate text fonts (size, type), highlighting
(bold, underlining, italics) and layout (paragraphing and
positioning);
• arranged in such a way that the students do not have to turn
pages to link sources to questions (stems to options); and,
• arranged in the paper in the order from easiest to hardest.
Test construction and item analysis workshop, November 2015
Capacity Development Workshop: Test and
Item Development and Design, Laos,
September 2016
36. One result of having a theatre in a community is more
a) jobs.
b) teachers.
c) stores.
d) crime.
Quick Quiz
What is the Problem with Item 1?
Test construction and item analysis workshop, November 2015
Capacity Development Workshop: Test and
Item Development and Design, Laos,
September 2016
37. In Charles Dickens’ novel A Christmas Carol, which characteristic
describes Ebenezer Scrooge?
a) Miserly.
b) Nervous.
c) Inquisitive.
d) All of the above.
Quick Quiz
What is the Problem with Item 2?
Test construction and item analysis workshop, November 2015
Capacity Development Workshop: Test and
Item Development and Design, Laos,
September 2016
38. The table below shows snowfall totals for Pokharan in February.
Based on the table, what percent of days in February had
snowfall of 5 cm or more?
a) Less than 20%.
b) Less than 30%.
c) More than 65%.
d) More than 75%.
Quick Quiz
What is the Problem with Item 3?
Test construction and item analysis workshop, November 2015
Capacity Development Workshop: Test and
Item Development and Design, Laos,
September 2016
39. Why should candy be eaten sparingly between meals?
a)Candy depletes energy.
b)Candy causes diabetes.
c)Candy causes headaches.
d)Candy dulls the appetite for other foods essential for proper
nutrition.
Quick Quiz
What is the Problem with Item 4?
Test construction and item analysis workshop, November 2015
Capacity Development Workshop: Test and
Item Development and Design, Laos,
September 2016
40. Quick Quiz (True or False or Don’t Know
[Need More Information])
A GOOD ITEM
1. Measures a specific learning objective.
2.Contains subject matter and vocabulary that is above the student’s
grade-level.
3. Has only one correct answer or clearly best answer.
4. Assesses trivial or obscure subject matter.
5. Free from grammatical clueing.
6. Is free of negative wording such as “not” or “none of the above.”
Test construction and item analysis workshop, November 2015
Capacity Development Workshop: Test and
Item Development and Design, Laos,
September 2016
41. Quick Quiz (True or False or Don’t Know
[Need More Information])
A GOOD ITEM
7. Assesses more than one concept.
8. Contains options that are opposite of one another.
9. Contains distractors that assess common errors or
misconceptions.
Test construction and item analysis workshop, November 2015
Capacity Development Workshop: Test and
Item Development and Design, Laos,
September 2016
42. Quick Quiz (True or False or Don’t Know
[Need More Information])
1. Distractors should be parallel in content, structure, and
length.
2. Cognitive level refers to the difficulty level of an item.
3. Items should be written so that the content in the item is
accessible to the widest range of students.
4. Parallelism refers to when a student from different ethnic,
sex, or cultural groups perform differently on an item.
5. Controversial items are often assessed with the multiple-
choice format.
Test construction and item analysis workshop, November 2015
Capacity Development Workshop: Test and
Item Development and Design, Laos,
September 2016
43. Quick Quiz (True or False or Don’t Know
[Need More Information])
6. Item fairness means that the item assesses all students at the
appropriate age and enrolled grade-level.
7. An example of bias is presenting a culturally stereotypical
situation in the item.
Test construction and item analysis workshop, November 2015
Capacity Development Workshop: Test and
Item Development and Design, Laos,
September 2016
Notas del editor
Background on mass, weight and density:
The mass of an object is a fundamental property of the object; a numerical measure of its inertia; a fundamental measure of the amount of matter in the object. Its SI unit is the kilogram and it is a constant. If the mass of an object is 1 kg on earth, it would be the same on the moon even though it would weigh only one-sixth as much as it’s weight on earth.
The weight of an object is defined as the force of gravity on the object and may be calculated as the mass times the acceleration of gravity, w = mg. Since the weight is a force, its SI unit is the newton.
How are weight and mass different?
To understand the differences we need to compare a few points:1) Mass is a measurement of the amount of matter something contains, while Weight is the measurement of the pull of gravity on an object.2) Mass is measured by using a balance comparing a known amount of matter to an unknown amount of matter. Weight is measured on a scale.3) The Mass of an object doesn't change when an object's location changes. Weight, on the other hand, does change with location.
A spring balance (balance X) is used to measure weight and a beam balance (balance Y) is used to measure mass. The science teacher is expected to have done lab work using these two balances when doing this concept. Hence, their inclusion in this diagram can appeal to kinesthetic learners too. Density is mass per volume. So lead, gold, platinum is denser than lithium -one of the least dense metals; honey is denser than water.