Insights and development of tests

Insights andInsights and
Development ofDevelopment of
TestsTests
Carlo Magno, PhDCarlo Magno, PhD
De La Salle University, ManilaDe La Salle University, Manila

Activity 1: Assessment SchemaActivity 1: Assessment Schema
Check-upCheck-up
Answer the following questions as a group. YourAnswer the following questions as a group. Your
answers should reflect your current practices inanswers should reflect your current practices in
assessing your students. Write your answers in a manilaassessing your students. Write your answers in a manila
paper and post it after.paper and post it after.
1. List down the things that you do when preparing to1. List down the things that you do when preparing to
write your test items?write your test items? (procedure)(procedure)
2. What are the things that you consider when writing2. What are the things that you consider when writing
your test items?your test items? (concepts)(concepts)
3. What further steps do you do after you have scored3. What further steps do you do after you have scored
and recorded the test papers?and recorded the test papers? (procedure)(procedure)
4. What other forms of assessment do you conduct4. What other forms of assessment do you conduct
aside from paper and pencil tests?aside from paper and pencil tests?

1. List down the things that you do when1. List down the things that you do when
preparing to write your test items?preparing to write your test items?
Prepare Table of Specifications (TOS)Prepare Table of Specifications (TOS)
Use the Taxonomy of Cognitive skillsUse the Taxonomy of Cognitive skills
(Bloom’s taxonomy)(Bloom’s taxonomy)
Conduct Item reviewConduct Item review

2. What are the things that you consider2. What are the things that you consider
when writing your test items?when writing your test items?
Learning objectivesLearning objectives
Curriculum/national standardsCurriculum/national standards
Needs of studentsNeeds of students
Higher order thinking skillsHigher order thinking skills
Test lengthTest length
Test instructionTest instruction
Test layoutTest layout
ScoringScoring

3. What further steps do you do after you3. What further steps do you do after you
have scored and recorded the test papers?have scored and recorded the test papers?
Item analysisItem analysis
– Item difficultyItem difficulty
– Item discriminationItem discrimination
Distracter analysisDistracter analysis
Reliability analysisReliability analysis
Validity analysisValidity analysis

4. What other forms of assessment do you4. What other forms of assessment do you
conduct aside from paper and pencil tests?conduct aside from paper and pencil tests?
Alternative forms of assessmentAlternative forms of assessment
– Performance-based AssessmentPerformance-based Assessment
– Authentic assessmentAuthentic assessment
– Portfolio assessmentPortfolio assessment

Types of Response FormatTypes of Response Format
Selected-response itemsSelected-response items
– Binary-choice itemsBinary-choice items
– Matching typeMatching type
– Multiple choice typeMultiple choice type
Constructed-responseConstructed-response
– Short-answer itemsShort-answer items
– Cloze testCloze test
– Essay itemsEssay items

Binary TypeBinary Type
1. Avoid the use of “specific determiners”1. Avoid the use of “specific determiners”
FAULTY: No picture-no sound in aFAULTY: No picture-no sound in a
television set may indicate a bad 5U4G.television set may indicate a bad 5U4G.
IMPROVED: A bad 5U4G tube in aIMPROVED: A bad 5U4G tube in a
television set will result in no picturetelevision set will result in no picture
sound.sound.
Why is the item faulty?

2. Base true-false items upon statements2. Base true-false items upon statements
that are absolutely true or false, withoutthat are absolutely true or false, without
qualifications or exceptions.qualifications or exceptions.
FAULTY: World War II was fought inFAULTY: World War II was fought in
Europe and the Far East.Europe and the Far East.
IMPROVED: The primary combatIMPROVED: The primary combat
locations in terms of military personnellocations in terms of military personnel
during World War II were Europe and theduring World War II were Europe and the
Far East.Far East.

3. Avoid negative stated items when3. Avoid negative stated items when
possible and eliminate all doublepossible and eliminate all double
negatives.negatives.
FAULTY: It is not frequently observed thatFAULTY: It is not frequently observed that
copper turns green as a result ofcopper turns green as a result of
oxidation.oxidation.
IMPROVED: Copper will turn green uponIMPROVED: Copper will turn green upon
oxidizing.oxidizing.

4. Use quantitative and precise rather4. Use quantitative and precise rather
than qualitative language where possible.than qualitative language where possible.
FAULTY: Many people voted for GloriaFAULTY: Many people voted for Gloria
Arroyo in the 2003 Presidential election.Arroyo in the 2003 Presidential election.
IMPROVED: Gloria Arroyo received moreIMPROVED: Gloria Arroyo received more
than 60 percent of the popular votes castthan 60 percent of the popular votes cast
in the Presidential election of 2003.in the Presidential election of 2003.

5. Avoid stereotypic and textbook statements.5. Avoid stereotypic and textbook statements.
FAULTY: From time to time efforts have beenFAULTY: From time to time efforts have been
made to explode the notion that there may be amade to explode the notion that there may be a
cause-and-effect relationship between arborealcause-and-effect relationship between arboreal
life and primate anatomy.life and primate anatomy.
IMPROVED: There is a known relationshipIMPROVED: There is a known relationship
between primate anatomy and arboreal life.between primate anatomy and arboreal life.

6. Avoid making the true items consistently6. Avoid making the true items consistently
longer than the false items.longer than the false items.
7. Avoid the use of unfamiliar or esoteric7. Avoid the use of unfamiliar or esoteric
language.language.
FAULTY: According to some peripateticFAULTY: According to some peripatetic
politicos, the raison d’etre for capital punishmentpoliticos, the raison d’etre for capital punishment
is retribution.is retribution.
IMPROVED: According to some politicians,IMPROVED: According to some politicians,
justification for the existence of capitaljustification for the existence of capital
punishment can be traced to the Biblicalpunishment can be traced to the Biblical
statement, “An eye for an eye.”statement, “An eye for an eye.”

8. Avoid complex sentences with many8. Avoid complex sentences with many
dependent clauses.dependent clauses.
FAULTY: Jane Austen, an AmericanFAULTY: Jane Austen, an American
novelist born in 1790, was a prolific writernovelist born in 1790, was a prolific writer
and is best known for her novel Pride andand is best known for her novel Pride and
Prejudice, which was published in 1820.Prejudice, which was published in 1820.
IMPROVED: Jane Austen is best knownIMPROVED: Jane Austen is best known
for her novel Pride and prejudice.for her novel Pride and prejudice.

9. It is suggested that the crucial elements of9. It is suggested that the crucial elements of
an item be placed at the end of the statement.an item be placed at the end of the statement.
FAULTY: Oxygen reduction occurs more readilyFAULTY: Oxygen reduction occurs more readily
because carbon monoxide combines withbecause carbon monoxide combines with
hemoglobin faster than oxygen does.hemoglobin faster than oxygen does.
IMPROVED: Carbon monoxide poisoning occursIMPROVED: Carbon monoxide poisoning occurs
because carbon monoxide dissolves delicatebecause carbon monoxide dissolves delicate
lung tissue.lung tissue.

Matching TypeMatching Type
Match List A with List B. You will be given oneMatch List A with List B. You will be given one
point for each correct match.point for each correct match.
List AList A List BList B
a. cotton gina. cotton gin a. Eli Whitneya. Eli Whitney
b. reaperb. reaper b. Alexander Graham Bellb. Alexander Graham Bell
c. wheelc. wheel c. David Brinkleyc. David Brinkley
d. TU54G tubed. TU54G tube d. Louisa May Alcottd. Louisa May Alcott
e. steamboate. steamboat e. None of thesee. None of these
•Directions failed to specify the basis for matching
•List are enumerated identically
•Responses not listed logically
•Lacks homogeneity
•Equal number of elements
•Use of “None of the above”

Matching TypeMatching Type
IMPROVED: Famous inventions are listed in the left-handIMPROVED: Famous inventions are listed in the left-hand
column and inventors in the right-hand column below.column and inventors in the right-hand column below.
Place the letter corresponding to the inventor in the spacePlace the letter corresponding to the inventor in the space
next to the invention for which he s famous. Each match isnext to the invention for which he s famous. Each match is
worth 1 point, and “None of these” may be the correctworth 1 point, and “None of these” may be the correct
answer. Inventors may be used more than once.answer. Inventors may be used more than once.
InventionsInventions InventorsInventors
__ 1. steamboat__ 1. steamboat a. Alexander Graham-Bella. Alexander Graham-Bell
__ 2. cotton skin__ 2. cotton skin b. Robert Fultonb. Robert Fulton
__ 3. sewing machine__ 3. sewing machine c. Elias Howec. Elias Howe
__ 4. reaper__ 4. reaper d. Cyrus McCormickd. Cyrus McCormick
e. Eli Whitneye. Eli Whitney
f. None of thesef. None of these

Multiple ChoiceMultiple Choice
1. Salvador Dali is1. Salvador Dali is
a. a famous Indian.a. a famous Indian.
b. important in international law.b. important in international law.
c. known for his surrealistic art.c. known for his surrealistic art.
d. the author of many avant-garde plays.d. the author of many avant-garde plays.
•It is recommended that the stem be a direct question.
•The stem should pose a clear, define, explicit, and
singular problem.

IMPROVED: With which one of the fineIMPROVED: With which one of the fine
arts is Salvador Dali associated?arts is Salvador Dali associated?
a. surrealistic paintinga. surrealistic painting
b. avant-garde theatreb. avant-garde theatre
c. polytonal symphonic musicc. polytonal symphonic music
d. impressionistic poetryd. impressionistic poetry

2. Milk can be pasteurized at home by2. Milk can be pasteurized at home by
a. heating it to a temperature of 130a. heating it to a temperature of 130oo
b. Heating it to a temperature of 145b. Heating it to a temperature of 145oo
c. Heating it to a temperature of 160c. Heating it to a temperature of 160oo
d. Heating it to a temperature of 175d. Heating it to a temperature of 175oo
•Include in the stem any words that might otherwise be
repeated in each response.

IMPROVED: The minimum temperatureIMPROVED: The minimum temperature
that can be used to pasteurize milk atthat can be used to pasteurize milk at
home is:home is:
a. 130a. 130oo
b. 145b. 145oo
c. 160c. 160oo
d. 175d. 175oo

3. Although the experimental research, particularly that by3. Although the experimental research, particularly that by
Hansmocker must be considered equivocal andHansmocker must be considered equivocal and
assumptions viewed as too restrictive, most testingassumptions viewed as too restrictive, most testing
experts would recommend as the easiest method ofexperts would recommend as the easiest method of
significantly improving paper-and-pencil achievementsignificantly improving paper-and-pencil achievement
test reliability totest reliability to
a. increase the size of the group being tested.a. increase the size of the group being tested.
b. increase the differential weighting of items.b. increase the differential weighting of items.
c. increase the objective of scoring.c. increase the objective of scoring.
d. increase the number of items.d. increase the number of items.
e. increase the amount of testing time.e. increase the amount of testing time.
Items should be stated simply and understandably,
excluding all nonfunctional words from stem and
alternatives.

IMPROVED: Assume a 10-item, 10-minuteIMPROVED: Assume a 10-item, 10-minute
paper-and-pencil multiple choice achievementpaper-and-pencil multiple choice achievement
test has a reliability of .40. The easiest way oftest has a reliability of .40. The easiest way of
increasing the reliability to .80 would be toincreasing the reliability to .80 would be to
increasedincreased
a. group sizea. group size
b. scoring objectivityb. scoring objectivity
c. differential item scoring weightsc. differential item scoring weights
d. the number of itemsd. the number of items
e. testing timee. testing time

4. None of the following cities is a state4. None of the following cities is a state
capital exceptcapital except
a. Bangora. Bangor
b. Los Angelesb. Los Angeles
c. Denverc. Denver
d. New Havend. New Haven
•Avoid negatively stated items

IMPROVED: Which of the following citiesIMPROVED: Which of the following cities
is a state capital?is a state capital?
a. Bangora. Bangor
b. Los Angelesb. Los Angeles
c. Denverc. Denver
d. New Havend. New Haven

5. Who wrote Harry Potter and the Goblet5. Who wrote Harry Potter and the Goblet
of Fire?of Fire?
a. J. K. Rowlinga. J. K. Rowling
b. Manny Paquiaob. Manny Paquiao
c. Lea Salongac. Lea Salonga
d. Mark Twaind. Mark Twain
•If possible the alternatives should be presented in some
logical, numerical, or systematic order.
•Response alternatives should be mutually exclusive.

IMPROVED: Who wrote Penrod?IMPROVED: Who wrote Penrod?
a. J. K. Rowlinga. J. K. Rowling
b. J. R. R. Tolkienb. J. R. R. Tolkien
c. V. Hugoc. V. Hugo
d. L. Carrold. L. Carrol

6. Which of the following statements6. Which of the following statements
makes clear the meaning of the wordmakes clear the meaning of the word
“electron”?“electron”?
a. An electronic toola. An electronic tool
b. Neutral particlesb. Neutral particles
c. Negative particlesc. Negative particles
d. A voting machined. A voting machine
e. The nuclei of atomse. The nuclei of atoms
•Make all responses plausible and attractive to the less
knowledgeable and skillful student.

IMPROVED: Which of the followingIMPROVED: Which of the following
phrases is a description of an “electron”?phrases is a description of an “electron”?
a. Neutral particlea. Neutral particle
b. Negative particleb. Negative particle
c. Neutralized protonc. Neutralized proton
d. Radiated particled. Radiated particle
e. Atom nucleuse. Atom nucleus

7. What is the area of a right triangle7. What is the area of a right triangle
whose sides adjacent to the right anglewhose sides adjacent to the right angle
are 4 inches long respectively?are 4 inches long respectively?
a. 7a. 7
b. 12b. 12
c. 25c. 25
d. None of the aboved. None of the above
•The response alternative “None of the above” should be
used with caution, if at all.

IMPROVED: What is the area of a rightIMPROVED: What is the area of a right
triangle whose sides adjacent to the righttriangle whose sides adjacent to the right
angle are 4 inches and 3 inchesangle are 4 inches and 3 inches
respectively?respectively?
a. 6 sq. inchesa. 6 sq. inches
b. 7 sq. inchesb. 7 sq. inches
c. 12 sq. inchesc. 12 sq. inches
d. 25 sq. inchesd. 25 sq. inches
e. None of the abovee. None of the above

8. As compared with the American factory8. As compared with the American factory
worker in the early part of the 19th century, theworker in the early part of the 19th century, the
American factory worker at the close of theAmerican factory worker at the close of the
centurycentury
a. was working long hoursa. was working long hours
b. received greater social security benefitsb. received greater social security benefits
c. was to receive lower money wagesc. was to receive lower money wages
d. was less likely to belong to a labor union.d. was less likely to belong to a labor union.
e. became less likely to have personal contacte. became less likely to have personal contact
with employerswith employers
Make options grammatically parallel to each other and
consistent with the stem.

IMPROVED: As compared with the AmericanIMPROVED: As compared with the American
factory worker in the early part of the century,factory worker in the early part of the century,
the American factory worker at the close of thethe American factory worker at the close of the
centurycentury
a. worked longer hours.a. worked longer hours.
b. had more social security.b. had more social security.
c. received lower money wages.c. received lower money wages.
d. was less likely to belong to a labor uniond. was less likely to belong to a labor union
e. had less personal contact with his employere. had less personal contact with his employer

9. The “standard error of estimate’ refer to9. The “standard error of estimate’ refer to
a. the objectivity of scoring.a. the objectivity of scoring.
b. the percentage of reduced errorb. the percentage of reduced error
variance.variance.
c. an absolute amount of possible error.c. an absolute amount of possible error.
d. the amount of error in estimatingd. the amount of error in estimating
criterion scores.criterion scores.
Avoid such irrelevant cues as “common elements” and “pat
verbal associations.”

IMPROVED: The “standard error ofIMPROVED: The “standard error of
estimate” is most directly related to whichestimate” is most directly related to which
of the following test characteristic?of the following test characteristic?
a. Objectivitya. Objectivity
b. Reliabilityb. Reliability
c. Validityc. Validity
d. Usabilityd. Usability
e. Specificitye. Specificity

10. What name is given to the group of10. What name is given to the group of
complex organic compounds that occur incomplex organic compounds that occur in
small quantities in natural foods that aresmall quantities in natural foods that are
essential to normal nutrition?essential to normal nutrition?
a. Caloriea. Calorie
b. Mineralsb. Minerals
c. Nutrientsc. Nutrients
d. Vitaminsd. Vitamins
In testing for understanding of a term or concept, it is
generally preferable to present the term in the stem and
alternative definitions in the options.

IMPROVED: Which of the followingIMPROVED: Which of the following
statements is the best description of astatements is the best description of a
vitamin?vitamin?

Short Answer TypeShort Answer Type
1. Require short, definite, clear-cut, and1. Require short, definite, clear-cut, and
explicit answersexplicit answers
FAULTY: Earnest Hemingway wrote______FAULTY: Earnest Hemingway wrote______
IMPROVED: The Old Man and the Sea wasIMPROVED: The Old Man and the Sea was
written by _______.written by _______.
Who wrote The Old man and the Sea?Who wrote The Old man and the Sea?

2. Avoid multimutilated statements2. Avoid multimutilated statements
FAULTY: _____ pointed out in ____ theFAULTY: _____ pointed out in ____ the
freedom of thought in America wasfreedom of thought in America was
seriously hampered by ___, ____, & __.seriously hampered by ___, ____, & __.
IMPROVED: That freedom of thought inIMPROVED: That freedom of thought in
America was seriously hampered by socialAmerica was seriously hampered by social
pressures toward conformity was pointedpressures toward conformity was pointed
out in 1830 by ______.out in 1830 by ______.

3. If several equal answers equal credit3. If several equal answers equal credit
should be given to each one.should be given to each one.
4. Specify and announce in advance4. Specify and announce in advance
whether scoring will take spelling intowhether scoring will take spelling into
account.account.

5. In testing for comprehension of terms and5. In testing for comprehension of terms and
knowledge of definition, it is often better toknowledge of definition, it is often better to
supply the term and require a definition than tosupply the term and require a definition than to
provide a definition and require the term.provide a definition and require the term.
FAULTY: What is the general measurement termFAULTY: What is the general measurement term
describing the consistency with which items in adescribing the consistency with which items in a
test measure the same thing?test measure the same thing?
IMPROVED: Define “internal consistencyIMPROVED: Define “internal consistency
reliability.”reliability.”

6. It is generally recommended that in completion6. It is generally recommended that in completion
items the blanks come at the end of theitems the blanks come at the end of the
statement.statement.
FAULTY: A (an) ________ is the index obtainedFAULTY: A (an) ________ is the index obtained
by dividing a mental age score by chronologicalby dividing a mental age score by chronological
age and multiplying by 100.age and multiplying by 100.
IMPROVED: The index obtained by dividing aIMPROVED: The index obtained by dividing a
mental age score by chronological age andmental age score by chronological age and
multiplying by 100 is called a (an) ________multiplying by 100 is called a (an) ________

7. Minimize the use of textbook expressions7. Minimize the use of textbook expressions
and stereotyped language.and stereotyped language.
FAULTY: The power to declare war isFAULTY: The power to declare war is
vested in ______vested in ______
IMPROVED: Which national legislative bodyIMPROVED: Which national legislative body
has the authority to declare war?has the authority to declare war?

8. Specify the terms in which the response is to be8. Specify the terms in which the response is to be
given.given.
FAULTY: Where does the Security Council of theFAULTY: Where does the Security Council of the
United Nations hold its meeting?United Nations hold its meeting?
IMPROVED: In what city of the United States doesIMPROVED: In what city of the United States does
the Security Council of the United Nations holdthe Security Council of the United Nations hold
its meeting?its meeting?
FAULTY: If a circle has 4-inch diameter, its areaFAULTY: If a circle has 4-inch diameter, its area
is_____is_____
IMPROVED: A circle has 4-inch diameter. Its areaIMPROVED: A circle has 4-inch diameter. Its area
in square inches correct to two decimal places,in square inches correct to two decimal places,
is _____is _____ Why is the item faulty?

9. In general, direct questions are9. In general, direct questions are
preferable to incomplete declarativepreferable to incomplete declarative
sentences.sentences.
FAULTY: Gold was discovered in CaliforniaFAULTY: Gold was discovered in California
in the year ___in the year ___
IMPROVED: In what year was goldIMPROVED: In what year was gold
discovered in California?discovered in California?

10. Avoid extraneous clues to the correct10. Avoid extraneous clues to the correct
answeranswer
FAULTY: A fraction whose denominator isFAULTY: A fraction whose denominator is
greater than its numerator is a _____greater than its numerator is a _____
IMPROVED: Fractions whose denominatorIMPROVED: Fractions whose denominator
are greater than their numerators areare greater than their numerators are
called _____called _____

ReferenceReference
Magno, C., & Ouano, J. (2010).Magno, C., & Ouano, J. (2010). DesigningDesigning
Written Assessment for student learningWritten Assessment for student learning..
Manila: Phoenix.Manila: Phoenix.

Activity 4: Insights onActivity 4: Insights on
AssessmentAssessment
1.1. After the activities, I realized that my testAfter the activities, I realized that my test
……
2.2. After the activity, now I know that …After the activity, now I know that …
3.3. After the activity, every time I constructAfter the activity, every time I construct
test, I will …test, I will …

Insights and development of tests

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (20)

Similar to Insights and development of tests

Similar to Insights and development of tests (20)

More from Carlo Magno

More from Carlo Magno (20)

Recently uploaded

Recently uploaded (20)

Insights and development of tests