1. Reverse Multiple-Choice
Method (RMCM)
Indu M. Anand
Mathematics Department
University of Massachusetts – Lowell
AMCOA Conference, February 9, 2012

2. RMCM -- The Method
Reverse Multiple-Choice Method:
 Presents content for Teaching or
Testing
 Introduced in Patent No. US
7,033,182
 Uses multiple-choice question format
 Formalizes reinforcement &
explanation
 Harnesses power of “wrong” answers

3. RMCM -- The Method (contd)
Requires examinee to:
 (i) examine each answer choice in the
context of the query;
 (ii) identify key information that makes
the answer correct or incorrect; then,
 (iii) change the given facts to make an
incorrect answer choice the "correct"
answer to changed query.

4. RMCM Method (contd)…
 Correct answer,  no modification
 For a “wrong” answer choice
Examinee:--
 Must examine answer choice
 Must understand why answer is
incorrect
 Must communicate why it is incorrect
 More efficient than Open format /
essay
 May be automated for grading

5. How it works – Examinee‟s
view…
Examinee, typically, is asked:
 To select correct answer
 Then --
 select one (or more) wrong answer
 change the query so that
selected wrong answer is the
correct answer for changed query
 Modification criteria may be provided

6. How it works – RMCM Question
 Question is key data structure of system.
 Components of the Question object are:
◦ Query – “Narrative” + Call of the question
◦ Answer choices – Multiple , One correct
others incorrect
 RMCM Questions:
◦ May be written from the scratch, or
◦ May be developed from other multiple-
choice questions
◦ Stored in persistent databanks of
questions

7. RMCM Question (contd)
◦ Special features of RMCM question:
◦ Fact Objects (FO‟s) = Facts / segments
of the query on which the answer turns;
FO = True or False for an answer
choice, depending on its relevance to
answer choice
◦ Fact Object Values (FV‟s) – Values
assigned to relevant FO‟s for original or
modified query.

8. Structure…
Test Question
Query
AnswerChoice
Body
Call
FactObject FactValue
Value Score
1
1..*
1..*

9. RMCM Question - Structure
Answer Choices should be closely
related
1
Answer choice
FO/FV
2 5
3 Query 4

10. How it works – Examiner‟s
view…
 Select incorrect answer choices
strategically in order to --
 Emphasize the facts provided in the
question
 Identify “Fact Objects” for each
answer choice;
 Provide “Fact Values” of the “True”
Fact Objects ( for each answer choice)
 Provide score / partial score for each
FO for each answer choice

11. RMCM – Examiner‟s view
(contd)
Examiner must:
 Define the Objectives for the question
Figure out what concept to assess
 Know where and how students miss the
concept
Form answer choices around common errors
 Work Backwards
Correlate facts / segments in query with
errors
 Determine importance of each FO
Apportion relative credit for the facts /
segments

12. How it works – System view
The RMCM system:
 Assists Examiner with
◦ Editorial support to write or import questions;
◦ Selecting “Fact Objects” for an answer choice
with (proprietary) segmentation software;
◦ Maintaining FO/FV databases, and credit
generation databases
◦ Generating computer-assisted queries
 Assists Examinee with navigation

13. How it works System view
(contd)
 Compare Examinee‟s answers with Examiner‟s
data
◦ Identifying FO‟s for selected wrong answer
choice +
◦ Fact Values for (True) FO‟s for selected wrong
answer
 Calculate score for question based on
comparison
◦ Identification, or not, of correct answer; and
◦ Identification of FO‟s for selected wrong answer
◦ Fact Values for FO‟s for the selected wrong

14. How the system works – Logic I

15. How the system works – Logic II

16. Example 1
What is the area of a rectangle with
sides 3 and 4?
A. 5
B. 12
C. 14
D. 16
E. 9

17. Screen mock up for Example
1

18. Screen mock up continued

19. Screen mock up continued

20. Screen mock up continued

21. Example 2
Divide and simplify the expression:
(10) 2 =
(A) 8
(B) 2 2
(C) 5
(D) 2 5
(E) 2 3
 This question is taken from REA‟s SAT preparation guidebook (1990)

22. Example 2 (contd)
 Correct answer is C.
 (A) would be the correct answer for
the expression: (10-2);
 (B) would be correct for the
expression: 2 * ( 5 - 1 );
 (D) would be correct for the
expression: (10 * 2);
 (E) would be correct for the
expression:
(10 + 2).

23. Example 3

24. Example 3 (contd)
What is the current in the 4 Ω resistor
while switch S is open?
(A) Is correct if 12 V is changed to 0 V;
(B) Is the correct answer;
(C) Is correct if 12 V is changed to 24 V;
(D) Is correct if 12 V is changed to 40 V;
(E) I correct if 12 V is changed to 60 V.

25. Example 3 & Discussion
 Other variants possible with different
sets of Fact Objects; e.g.,
 Resistors with other resistance values;
or
 Changes of both resistance and
voltage.
 One interesting case is where a Fact
Object is “Switch S is open or closed.”
 In general, FO holds information –
may be numeric or non-numeric
 Examples: Mathematical operation;
Word order in a sentence; Sequence
of lab activity

26. Example 4 – Math word
problem*
 The sophomore class plans to sell T-
shirts with the school‟s name on them.
The cost of each T-shirt alone is $3.50,
and the printing cost of each shirt is
$0.75. If the class plans on selling each
printed T-shirt for $11, what expression
can you use to calculate the class profit
for selling n printed T-shirts?
 A. 11.00 – (3.50 + 0.75)n
 B. 11.00n – (3.50 + 0.75)
 C. 11.00 – 3.50 – 0.75n
 D. (11.00 – 3.50 – 0.75)n
* From 2001 MCAS test for 10th grade

27. Example 4; non-numeric FO
 The sophomore class plans to sell T-
shirts with the school‟s name on them.
The cost of each T-shirt alone is
$3.50, and the printing cost of each
shirt is $0.75. If the class plans on
selling each printed T-shirt for
$11, what expression can you use to
calculate the class profit for selling n
printed T-shirts?
 A. 11.00 – (3.50 + 0.75)n
 B. 11.00n – (3.50 + 0.75)
 C. 11.00 – 3.50 – 0.75n
 D. (11.00 – 3.50 – 0.75)n

28. Non-numeric FO highlighted
 The sophomore class plans to sell T-
shirts with the school‟s name on them.
The cost of each T-shirt alone is
$3.50, and the printing cost of each
shirt is $0.75. If the class plans on
selling each printed T-shirt for
$11, what expression can you use to
calculate the class profit for selling n
printed T-shirts?
 A. 11.00 – (3.50 + 0.75)n
 B. 11.00n – (3.50 + 0.75)
 C. 11.00 – 3.50 – 0.75n
 D. (11.00 – 3.50 – 0.75)n

29. Other question FO‟s
highlighted
 The sophomore class plans to sell T-
shirts with the school‟s name on them.
The cost of each T-shirt alone is
$3.50, and the printing cost of each
shirt is $0.75. If the class plans on
selling each printed T-shirt for $11,
what expression can you use to
calculate the class profit for selling n
printed T-shirts?
 A. 11.00 – (3.50 + 0.75)n
 B. 11.00n – (3.50 + 0.75)
 C. 11.00 – 3.50 – 0.75n
 D. (11.00 – 3.50 – 0.75)n

30. Examiner‟s selection & save of
FO
 The sophomore class plans to sell T-
shirts with the school‟s name on them.
The cost of each T-shirt alone is
$3.50, and the printing cost of each
shirt is $0.75. If the class plans on
selling each printed T-shirt for $11,
what expression can you use to
calculate the class profit for selling n
printed T-shirts?
 A. 11.00 – (3.50 + 0.75)n
 B. 11.00n – (3.50 + 0.75)
 C. 11.00 – 3.50 – 0.75n
 D. (11.00 – 3.50 – 0.75)n
selling each printed T-shirt

31. RMCM query modifications…
 A: Change „selling each printed shirt‟ to
„selling all printed shirts.‟
 B: Change „cost of each T-shirt‟ to „cost of all
T-shirts,‟ and change „printing cost of each
shirt‟ to „printing cost of all shirts.‟
 C: Change „cost of each T-shirt‟ to „cost of all
T-shirts,‟ and change „selling each printed
shirt‟ to „selling all printed shirts.‟
 D: (may indicate that student lacks
understanding of the material tested and may
lead to complete loss of credit for the
question).

32. Another RMCM embodiment of
Example 4…
 Examinee Task:
◦ choose an answer that you believe
is incorrect.
◦ find facts /segments from LIST, so
that chosen incorrect answer
becomes correct answer to
changed question
◦ LIST: --„cost of each T-shirt;‟
„alone;‟ „$3.50;‟ „printing cost of
each shirt;‟ „$0.75;‟ „selling each
printed shirt;‟ „$11;‟ „profit;‟ „selling n
printed shirts‟

33. ◦ LIST: „cost of each T-shirt;‟ „alone;‟
„$3.50;‟ „printing cost of each shirt;‟
„$0.75;‟ „selling each printed shirt;‟
„$11;‟ „‟profit;‟ „selling n printed
shirts.‟
Expected answers:
 A: „selling each printed shirt.‟
 B: „cost of each T-shirt‟ and „printing cost of each
shirt.‟
 C: „cost of each T-shirt‟ and „selling each printed
shirt.‟
 D: (test-taker lacks understanding of subject

34. Wrap up - Advantages of RMCM
 Multiple-choice format used for
RMCM:--
 Allows context definition economically
 Questions may be reused
 Scoring may be automated
 Other advantages of automation e.g.
computer-aided generation of queries
 Forces analysis/ compares / contrasts
of similar and related concepts

35. Wrap up - Disadvantages
mitigated by RMCM
 Unlike traditional Multiple-choice
RMCM:
 Discourages answering without careful
reading
 Encourages analysis of questions
 Finds depth of student‟s learning
 Databank of questions has lasting
value

36. Disadvantages of open format/
essay mitigated by RMCM
◦ Encourages identification of context
◦ Discourages padding of answers
with irrelevant writing
◦ More efficient
◦ Computer-aided assessment
possible

37. Invitation
 If you would like to get involved --
 Effort is underway to:
◦ Assemble groups of educators / others
◦ Assemble RMCM Questions databanks
◦ Develop self-study courses
◦ Develop instructor-led courses
You are also invited to contribute to further
design and development of RMCM
software

39. Thank You!
Contact
Indu M. Anand
 indu_m_anand@yahoo.com
 indu_anand@uml.edu