The outline of the standard method to develop an SAQ.

1. DEVELOPING SHORT ANSWER
QUESTIONS (SAQs)
Presented in a workshop at Quaid-e-
Azam Medical College, Bahawalpur
Facilitator: Prof. Javed Iqbal

2. Objective
To write quality Short Answer Questions

3. Overview
• Brief introductory presentation
• Presentation and critique of submitted SAQs
• Tea break
• Develop new SAQs in groups
• Presentation and critique

4. Criteria of a good item
• Objective
• Valid (Content validity)
• Reliable
• Feasible
• Positive educational impact

5. Levels of Cognition
• Evaluation
• Synthesis
• Analysis
• Application
• Comprehension
• Knowledge

6. Two basic questions
• Why CPSP has decided to switch over from
Short Essay Questions (SEQs) to Short Answer
Questions?
• Why CPSP does not have both MCQ papers?

7. Answer to Q.1
• The very word ‘Essay’ invites to write more
and tests:
– Organization of thought
– Knowledge of English language
– Writing skills
Instead the purpose is to test:
– Interpretation, Reasoning & Problem-solving
Skills

8. Answer to Q.2
• Very presence of ‘Best Answer’ among the
options of an MCQ has a cueing effect
• An MCQ tests (or provide solution to) only one
aspect of a problem and that too without
justification for arriving to that solution
Whereas
We may need answers for different aspects
of same problem with reasons!

9. LEVEL OF KNOWLEDGE TESTED
MCQs
SAQs
Breadth of knowledge
Depth of knowledge

10. SHORT ANSWER QUESTIONS (SAQs)
• Supply type item (a form of open-ended
question) with no cueing
• Response restricted to point-wise answers
• More objective as point-wise answers make
agreement on key easier
Norman GR, Van der Vleuten CPM, Newble DI (eds.) International Handbook of Research in Medical
Education. UK. Kluwer Academic Publishers; 2002. Chap. 20 , Assessment of Knowledge with Written Test
Forms: pp.664-6

11. LIMITATIONS OF SAQs
• Appropriate selection of words is essential
• Construction of vignette and the key require
careful deliberation.
• If breadth of knowledge is to be tested, MCQs
are a better option.
Amin Z, Seng CY, Eng KH. Practical Guide to Medical Student Assessment. Singapore. World Scientific,2006. Chap.8, Short Answer Questions
(SAQs): pp. 34-6
Norman GR, Van der Vleuten CPM, Newble DI (eds.) International Handbook of Research in Medical
Education. UK. Kluwer Academic Publishers; 2002. Chap. 20, Assessment of Knowledge
with Written Test Forms: pp.664-6

12. COMPARISON BETWEEN SEQ AND SAQ
S# Characteristic SEQ SAQ
1. Type
2. Division of
question
3. Response
4. Tests
5. Objectivity
6. Content Validity
7. Inter-rater
reliability
8. Key construction
9. Educational impact

13. COMPARISON BETWEEN SEQ AND SAQ
S# Characteristic SEQ SAQ
1. Type Supply type Supply type
2. Division of
question
May be divided Divided into 3-4 small tasks
3. Response Greater freedom Restricted point-wise answer
4. Test Higher levels of
cognition
Reasoning & Problem solving
skills
5. Objectivity Low High
6. Content Validity Low High
7. Inter-rater
reliability
Low High
8. Key construction Difficult Easier
9. Educational impact Writing more will earn
high scores
Write to the point answers

14. STEPS FOR WRITING SAQs
• Identify topic/concept that requires exploring of different
aspects such as diagnosis along with reasons to arrive to
the diagnosis.
• Construct clinical vignette on the concept and link
questions to the vignette
• Frame 3-4 questions on specific tasks you want
candidates to perform
•
• Select appropriate wordings for each question

15. WORDINGS FOR GOOD SAQs
The wording of each question should:
• Be simple, clear and tests an important
aspect
• Select appropriate words to restrict the
response e.g. Name 3 preliminary
investigations instead of saying ‘what
investigations you will order?’ Give one
reason for each investigation.

16. WORDS TO BE AVOIDED
Avoid words that invite lengthy answers, like:
– Discuss
– Describe
– Explain
– Outline
– Comment
– Write a short note

17. DEVELOPMENT OF KEY
•Instructions should include a mechanism to
award marks to the required number of answers
• Develop a key according to the key features of
the answer for each question
• Assign weight (marks) to each component
answer according to its importance

18. Example of an SAQ
You are a medical officer in OBGY. You have been asked to
review a 28 years old woman , para 2 who has presented
with a 6 weeks’ amenorrhea & slight vaginal bleeding. She
is haemodynamically stable. Ultrasound shows a complex
right adnexal mass, 3x2 cms in size. There is no pregnancy
sac in the uterus.
Case Vignette
Exam: Final fellowship
Importance: Essential

19. Example No.1 of an SAQ (contd…)
Q1. What is the most likely diagnosis?
(2 marks)
Q2. Name three preliminary investigations you will
like to perform at this stage.
(3 marks)
Q3. What will be immediate steps in the
management of this patient? (5 marks)
Questions

20. Example No.1 of an SAQ (contd…)
Ans (1) Tubal pregnancy 2 marks
Ans (2) 3 preliminary investigations are:
• Complete blood count
• Blood group and RH factor
• Serum B-HCG. 3 marks
Ans (3) lmmediate steps in management are:
• Save I/V line 5 marks
• Arrange two units of blood
Key

21. • Medical management by methotrexate 50 mgm I/M if
white cell and platelet count are normal, and serum B-
HCG are <15000mIU/ml.
• Serial monitoring by B-HCG level every 48 hrs. Expect
15% drop from day 4-7 after injection
• Laparoscopy/laparotomy if initial B-HCG is above
15000mIU/ml or if the patient becomes haemo-
dynamically unstable.
Ref:Ectopic Pregnancy.Washington:American College of Obstetricians and Gynecologists.19900ACOG Technical Bulletin No 150.
Example No.1 of an SAQ (contd…)
Key