FAQ - Can you give me some advice on how to write high quality multiple choice and extended matching questions?
Answer
Item writing: multiple choice questions
The following is summarised from NBME handbook by Case and Swanson.
Multiple choice questions (MCQs) can be presented in two main formats:
1. True/false format
True/false items require an examinee to select all the options that are true. Problems arise if the options are not either completely true or completely false. Stems (the lead in statement or question) must be clear and unambiguous. Imprecise or vague terms such as: 'is important', 'is associated with', 'is useful for', 'usually' and 'frequently' should be avoided as should words that provide cueing such as 'may' or 'could be'.
By their nature these tests assess the recall of a single isolated fact. This combined with the problematic nature of writing this style of MCQ has led to many experts rejecting them in favour of one best answer MCQs.
2. One best answer format
This style of test is recommended for assessing the application of knowledge therefore many are written as clinical scenarios, describing findings, test results etc.
STRUCTURE: This question type consists of a stem (e.g. a clinical case presentation), a lead in question and a series of choices (typically one correct answer and four distractors).
IMPORTANT POINTS:
(i) Choices should be consistent, that is, they all fall into the same category as the correct answer (e.g. all diagnoses, drugs, treatments etc)
(ii) It is important to write options that are grammatically consistent and logically compatible with the stem; and they should be listed in alphabetical or logical order
(iii) Distractors should be plausible and the same relative length as the correct answer (iv) Avoid absolute and vague terms
(v) Keep stems long and options short and
(vi) to determine if a question is clear cover up the options and see if it is possible to answer.
CONTENT: When testing basic or clinical science knowledge the use of patient or lab vignettes is recommended. Items should focus on an important concept e.g. a common or potentially life threatening clinical problem. Application of knowledge rather than basic recall can be ensured by designing the item to require the examinee to reach a conclusion, make a prediction or select a course of action. Using a clinical vignette different clinical topics can be assessed such as health and health maintenance, mechanisms of disease, diagnosis, and management.
Negative marking: Negative marking is almost unknown outside UK medical schools and public examination boards do not use it. It is used to discourage guessing although candidates who do guess are statistically likely to guess wrongly as this is the function of the distractors. It penalises candidates who lack confidence even though they may have an equal knowledge and understanding of the subject matter as more confident individuals.
For further details please see FAQ:I have heard that negative marking of multiple-choice questions can have undesirable results. What is the research on this?
Item writing: extended-matching questions
Extended-matching items are similar to one-best MCQs, from which they were developed. They have a question stem, but it is usually longer and more detailed than in MCQs. There are also more options, typically 8 to 10, but sometimes many more. Even more than one-best MCQs, extended-matching items test the ability to analyse, interpret and apply clinical information. Information is presented in the form of clinical vignettes, which reinforce the clinical relevance of the information and require students to integrate several pieces of information.
An extended-matching question (EMQs) is made up of four components: a theme, a lead in, a number of options and a number of item stems. The theme relates to the topic which is being addressed it could be anatomic sites, cell types, pathogens etc. The options form the list of potential answers. The item stem is written as a clinical vignette, which describes a patient in a clinical situation. Finally, the lead in statement provides directions to the examinee- it indicates the relationship between the stems and options e.g. for each patient described below, select the most likely diagnosis. The process for writing these items is to 1) identify a clinical problem, 2) list possible diagnoses that could cause the clinical problem, and 3) write a vignette to describe the problem. Typically, there are more options on the list of answers for an EMQ than there are questions. This helps reduce respondents guessing.
Rules for writing a good EMQ are analogous to those for one-best answer items. The stems should be long and similar in structure. The list of options should be single words or short phrases and should be listed in alphabetical or logical order. Questions can be made more or less difficult by altering the number of options (to 20 or more) and the degree of discrimination between options. Final checks should confirm there is only a single best answer for each question, that there are at least four reasonable distractors for each item and that the question can be answered with the options hidden.
Disclaimer: This FAQ was written by Dr Sarah Marshall and does not reflect an official endorsement by the HEA or any other organisation. Any questions or queries should be sent to: enquiries@medev.ac.uk
Last updated: 04 July 2011
