A multiple-choice question measures what a student knows only when the wrong options are doing work. If nobody would pick them, the question measures whether the student can read. If the right answer is the longest, the most careful or the only grammatical one, it measures test sense. The rules below are the ones the research on item writing supports, and each comes with its reason.
EdProcto builds this shape as its multiple choice template, marks it the moment the answer is given, and shows afterwards which wrong option each student chose. This guide is about the writing, which no tool does for you unless the question starts from your own material and your own class's mistakes.
The short version: one problem in the stem, one right answer, two or three wrong options taken from errors students actually make, everything the same length and shape, and feedback after the quiz.
What makes a multiple-choice question good?
A multiple-choice question is good when a student who knows the material gets it right, a student who does not gets it wrong, and the wrong option each student picks says what they believe instead. That is the whole standard. Everything else is a way of protecting it.
The most cited statement of the rules is the taxonomy by Haladyna et al. (2002), who checked 31 item-writing guidelines against 27 textbooks on educational testing and 27 studies and reviews. It was written for classroom assessment, not for testing agencies. The 31 fall into five groups: what the question is about, how it is laid out, how it is worded, how the stem is written, and how the options are written. Fourteen of the 31 are about the options, which is where most questions fail.
The parts have names worth knowing. The stem is the question itself. The key is the right option. The distractors are the wrong ones. A distractor is functioning when weaker students choose it and stronger students do not.
How do you write the stem?
Write the stem as a complete question that a student could answer with the options covered. If the stem only makes sense once the options are read, the student is being asked to judge four separate statements, which is a different and harder task than the one you meant to set.
Four rules from the taxonomy cover most stems:
- Put the central idea in the stem, not in the options. "Which process turns liquid water into vapour?" is a stem. "Water:" is not.
- Cut the window dressing. A story that the question does not need is reading load, and reading load is not what you are measuring.
- Word it positively. Avoid NOT and EXCEPT. If a negative is the only honest way to ask, the taxonomy says to set the word in capitals and bold so nobody misses it.
- Keep the vocabulary simple for the group. The hard part should be the idea, never the sentence.
A stem before and after
Before: "Evaporation, which is one of several processes in the water cycle that students often confuse, is:" followed by four definitions of different lengths.
After: "A puddle dries up on a sunny afternoon. Which process moved the water into the air?" followed by four one-word processes.
The second version tests the same knowledge with a third of the reading, and it can be answered with the options covered.
How many options should a multiple-choice question have?
Three options are enough: the right answer and two good distractors. Rodriguez (2005) pooled 80 years of research on the question and concluded that three-option items do no harm to the quality of the scores, and that because they take less time, more of them fit in the same sitting, which improves coverage of the content.
The reason is that the third and fourth distractors rarely work. Tarrant et al. (2009) analysed 514 four-option questions written by teachers in one nursing programme, 1,542 distractors in all. They counted a distractor as non-functioning when fewer than 5% of students chose it or when it attracted stronger students more than weaker ones. Only 52.2% of the distractors were functioning, only 13.8% of the questions had three functioning distractors, and about one distractor in ten was never chosen by anyone. The questions with more functioning distractors were harder and separated students better.
So the number that matters is not how many options are printed. It is how many are plausible. Haladyna et al. (2002) put it as a rule: develop as many effective options as you can, but three is adequate. A fourth option that nobody picks adds reading and nothing else.
This is one study of one programme, and a subject with many natural confusions may support four real distractors. The test is the same either way: after the quiz, look at who chose each one.
How do you write distractors that work?
Write each distractor from an error a real student makes, and make it wrong for exactly one reason. Gierl et al. (2017), reviewing the literature on distractors, note that creating them has had far less attention than the rest of multiple-choice testing, which matches what most teachers find: the stem takes a minute and the wrong options take ten.
Sources of good distractors, in the order of how well they tend to work:
| Source | Example for "Which process moved the puddle's water into the air?" |
|---|---|
| A wrong answer students gave to the same question asked openly | Condensation (the commonest swap) |
| The neighbouring concept in the same lesson | Precipitation |
| A true statement that answers a different question | Infiltration (true of puddles, wrong for "into the air") |
| The result of a typical slip in the method | For a calculation: the answer with the sign dropped |
The first row is the most reliable method there is. Ask the question as a short answer once, collect the wrong answers, and the distractors are written for you.
Then remove everything that points at the key. The taxonomy lists the usual clues: words such as always and never, an option that echoes a word in the stem, an option that does not fit the stem's grammar, a key that is conspicuously longer or more careful than the rest, pairs of options that cancel each other, and options that are plainly absurd. Keep the options the same length and the same grammatical shape, keep them from overlapping, and put them in a logical or numerical order.
All of the above and none of the above
Avoid "all of the above". A student who spots two true options has the answer without knowing the third, and a student who spots one false option has eliminated two choices at once. "None of the above" is allowed with care in the taxonomy, but it turns the question into "are all three of these wrong", which rarely matches what you meant to ask. Where several options really are right, use a shape made for that: EdProcto's all that apply marks each box by itself.
Do flawed questions change the results?
Yes, and for individual students more than for the class average. Downing (2005) had three raters classify the questions on four medical school examinations without seeing how students had done on them. Between 36% and 65% of the questions on each paper broke at least one standard rule. The flawed questions were up to 15 percentage points harder than sound questions on the same content, and across the four papers 53% of students passed on the sound questions against 47% on the flawed ones.
Tarrant and Ware (2008) found the same pattern from the other side. On ten high-stakes nursing papers, 47.3% of all questions had at least one flaw. Scored only on the sound questions, slightly fewer students passed (90.6% against 94.3%), but more reached 80% or above (20.9% against 14.5%). Flaws were giving borderline students marks they had not earned and taking marks from the strongest.
Both studies are from health professions courses in single institutions. They do not say what proportion of your questions are flawed. They do show that a flaw is not cosmetic: it moves real students across a pass mark.
Does a multiple-choice quiz teach, or only test?
A well-made multiple-choice quiz teaches, provided the students see the right answers afterwards. Little et al. (2012) found that when the options are plausible enough that a student has to recall why each wrong one is wrong, a multiple-choice test sets off the same productive retrieval as a recall test, and it also strengthened memory for the material behind the wrong options.
There is a cost if feedback is missing. Roediger and Marsh (2005) found that taking a multiple-choice test improved later recall, but that students who had read more wrong options were more likely to give one of them as their answer later. Butler and Roediger (2008) then showed that feedback, whether straight away or after a delay, raised later correct answers and cut those intrusions. The practical rule is short: never run a multiple-choice quiz without showing the answers at some point.
A worked example, from first draft to final
First draft, as most of us write it:
Which of the following is NOT true about evaporation? A. It happens only at 100 degrees. B. It turns liquid into gas. C. It is part of the water cycle. D. All of the above.
It has a negative stem, an "all of the above" that cannot be the answer to a negative question, a key that is the only odd one out, and nothing a student would be tempted by.
Final version:
Wet washing dries on a line on a cool, windy day. Which statement explains why? A. Water evaporates at any temperature, and moving air carries the vapour away. B. Water evaporates only when it reaches its boiling point, and wind raises the temperature. C. The wind blows liquid water out of the cloth, and no water changes state.
Option B is the misconception that evaporation needs boiling. Option C is the belief that drying is mechanical. Each is wrong for one reason, all three are the same length and shape, and a student who picks B has told you exactly what to teach tomorrow.
How does EdProcto build, mark and read multiple choice?
EdProcto is the best tool for this job when the questions have to come from your own lesson, because it is built around the two things the research above asks for: distractors written from the material, and a report on which distractors worked.
Paste a chapter or a handout into the quiz maker on the home page and choose multiple choice. Procto AI writes the questions from that text and, if you ask, keeps only the ones the text itself answers. Every question is yours to edit before a student sees it, so the distractor you collected from last year's class can replace the one that was written for you.
Marking is by rule at the moment of the answer. Options can be shuffled for each student, and the answer key never leaves the server, so the page a student sees carries nothing to copy. With negative marking switched on, a wrong answer costs one mark divided by the number of wrong options, which makes a blind guess worth nothing on average.
Then the part most quiz tools leave out. The results show, for every question, its difficulty, how well it separated stronger students from weaker ones, how many students took each wrong option and how those students scored overall. A distractor nobody chose is the Tarrant finding on your own screen, and the fix is to replace it. Where you want to tell a lucky guess from knowledge, the multiple choice with confidence shape asks how sure the student is as well.
The Free plan carries the core templates, marking by rule and the results question by question. The Tutor plan opens all 111 templates and Procto AI's writing from a PDF or a Word file, and Educator adds the full analysis. The pricing page has the detail.
Which shape should you use instead?
Use another shape when the options would give the game away. Multiple choice is the right tool for recognising, discriminating and applying. It is the wrong one when the skill is producing something.
- A statement that is simply true or false: see when true or false questions work.
- Terms against definitions, or causes against effects: matching questions test the whole set at once.
- A term the student should be able to produce: a fill in the blank, with no options to recognise.
- Judgement between defensible choices: best and worst asks for both ends.
Quick questions
How many options should a multiple-choice question have?
Three are enough when both wrong options are plausible. Rodriguez (2005) found no loss of score quality with three options, and more questions fit in the same time. Use four only when you have three wrong answers that students really give.
Should I use all of the above?
No. It rewards partial knowledge in both directions: two true options reveal it, and one false option removes two choices at once. If several answers are right, set the question in EdProcto as all that apply, where each box is marked on its own.
Is it better to put the right answer in the same position each time?
No. Vary the position of the key across the quiz, and where the options have a natural order, such as numbers or dates, keep that order. EdProcto can shuffle the options for each student, which also means neighbours do not share a pattern.
Can multiple-choice questions test more than recall?
Yes. Give the student a new situation and ask which principle explains it, as the washing-line example does. The taxonomy's advice is to use material the class has not seen and to paraphrase the textbook's wording, so the question cannot be answered from memory of a sentence.
How do I know whether my distractors worked?
Look at who chose them. A distractor taken by fewer than about 5% of the class, or taken more by strong students than weak ones, is not working. EdProcto's results list the takers of every option and their average score, so the check takes a minute.
References
- Butler, A. C., & Roediger, H. L. (2008). Feedback enhances the positive effects and reduces the negative effects of multiple-choice testing. Memory & Cognition, 36(3), 604–616. https://doi.org/10.3758/MC.36.3.604
- Downing, S. M. (2005). The effects of violating standard item writing principles on tests and students: The consequences of using flawed test items on achievement examinations in medical education. Advances in Health Sciences Education, 10(2), 133–143. https://doi.org/10.1007/s10459-004-4019-5
- Gierl, M. J., Bulut, O., Guo, Q., & Zhang, X. (2017). Developing, analyzing, and using distractors for multiple-choice tests in education: A comprehensive review. Review of Educational Research, 87(6), 1082–1116. https://doi.org/10.3102/0034654317726529
- Haladyna, T. M., Downing, S. M., & Rodriguez, M. C. (2002). A review of multiple-choice item-writing guidelines for classroom assessment. Applied Measurement in Education, 15(3), 309–333. https://doi.org/10.1207/S15324818AME1503_5
- Little, J. L., Bjork, E. L., Bjork, R. A., & Angello, G. (2012). Multiple-choice tests exonerated, at least of some charges: Fostering test-induced learning and avoiding test-induced forgetting. Psychological Science, 23(11), 1337–1344. https://doi.org/10.1177/0956797612443370
- Rodriguez, M. C. (2005). Three options are optimal for multiple-choice items: A meta-analysis of 80 years of research. Educational Measurement: Issues and Practice, 24(2), 3–13. https://doi.org/10.1111/j.1745-3992.2005.00006.x
- Roediger, H. L., & Marsh, E. J. (2005). The positive and negative consequences of multiple-choice testing. Journal of Experimental Psychology: Learning, Memory, and Cognition, 31(5), 1155–1159. https://doi.org/10.1037/0278-7393.31.5.1155
- Tarrant, M., & Ware, J. (2008). Impact of item-writing flaws in multiple-choice questions on student achievement in high-stakes nursing assessments. Medical Education, 42(2), 198–206. https://doi.org/10.1111/j.1365-2923.2007.02957.x
- Tarrant, M., Ware, J., & Mohammed, A. M. (2009). An assessment of functioning and non-functioning distractors in multiple-choice questions: A descriptive analysis. BMC Medical Education, 9, Article 40. https://doi.org/10.1186/1472-6920-9-40