How to write good distractors for multiple-choice questions

The stem took a minute and the key took ten seconds. Then come three empty boxes, and the temptation to fill them with anything that is wrong. Good distractors for multiple-choice questions are not found that way.

A distractor is a hypothesis about how a student goes wrong: someone who makes this mistake will land here. EdProcto's multiple choice template runs the question, and its results test each hypothesis by showing how many students chose each option.

The short version: start from real errors, write one distractor for each, make every option look like the key, and let the results decide which stay.

What is a distractor supposed to do?

A distractor is supposed to attract the students who hold one wrong idea, and nobody else. Then nobody finds the key by crossing out nonsense, and each wrong choice names an error you can teach against.

Plausible wrong answers also teach. Little et al. (2012) found that multiple-choice tests with plausible alternatives improved later recall of information related to the wrong options, which cued-recall tests did not.

The rules for the stem and the key are in how to write multiple-choice questions that measure. This guide covers the wrong answers alone, the subject Gierl et al. (2017) review in full.

Where do good distractors come from?

Good distractors come from errors, and there are five places to look, listed in the order we would try them. No study we read ranks all five against one another, so the order is practical advice.

1. Answers students gave to the open version

Ask the question with no options first, as a short answer, and tally what comes back. A biology example with an invented tally: 28 students are asked why antibiotics will not cure a cold. Eleven write that a cold is a virus, eight that you would become immune, five that a cold is not serious enough, and four give scattered answers or none. The two wrong answers that recur become the distractors, reworded to match the key:

Why will antibiotics not cure a cold? A. A cold is caused by a virus, and antibiotics act only on bacteria. B. A person who takes antibiotics for a cold becomes immune to them. C. A cold is too mild an illness for antibiotics to act on.

2. The result of one named wrong step

For a calculation, work the problem with one deliberate error and use what comes out. A maths example: a jacket costs £37.20 after a 20% reduction. What did it cost before?

• The correct step, dividing by 0.80: 37.20 ÷ 0.80 = £46.50.
• Adding 20% of the sale price: 37.20 × 1.20 = £44.64.
• Taking 20% off a second time: 37.20 × 0.80 = £29.76.

Our guide to questions above recall does the same with steps of reasoning.

3. The neighbouring concept

Use the idea taught beside the right one. A psychology example:

A driver fastens her seat belt to stop the warning chime, and next time she fastens it sooner. Which process is this? A. Negative reinforcement B. Positive reinforcement C. Positive punishment

The key is A: something unpleasant stops, and the behaviour grows. B belongs to the student who thinks all reinforcement adds a reward, C to the one who hears an unpleasant chime and thinks of punishment.

4. A true fact that answers another question

A sentence remembered from the notes tempts even when it does not answer the stem. A history example:

Which statement explains why East Germany built the Berlin Wall in August 1961? A. East Germans were leaving for the West through Berlin in large numbers. B. Germany had been divided into four occupation zones in 1945. C. West Berlin had been supplied by air during the blockade of 1948 and 1949.

All three are true, and only A answers the stem. Take care: if a strong student could defend B, the question has two keys.

5. The common-sense answer the lesson was meant to replace

Some lessons exist to overturn a belief students arrive with, so put that belief among the options. A physics example:

A 1 kg steel ball dropped from a balcony lands after 1.0 s. A 2 kg steel ball is dropped from the same height. Air resistance is negligible. When does it land? A. After 0.5 s B. After 0.9 s C. After 1.0 s

A is the everyday belief at full strength, B the student half persuaded. Sadler (1998) gave 1,250 students in grades 8 to 12 an astronomy test whose distractors matched ideas recorded in qualitative studies of children's thinking. His abstract, all we could read, calls them highly attractive.

How many distractors should you write?

Write one for every real error you have found and keep the best two or three, because most questions do not support more. The series' main guide, linked above, gives the classic studies behind the three-option rule, and four others agree.

• A medical school's papers. Kilgour and Tayyaba (2016) examined 480 five-option questions, each sat by about 269 students. Of 1,920 distractors, 55.3% were chosen by fewer than 5% of students and 17.8% by nobody. Only 7.1% of questions had four working distractors.
• A revised pool. Testa et al. (2018) analysed 635 distractors in 164 statistics questions that lecturers wrote and two of the authors revised. Counting a distractor as working when at least 5% chose it and its takers were the weaker students, 68.5% worked.
• Professionally written tests. In 840 five-option questions from credentialing tests, Raymond et al. (2019) found that most had one or two functional distractors by the 5% rule, and that 47.3% had three or four by an index that allows for difficulty. We could read only the abstract.
• Filler does harm. Among 1,925 questions by 50 veterinary faculty, Rush et al. (2016) found that four distractors discriminated no better than three (0.186 against 0.182). Questions judged to contain an implausible distractor were easier (95% correct against 80.1%) and separated students less well (0.09 against 0.20).

So stop at the last real error. None of the four samples comes from a school classroom.

Can a machine suggest the distractors?

A language model can suggest distractors that are valid, but in the one study we read it was worse than people at predicting the errors students make. Feng et al. (2024) had large language models write the three distractors for maths questions aimed at students aged 10 to 13, then asked two graduate students with teaching experience to rate machine and human distractors for 20 questions, mixed together, from 1 to 5. Machine distractors averaged 3.28 for mathematical validity against 3.99 for human ones, and 2.68 against 3.72 for how likely a student was to choose them. The authors call the sample small.

Treat a machine draft as a first list and your class's errors as the edit. In EdProcto, Procto AI writes the questions from your material, and every question is yours to change.

How do you stop the options giving the key away?

Make every option look as though it could be the key: a distractor built on a real error still fails if its form marks it as wrong. Among the multiple-choice questions Rush et al. (2016) rated, the key was the longest option in 20.6%, 15.4% carried a grammatical clue and 5.9% used absolute terms.

• Same length. Trim the key or lengthen the others.
• Same grammar. Every option completes the stem.
• Same specificity. If the key has a unit, a date or a qualifier, they all do.
• No overlap. If "under 10" is right, so is "under 20". Avoid opposites too: students who see two mutually exclusive options conclude that one is the key, as Rush et al. note.
• No "always" or "never". Test-wise students read absolutes as false.
• A plausible order. Numbers ascending, dates in sequence, otherwise alphabetical.
• Numbers that came from somewhere. A key of 46.50 among 40, 50 and 60 is the only untidy figure, and a key with distractors ten below and ten above sits mid-pattern. Take each number from a wrong step.

How do you test a set of distractors before the quiz?

Run three checks before any student sees the question.

1. Cover the stem. Read the options alone. If you can pick the key, the form is giving it away.
2. Ask a colleague to guess the key without the stem. Then show the stem and ask which wrong option students will find least convincing.
3. Name the error beside each distractor. One with no note is filler.

A colleague helps, and several help more. Rogausch et al. (2010) asked 37 experienced clinicians to mark the least plausible options in 55 questions from a Swiss graduation examination. On the 22 questions where only one or two distractors had drawn under 5% of candidates, 61% of the clinicians on average marked one of those, and the majority view did so for 18 of the 22.

How do you read the results and retire a distractor?

Count who chose each option, check how those students scored overall, and retire a distractor only when both say it is idle.

1. Open the question in EdProcto's results. For a question with one right choice they give, for every option, the number of students who took it, their share of the submissions, and their mean score on the whole quiz.
2. Count the takers. The usual research rule calls a distractor non-functioning when under 5% of students choose it (Kilgour and Tayyaba, 2016), and Rush et al. (2016) advise rewriting or removing those. In a class of 30, 5% is one and a half students, so pool classes or years before judging.
3. Allow for an easy question. When over 90% answer a five-option question correctly, at most one distractor can reach 5% (Raymond et al., 2019). Testa et al. (2018) used a simple benchmark: the percentage who got the question wrong, divided by the number of distractors. At 92% correct with three distractors that is 8 ÷ 3, about 2.7% each.
4. Compare the means. A working distractor's takers score lower on the whole quiz than the students who chose the key, the test Testa et al. (2018) apply. If they score as high, suspect a second defensible answer or a wrong key.
5. Decide. Keep it, reword it, replace it, or run the question with one option fewer.

EdProcto writes two findings as sentences: when a wrong option with three or more takers drew more students than the key, and when ten or more students have scores and an option drew nobody. The item analysis guide explains the other figures.

Bad distractors and the repair for each

What does a distractor nobody chose tell you?

A distractor nobody chose tells you one of three things, and only the first is a fault: it was never plausible, the question was easy for this class, or the teaching worked.

If the error filled last year's open answers and nobody takes it after your lesson, the empty row is the result you taught for. In Sadler (1998), instruction appeared to strengthen support for such distractors before it declined, so a distractor's share moves as a class learns. Rogausch et al. (2010) add that candidates who took rarely chosen distractors scored lower than those who took popular ones.

Before deleting, ask whether the error was ever real. If it was, keep the distractor for the next cohort or a before-and-after test.

How does EdProcto help with distractors?

EdProcto is the best tool for distractors because it works at both ends of the job: it gathers the open answers that distractors come from, and afterwards it reports how many students chose each option and how they scored. In the medical school study above, over half the distractors drew under 5% of students, and only the results showed which.

To collect, set the open version as a short answer and read what the class writes. To draft, paste your material into the quiz maker on the home page and choose multiple choice, one of the templates: Procto AI writes the questions from that text, and you swap in the errors you collected.

Options can be shuffled for each student; leave that setting off when the options are numbers in order. With negative marking on, a wrong choice costs the question's marks divided by the number of wrong options: a third with three distractors, a half with two.

On Free you get the core templates, marking by rule and the results for each question. Tutor opens every template. The full analysis, with the distractor report, is on Educator. See pricing.

Quick questions

How many distractors should a multiple-choice question have?

Two or three that work. In Rush et al. (2016), questions with four distractors discriminated no better than those with three.

What makes a distractor plausible?

A plausible distractor is the answer a student reaches by one real mistake: a wrong step, a neighbouring idea or an everyday belief. If you cannot name the mistake, the option is filler.

Should every question have the same number of options?

No. Let the count follow the number of real errors.

How do I know when to replace a distractor?

When almost nobody chooses it across enough students to judge, or when its takers score as high as the key's. EdProcto's results show both figures for every option.

References

• Feng, W., Lee, J., McNichols, H., Scarlatos, A., Smith, D., Woodhead, S., Ornelas, N., & Lan, A. (2024). Exploring automated distractor generation for math multiple-choice questions via large language models. In Findings of the Association for Computational Linguistics: NAACL 2024 (pp. 3067–3082). Association for Computational Linguistics. https://doi.org/10.18653/v1/2024.findings-naacl.193
• Gierl, M. J., Bulut, O., Guo, Q., & Zhang, X. (2017). Developing, analyzing, and using distractors for multiple-choice tests in education: A comprehensive review. Review of Educational Research, 87(6), 1082–1116. https://doi.org/10.3102/0034654317726529
• Kilgour, J. M., & Tayyaba, S. (2016). An investigation into the optimal number of distractors in single-best answer exams. Advances in Health Sciences Education, 21(3), 571–585. https://doi.org/10.1007/s10459-015-9652-7
• Little, J. L., Bjork, E. L., Bjork, R. A., & Angello, G. (2012). Multiple-choice tests exonerated, at least of some charges: Fostering test-induced learning and avoiding test-induced forgetting. Psychological Science, 23(11), 1337–1344. https://doi.org/10.1177/0956797612443370
• Raymond, M. R., Stevens, C., & Bucak, S. D. (2019). The optimal number of options for multiple-choice questions on high-stakes tests: Application of a revised index for detecting nonfunctional distractors. Advances in Health Sciences Education, 24(1), 141–150. https://doi.org/10.1007/s10459-018-9855-9
• Rogausch, A., Hofer, R., & Krebs, R. (2010). Rarely selected distractors in high stakes medical multiple-choice examinations and their recognition by item authors: A simulation and survey. BMC Medical Education, 10, Article 85. https://doi.org/10.1186/1472-6920-10-85
• Rush, B. R., Rankin, D. C., & White, B. J. (2016). The impact of item-writing flaws and item complexity on examination item difficulty and discrimination value. BMC Medical Education, 16, Article 250. https://doi.org/10.1186/s12909-016-0773-3
• Sadler, P. M. (1998). Psychometric models of student conceptions in science: Reconciling qualitative studies and distractor-driven assessment instruments. Journal of Research in Science Teaching, 35(3), 265–296. https://doi.org/10.1002/%28SICI%291098-2736%28199803%2935:3%3C265::AID-TEA3%3E3.0.CO;2-P
• Testa, S., Toscano, A., & Rosato, R. (2018). Distractor efficiency in an item pool for a statistics classroom exam: Assessing its relation with item cognitive level classified according to Bloom's taxonomy. Frontiers in Psychology, 9, Article 1585. https://doi.org/10.3389/fpsyg.2018.01585