Sign up free

Multiple choice is not the lazy option: what the evidence says

An opinion argued from published studies on what a well-written multiple-choice question measures, what it costs to write, and where the format falls short.

The EdProcto editorial teamWritten and checked against the research it cites. Last checked . How the guides are made

Guides · Writing questions that measure · · 12 min read

Is multiple choice the lazy option for a teacher setting a test?

Multiple choice is not the lazy option when the question is well written. That is our opinion, argued from published studies. Such questions can ask for reasoning, their scores largely agree with written-answer scores on the same content, and competitive wrong options help memory. The labour moves from marking to writing. Writing, speaking and building an argument still need other formats.

This is an opinion piece. Our opinion is that multiple choice is not the lazy option: a well-written multiple-choice question is slow to write, can ask for reasoning, and earns its place beside written answers. The studies under it are published, and each is named.

EdProcto publishes this guide and builds multiple choice as one of its question shapes, so we have an interest. For that reason the case against comes first, in its strongest form.

The sources were all read on 10 October 2026; "abstract read" or "index record read" marks one where no more was open. For the craft itself, see how to write multiple-choice questions that measure.

Why is multiple choice called the lazy option?

Because it marks itself, because most of the questions in use ask for recall, and because students who expect only options study differently. All three charges have evidence behind them.

The case against, at its strongest

  • It shapes what is taught. Frederiksen (1984) argued that because the multiple-choice tests in wide use do not measure the more complex skills, those skills go untaught, and that dearer formats repay their cost through what they lead teachers to teach (index record read).
  • It shapes how students study. Stanger-Hall (2012), read in full, compared two sections of one introductory biology course. One sat exams of 50 multiple-choice questions, the other 30 of them and three or four short written answers. In both, 25% to 30% of every exam asked for application or analysis. Before any exam, 72% of the mixed section said they saw the value of learning at every level, against 57%. On the 29 higher-level multiple-choice questions that both sections met in the final, the mixed section scored 64.4% and the other 59.5%. Asked whether an exam of options alone hinders critical thinking, she answers "a convincing yes". Students were not assigned at random.
  • In practice it is mostly recall. Vanderbilt et al. (2013) coded every first-year examination at one American medical school. In 19 of the 22, more than half the questions were coded as recall.

We accept two parts of this. Students prepare for the exam they expect, and most multiple-choice questions in use are recall, as our guide to questions above recall also reports. But neither fact belongs to the format: both belong to how a question is written and to what is set beside it.

Can a multiple-choice question test higher-order thinking?

Yes, as far as application, analysis and judgement between options.

  • Level is not difficulty. Tractenberg et al. (2013) had three cognitive psychologists rate 252 multiple-choice questions from a graduate physiology course on a six-level scale of complexity. For 88% of the questions, complexity was statistically independent of difficulty, and changes were identified that would lift some questions one level without making them harder (abstract read).
  • Candidates reason, and also play the test. Surry et al. (2017) asked 14 people, from medical students to staff physicians, to think aloud after sets of clinical vignette questions. What they described matched the analytical and non-analytical reasoning expected in practice, alongside test-taking moves that seem out of place there (abstract read).
  • One school, one format, a year apart. In the second year of the school that Vanderbilt et al. (2013) studied, 17 of 23 examinations were mostly application.
  • The critic's own measure. The gain in Stanger-Hall (2012) was found with multiple-choice questions: her 29 higher-level ones were sensitive enough to tell the two sections apart.

Hift (2014), reviewing the literature for final examinations in medicine, concludes that a well-built multiple-choice question tests the higher cognitive functions as well as a well-built open one.

A recall question and its higher-order twin

The example is ours, not from a study.

Which average is the middle value of a set placed in order? A. The median B. The mean C. The mode

Nine houses on a street sold for between £180,000 and £240,000, and a tenth for £2.1 million. An agent wants one figure for a typical sale. Which should she quote? A. The median, because one extreme sale hardly moves it. B. The mean, because it uses all ten prices. C. The mode, because it is the price that occurs most often.

The first is answered from a remembered definition. The second puts the definitions to work, and each wrong option is a true statement about its average that fails for this purpose. The work of writing it is in B and C.

Do multiple-choice and written-answer scores agree?

Largely, when both ask about the same content, and least where the written answer is an essay. Researchers call a written answer a constructed response.

  • Rodriguez (2003) searched for studies that compared the two formats and found 67, of which 29 reported 56 correlations. Corrected for measurement error, the correlation approached one where both formats were built on the same question. It was significantly lower where they were not, for essays most of all (abstract read).
  • Lukhele et al. (1994) fitted item response models to two Advanced Placement examinations: chemistry in 1989, sat by 18,462 students, and United States history in 1988, sat by 82,842. The constructed-response sections added little information to what the multiple-choice sections already gave (abstract and index record read).

Where the scores part company

Agreement on average is not agreement for every student.

  • Kennedy and Walstad (1997) simulated replacing the written part of the 1991 Advanced Placement economics examinations with more multiple-choice questions. The swap misclassified a number of students that was small but statistically significant (index record read).
  • Bridgeman and Lewis (1994) matched Advanced Placement scores with first-year grades at 32 colleges. Multiple-choice scores in American history and biology tracked the first-year grade average more closely than essay scores did. In history courses, though, women's grades and essay scores matched men's, while men's multiple-choice scores stood nearly half a standard deviation higher (abstract read).

So the two formats mostly rank students alike, and a paper of options alone can still misplace some of them. That is a reason to set both, not to drop either.

Does answering multiple-choice questions help students remember?

As retrieval practice it does, when the wrong options compete with the right one; a question padded with look-alikes may teach nothing.

Little and Bjork (2015) had people answer multiple-choice questions on material they had studied, with wrong options that were either competitive or not. On a later recall test they did better on new, related questions whose answers had earlier stood among the competitive wrong options, and no better where those options had been uncompetitive. Competitive options were no more likely to come back later as mistakes. The pattern held in two experiments, one with corrective feedback and one without (abstract read).

The best evidence against comes from a lecture hall. Greving and Richter (2018), read in full, gave students ten minutes at the end of each of seven psychology lectures to answer short-answer questions, answer multiple-choice questions or read summary statements. Tests one, 12 and 23 weeks later showed a gain over reading for the short answers students could retrieve, and none for multiple choice. Their multiple-choice questions were the short-answer ones with four options attached: for prosopagnosia, blindness to faces, shapes, colours or objects. No feedback followed, about two thirds were answered correctly, and the authors themselves suggest such questions asked for little effort of retrieval.

Our reading is that the lecture-hall study tested the lazy kind of question and found that it did not teach. The abstract of Little and Bjork (2015) names no classroom, so the step from competitive options to a gain in a course is our inference. Our guide to retrieval practice has the wider evidence.

What does a good wrong option cost to write?

Time: the labour of multiple choice sits in the writing, and most of that in the wrong options.

  • Time. A testing company timed 75 subject experts drafting questions for a health licensing examination. A first draft of one four-option question, with a reason for every option and a reference for the key, took 52 minutes on average, before the reviews that follow (Meazure Learning, 2014). It is one company's count of its own writers, not a peer-reviewed study.
  • Money. A panel of the National Academies of Sciences, Engineering, and Medicine (2022) reports that creating and reviewing one selected-response question for the national assessment of the United States costs 1,000 to 2,500 US dollars, against 1,500 to 3,500 for a constructed-response one. Other testing programmes, it adds, typically spend from hundreds of dollars to under 3,000 on such a question, piloting included.

No teacher pays a thousand dollars for a question, and at that scale a written-answer question costs more to create, not less. The figures show where careful organisations find the cost: in writing, review and trial. The table is our own comparison, for a teacher's own questions, of multiple choice versus written answers.

Multiple choice A written answer
Writing the question Slow: a key, and a real error behind every wrong option Quick: a prompt, then a rubric
Marking By rule, with no reader, as in EdProcto A reader, one answer at a time
Consistency One key for every paper Rests on the rubric and on moderation
What you learn Which wrong option each student chose How the student puts it in their own words
Using it again The writing is paid for once The marking is paid for at every sitting

The format is lazy only when the first row is skipped. How to write distractors that work gives the method.

Where does multiple choice honestly fall short?

On anything a student must produce: a piece of writing, a spoken answer, an argument built from nothing.

  • Writing. Choosing the best of four sentences is not composing one. In Rodriguez (2003) the formats agreed least where the written answer was an essay.
  • Speaking. Pronunciation, fluency and the power to explain aloud exist only in a voice.
  • Producing an argument. Options hand the student the claims to weigh. Stanger-Hall (2012) notes that her multiple-choice format left the top level, synthesis, unassessed.
  • Expectation. Students who expect only options prepare to recognise. This part of the case against stands, and in the one course studied, three or four written answers an exam were enough to change it.
  • Cueing. Options can prompt an answer the student could not have produced: see single-best-answer questions.

For these, set a shape in which the student makes the answer. EdProcto carries Short answer ("One to three sentences, graded against a model answer or a rubric."), Spoken answer ("A recorded answer, assessed for what it says.") and Defend a position, whose four parts are "Claim", "Evidence", "Reasoning" and "Counter-argument and rebuttal". Between choosing and producing sits Choose and justify: "A choice, then the reason for it. Tests why, not only what."

How do you set multiple choice so that it is not lazy?

Spend the time saved on marking before the quiz, on the options.

  1. Decide what the student must do. If the answer has to be produced, choose another shape.
  2. Ask about a case the class has not met. The house prices above, not the definition.
  3. Take every wrong option from a real error. If you cannot name the mistake behind one, cut it.
  4. Show the answer and the reason afterwards. In EdProcto, "Show the correct answer" and "Show the explanation" are each set to "At once", "After submitting" or "Never".
  5. Keep one produced answer in the paper. It changes what students prepare for.
  6. Read which options were chosen. See distractor analysis.

How does EdProcto run multiple choice, and what sits beside it?

EdProcto is the best tool for multiple choice that is not lazy, because it gives the format the two things the studies above ask for: wrong options that can be counted after the quiz, and produced answers on offer beside the options. In Stanger-Hall (2012), a few written answers an exam changed how a whole section studied.

The shape. "Multiple choice": "One right answer among options." It is marked by rule.

A reason beside the choice. In Choose and justify the rules mark the choice for half the points. The reason is the other half and waits for a reader.

The results. Every plan shows "Each question" with "Got it", the share who were right. On the Educator plan the full analysis adds, for each option, "Chose it" and "Their mean".

The plans. Free gives 10 quizzes a month for 30 students each, the core templates and marking by rule, and Procto AI writes 20 questions a month. Tutor reaches 100 students a quiz, with all 111 templates and 50 questions a month from Procto AI. Educator reaches 500 students a quiz and adds the full analysis. See pricing.

Quick questions

Is multiple choice the lazy option?

Not when it is well written, in our opinion. The marking is quick and the writing is slow, with most of the work in the wrong options.

Can multiple-choice questions test higher-order thinking?

Up to application, analysis and judgement between options, yes. In Tractenberg et al. (2013) the level of a question was independent of its difficulty for 88% of 252 questions.

Are multiple-choice tests as valid as written exams?

On the same content the scores largely agree (Rodriguez, 2003), least for essays. Bridgeman and Lewis (1994) show that options alone can still rank groups differently from written work.

How long does it take to write a good multiple-choice question?

One company timed its expert writers at 52 minutes for a first draft (Meazure Learning, 2014). In a class quiz, too, the wrong options take longest.

What should I set when the student has to write, speak or argue?

A shape in which the answer is produced. Among EdProcto's templates those include Short answer, Spoken answer and Defend a position, and Choose and justify adds a reason to a choice.

References

Multiple choice

One right answer among options.

In the builder now

See Multiple choice

Read next