Fifteen of thirty students picked one option, and the key marks it wrong. When half the class chose the same wrong answer, something sent them there, and it is one of five things: a shared misconception, a wrong key, an ambiguous question, a second defensible answer, or content that was never taught.
In three of the five, the students who chose it have earned the mark. So read the row before a score goes out or a lesson is planned.
EdProcto, which publishes this guide, shows how many students chose each option and how they scored on the whole quiz, in the full analysis of its results. Our guide to distractor analysis reads every option of a question; this one stays with the popular one.
What can it mean when half the class chose the same wrong answer?
It can mean five things, and they differ in who was wrong: the class, the quiz or nobody yet.
| Cause | What happened | Who was wrong |
|---|---|---|
| A shared misconception | The class holds one wrong idea, and the option is where it leads | The class, after being taught |
| A wrong key | The option is right, and the quiz marks it wrong | The quiz |
| An ambiguous question | The stem reads two ways, and on the second the option is right | The quiz |
| A second defensible answer | The stem reads one way, and two options satisfy it | The quiz |
| Something never taught | The class could not know, and took the everyday answer | Nobody yet |
The counts cannot choose between them. They rule out only blind guessing: by our arithmetic, thirty students picking at random among four options put fifteen or more on any single one in about one class in ninety.
Nor does a pile prove that the point was taught and lost. Halloun and Hestenes (1985) measured what students believed about motion on entering a first college physics course: those common-sense beliefs had a large effect on how they performed, and conventional teaching changed them little. A class can bring its wrong answer to a topic you have not reached.
How do you tell the causes apart from who chose it?
Compare the students who chose the option with those who chose the key. In EdProcto's results, pressing the row of a multiple choice question opens a table headed "Option", "Chose it" and "Their mean": each option's takers, and their average score on the whole quiz.
| "Their mean" for the option, beside the key's | Who chose it | Causes still standing |
|---|---|---|
| Lower, by more than the question is worth | Your weaker students | A misconception, if the point was taught |
| Higher | Your stronger students | A wrong key, or a second answer the strongest preferred |
| About level | Students like the key's | An ambiguous question, a second defensible answer, something never taught, or a misconception the whole class holds |
Take the question's own points off first
"Their mean" includes the question itself: students who chose the key were given its points, and the others were not. On ten one-point questions that is ten points in the key's favour.
So a lead smaller than the question is worth is no lead: on the rest of the quiz the option's takers did as well, or better. Researchers make the same correction: Shoji et al. (2021) set each response to one physics question against the total on the others.
How large a gap is real?
In one class, treat a gap under about 16 points as a hint. By our arithmetic, where scores have a standard deviation of 20 points, the gap between the means of fifteen students and of ten has a standard error of about 8.
"Separates" below zero says the top of the class did worse on the question than the bottom. And the smaller options show who was left: if their takers score like everyone else, nobody knew; if far lower, the two leading options held every student who knew something.
How do you settle what the scores leave open?
Read the question and ask the students, because scores cannot tell an ambiguous stem from a second right option, or an untaught topic from a belief the whole class holds.
- Write down the reading that makes the option right. If it takes a second reading of the stem, the question is ambiguous. If the option is right on the only reading, it is a second defensible answer. If no honest reading rescues it, the question is sound.
- Find the lesson. Put a finger on the slide, page or task where the point was taught. If you cannot, it was never taught.
- Ask three students who chose it why, before telling them it is wrong. Reviewing 273 articles on diagnosing misconceptions in science, Gurel et al. (2015) found interviews used in 53%, open-ended tests in 34%, multiple-choice tests in 32% and multiple-tier tests in 13%: researchers who need the reason mostly ask for it. To have the reasons in writing next time, set the question as choose and justify: EdProcto puts a box for the reason under the choice.
- Read the question before it. Where two questions share a stem, or one builds on the other's answer, a slip in the first is carried into the second.
What did research find about wrong answers that many students share?
It found that most follow a known idea, that some follow the question, and that a right answer can hide the same idea. Each source was opened on 10 October 2026; of every study in this guide except Chen et al. (2020) we read the abstract only.
- Most follow a known idea; some follow the question. Wells et al. (2019) analysed 4,509 pretest and 4,716 post-test responses to the Force Concept Inventory, a physics test. The incorrect answers fell into 9 groups before teaching and 11 after. Most could be mapped onto the misconceptions the options were written from; the rest came from questions that share a stem.
- A right answer can hide the wrong idea. On one question of that test, about atmospheric pressure, the share right fell after teaching in some Japanese classes. Shoji et al. (2021) read 429 written explanations from 358 respondents: many who chose the key were not aware of atmospheric force, and many who chose wrongly had been taught about air pressure without understanding it.
- Knowing the popular wrong answer goes with better learning. Chen et al. (2020) gave 2,749 high school students of 79 biology teachers 29 five-option questions designed to reveal misconceptions. Allowing for pretest scores, students did better at the end of the year on a question their teacher could answer, and better still when the teacher could also predict the commonest wrong answer, most of all where one wrong answer stood out. The design is correlational.
A search on the day of writing found no study that sorted popular wrong answers on classroom quizzes into these causes and counted them: the three-way sort above is our reasoning from how the figures are built.
What should you do in each case?
Fit the remedy to the cause: re-mark a wrong key, credit both answers when each holds up, drop what was never taught, and reteach a misconception.
| Cause | This time | The question next time |
|---|---|---|
| A shared misconception | The marks stand; reteach the idea another way | Keep it |
| A wrong key | Correct the key and press "Re-mark this quiz" in EdProcto's results | Keep it, corrected |
| An ambiguous question | Credit both readings | Rewrite the stem |
| A second defensible answer | Credit both options | Replace the option, or add the condition that rules it out |
| Something never taught | Drop the question; teach the point for the first time | Keep it for afterwards |
In EdProcto:
- Re-mark. Correct the key in the builder; then, on the results page, "Re-mark this quiz" and "Yes, re-mark". The card says: "Every attempt handed in is marked again from the questions and keys as they stand now. A mark you or Procto AI gave is kept."
- Drop. Delete the question in the builder and re-mark: it leaves every total. On an exam that counts, read when to throw a question out first.
- Credit both. From the "Students" list, open each student who chose the second answer, press "Add a note, or set the points by hand" under it and give the points in "Points". A later re-mark keeps them.
- Reteach. A reteaching plan from one quiz takes it from here, and finding and fixing misconceptions has a ten-minute routine.
A worked example: one row of counts in four classes
The four classes are invented as an illustration, and each figure came out of EdProcto's analysis code, run on their answers. Each class of thirty sat ten four-option questions, one point each, and averaged 55.7%. On Q7 each gave the same counts: fifteen on one wrong option, ten on the key, three and two on the others. The percentages are "Their mean".
| Class | The key's ten | The fifteen | The other five | "Separates" | Top of the page |
|---|---|---|---|---|---|
| A | 80% | 50% | 26.7% and 20% | 0.88 | A shared wrong answer |
| B | 52% | 68% | 26.7% and 25% | −0.21 | Check the key |
| C | 67% | 60% | 20% and 20% | 0.39 | A shared wrong answer |
| D | 60% | 53.3% | 53.3% and 55% | 0.10 | A shared wrong answer |
Every row carried one sentence, here class A's:
More students chose "Less than one half" than the right answer on Q7. Either the key is wrong or the class shares a misconception.
Class A. A fair coin has landed heads five times running: what is the chance of heads next? Less the question's ten points, the key's mean is 70 against 50. The fifteen are the weaker students, probability was taught last week, and "Less than one half" is the belief that a run must even out. A misconception: reteach.
Class B. Which planet is nearest the Sun? The key says Venus and the fifteen said Mercury. Less the question's points the fifteen lead, 68 to 42, and the row added: "On Q7 the weaker students did better than the stronger ones. That usually means a wrong key or an ambiguous wording." A slip of the key: correct it and re-mark.
Class C. Which of these is a noble gas? The key is neon and the fifteen chose helium: less the question's points, 57 against 60, level. The five on nitrogen and oxygen averaged 20%, so the two leading options held everyone who knew the topic. Helium is a noble gas too: credit both, and replace one option.
Class D. Why does the Moon show phases? The fifteen chose "The shadow of the Earth covers part of it". Every option's takers look like the class, 50 against 53.3, 53.3 and 55, and the row added: "Q7 does not separate the students who know the material from those who do not." The unit on the Moon starts next week: drop the question and keep it for afterwards.
The top line makes the urgent cut, key or class, from "Separates" and the means as printed; the table and the checks above make the finer ones.
How do you write the next question so each wrong option names one mistake?
Write the mistake beside each wrong option before the quiz is set: with such diagnostic distractors, a count of takers is a count of a mistake.
- One mistake to an option. In class A's question, "Less than one half" is the run that must even out, "More than one half" is the streak that must go on, and "One in sixty-four" is the chance of six heads in a row.
- No mistake that lands on the key. "What is 2²?" cannot catch the student who doubles, because doubling gives 4. Ask for 3²: doubling gives 6.
- No two mistakes on one option. If two wrong methods give the same number, change the numbers until they part.
- Wrong options from what students wrote. How to write distractors that work lists five places to find them.
How does EdProcto show a popular wrong answer?
EdProcto is the best tool for reading a popular wrong answer when the scores go out tomorrow, because it sets the two figures that sort the causes, "Chose it" and "Their mean", beside every option, and files the question under the key or the class before you open it. In the worked example it sent class B to the key at once.
Beside the question, under "What the numbers say":
- The "More students chose" sentence, where a wrong option drew three students or more, and more than the key did. An option that only ties with the key draws no sentence and points at nothing: read a tied row yourself.
- "Nobody chose "…" on Q7. A wrong answer nobody picks is doing no work; make it plausible." For a wrong option with no takers, once ten sittings have scores.
At the top, under "What it says about the teaching":
- "A shared wrong answer on Q7: the class was taught something, and it was not this. Worth a minute in the next lesson." Nothing points at the key.
- "Check the key on Q7: the stronger students got it wrong more often than the weaker ones." "Separates" is below zero.
- "Check the key on Q7: the students who chose the most popular wrong answer scored higher on the rest of the quiz than those who chose the keyed one." Its takers' mean is the higher on the whole quiz, or leads on the rest of the quiz by more than chance would give.
- "Check the key on Q7: nobody chose the keyed answer." The key drew nobody.
Each is a reason to reread the question, never a verdict.
The plans. Each option's takers and these sentences come with the full analysis, from the Educator plan up, which takes 500 students a quiz. On Free, with thirty students a quiz, and Tutor, with 100, the results give each question's share right. See pricing.
Quick questions
What does it mean when most of the class picks the same wrong answer?
A shared misconception, a wrong key, an ambiguous question, a second defensible answer, or content never taught. How its takers scored, beside the key's, narrows the list.
Is a popular wrong answer always a misconception?
No. In Wells et al. (2019), some groups of wrong answers on a physics test came from questions that share a stem, not from misconceptions.
Should I re-mark, drop or keep the question?
Re-mark for a wrong key, give the mark to both answers when each holds up, drop what was never taught, and keep a question that found a misconception.
Does EdProcto say whether the key is wrong or the class is?
It says which way the scores point. The full analysis in EdProcto, on the Educator plan, prints "Check the key" where the stronger students did worse on the question or the option's takers outscored the key's, and "A shared wrong answer" otherwise.
References
- Chen, C., Sonnert, G., Sadler, P. M., & Sunbury, S. (2020). The impact of high school life science teachers' subject matter knowledge and knowledge of student misconceptions on students' learning. CBE Life Sciences Education, 19(1), Article 9. https://pmc.ncbi.nlm.nih.gov/articles/PMC8697650/
- Gurel, D. K., Eryilmaz, A., & McDermott, L. C. (2015). A review and comparison of diagnostic instruments to identify students' misconceptions in science. Eurasia Journal of Mathematics, Science and Technology Education, 11(5), 989–1008. https://doi.org/10.12973/eurasia.2015.1369a
- Halloun, I. A., & Hestenes, D. (1985). The initial knowledge state of college physics students. American Journal of Physics, 53(11), 1043–1055. https://www.compadre.org/portal/items/detail.cfm?ID=2658
- Shoji, Y., Munejiri, S., & Kaga, E. (2021). Validity of Force Concept Inventory evaluated by students' explanations and confirmation using modified item response curve. Physical Review Physics Education Research, 17(2), Article 020120. https://doi.org/10.1103/PhysRevPhysEducRes.17.020120
- Wells, J., Henderson, R., Stewart, J., Stewart, G., Yang, J., & Traxler, A. (2019). Exploring the structure of misconceptions in the Force Concept Inventory with modified module analysis. Physical Review Physics Education Research, 15(2), Article 020122. https://doi.org/10.1103/PhysRevPhysEducRes.15.020122