A student who does not know the answer to a yes or no question can guess or leave it. Guess, and half the time the score says they knew. A don't know option gives them a third choice, and gives the teacher something a guess never does: a gap the student has owned up to.
Whether it belongs on a test depends on what the score is for and what a wrong answer costs. This guide covers both, with the research on who loses by it and one example scored five ways. EdProcto carries the shape as its yes, no or maybe template, and a section below says exactly how its third option is marked.
Why would a test have a don't know option?
A test has a don't know option because guessing inflates scores and hides the gaps a teacher needs to see. A student who knows four of ten yes or no questions and guesses the other six expects seven right, and the three lucky marks look exactly like knowledge.
What a don't know answer tells you that a wrong answer does not
A don't know answer marks a gap the student already knows about. A wrong answer is either a guess that missed or a belief that is mistaken, and the score cannot say which. A gap needs teaching. A mistaken belief has to be dislodged first, as our guide to finding misconceptions in a quiz describes.
A don't know is not always a blank. Muijtjens et al. (1999) had medical students at Maastricht answer true-false tests with a don't know option, then replace every don't know with true or false. On those items they were right 4.5% and 5.9% more often than pure guessing would give, in the two year groups tested. Read a don't know as a real gap, with a little unclaimed knowledge under it.
How is a don't know answer scored?
A don't know answer is scored in one of three ways: as nothing while a wrong answer also costs nothing, as nothing while a wrong answer costs marks, or as a part mark. The first is number-right scoring and the other two are formula scoring. Kanzow et al. (2023), whose scoping review found 21 scoring rules in 258 sources, give the formulas, with R the number right, W the number wrong, O the number omitted or answered don't know, and n the number of answers to choose between. A right answer earns one mark under all three.
| Rule | Formula | Wrong | Don't know | A blind guess expects |
|---|---|---|---|---|
| Number-right | R | 0 | 0 | 0.5 |
| Rights minus wrongs | R − W/(n − 1) | −1 | 0 | 0 |
| Credit for omissions | R + O/n | 0 | 0.5 | 0.5 |
The figures are for a yes or no question, where n is 2. The review dates the penalty on two-option questions to 1920 and the credit for omissions to 1951. On four options the penalty falls to a third of a mark and the credit to a quarter.
Under number-right scoring the option is a trap for the honest: a guess expects half a mark and a don't know earns nothing, so a student after the best score never chooses it. The option is used truthfully only when nothing rides on the score, or when a wrong answer costs more than an admission.
What did research find about formula scoring and number-right scoring?
Research found a trade, and not a settled one: formula scoring tends to give steadier scores and number-right scoring less biased ones.
Lord (1975) found formula scoring clearly superior under one assumption about how students omit: only where they would otherwise guess blindly. Students do not behave that way. Bliss (1980) gave 168 elementary pupils a mathematics test with instructions to avoid wild guessing, then had them complete every question in a different coloured pencil: pupils, the more able especially, had omitted too many. In the Maastricht study the two scoring methods differed by 2.5% and 3.4% of the score on average, against a standard deviation of 11%, which the authors read as the bias against students less willing to guess. Reliability was higher under formula scoring: 0.72 against 0.66, and 0.74 against 0.66.
Ravesloot et al. (2015) ran a randomised crossover with 337 Dutch radiology residents, on true-false items scored one for right, minus one for wrong and nothing for don't know. Reliability again favoured formula scoring, 0.74 against 0.62 on average. But after allowing for length of training, the number-right score explained 27% of the variation in formula scores, and how often a resident chose don't know, with its interaction with gender, a further 8%. The authors call the decision a trade between random and systematic error.
Reviewers are not unanimous. Frary (1988) judged formula scoring inappropriate for most classroom testing, though possibly desirable on speeded tests and difficult tests with low pass marks. Budescu and Bar-Hillel (1993) showed that for a student out to maximise the score, answering is never worse than omitting, and endorsed number-right scoring. Burton (2002), reading three studies of medical students, argued the other way: blind guessing under number-right scoring generally harms reliability more than some students' over-caution does under negative marking.
Who is penalised by a don't know option?
The student penalised is the cautious one who half-knows the answer and says don't know when answering would have paid. Under rights minus wrongs on a yes or no question, answering pays whenever the chance of being right is above one half, so every don't know from a student who leans one way gives marks away.
Most studies of who they are look at gender, and they do not agree.
- Ben-Shakhar and Sinai (1991): in two Israeli aptitude batteries, for ninth graders and for university applicants, female candidates omitted more questions in most subtests. Correcting scores for guessing narrowed the male advantage, but only slightly.
- Baldiga (2014): in an experiment with practice questions from SAT II history tests, everyone answered every question when wrong answers cost nothing. With a penalty, women answered significantly fewer than men, with no difference in knowledge or confidence. It was one subject, and not a real examination.
- Kelly and Dennick (2009): across 359 assessments in the first two years of medicine at Nottingham, one with true-false-abstain questions had 16.7 times the odds of a significant advantage for men. In the fifteen such exams with the largest gaps, women abstained 3% more. The authors had no other explanatory variables.
- Ravesloot et al. (2015): women chose don't know slightly more often and lost nothing by it overall. Residents with more knowledge lost most by choosing it.
- Coffman and Klinowski (2020): when Chile's national college entry examination dropped its penalties for wrong answers, a large gender gap in questions skipped shrank and gaps in performance narrowed, mainly among high performers.
So the penalty falls on caution, and how much depends on the setting. None of these is a study of a school classroom quiz.
When should you offer a don't know option?
Offer a don't know option when the purpose is to find out what a class knows and nothing rides on the score, and leave it off when the score counts.
| Situation | Offer it? | Why |
|---|---|---|
| A baseline test or diagnostic before a unit | Yes, at zero, with no penalty | You want the map: EdProcto's results count the not sure answers on each question |
| A graded test, number-right | No | A guess expects half a mark and the option nothing |
| A graded test with a penalty | Rarely | The score then measures caution along with knowledge |
How to word the instruction
Word the instruction so that the student knows what each answer scores and when the third is the right choice. Frary (1988) gives the standard form for formula scoring: omit only where the answer would be a sheer guess, and answer whenever an option can be ruled out. Budescu and Bar-Hillel (1993) found adequate instructions very hard to give, so keep them short.
- Say what the result is for. "This is to plan next week. It does not count."
- Say what each answer scores. "Not sure scores nothing, and so does a wrong answer."
- Say when to choose it. "Choose not sure if you would be tossing a coin. If you lean one way, answer."
- Name the option for what it means. "Not sure" is an admission, and "maybe" can read as a third answer. In EdProcto's yes, no or maybe shape you type the options yourself.
- If wrong answers cost marks, say how much.
How do three students with the same number right differ? A worked example
Three students with the same number right differ in what they got wrong and what they admitted, and only some scoring rules show it. Take a ten-question diagnostic, yes or no with a not sure option, on which three students each have six right. The scores were computed in code for this guide, the two EdProcto rows by running EdProcto's marking function on the thirty answers.
| Student A | Student B | Student C | |
|---|---|---|---|
| Right, wrong, not sure | 6, 4, 0 | 6, 2, 2 | 6, 0, 4 |
| Number-right, R | 6 | 6 | 6 |
| Rights minus wrongs, R − W | 2 | 4 | 6 |
| Credit for omissions, R + O/2 | 6 | 7 | 8 |
| EdProcto, "Negative marking" off | 6 | 6 | 6 |
| EdProcto, "Negative marking" on | 4 | 4 | 4 |
Number-right cannot tell the three apart. The two formula rules can, and they always agree on the order: on ten questions the credit rule is the penalty rule plus ten, halved.
- Student A never chose not sure. The four wrong answers may be guesses or beliefs. Where much of the class missed the same question, treat it as a misconception. Ask A to explain one answer aloud.
- Student B has two answers to put right and two gaps to fill. Start with the wrong answers.
- Student C knows where the gaps are. Teach those four topics. Then ask C for a best guess on each: if most come out right, the trouble is confidence, and C is the student a penalty would cost marks.
How does EdProcto mark a yes, no or not sure question?
EdProcto marks the third option as an ordinary option: the key is one of the options you typed, and any other choice, the unsure one included, is marked wrong. Nothing in the marking treats it as a special answer. EdProcto is the best tool for a diagnostic with a not sure option when the questions have to come from your own material, because its results count the unsure answers on each question apart from the wrong ones: the split between a gap and a misconception that the option exists to show.
Building it. The yes, no or maybe template is a single-choice question marked by rule. In the builder you type each option, mark the right one and remove any box you do not use. Or paste a chapter into the quiz maker on the home page, pick the shape, and Procto AI drafts questions for you to edit: check the options and the key on each. Shuffling the options is ruled out for this shape, so put the unsure one last.
Scoring it. Leave "Negative marking" off: a right answer earns the question's points and every other answer earns zero. With it on, the setting's own hint reads "A wrong choice loses points: one over the number of wrong options." Not sure is one of this shape's two wrong options, so a wrong yes or no loses half the points and so does not sure. Only an unanswered question loses nothing. That is why all three students fall to 4, and a coin toss between yes and no still expects a quarter of a mark.
For rights minus wrongs proper, write the question as true or false with two options, switch "Negative marking" on, leave "Allow skipping a question" on and tell students that an unanswered question is the don't know. On the same three students EdProcto's marking then gives 2, 4 and 6. For a finer scale than sure or unsure, see confidence-based marking.
Reading it. The analysis lists each option of a question with how many students chose it and their mean score. Many on not sure is a gap to teach. Many on the wrong answer is a misconception or a wrong key, so check the key first. Two automatic findings were written for wrong options in general: "Nobody chose" the unsure option is good news here, and where it drew as many students as the right answer, read the finding's "misconception" as a shared gap.
Free carries the core templates, marking by rule and the results question by question. Tutor opens all 111 templates. Educator adds the full analysis, with the option counts. Pricing lists every plan.
Quick questions
Should a test have an I don't know option?
On a diagnostic, yes, scored at zero with no penalty for wrong answers. On a graded test, no: without a penalty a guess beats it on average, and with one, cautious students score below what they know.
What is formula scoring?
Formula scoring takes marks off for wrong answers, or adds part marks for omitted ones, so that a blind guess gains nothing over an omission on average. The common form is rights minus wrongs divided by one less than the number of options (Kanzow et al., 2023), so in multiple choice with four options a wrong answer costs a third of a mark.
How do you score a true false don't know question?
One for right and nothing for don't know. A wrong answer scores nothing on a diagnostic and minus one under formula scoring, as on the Dutch progress test studied by Ravesloot et al. (2015).
Does EdProcto give zero for a not sure answer?
Yes, with "Negative marking" off: in EdProcto's yes, no or maybe shape the unsure option is an ordinary option. With the setting on it loses half the points, the same as a wrong answer, and only a blank loses nothing.
References
- Baldiga, K. (2014). Gender differences in willingness to guess. Management Science, 60(2), 434–448. https://doi.org/10.1287/mnsc.2013.1776
- Ben-Shakhar, G., & Sinai, Y. (1991). Gender differences in multiple-choice tests: The role of differential guessing tendencies. Journal of Educational Measurement, 28(1), 23–35. https://doi.org/10.1111/j.1745-3984.1991.tb00341.x
- Bliss, L. B. (1980). A test of Lord's assumption regarding examinee guessing behavior on multiple-choice tests using elementary school students. Journal of Educational Measurement, 17(2), 147–152. https://doi.org/10.1111/j.1745-3984.1980.tb00823.x
- Budescu, D., & Bar-Hillel, M. (1993). To guess or not to guess: A decision-theoretic view of formula scoring. Journal of Educational Measurement, 30(4), 277–291. https://doi.org/10.1111/j.1745-3984.1993.tb00427.x
- Burton, R. F. (2002). Misinformation, partial knowledge and guessing in true/false tests. Medical Education, 36(9), 805–811. https://doi.org/10.1046/j.1365-2923.2002.01299.x
- Coffman, K. B., & Klinowski, D. (2020). The impact of penalties for wrong answers on the gender gap in test scores. Proceedings of the National Academy of Sciences, 117(16), 8794–8803. https://doi.org/10.1073/pnas.1920945117
- Frary, R. B. (1988). Formula scoring of multiple-choice tests (correction for guessing). Educational Measurement: Issues and Practice, 7(2), 33–38. https://doi.org/10.1111/j.1745-3992.1988.tb00434.x
- Kanzow, A. F., Schmidt, D., & Kanzow, P. (2023). Scoring single-response multiple-choice items: Scoping review and comparison of different scoring methods. JMIR Medical Education, 9, Article e44084. https://doi.org/10.2196/44084
- Kelly, S., & Dennick, R. (2009). Evidence of gender bias in True-False-Abstain medical examinations. BMC Medical Education, 9, Article 32. https://doi.org/10.1186/1472-6920-9-32
- Lord, F. M. (1975). Formula scoring and number-right scoring. Journal of Educational Measurement, 12(1), 7–11. https://doi.org/10.1111/j.1745-3984.1975.tb01003.x
- Muijtjens, A. M., van Mameren, H., Hoogenboom, R. J., Evers, J. L., & van der Vleuten, C. P. M. (1999). The effect of a 'don't know' option on test scores: Number-right and formula scoring compared. Medical Education, 33(4), 267–275. https://doi.org/10.1046/j.1365-2923.1999.00292.x
- Ravesloot, C. J., Van der Schaaf, M. F., Muijtjens, A. M. M., Haaring, C., Kruitwagen, C. L. J. J., Beek, F. J. A., Bakker, J., Van Schaik, J. P. J., & Ten Cate, T. J. (2015). The don't know option in progress testing. Advances in Health Sciences Education, 20(5), 1325–1338. https://doi.org/10.1007/s10459-015-9604-2