A multiple-answer question lists several options, makes more than one of them right, and asks the student to tick every right one. It also goes by select all that apply, multiple response and, in a close relative, multiple true-false.
Writing one is quick. Scoring it is the hard part: a five-option question can be answered in 32 ways, most of them partly right, and the published rules disagree about what each is worth. In the worked example below, one student's ticks earn anything from 0 to 8 marks out of 10.
EdProcto builds the shape as its multiple answers template and offers two of those rules.
What does a multiple-answer question measure that a single-answer question does not?
A multiple-answer question measures whether a student can judge every option on its own merits, so it records knowing part of a topic as something different from knowing all of it or none. In a single-answer question the options compete: find the best and the rest can be ignored. Here no option settles another, so the student cannot spot the answer at a glance or skip the other options (Kanzow et al., 2023).
The scores can be more reliable too. Kreiter and Frisbie (1989) set a college midterm in two forms with parallel content, and the multiple true-false form gave higher reliabilities than the multiple-choice form.
Three formats share the name
Select all that apply has tick boxes: the student ticks the right options and leaves the rest, so an empty box can mean "wrong" or "not sure". EdProcto's template is this form. In multiple true-false every statement is marked true or false. In Pick-N the student is told how many options are right (Kanzow et al., 2023).
Most scoring research used the last two. A rule moves across to tick boxes with one catch: if it pays for a correct "false", it pays for a box nobody touched. The guide to select all that apply questions covers the studies of what a single-answer question hides.
How do you write a multiple-answer question?
Write one stem about one thing, four to six options that are each right or wrong on their own, and make no more than half of them right.
- Ask one question of every option. "Which of these angles are obtuse?" is one test applied five times. "Which of these statements about angles are true?" is five unrelated questions, better set as true or false questions.
- Make each option stand alone. It must be right or wrong whatever the student decides about the others. Opposite pairs fail, because one decision fills two boxes.
- Use four to six options. The studies cited here used four statements (Lahner et al., 2018), five or six options (Betts et al., 2022) and four to eight (Domnich et al., 2015). None tested which number is best.
- Keep the right options to half or fewer, and vary the number. With more right than wrong, ticking every box starts to pay, as the section on guessing shows.
- Build it. In EdProcto the shape is multiple answers. Give the key at least one right option: a question with none is not marked by rule and waits as pending.
Why "all of the above" has no place here
"All of the above" is an answer the student can already give by ticking every box. As an extra option its truth depends on the other boxes, so one piece of knowledge is counted twice. "None of the above" is the empty answer in disguise, and "A and C only" makes the question single-answer again. If the options really compete, write a multiple-choice question.
How do you score a multiple-answer question?
Score it with one of four kinds of rule: all or nothing (dichotomous scoring, in the research), credit for every box judged correctly, right ticks set against wrong ticks, or a threshold. They differ on what an untouched wrong option earns and what a wrong tick costs.
Kanzow et al. (2023) count 41 published methods and applied all but one to the same examination answers: on the multiple true-false questions, all or nothing averaged 0.51 of the marks and credit per statement 0.86.
One answer scored under eight rules
Which of these angles are obtuse? Select every one. 35°, 90°, 95°, 170°, 180°
Two are right, 95° and 170°; the right angle and the straight angle sit on the edges of the definition, where students slip. One student ticks 95°, 170° and 180°: two right ticks and one wrong. Counting an empty box as "not obtuse", four of five judgements are right. The question is worth 10, and each mark was computed in code from the rule as its source states it.
| Rule | How the mark is made | This answer | Out of 10 |
|---|---|---|---|
| All or nothing | All for the exact set, otherwise zero | Not exact | 0 |
| Ripkey's rule | Share of right options ticked; zero if ticks outnumber right options | 3 ticks, 2 right options | 0 |
| PS50 | Half for over half the judgements right, all for every one | 4 of 5 | 5 |
| Plus/minus (EdProcto, partial credit on) | Right ticks minus wrong ticks, over the right options, never below zero | (2 − 1) ÷ 2 | 5 |
| Judgements right minus wrong | Right judgements minus wrong ones, over all options | (4 − 1) ÷ 5 | 6 |
| Right share less wrong share | Share of right options ticked minus share of wrong options ticked | 2 ÷ 2 − 1 ÷ 3 | 6.67 |
| Balanced rule | Share of right options ticked, less a penalty only for ticks beyond their number | 1 − (3 ÷ 5 − 2 ÷ 5) ÷ (3 ÷ 5) | 6.67 |
| Credit per option | An equal share for each box judged rightly | 4 ÷ 5 | 8 |
PS50 and credit per option are as Lahner et al. (2018) define them, plus/minus and Ripkey's rule as Betts et al. (2022) do, and the other three as Domnich et al. (2015) do. Plus/minus is also how the NCLEX nursing examination scores questions where candidates may select an unspecified number of options (National Council of State Boards of Nursing, n.d.).
The same three ticks earn 0, 5, 6, 6.67 or 8. Credit per option pays for the two wrong options left alone. Plus/minus pays nothing for those and charges a wrong tick the value of a right one. Betts et al. (2022) judged Ripkey's rule a better fit for questions that say how many to pick.
Which scoring method gives the most reliable marks?
Partial credit gives more reliable marks than all or nothing in most studies that have compared them, and no single partial-credit rule leads consistently.
Lahner et al. (2018) have the clearest figures, from 37 medical examinations with 998 four-statement multiple true-false questions. Reliability was 0.75 under PS50 and under credit per option, and 0.70 under all or nothing. As a workload, reaching 0.8 would take 74 or 75 questions with partial credit and 103 without.
Domnich et al. (2015) scored nine tick-box questions from 286 secondary school students under eight rules. All or nothing gave a coefficient of 0.42 and the partial rules 0.53 to 0.71: Ripkey's rule highest, the balanced rule 0.68, credit per option 0.66. The authors put the low figures down to the short test.
On 2,243 select all that apply questions from a licensing examination, Betts et al. (2022) found that every partial-credit method gave substantially more information per unit of time than all or nothing. They judged plus/minus and credit per option the most promising, plus/minus above all.
Where the studies disagree, and their limits
Tsai and Suen (1993) compared six methods on multiple true-false questions and found no statistically significant differences in reliability or discrimination. For ranking students, they concluded, no method is demonstrably better, but the method changes how hard the test looks, which matters wherever there is a pass mark.
Most of these studies come from medical and health education, most used true-false or Pick-N forms, and none read for this guide reports a classroom reliability coefficient for plus/minus itself.
Albanese and Sabers (1988) add a caution: answers to options under one stem correlated 2.6 to 7.0 times more strongly than answers under different stems, so counting each box as a separate question overstates reliability. EdProcto works out Cronbach's alpha over whole questions; see item analysis for teachers.
How much does guessing pay under each method?
Guessing pays most under credit per option, where a coin flip on every box averages half marks, and least under rules that set a wrong tick against a right one. The angle question again, worth 10, for three students who know nothing:
| Rule | Ticks nothing | Ticks every box | Coin flip per box, on average |
|---|---|---|---|
| All or nothing | 0 | 0 | 0.31 |
| Ripkey's rule | 0 | 0 | 1.56 |
| PS50 | 5 | 0 | 2.66 |
| Plus/minus | 0 | 0 | 1.09 |
| Judgements right minus wrong | 2 | −2 | 0 |
| Right share less wrong share | 0 | 0 | 0 |
| Balanced rule | 0 | 0 | 2.71 |
| Credit per option | 6 | 4 | 5 |
The last column is the mean over all 32 possible answers. Under credit per option, 16 of the 32 earn 6 marks or more, and so does an untouched question, because three wrong options were "correctly" left alone. Under plus/minus, one answer in 32 reaches 6.
Pomplun and Omar (1997) found that students who guessed on tick-box questions tended to leave a box empty, not tick it, so a rule that pays for empty boxes pays for the commonest guess. Credit per option and PS50 were defined for true-false marking, where every statement is marked. Even there a coin-flipper averages half the marks under credit per statement (Kanzow et al., 2023), which puts a 60% pass mark ten points above chance.
Two rules average zero only by letting a question's mark fall below zero, to minus 10 here, which Kanzow et al. (2023) advise against, citing legal reasons. Plus/minus stops at zero.
Keep the right options to half or fewer
Under plus/minus, ticking every box earns the right ticks less the wrong ones. With two right out of five that is 2 − 3, held at zero. Three right out of four earns 67% of the marks, three out of five 33% and four out of five 75%. So make one or two right out of four or five, and up to three out of six.
How does EdProcto score a multiple-answer question?
EdProcto scores a multiple-answer question all or nothing until you switch on the setting "Partial credit", and with it on uses plus/minus as a share of the question's points.
For this shape EdProcto is the best tool when a mark must hold against both the blank answer and the student who ticks everything. Its partial-credit rule is plus/minus, the method Betts et al. (2022) thought held the best promise for select all that apply, and in the guessing table it pays zero for both blanket answers without ever taking a question below zero.
Multiple answers and all that apply, where the count of right options is not given, are both in the builder and marked identically.
"Partial credit" is a grading setting, off until you turn it on. Its note begins "On multiple-answer, ordering, matching, sorting and best-and-worst questions", so one switch covers those shapes across the quiz.
- Off: full points when every right option is ticked and no wrong one, otherwise zero.
- On: right ticks minus wrong ticks, divided by how many options are right, times the question's points, rounded to the hundredth, never below zero.
On the angle question at 10 points with partial credit on, 95° and 170° earn 10. Adding 180° earns 5, and 95° alone earns 5. Ticking 95° with 90° earns 0, and so do every box and no box. With it off, only the first answer earns anything. A missed right option and a wrong tick cost the same.
One limit to design around: the rule protects against ticking everything only while the right options are no more than half. With three right options of four, every box ticked still earns two thirds of the points, so keep the right options to half the list or fewer.
Marks are made on the server at hand-in, so settle the key first. A key corrected afterwards takes effect on the earlier attempts when you press "Re-mark this quiz" on the results page.
In the analysis each multiple-answer question has its difficulty (the average share of its points earned), its discrimination (how far the top 27% of the class outscored the bottom 27% on it) and its median time. The count of students behind each option is drawn for single-answer questions only; for a verdict statement by statement, set the statements as true or false questions.
To start from your own text, put a chapter in the quiz maker on the home page, pick multiple answers, and Procto AI writes a first draft for you to edit. Free has ten quizzes a month with the core templates, marked by rule. Tutor has all 111 templates and Educator the full analysis, discrimination and reliability included; see pricing.
Which scoring method should you choose?
Choose plus/minus for most classroom quizzes, all or nothing only where a partly right answer is no use, and credit per option only with true-false marking and a pass mark raised to match.
All or nothing needs a pass mark of its own: in Domnich et al. (2015) the same answers averaged 33% under all or nothing and 54% to 64% under four of the partial rules. Credit per statement needs one that allows for the 50% chance score. Whatever the rule, tell students before the quiz whether a wrong tick costs marks, and keep to one rule across a course.
Quick questions
How do you score select all that apply questions?
Give partial credit in which a wrong tick cancels a right one and the question cannot go below zero. Without a cost for wrong ticks, ticking every box collects every mark. In EdProcto, switch on "Partial credit" for multiple answers.
Is partial credit more reliable than all or nothing?
In most studies, yes: Lahner et al. (2018) found 0.75 with partial credit against 0.70 without. Tsai and Suen (1993), though, found no significant difference between six methods.
How many right answers should a multiple-answer question have?
No more than half the options, and a different number from question to question. Past half, ticking everything starts to earn marks under plus/minus.
Does negative marking apply to multiple-answer questions in EdProcto?
No. "Negative marking" acts on single-answer questions. A multiple-answer mark is the same with it on or off, because its deduction is inside "Partial credit".
References
- Albanese, M. A., & Sabers, D. L. (1988). Multiple true-false items: A study of interitem correlations, scoring alternatives, and reliability estimation. Journal of Educational Measurement, 25(2), 111–123. https://doi.org/10.1111/j.1745-3984.1988.tb00296.x
- Betts, J., Muntean, W., Kim, D., & Kao, S. (2022). Evaluating different scoring methods for multiple response items providing partial credit. Educational and Psychological Measurement, 82(1), 151–176. https://doi.org/10.1177/0013164421994636
- Domnich, A., Panatto, D., Arata, L., Bevilacqua, I., Apprato, L., Gasparini, R., & Amicizia, D. (2015). Impact of different scoring algorithms applied to multiple-mark survey items on outcome assessment: An in-field study on health-related knowledge. Journal of Preventive Medicine and Hygiene, 56(4), E162–E171. https://pmc.ncbi.nlm.nih.gov/articles/PMC4753817/
- Kanzow, P., Schmidt, D., Herrmann, M., Wassmann, T., Wiegand, A., & Raupach, T. (2023). Use of multiple-select multiple-choice items in a dental undergraduate curriculum: Retrospective study involving the application of different scoring methods. JMIR Medical Education, 9, Article e43792. https://doi.org/10.2196/43792
- Kreiter, C. D., & Frisbie, D. A. (1989). Effectiveness of multiple true-false items. Applied Measurement in Education, 2(3), 207–216. https://doi.org/10.1207/s15324818ame0203_2
- Lahner, F.-M., Lörwald, A. C., Bauer, D., Nouns, Z. M., Krebs, R., Guttormsen, S., Fischer, M. R., & Huwendiek, S. (2018). Multiple true-false items: A comparison of scoring algorithms. Advances in Health Sciences Education, 23(3), 455–463. https://doi.org/10.1007/s10459-017-9805-y
- National Council of State Boards of Nursing. (n.d.). NCLEX FAQs: How are items scored? Retrieved October 9, 2026, from https://www.nclex.com/faqs.page
- Pomplun, M., & Omar, M. H. (1997). Multiple-mark items: An alternative objective item format? Educational and Psychological Measurement, 57(6), 949–962. https://doi.org/10.1177/0013164497057006005
- Tsai, F.-J., & Suen, H. K. (1993). A brief report on a comparison of six scoring methods for multiple true-false items. Educational and Psychological Measurement, 53(2), 399–404. https://doi.org/10.1177/0013164493053002008