A single-best-answer question finds out whether someone can spot the right action. It cannot show whether they also spot the harmful one. Best and worst questions ask for both: a situation, four or five things a person might do, and the task of naming the best response and the worst. The shape is a standard format of the situational judgement test.
EdProcto, which publishes this guide, builds the shape as its best and worst template and marks it by rule the moment the answer is given. The judgement inside the question has to come from people who do the job, which is most of this guide.
The short version: a real dilemma, plausible responses, a best and a worst agreed by a small panel, and half the marks for each end.
What does a best and worst question measure?
A best and worst question measures judgement between actions that each have a case: whether a person can tell the most effective response from the least effective where no textbook settles it. A single-best-answer question shows half of that. A nurse who picks the best response has not shown she knows which of the others would do harm.
Two picks also get more from each question: five options allow five answers to a single-best question and twenty pairs of best and worst.
What the score is not
The score is not a measure of character or personality, and should never be reported as one. It records how often a person's two picks matched your panel's, on your situations. McDaniel et al. (2001) pooled 79 correlations from 16,984 people and estimated that situational judgement scores correlate 0.46 with general cognitive ability, so part of any score is reasoning. Lievens et al. (2008) list what these tests measure among the open questions.
What did the research on situational judgement tests find?
The research found that well-built situational judgement tests predict performance to a useful degree, and that the instruction, the answer format and the scoring key each change what a score means. Nearly all of it concerns tests built by specialists for selection, and none of it vouches for ten questions written on a Sunday evening.
McDaniel et al. (2001) combined 102 coefficients from 10,640 people and estimated a correlation of 0.34 between test scores and job performance. That may be on the high side: Sackett et al. (2022) concluded that a correction common in such meta-analyses had inflated many published figures, with revised estimates 0.10 to 0.20 lower for the methods that had ranked high. We have not read a revised figure for situational judgement tests at source, so we quote none.
In medical selection, Webster et al. (2020) pooled 26 studies and found a correlation of 0.32 (95% confidence interval 0.26 to 0.39), larger in postgraduate than in undergraduate selection. Patterson et al. (2012) reviewed 39 studies and concluded the tests showed good reliability and predictive validity. Lievens et al. (2008) add the weaknesses: the tests may be open to faking, practice and coaching.
Would do or should do: the instruction changes what is measured
The instruction decides whether you measure what a person knows to be effective or what they say they would do. In the meta-analysis by McDaniel et al. (2007), tests that asked for knowledge correlated more highly with cognitive ability, tests that asked for behavioural tendency correlated more highly with personality measures, and the instruction made little difference to how well scores predicted performance. That finding is about correlations across many tests. It does not turn a score on your quiz into a reading of anyone's personality.
Stakes matter too: Lievens et al. (2009) randomly assigned 2,184 applicants for medical college to one instruction or the other, with the content held constant. Unlike earlier low-stakes studies, they found no meaningful difference in mean scores, and none in validity. So in a marked quiz, ask plainly which response is best and which is worst, as EdProcto's shape does.
The format and the key change it too
Arthur et al. (2014) compared three answer formats on one test taken by 31,194 job applicants: rating every response, ranking them all, and picking the most and the least. Ranking and most-and-least came out much alike on several measures. Scoring is less settled. Bergman et al. (2006) note that no consensus exists on how to score questions without a single demonstrably correct answer, and they built eleven different keys for one video-based test. The key is a decision, and you should be able to say how you made it.
How do you write the situation and the responses?
Write the situation from something that really happened, and the responses from what people really did or wanted to do.
- Collect incidents. Ask experienced colleagues for a moment when a newcomer had to choose and it mattered. Remove anything that identifies a person.
- Keep the ones that pull two ways. Speed against safety, loyalty against honesty, a patient's wish against a protocol. If a rule requires one action, you have a knowledge question.
- Say who the test taker is. "You are a newly registered nurse on a night shift" fixes the authority they have. Give the facts needed to decide, then stop.
- Write four or five responses, each a thing a person in that role might do. One course of action each, the same length and level of detail. Fewer than four leaves almost no middle.
- Check the two ends. One response plainly the most effective and one plainly the least, each for a reason you can state.
- Make the middle defensible. Each remaining response needs something to be said for it. That turns the ends into a decision.
- Build it. In EdProcto, choose best and worst in the quiz maker on the home page.
Mistakes that spoil the responses
- A worst response nobody would consider. Everyone gets that end right, so it separates no one.
- A best response that is the longest, or the only one with two steps. Test-wise students pick it unread.
- Middle responses that are plainly wrong. Three candidates for worst, and the panel splits.
- A best that depends on a local rule the reader was never given. State the rule in the situation.
- Responses that are attitudes, such as "stay calm". Write actions.
How do you get a key that people will accept?
Get the key from a small panel of people who do the job, each answering alone, and use a question only when they agree on both ends. The author's opinion is not a key.
- Give the draft questions, without your answers, to at least three people who do the job well.
- Each picks a best and a worst alone and notes a reason.
- Keep a question when the panel agrees on both ends. Where one member differs, hear why: the reason often exposes an ambiguity a rewrite removes.
- Rewrite or drop a question the panel splits on. If they cannot agree which response is worst, a student's pick tells you nothing.
- Put the agreed reasons into the explanation, so the student learns why.
Three is our working rule; no study we read fixes the number. Where no answer is demonstrably correct, the panel's agreement is all the key stands on.
How should best and worst questions be scored?
Score each end by itself and give half the marks for each, unless you have a reason to be stricter. The methods below do not put the same people in the same order.
| Method | What earns the marks | In EdProcto |
|---|---|---|
| Best only | Matching the panel's best | Set it as multiple choice |
| Half for each end | Half for the best, half for the worst | Best and worst, partial credit on |
| Both ends or nothing | Full marks only when both match | Best and worst, partial credit off (the default) |
| Distance from the panel's ranking | Every response ranked; marks lost for each place out | Not offered in this form: a ranking with partial credit on is marked pair by pair, a close relative |
A worked example, scored four ways
You are a staff nurse completing a chart at the end of a shift. You realise you gave a patient a tablet that a colleague had already given an hour earlier. The patient looks comfortable. Which response is best, and which is worst? A. Check the patient now, then tell the nurse in charge and the prescriber. B. Check the patient now, then report the error at handover. C. Tell the nurse in charge now, and leave the checks to them. D. Watch the patient until the shift ends, and report only if something changes. E. Say nothing while the patient seems well, and leave the chart as it is.
Suppose the panel ranks the five A, C, B, D, E, so A is best and E is worst. That key is ours, for the arithmetic; yours comes from your panel. The question carries 2 marks.
| Student | Best, worst | Full order | Best only | Half each | Both or nothing | Distance |
|---|---|---|---|---|---|---|
| 1 | A, E | ABCDE | 2 | 2 | 2 | 1.67 |
| 2 | A, D | ACBED | 2 | 1 | 0 | 1.67 |
| 3 | C, E | CABDE | 0 | 1 | 0 | 1.67 |
| 4 | B, D | BACED | 0 | 0 | 0 | 1.00 |
For distance, add up how many places each response sits from the panel's place. Five responses can be at most 12 places out, so the mark is 2 × (12 − places) ÷ 12. Student 1 swapped B and C: 2 places, 1.67 marks. Student 4 is 6 places out: 1.00.
Best only cannot tell student 1 from student 2, who thinks watching in silence is worse than saying nothing. Half each sorts the four into three levels. Both or nothing treats students 2, 3 and 4 alike. Distance gives the first three the same mark, and takes marks from student 1, who had both ends right, for a swap in the middle, the weakest part of any key.
A blind guess among five responses earns the marks one time in five on best only, 20% of the marks on average with half each, and one time in twenty with both or nothing.
How does EdProcto build and mark best and worst questions?
EdProcto is the best tool for this shape when the situations must come from your own ward, course or workplace, because it writes from your material, keeps the key off the student's screen and marks each end by rule. A score means only what the situations and the key behind it mean, and here both are yours.
Paste a protocol, a policy or incident notes into the quiz maker and choose best and worst. Procto AI writes the questions from that text, and each is yours to edit before a student sees it. Treat a drafted key as a proposal for your panel.
Each question holds its responses, the one that is best and the one that is worst, and the student marks one of each. Half the credit belongs to each end. Partial credit is off by default, and with it off the question earns its marks only when both ends match. Switch it on and one matching end earns half the marks. Negative marking does not apply to this shape. The key never leaves the server, and the marks are made there.
For candidates, the hiring page describes a situational judgement kit: workplace situations on safety, honesty, priorities, customers and policy, each with the best response and the reason. That kit asks for the best response only.
Free carries the core templates, marking by rule and results question by question. Tutor opens all 111 templates and Procto AI writing from a PDF, a Word file or your notes. Educator adds the full analysis. See pricing.
When should you use a case study or plain multiple choice instead?
Use another shape when one action is simply right, when the judgement unfolds over time, or when the whole order matters.
- A rule requires one action. A dose or a step in a protocol is knowledge, and a multiple-choice question written to measure tests it more cleanly.
- The situation develops. A case study carries one scenario across several questions, each answer revealing more, with no going back.
- The whole order matters. Use ranking or rank by likelihood. With partial credit on, EdProcto credits every pair of items ranked the right way round and takes away every pair ranked the wrong way round; with it off, only the complete order earns the marks.
- The reason matters as much as the pick. Use choose and justify: the choice is marked by rule for half the points, and the reason typed under it, read by you or by Procto AI when you ask, is the other half.
- The skill is saying it to a person. Use a roleplay, a spoken conversation with a character, instead.
Quick questions
Should the instruction say "would do" or "should do"?
In a marked quiz, ask which response is best and which is worst. With a place at medical college at stake, Lievens et al. (2009) found no meaningful difference in scores or validity between the two instructions.
How does EdProcto mark a best and worst question?
EdProcto marks its best and worst shape by rule: half the credit for each end. With partial credit off, the default, both must match the key to earn the marks. With it on, one matching end earns half.
Does a best and worst score measure character?
No. It counts agreement with a panel's key on particular situations, and researchers still debate what lies behind such scores (Lievens et al., 2008). Report it as a score on this test.
Can best and worst questions be used in hiring?
Yes, as one input beside a test of job knowledge. How a test may be used in selection is a matter of law that differs by country, which this guide does not advise on.
References
- Arthur, W., Glaze, R. M., Jarrett, S. M., White, C. D., Schurig, I., & Taylor, J. E. (2014). Comparative evaluation of three situational judgment test response formats in terms of construct-related validity, subgroup differences, and susceptibility to response distortion. Journal of Applied Psychology, 99(3), 535–545. https://doi.org/10.1037/a0035788
- Bergman, M. E., Drasgow, F., Donovan, M. A., Henning, J. B., & Juraska, S. E. (2006). Scoring situational judgment tests: Once you get the data, your troubles begin. International Journal of Selection and Assessment, 14(3), 223–235. https://doi.org/10.1111/j.1468-2389.2006.00345.x
- Lievens, F., Peeters, H., & Schollaert, E. (2008). Situational judgment tests: A review of recent research. Personnel Review, 37(4), 426–441. https://doi.org/10.1108/00483480810877598
- Lievens, F., Sackett, P. R., & Buyse, T. (2009). The effects of response instructions on situational judgment test performance and validity in a high-stakes context. Journal of Applied Psychology, 94(4), 1095–1101. https://doi.org/10.1037/a0014628
- McDaniel, M. A., Hartman, N. S., Whetzel, D. L., & Grubb, W. L. (2007). Situational judgment tests, response instructions, and validity: A meta-analysis. Personnel Psychology, 60(1), 63–91. https://doi.org/10.1111/j.1744-6570.2007.00065.x
- McDaniel, M. A., Morgeson, F. P., Finnegan, E. B., Campion, M. A., & Braverman, E. P. (2001). Use of situational judgment tests to predict job performance: A clarification of the literature. Journal of Applied Psychology, 86(4), 730–740. https://doi.org/10.1037/0021-9010.86.4.730
- Patterson, F., Ashworth, V., Zibarras, L., Coan, P., Kerrin, M., & O'Neill, P. (2012). Evaluations of situational judgement tests to assess non-academic attributes in selection. Medical Education, 46(9), 850–868. https://doi.org/10.1111/j.1365-2923.2012.04336.x
- Sackett, P. R., Zhang, C., Berry, C. M., & Lievens, F. (2022). Revisiting meta-analytic estimates of validity in personnel selection: Addressing systematic overcorrection for restriction of range. Journal of Applied Psychology, 107(11), 2040–2068. https://doi.org/10.1037/apl0000994
- Webster, E. S., Paton, L. W., Crampton, P. E. S., & Tiffin, P. A. (2020). Situational judgement test validity for selection: A systematic review and meta-analysis. Medical Education, 54(10), 888–902. https://doi.org/10.1111/medu.14201