A quiz question has a shape before it has any words: a row of options, an empty box, a picture to click, a microphone. Which question shape to use depends on what you want to find out, and the choice decides what the score will mean.
EdProcto, which publishes this guide, files its 111 templates in twelve categories. This page maps them by the act each asks for, and links every shape that has a guide of its own.
The short version: ask for the act you taught, count the answers a person must read, and give every student a way in.
How do you choose the shape of a question?
Choose the shape whose answer is the same act as the thing learned. A student taught to name a capital should type it, and one taught to pronounce a phrase should say it. Seven acts cover most teaching: choosing, producing, ordering, pointing, listening, speaking and showing working.
The shape does not set the level of thinking. Schuwirth and van der Vleuten (2004), reviewing written assessment in medicine, separate what a question asks from how its answer is recorded and conclude, "The response format is less important in this respect than the stimulus." A choice on a new case can demand more than an essay asking for a list: see questions above recall.
What does the research say about question format?
Research finds that choosing and producing overlap more than teachers expect, that options cue, and that a produced answer is paid for in marking. We read Sam et al. (2019) in full, the W3C standard's own criteria, and the abstracts of the rest.
Do selected response and constructed response measure the same thing?
Largely, when the question is the same. Rodriguez (2003) found 29 studies holding 56 correlations between selected response (an answer chosen) and constructed response (an answer produced). Built on one stem, the two correlated close to one once measurement error was allowed for. Where the questions differed the figure fell significantly, furthest for essays.
The dispute is over what follows. Schuwirth and van der Vleuten (2004) would keep open-ended questions only for what a choice cannot test, citing lost reliability and cost. Sam et al. (2019) conclude that options flatter what students know.
How much do options cue the answer?
A good deal. In Sam et al. (2019), 1,417 volunteers in their final year at 20 UK medical schools met 50 clinical questions twice: first typing one to five words, then picking from five options. The options were worth 21 percentage points. After a miss on the typed version, students chose correctly 42.7% of the time, where blind guessing gives 20%. It was a practice test with the questions fresh in mind, which the authors say may have lifted the figure.
Does the format change what students remember?
Less than its reputation suggests. Smith and Karpicke (2014) ran four experiments with 372 students, who read texts, then practised with short-answer questions, multiple-choice questions or both. A week later every kind of practice had beaten study alone, and in three experiments typing held little or no advantage over choosing. It came out ahead only in the fourth, where more of the typed answers were recalled.
What does marking cost, and how reliable is it?
A mark by rule costs nothing and never varies, while a reader costs time and does. Short typed answers sit between: in Sam et al. (2019) a computer matched answers to an accepted list and clinicians reviewed the rest, 24.5 hours in all, or 1.25 seconds an answer.
Long answers are where readers part. In 233 essay papers from three final surgery examinations, Anatol and Hariharan (2009) found that two examiners gave an answer the same mark on 46.3% of occasions, and marks within 5% of each other on 90.7%.
Which formats shut some students out?
Dragging, audio and images each shut some students out unless there is a second way to answer. The web accessibility standard expects everything to work from a keyboard, and since version 2.2 asks that whatever is done by dragging can also be done with single taps or clicks (World Wide Web Consortium, 2024). It asks for a text alternative to pictures and sound, except that a test which words would spoil need only say what the content is. So a listening question can be legitimate and still shut a deaf student out. We found no study comparing shapes for students who use a screen reader.
What is each of the twelve categories for?
Each is for one job. The sections keep EdProcto's order and open with its own line for the category. Every shape is in the builder.
Selection
"A choice between options." Twelve shapes, all marked by rule. Its one job is comparison: every student meets the same options, so a cohort reads side by side. Slow to write, since tempting wrong options take longer than the stem and their number matters; free to mark. The mistake is using it where the skill is producing. Guides: Multiple choice, Multiple answers, True or false, Yes, no or maybe, Best and worst, Multiple choice with confidence, All that apply, Pick the picture, Rank by likelihood.
Text input
"A typed answer." Ten shapes. Its one job is recall with no option to recognise. Five are marked by rule against the answers you list, Cloze passage and One word among them; Short answer, Translate, Define the term and Summarise wait for a reader, and Choose and justify is half each. Quick to write. The mistake is one accepted answer where students have five right ways to say it.
Pair and order
"Match, sort, sequence." Nine shapes. Its one job is structure: what goes with what, and what follows what. Eight are marked by rule, all or nothing unless "Partial credit" is on; Sort and justify adds a reason for a reader. Cheap to write. The mistake is a sequence with two defensible orders. Guides: Matching pairs, Reorder the sentences, Reorder the code, Sequence the process, Timeline, Sort into groups, Ranking.
Spatial and visual
"Draw, label, locate." Fourteen shapes, all marked by rule but Annotate the image, which waits for you. Its one job is place and structure: a click on an unlabelled picture shows where, and a diagram built from its parts shows how they connect. The cost is a clean image and a region you draw. The mistake is a picture that keeps its labels. Guides: Hotspot, Compare two images, Continue the pattern.
Media prompts
"Audio, video or an image as the question." Nine shapes. Its one job is comprehension of something heard or watched. Six are marked by rule, Listen once and Watch and answer among them; Listen and answer, Caption it and Identify the source wait for a reader. The cost is the recording. The mistake is a question the wording answers without the clip.
Memory and recall
"Shown, hidden, asked." Five shapes, all marked by rule. Its one job is asking after the material has left the screen, so the answer comes from what the student holds. Cheap to write once the seconds are chosen. The mistake is a reveal too short to read. One guide covers Memory recall, Detail spotting and the other three.
Voice and performance
"Spoken or recorded answers." Eight shapes. Its one job is the skill that is itself spoken or performed. Quick to write and the dearest to mark: most recordings wait for you, and two minutes from each of thirty students is an hour of listening. Speak and compare is marked by rule against the words, and a Roleplay is a conversation with a character, read by you or by Procto AI. The mistake is a recording with no criteria. Guides: Spoken answer, Sing or play, Presentation.
Handwritten and files
"A photo, a scan or a file as the answer." Six shapes. Quick to set, slow to mark: each file is marked by you. Its one job is to bring work done on paper or at a bench into the same timed attempt. The mistake is a vague instruction, which returns files nobody can mark. Guides: Handwritten work, Redline a document.
Specialised input
"Maths, code, music, chemistry." Eleven shapes. Its one job is an answer in the subject's own notation. Nine are marked by rule: Maths expression and Chemical equation sign for sign against the forms you list, Estimate inside a tolerance, Read code as a choice, Write code in JavaScript by the tests it passes, and Build the equation, Music notation, Spreadsheet cells and Highlight the text against your key. Show your working and Annotate code wait for a reader. The mistake is listing one form of an expression when several are equal: no algebra is done.
Young learners
"Large targets and little reading." Eight shapes, all marked by rule. Its one job is to reach a child who cannot yet read a stem or type an answer: Big buttons to tap, letter tiles in Spell it. The cost is pictures and a recorded prompt. The mistake is a shrunk exam.
Graduate and doctoral
"Research, defence, peer review." Six shapes. Its one job is an extended argument held to a rubric: Defend a position asks for "Claim", "Evidence", "Reasoning" and "Counter-argument and rebuttal". It costs a rubric to write, and every answer waits for a reader: you, or Procto AI when you press "Ask Procto AI" on it. The mistake is a rubric so general that every answer is average. One guide covers all six.
Whole-quiz structures
"How the questions of a quiz relate to one another." Thirteen shapes. These shape the quiz, not a question, and each question inside is marked by its own kind. Their one job is order: a Case study unfolds one scenario, a Scaffolded sequence sets one problem in steps, a Sandwich puts the hard questions in the middle. The cost is planning the structure first. The mistake is a case whose questions do not need the scenario. Also: Mixed media quiz and Exit ticket. The rest walk each student their own way: a Boss question, Adaptive branching, a Mastery loop, a Diagnostic funnel, Before and after, Spiral review, Layered difficulty and a Staged reveal.
Which shape answers which question?
Find what you want to know; beside it are the category and one ready shape, linked to its page on EdProcto.
| What I want to know | Category | Shape |
|---|---|---|
| Can they tell right from nearly right? | Selection | Multiple choice |
| Can they produce the term unprompted? | Text input | One word |
| Can they explain why? | Text input | Short answer |
| What goes with what? | Pair and order | Matching pairs |
| What follows what? | Pair and order | Sequence the process |
| Where is it? | Spatial and visual | Hotspot |
| Do they follow it at first hearing? | Media prompts | Listen once |
| What do they hold once it has gone? | Memory and recall | Memory recall |
| Can they say it aloud? | Voice and performance | Spoken answer |
| Can they do it by hand? | Handwritten and files | Handwritten work |
| Can they set out the method? | Specialised input | Show your working |
| What does a child know before reading? | Young learners | Listen and point |
| Can they argue from evidence? | Graduate and doctoral | Defend a position |
| Can they use it as a situation changes? | Whole-quiz structures | Case study |
How do you mix shapes in one quiz?
Mix shapes by five rules, which make mixing question types a matter of evidence, not variety.
- One shape for each act. List what the unit taught students to do, and give each act its shape.
- Like with like. Keep questions of one shape together, so each way of answering is learned once.
- Typing before choosing. Where two questions touch one fact, ask the typed one first, because options cue (Sam et al., 2019). In EdProcto, leave "Shuffle the questions" and "Going back allowed" off for that quiz.
- Count the reading before you publish. Multiply the questions that wait for a reader by the class.
- Name the equivalent in advance. For each drag, clip and picture, decide the question in another shape, worth the same marks.
How would you choose shapes for a twelve-question unit quiz?
Start from the unit's acts, give each a shape, then count the marking. Take thirty students aged 13 to 14 after a unit on electric circuits.
| Questions | The act | Shape |
|---|---|---|
| 1 and 2 | Name a unit and a component | One word |
| 3 to 6 | Tell a true statement about current from a common wrong one | Multiple choice |
| 7 | Pair each quantity with its unit | Matching pairs |
| 8 | Find the voltmeter on an unlabelled diagram | Hotspot |
| 9 and 10 | Work out a current | Estimate |
| 11 | Set out a two-step calculation | Show your working |
| 12 | Say which bulb is brighter, and why | Choose and justify |
Seven acts, seven shapes, typing before choosing. The first ten are marked by rule at hand-in. Question 11 and the reason in question 12 wait for a reader, sixty pieces of reading, and in an exam, question 12 shows "to grade" and counts as zero until its reason is marked. A student who cannot use a pointer gets question 8 as a choice of what a voltmeter is connected across.
How do you find and use the shapes in EdProcto?
EdProcto is the best tool for choosing a shape, because its 111 shapes are filed in the twelve categories above and each card says how the shape is marked before you write a word. Every one is in the builder.
- Open the "Templates" page of the dashboard. Pick a category from the list, "All" and then the twelve with a count beside each, or type in "Search templates", which looks in every shape's name and one-line description.
- Read the card: the shape's line, how it is marked ("by rule", "by you") and "Use".
- Press "Use" to start a quiz in that shape. In the builder, "Add a question" adds one in the quiz's own shape, and "Another shape" opens the list "Which shape?" with the 98 question shapes, so one quiz can mix them.
- Before publishing an exam, press "Preview as student" and try every question with the keyboard alone.
Free carries the core templates and marking by rule, Tutor opens all 111 templates and grading by Procto AI against your rubric, and Educator adds the full analysis: see pricing.
Quick questions
What are the main types of quiz questions?
Two families: selected response, where the student chooses, and constructed response, where the student produces. EdProcto files its shapes in twelve categories of templates.
Is multiple choice worse than short answer?
Not as a rule. On the same question the two measure nearly the same thing (Rodriguez, 2003), though options were worth 21 percentage points in Sam et al. (2019).
How many question types should one quiz use?
As many as the unit has acts to show: seven in the worked example's twelve questions. We found no study that set a number.
Which question types are marked automatically?
In EdProcto, choices, typed words, gaps, numbers, lists, orders, pairs, sorts and a point on an image are marked by rule. Free text, working, files and recordings wait for a reader.
References
- Anatol, T., & Hariharan, S. (2009). Reliability of the evaluation of students' answers to essay-type questions. West Indian Medical Journal, 58(1), 13–16. https://pubmed.ncbi.nlm.nih.gov/19565993/
- Rodriguez, M. C. (2003). Construct equivalence of multiple-choice and constructed-response items: A random effects synthesis of correlations. Journal of Educational Measurement, 40(2), 163–184. https://doi.org/10.1111/j.1745-3984.2003.tb01102.x
- Sam, A. H., Westacott, R., Gurnell, M., Wilson, R., Meeran, K., & Brown, C. (2019). Comparing single-best-answer and very-short-answer questions for the assessment of applied medical knowledge in 20 UK medical schools: Cross-sectional study. BMJ Open, 9(9), Article e032550. https://doi.org/10.1136/bmjopen-2019-032550
- Schuwirth, L. W. T., & van der Vleuten, C. P. M. (2004). Different written assessment methods: What can be said about their strengths and weaknesses? Medical Education, 38(9), 974–979. https://doi.org/10.1111/j.1365-2929.2004.01916.x
- Smith, M. A., & Karpicke, J. D. (2014). Retrieval practice with short-answer, multiple-choice, and hybrid tests. Memory, 22(7), 784–802. https://doi.org/10.1080/09658211.2013.831454
- World Wide Web Consortium. (2024). Web Content Accessibility Guidelines (WCAG) 2.2 (W3C Recommendation). https://www.w3.org/TR/WCAG22/