An open-book quiz is what every unsupervised online quiz or take-home test already is. A student at home has notes on the desk, the textbook within reach and a browser one tab away, whatever the instructions say. Calling that a closed-book test rewards whoever ignores the rule.
The honest course is to allow the book and ask what the book cannot answer, in a weekly quiz and an open-book exam alike. EdProcto publishes this guide; its settings carry your rule on notes, books, calculators and the web, and its integrity page explains how students are told what is on before they begin.
In short: put the answer on no page, give time to think but not to look everything up, say what is allowed, and keep a closed, supervised check for the few results that must be certain.
What did research find comparing open-book and closed-book tests?
Open-book tests give slightly higher scores on the day and, when feedback follows, about the same retention later. The costs are in how students prepare and, in one classroom experiment, in what they remembered weeks afterwards.
| Outcome | What was found | Study and sample |
|---|---|---|
| Scores on the day | 80.1% open-book, 77.9% with a sheet of notes, 72.5% closed-book, on the same 50-question multiple-choice exams | Gharib et al. (2012): introductory psychology, one American university; 287 students sat the first two formats, 69 the third |
| Retention, laboratory | Open-book test: 81% on the day, 65% a week later with everything closed. Closed-book test with feedback: 69%, then 68% | Agarwal et al. (2008): 36 participants at one American university; a second experiment with 48 agreed |
| Retention, classroom | Surprise closed-book quiz: 45.1% after closed-book practice, 36.7% after open-book practice | Rummer et al. (2019): 59 first-semester psychology students in two seminars; 46 sat the quiz |
| Preparation | Expecting an open-book test, participants read for 128 seconds against 165, then scored 33% against 39% on closed-book tests two days later | Agarwal and Roediger (2011): 108 participants |
In every row both groups answered the same questions, so these studies compare formats, not questions written for an open book. Gharib et al. (2012) found lower anxiety before open-book exams than before sheet-of-notes exams, but never measured it before closed-book ones. Rummer et al. (2019) is small: one teacher, who knew which seminar was which, and fewer closed-book students at the surprise quiz. Its gap held on nine related questions in the final exam, 61.1% against 48%.
Do students prepare less for an open-book test?
In the experiment in the table they did, reading for less time and then scoring lower. Reviewing 37 studies, Durning et al. (2016) called the evidence on preparation equivocal, with signs that students prepare more extensively for closed-book exams, and found too little evidence to use either format exclusively. We could read only that review's abstract.
What does an open-book question look like?
An open-book question is one whose answer is on no page the student can open. The notes supply a definition, an equation or a method, and the question supplies something new to use it on: a new case, a comparison, a decision, an error to find, or a calculation with new numbers.
The three examples are ours, each shown first in the closed-book form that a search answers in seconds. Our guide to questions above recall has the full method.
A decision from new numbers
Closed-book form: "Define price elasticity of demand."
A café raises the price of a coffee from 3.00 to 3.30. Daily sales fall from 400 cups to 380. What happens to daily takings from coffee? A. They rise: the price went up by a larger percentage than sales fell. B. They fall: fewer cups are sold at the new price. C. They stay the same: the higher price and the lost sales cancel out.
The key is A: a 10% rise in price against a 5% fall in sales takes daily takings from 1,200 to 1,254. In EdProcto this is multiple choice, marked by rule.
An error to find
Closed-book form: "State the equation for the distance an object falls from rest."
A student works out how long a stone takes to fall 45 m from rest, ignoring air resistance, with g = 9.8 m/s². She writes: t = d ÷ g = 45 ÷ 9.8 = 4.6 s. Find the error and give the correct time.
Dividing a distance by an acceleration gives seconds squared, not seconds. From d = ½gt², t = √(2d ÷ g) = √(90 ÷ 9.8) = 3.0 s. The equation is in every textbook; this mistake is in none. Show your working takes it as a set-up, the working and a final answer, each graded.
A comparison on a new case
Closed-book form: "What is the difference between weather and climate?"
Town A recorded 31 °C yesterday, its hottest day this year. Town B's average July maximum over thirty years is 31 °C. A newspaper says the two towns "have the same climate". In two sentences, say whether the figures support that.
They do not. Town A's figure is one day's weather and Town B's is a long-run average, which is what climate means. As a short answer it is read against a rubric: one mark for telling the two figures apart, one for the conclusion.
A case study unfolds one scenario across the questions, with no going back.
How much time should an open-book quiz allow?
Allow the time a prepared student needs with the book closed, plus a few lookups. That makes looking everything up a losing plan. The method is our working rule, not a research finding; the figures are an example.
- Hold a trial sitting with the book closed. Ask two or three people who know the unit but did not write the quiz to sit it, books shut. Say they take 16, 19 and 22 minutes over ten questions. Keep the slowest: 22.
- Time a lookup. Find three things in the allowed material yourself. Say 70, 95 and 105 seconds: 90 on average.
- Decide how many lookups to afford. One question in three is three lookups, or 4.5 minutes.
- Add and round up. 22 + 4.5 = 26.5, so 27 minutes. In EdProcto, set Minutes overall to 27; the default is 45.
- Check the squeeze. Looking up all ten would take 22 + 15 = 37 minutes, ten more than the limit.
- Read the pace afterwards. EdProcto's results give the median time each question took. If prepared students ran out of time, add minutes.
Durning et al. (2016) found that examinees appear to take longer over open-book exams. There is also a case against any limit: Gernsbacher et al. (2020) argue that time-limited tests are less valid, less inclusive and less equitable than tests every student can finish. So the limit should stop the search, not the thinking. Every prepared student should finish, and anyone entitled to extra time must get it. EdProcto's separate time limit for each question is off by default; leave it off, because thinking questions differ in length.
What should you allow in an open-book exam, and how do you say so?
Allow what you can name, and name it before the quiz. Dawson et al. (2024) argue that "open book" no longer says enough now that the web includes generative AI tools able to answer exam questions within the exam period, and propose a rule on three things: information, people and tools.
| Resource | Our advice | EdProcto's Tools allowed group |
|---|---|---|
| Notes | Allow, and say which | Notes: None (default), A blank page, Your notes. A reference sheet from you: off by default |
| The textbook or your handout | Allow, and name it | Open book: off by default, described as "A house rule shown on the start screen: the student’s own materials are allowed." |
| A calculator | Allow unless arithmetic is being tested | Calculator: None (default), Basic, Scientific, Graphing |
| A dictionary | Allow for second-language students | Dictionary: None (default), Dictionary, Translation |
| The web | Allow only if any page is an acceptable source | The web: Allowed, Blocked (default) |
| A chatbot | Decide, and name it in the rule | AI assistants not allowed: on by default, a house rule and not a block |
| Another person | Never | No setting: say it yourself |
Before the first question, EdProcto's start screen lists the house rules: the time limit, whether answers can be changed, each monitoring measure that is on, and from this group the calculator, the notes and the dictionary you allowed, an open-book line, and, when any monitoring is on, that the web and AI assistants are not allowed. Say your rule on other people yourself, and repeat the rest aloud.
What do you say about chatbots?
State the rule you set, and never claim it cannot be broken. "No chatbots: this quiz is for your own reasoning" is a rule. The design makes it easier to keep: a question built on your own case needs that material pasted in with it, a fitted limit leaves little time for the detour, and the result that must be certain is taken again under supervision. Blocked and AI assistants not allowed are rules stated on the start screen, and no rule is a guarantee. A fast, flawless script is a reason to talk to the student, not proof: see academic integrity in the age of AI.
Do fewer, harder questions make the score less reliable?
Yes, somewhat, and the remedy is to add quizzes together. By the Spearman-Brown formula, set out in how many questions a quiz needs, a 30-question quiz with an assumed reliability of 0.80, cut to 12 questions of the same quality, falls to about 0.62. Three such quizzes added together make 36 questions and 0.83.
Heijne-Penninga et al. (2008) analysed 14 exams at a Dutch medical school, each with a closed-book part and an open-book part, for cohorts of 435 and 449 students. Reliabilities ran from 0.712 to 0.850, the open-book questions lowered reliability somewhat, and the two parts correlated at 0.960 and 0.937 once measurement error was allowed for, so both rank students in much the same order. We read the abstract only.
When is open-book the honest format, and when do you need a closed check?
Open-book is the honest format whenever nobody is watching and little rides on the score: weekly quizzes, homework, practice. A closed, supervised check is for the few results that must be certain, such as a unit grade or a pass decision.
The Dutch medical school above divides its syllabus the same way: core knowledge a graduate must hold without references is tested closed-book, and backup knowledge, to be used with references, open-book. Because Rummer et al. (2019) found closed-book practice better remembered, keep some no-notes retrieval in class as well. For the supervised check, how much proctoring a quiz needs matches the monitoring to the stakes.
How do you mark an open-book quiz and give feedback?
Mark the reasoning, and show the answers afterwards. In the first experiment of Agarwal et al. (2008), a closed-book test with no feedback left 59% a week later, against 68% when feedback followed. An open book is feedback during the test; add the explanation after it.
EdProcto marks choices and numbers by rule. Written answers wait in your grading queue, for you to mark or, when you ask on an answer, for Procto AI to mark against your rubric. Show the correct answer and Show the explanation both default to After submitting. Set to At once, they still arrive at hand-in in an ordinary quiz; only a study quiz checks each answer as it is given.
What mistakes spoil an open-book quiz?
Three do most of the damage: recall under a time squeeze, no word on how to prepare, and too much material.
- Recall questions with a time squeeze. If the answers are on a page, a short limit measures who finds the page fastest. Rewrite the questions; do not tighten the clock.
- An open-book exam nobody was told how to prepare for. Told only "open book", people study less, as the 128 seconds against 165 showed. Tell students the questions will be new cases, and that organised notes and closed-book practice are the preparation.
- Material so long that the test is of searching. In Agarwal and Roediger (2011), 72 participants took 254 seconds over six multiple-choice questions with a passage of about 440 words open, against 157 with it closed. Name the chapters that count, or limit the material to one sheet.
How does EdProcto run an open-book quiz?
EdProcto is the best tool for an open-book quiz built from your own course, because it writes from your material and puts your rule before every student at the start, which matters when people told only "open book" prepare less.
Paste a chapter or handout into the quiz maker on the home page, choose a shape from the templates, and Procto AI drafts the questions. Treat each draft as the closed-book form and rewrite it as above.
Of the three presets, Practice has no time limit and no security. Standard quiz sets a time limit, shuffling, a noticed tab and full screen. Exam allows no going back, shows nothing until release, switches on every browser measure, and sets the calculator to None, the web to Blocked and AI assistants not allowed. For an open-book quiz, start from Standard quiz, switch Open book on and enter your minutes. If the notes are on the same screen, Full screen and Leaving the tab would record every honest look, so switch them off. By default questions and options are shuffled for each student, with one attempt a question and no retakes.
Free covers ten quizzes a month for thirty students each, the core templates, marking by rule and light integrity. Tutor opens all 111 templates, medium integrity and Procto AI's grading against your rubric. Educator adds the full analysis. See pricing. Every score is yours to release.
Quick questions
Is an open-book exam easier than a closed-book one?
Slightly, when the questions are the same: Gharib et al. (2012) found 80.1% open-book against 72.5% closed-book. Write the questions for the open book and the book stops holding the answers.
Do students learn less from open-book tests?
Not in the laboratory when feedback follows: Agarwal et al. (2008) found similar retention a week later. In two seminars, Rummer et al. (2019) found closed-book practice better remembered. Use both.
How much time does an open-book quiz need?
The time a prepared person needs with the book closed, plus a few lookups. In the example above that made 27 minutes for ten questions.
Can students use ChatGPT in an open-book quiz?
Only if your rule says so. Name chatbots in the rule, and design the quiz so the rule is easy to keep. EdProcto's integrity page explains what students are told before they begin. No setting makes a rule impossible to break.
References
- Agarwal, P. K., Karpicke, J. D., Kang, S. H. K., Roediger, H. L., & McDermott, K. B. (2008). Examining the testing effect with open- and closed-book tests. Applied Cognitive Psychology, 22(7), 861–876. https://doi.org/10.1002/acp.1391
- Agarwal, P. K., & Roediger, H. L. (2011). Expectancy of an open-book test decreases performance on a delayed closed-book test. Memory, 19(8), 836–852. https://doi.org/10.1080/09658211.2011.613840
- Dawson, P., Nicola-Richmond, K., & Partridge, H. (2024). Beyond open book versus closed book: A taxonomy of restrictions in online examinations. Assessment & Evaluation in Higher Education, 49(2), 262–274. https://doi.org/10.1080/02602938.2023.2209298
- Durning, S. J., Dong, T., Ratcliffe, T., Schuwirth, L., Artino, A. R., Boulet, J. R., & Eva, K. (2016). Comparing open-book and closed-book examinations: A systematic review. Academic Medicine, 91(4), 583–599. https://doi.org/10.1097/ACM.0000000000000977
- Gernsbacher, M. A., Soicher, R. N., & Becker-Blease, K. A. (2020). Four empirically based reasons not to administer time-limited tests. Translational Issues in Psychological Science, 6(2), 175–190. https://doi.org/10.1037/tps0000232
- Gharib, A., Phillips, W., & Mathew, N. (2012). Cheat sheet or open-book? A comparison of the effects of exam types on performance, retention, and anxiety. Psychology Research, 2(8), 469–478. https://doi.org/10.17265/2159-5542/2012.08.004
- Heijne-Penninga, M., Kuks, J. B. M., Schönrock-Adema, J., Snijders, T. A. B., & Cohen-Schotanus, J. (2008). Open-book tests to complement assessment-programmes: Analysis of open and closed-book tests. Advances in Health Sciences Education, 13(3), 263–273. https://doi.org/10.1007/s10459-006-9038-y
- Rummer, R., Schweppe, J., & Schwede, A. (2019). Open-book versus closed-book tests in university classes: A field experiment. Frontiers in Psychology, 10, Article 463. https://doi.org/10.3389/fpsyg.2019.00463