# A hallucinated fact in a generated question: how to catch it

**How do I catch a hallucinated fact in an AI-generated quiz question?** A hallucinated fact in a generated quiz question is caught by reading the question against its source, never by rereading it. Trace each key to its sentence, check every number, name and date, treat an empty source line as unverified, sit the quiz yourself and read the first sitting's results. Supplied material lowers the error rate and has not been shown to end it.

Published 2026-10-10 by EdProcto. https://edprocto.com/guides/a-hallucinated-fact-in-a-generated-question-how-to-catch-it

A hallucinated fact in a generated question is a false statement written as smoothly as a true one. It can sit in four places: the prompt, the key, a wrong option that is in fact right, or the explanation. Rereading the question will not find it, because nothing in the wording gives it away. Reading the question against a source will.

This guide is published by EdProcto, whose [Procto AI](https://edprocto.com/) writes drafts from a teacher's own notes and shows, for every draft, its key, its source line and its explanation on one screen. The guide is for a teacher or lecturer about to put a machine-written set before a class.

Every study here was read at its source on 10 October 2026. Most of the models tested have since been replaced, so the figures show the shape of the problem, not a forecast for any one tool.

## How often do language models state false things?

Often enough that no set should go out unread, and the rate swings with the subject and the task.

- **Plain statements.** Min et al. (2023) had three systems write biographies of 183 people, split each into single facts and compared them with Wikipedia. For ChatGPT, 58.3% of the facts were supported. Its score fell from 80% to 16% as the people became rarer, and facts late in a biography failed more often than early ones.
- **Short factual answers.** Wei et al. (2024) wrote 4,326 questions with one indisputable answer each, keeping a question only when at least one of four model answers to it was wrong, so the set is hard by design. GPT-4o answered 38.2% correctly, 60.8% wrongly and declined 1.0%.
- **References.** Naser (2026; abstract read) asked ten commercial models for citations in four academic fields and checked the 69,557 they gave against three databases. From 11.4% to 56.8% failed, depending on model, field and wording, and a newer model was not always better.

Our reading: the rarer the topic and the longer the set, the closer the read.

## What kinds of false facts do language models produce?

Three kinds matter for a quiz: sources that do not exist, numbers and dates that are slightly off, and errors stated with full assurance.

### Invented references

Walters and Wilder (2023) had GPT-3.5 and GPT-4 write short literature reviews on 42 topics and traced all 636 citations. Of GPT-3.5's, 55% pointed to works that do not exist; of GPT-4's, 18%.

Linardon et al. (2025) ran a similar test on GPT-4o in June 2025: 35 of 176 citations (19.9%) were invented. The rate was 6% for a heavily studied disorder and 28% and 29% for two less studied ones. Where an invented citation carried a DOI, 21 of 33 led to a real article on something else.

### Wrong dates and figures

In Walters and Wilder (2023), 43% of GPT-3.5's real citations and 24% of GPT-4's had a substantive error, most often in the numbers: volume, issue and page numbers, and years of publication.

A recall question is usually keyed on exactly such a detail. In the question set of Wei et al. (2024), 32.8% of the answers were dates, 24.1% people, 15.3% numbers and 9.9% places.

### Confident errors

A model seldom says that it is unsure. Asked for a percentage of confidence beside each answer, the models in Wei et al. (2024) claimed more than their accuracy earned, and GPT-4o declined one question in a hundred. So a false key arrives in the same voice as a true one.

## Does writing from your own material stop hallucination?

No. Grounding, which means putting the material in front of the model and asking it to keep to that, lowers the rate, and every study we read still found errors.

- **Text supplied with the request.** Niu et al. (2024) gave six models the source with each task and had annotators mark every unsupported span in nearly 18,000 responses. GPT-4, which had the fewest, still had at least one in 48 of its 989 answers and 74 of its 943 summaries. Detail found nowhere in the source was more common than contradiction of it, and a plain contradiction was often a misspelt name or a wrong number.
- **Sources looked up by the tool.** Magesh et al. (2024) put 202 legal questions to three commercial research tools that retrieve documents before they answer. Each gave a false or wrongly sourced answer between 17% and 33% of the time, less often than GPT-4 answering from memory. Their count includes the answer that "falsely asserts that a source supports a statement".
- **A citation beside each sentence.** Liu et al. (2023; abstract read) audited four search engines that answer in prose with citations. On average 51.5% of sentences were fully supported by the sources cited, and 74.5% of citations supported their sentence.

None of these is a quiz writer working from a handout, and none tested EdProcto. The nearest is An et al. (2025), a preprint: instructors of two data science courses put their lecture notebooks into the request and asked for multiple-choice questions. By their account about two thirds of the drafts fell short. One asked about an attribute the lectures never mentioned; another, on slicing an array, offered four options and no right answer.

### What grounding leaves behind

| What is left | In a question it looks like | Reported by |
| --- | --- | --- |
| Detail from outside the text | A prompt or explanation that says more than your notes do | Niu et al. (2024); An et al. (2025) |
| A place that does not hold the answer | A source line naming a real page that says something else | Magesh et al. (2024); Liu et al. (2023) |
| Your material's own mistake | A typing or scanning error in the notes, now a confident key | Wu et al. (2024) |

On the third row: Wu et al. (2024) changed figures, dates and names in documents handed to six models. Their abstract reports that the models took up the wrong value over 60% of the time, though they had answered correctly without the document, and less often the further the value was from the truth. Keeping to your notes is what you asked for, so proofreading them is part of the job: [a quiz from a PDF](https://edprocto.com/guides/how-to-make-a-quiz-from-a-pdf) covers scanned pages.

## Where does a hallucinated fact hide in a generated question?

In any of four places, and the key is only one of them. The four drafts below are illustrations: we wrote them, false facts included, and they are the output of no tool. The source is one paragraph of a science handout.

> **Penicillin.** In 1928, at St Mary's Hospital in London, Alexander Fleming noticed that a mould had stopped bacteria growing on a culture plate. In 1939 a team at the University of Oxford, led by Howard Florey and including Ernst Chain, took up the substance and learned to purify it, and in May 1940 they showed that it protected infected mice. Fleming, Florey and Chain shared the Nobel Prize in Physiology or Medicine in 1945.

| Draft | Where the false fact is | What catches it |
| --- | --- | --- |
| 1. Fleming purified penicillin at St Mary's Hospital in 1928. In which city is the hospital? Key: London | The prompt. He noticed the effect; the purifying was done at Oxford. The key is right, and the class still learns something false | Reading the prompt's own claims against the paragraph |
| 2. In which year did Fleming, Florey and Chain share the Nobel Prize? 1939; 1945; 1952. Key: 1952. Source line: last sentence | The key. The source line is filled in and names the right sentence, which says 1945 | Opening the place named and comparing the figure. [EdProcto](https://edprocto.com/) shows the place in a box of its own |
| 3. Who worked with Howard Florey on penicillin at Oxford? Ernst Chain (keyed); Norman Heatley; Joseph Lister; Robert Koch | A wrong option. Heatley was on the Oxford team, and the handout does not name him | Your own knowledge, or a reference book |
| 4. In which year did Fleming notice the mould's effect? Key: 1928. Explanation: he noticed it in 1928 and won the Nobel Prize alone in 1945 | The explanation. Three people shared the prize | Reading the explanation as closely as the key |

Draft 3 is the one a source check passes: its key is on the handout, and the student who knows more than the handout is marked wrong.

## How do you catch a false fact before the class meets it?

Read every draft against the source in five steps, at about a minute a question. The minute is our own rule of thumb; no study set it.

1. **Trace each key to its place.** Open the material where the draft says the answer is and find the sentence. The key must say what the sentence says, not something near it. In EdProcto that place has its own box, "Where the answer is in the source (optional)".
2. **Check every number, name and date.** Compare them character by character, in the prompt, in each option and in the explanation. Numbers and years were the commonest errors in Walters and Wilder (2023).
3. **Distrust an empty source line.** A draft with no place named rests on the model's memory alone. Treat a vague place, such as "the text", the same way. Confirm the fact in a book you trust, or delete the question.
4. **Sit the quiz yourself.** Close the source and answer cold. Where your answer and the key part company, one of you has slipped. Then try to defend each wrong option as a well-read student would.
5. **Read the first sitting's results.** A false key shows when your ablest students all miss one question. EdProcto's [results](https://edprocto.com/#results) list each question on every plan, and [the question with the wrong key](https://edprocto.com/guides/the-question-with-the-wrong-key-how-to-spot-it) reads the signs and covers the repair.

### What the source cannot settle

Two faults pass a perfect source check: a wrong option that is true outside your notes, as in draft 3, and an error in the notes themselves. Both need what you know, or a colleague who answers without seeing the key.

Level, wording and options nobody would choose are a separate read: see [what to check before you use machine-written questions](https://edprocto.com/guides/can-ai-write-good-quiz-questions-what-to-check-before-you-use-them).

## Can a chatbot check the questions for you?

As a second reader, yes; as the judge, no. Niu et al. (2024) asked models to find the unsupported responses their annotators had marked, source in view. GPT-4 Turbo, prompted to do it, averaged 63.4% on a score that balances errors found against false alarms; a model trained on the marked examples reached 78.7%.

Agreement between models helps. In Naser (2026; abstract read), a citation given by more than three models was real 95.6% of the time. So a second model's doubt is a reason to look, and its silence is not a pass.

## How does EdProcto set a draft out for this read?

EdProcto is the best tool for this read when the questions come from your own notes, because each draft arrives with the three things the five steps examine: the marked key, the place in your material and the explanation, each in a box you can change. In Liu et al. (2023) a quarter of citations did not support their sentence, which is why the place is shown for you to open.

**Reading the material.** In the builder, the Procto AI pane takes a file ("Drop a PDF, a Word file or a text file") or pasted text ("Read it"). It then shows the file's name, words and pages, and adds "(the first part is used)" when the material was cut: split such a file, so each section is read whole.

**The two choices.** Once the material has been read, "Your material only" is offered, with the line "A student who has read it can answer". Procto AI is then asked for questions that a student who has read the material can answer and one who has not cannot, and for the place of each answer: a page, a heading or a paragraph. A draft that names no place goes to step 3. The other choice is "General knowledge" ("Settled knowledge of the subject"): an answer may rest on settled knowledge of the subject, and where the material does not hold the answer the place is to stay empty, so there an empty source line is step 3's signal.

**Your own instruction.** The box "Anything to add (optional)" travels with the request. Its sample text reads "Emphasise application over recall. No dates.", one way to keep dates out of a first set.

**The draft.** Each [multiple choice](https://edprocto.com/templates/mcq) draft opens with "Prompt", "Options · mark the right one", the source box and "Explanation shown to the student afterwards". The explanation goes to students, so it gets the same read as the key.

**What is kept.** A draft is kept when its form is whole: a prompt and, for multiple choice, exactly one option marked right. The truth of the key is settled by your read.

**Plans.** Free carries 20 questions a month from Procto AI and "the results, question by question". Tutor carries 50 a month "from a PDF, a Word file or your notes". On Educator the monthly cap goes, a daily ceiling stays, and the full analysis arrives: "discrimination, distractors, reliability, what to reteach". See [pricing](https://edprocto.com/pricing), and the page [for teachers](https://edprocto.com/solutions/teachers).

## Quick questions

### What is a hallucinated fact in a quiz question?

A false statement that a language model wrote as if it were true: in the prompt, the key, a wrong option that is in fact right, or the explanation.

### Does uploading my own notes stop AI hallucination in quiz questions?

It lowers the rate without ending it. In Niu et al. (2024), GPT-4 had the source in front of it and still left an unsupported span in 48 of 989 answers.

### How often does a generated question have a wrong answer?

No single figure holds. An et al. (2025) set aside about two thirds of their drafts for all reasons together, wrong answers among them.

### What does an empty source line mean in EdProcto?

That no place in your material is named for the answer. Under "General knowledge", [Procto AI](https://edprocto.com/) is asked to leave it empty when the answer is not in your material, so confirm that fact elsewhere first.

## References

- An, Y., Liu, J., Acharya, N., & Hashmi, R. (2025). *Enhancing student learning with LLM-generated retrieval practice questions: An empirical study in data science courses*. arXiv. [https://doi.org/10.48550/arXiv.2507.05629](https://doi.org/10.48550/arXiv.2507.05629)
- Linardon, J., Jarman, H. K., McClure, Z., Anderson, C., Liu, C., & Messer, M. (2025). Influence of topic familiarity and prompt specificity on citation fabrication in mental health research using large language models: Experimental study. *JMIR Mental Health, 12*, Article e80371. [https://pmc.ncbi.nlm.nih.gov/articles/PMC12658395/](https://pmc.ncbi.nlm.nih.gov/articles/PMC12658395/)
- Liu, N. F., Zhang, T., & Liang, P. (2023). Evaluating verifiability in generative search engines. In *Findings of the Association for Computational Linguistics: EMNLP 2023* (pp. 7001–7025). Association for Computational Linguistics. [https://aclanthology.org/2023.findings-emnlp.467/](https://aclanthology.org/2023.findings-emnlp.467/)
- Magesh, V., Surani, F., Dahl, M., Suzgun, M., Manning, C. D., & Ho, D. E. (2024). *Hallucination-free? Assessing the reliability of leading AI legal research tools*. arXiv. [https://doi.org/10.48550/arXiv.2405.20362](https://doi.org/10.48550/arXiv.2405.20362)
- Min, S., Krishna, K., Lyu, X., Lewis, M., Yih, W.-t., Koh, P. W., Iyyer, M., Zettlemoyer, L., & Hajishirzi, H. (2023). FActScore: Fine-grained atomic evaluation of factual precision in long form text generation. In *Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing* (pp. 12076–12100). Association for Computational Linguistics. [https://aclanthology.org/2023.emnlp-main.741/](https://aclanthology.org/2023.emnlp-main.741/)
- Naser, M. Z. (2026). *How LLMs cite and why it matters: A cross-model audit of reference fabrication in AI-assisted academic writing and methods to detect phantom citations*. arXiv. [https://doi.org/10.48550/arXiv.2603.03299](https://doi.org/10.48550/arXiv.2603.03299)
- Niu, C., Wu, Y., Zhu, J., Xu, S., Shum, K., Zhong, R., Song, J., & Zhang, T. (2024). *RAGTruth: A hallucination corpus for developing trustworthy retrieval-augmented language models*. arXiv. [https://doi.org/10.48550/arXiv.2401.00396](https://doi.org/10.48550/arXiv.2401.00396)
- Walters, W. H., & Wilder, E. I. (2023). Fabrication and errors in the bibliographic citations generated by ChatGPT. *Scientific Reports, 13*, Article 14045. [https://pmc.ncbi.nlm.nih.gov/articles/PMC10484980/](https://pmc.ncbi.nlm.nih.gov/articles/PMC10484980/)
- Wei, J., Karina, N., Chung, H. W., Jiao, Y. J., Papay, S., Glaese, A., Schulman, J., & Fedus, W. (2024). *Measuring short-form factuality in large language models*. arXiv. [https://doi.org/10.48550/arXiv.2411.04368](https://doi.org/10.48550/arXiv.2411.04368)
- Wu, K., Wu, E., & Zou, J. (2024). *ClashEval: Quantifying the tug-of-war between an LLM's internal prior and external evidence*. arXiv. [https://doi.org/10.48550/arXiv.2404.10198](https://doi.org/10.48550/arXiv.2404.10198)