# AI detectors are not evidence: studies and false positives

**Can an AI detector score be used as evidence that a student used a chatbot?** AI detectors are not evidence of misconduct. Independent tests found them easy to defeat by paraphrasing and liable to flag honest writers, the makers' own pages say a score is a prompt for a conversation and not grounds to punish, and regulators say it is never sole proof. Rest any decision on evidence the student can see and answer.

Published 2026-10-10 by EdProcto. https://edprocto.com/guides/ai-detectors-are-not-evidence-the-studies-and-the-false-positives

AI detectors are not evidence. A detector reads a piece of writing and returns an estimate of how machine-like its style is. It cannot show where the text came from, and nobody, the student included, can check how it reached its figure.

This guide is for the teacher or lecturer holding a flagged essay, and for the department writing its procedure. The first detector studies are in [academic integrity in the age of AI](https://edprocto.com/guides/integrity-in-the-ai-era-what-still-proves-a-student-learned) and are not repeated here.

EdProcto publishes this guide. It makes the thing that belongs beside a doubted piece of work: a short quiz taken under [conditions everyone knows](https://edprocto.com/integrity).

## Are AI detectors accurate enough to act on?

No. Tested by people who do not sell them, detectors miss much machine text, are easy to evade, and still flag some honest writing.

Perkins et al. (2024) put 805 samples through six detectors. They report accuracy that was already low, at 39.5%, and a reduction they give as 17.4% once the text had been altered with simple techniques meant to evade detection. They conclude that the tools cannot currently be recommended for deciding whether a student broke the rules (abstract read).

Dugan et al. (2024) built a harder test: over six million generated texts, with 11 kinds of attack. The 12 detectors they tried were easily fooled by attacks, by changed generation settings and by models a detector had not met, though many such tools claim accuracy of 99% or more (abstract read).

### How easily does paraphrasing defeat detection?

Very easily, in the tests published so far. Krishna et al. (2023) built a paraphrasing model and passed machine text through it. One detector's catch rate fell from 70.3% to 4.6% with its false positive rate held at 1%, and the rewritten text also slipped past GPTZero, a watermark and OpenAI's classifier (abstract read).

So a student who sets out to cheat can lower the score, and one who wrote every word cannot.

### Are the newest detectors better?

Some appear to be. In a working paper, Jabarian and Imas (2025) compared three commercial detectors with one open-source model. The commercial ones did better, and one, Pangram, kept both kinds of error near zero, even on passages of 50 words or fewer and on text run through "humaniser" tools (abstract read).

A better detector is still not evidence. An error rate measured on a researcher's collection is not the rate in your class, the student cannot test the score that accuses them, and no tool can say which flagged essay is the mistake.

## What do the makers say about error rates and intended use?

Neither maker we read says its score proves misconduct. Both were read on their own sites on 10 October 2026.

| Maker | What it says about errors | What it says a score is for | What it does well |
| --- | --- | --- | --- |
| Turnitin | Chechitelli (2023): fewer than one human-written document in 100 is flagged, counting documents scored above 20% AI; about four highlighted sentences in 100 are human; false positives are commoner under 20%, so that range carries an asterisk; nothing under 300 words is scored | Opening a conversation with the student; its current page says the checker shows when AI may have been used (Turnitin, n.d.) | Publishes its error rates and marks the range it trusts least |
| GPTZero | GPTZero (n.d.): errors run both ways; accuracy is highest for a whole document and lowest for a sentence; text heavily changed after generation is outside its training. Where it reports high confidence, 99.1% of human articles are classed as human | One part of a wider look at the work: "these results should not be used to punish students" | Says so plainly, and offers a report that replays how a document was typed |

Turnitin's help-centre pages would not open for us, so its figures come from a 2023 post and its current AI writing page; newer ones may exist.

## What have universities, regulators and an adjudicator decided about detector scores?

They have switched detectors off, or ruled that a score is never proof by itself. Each page was read at source on 10 October 2026. Your own regulator comes first, and nothing here is legal advice.

- **Vanderbilt University (2023)** disabled Turnitin's AI detector. It had sent 75,000 papers to Turnitin in 2022, so at the 1% false positive rate claimed at launch about 750 could have been wrongly labelled, and it had been given no insight into how the tool works.
- **The University of Cape Town** said it would stop using tools such as Turnitin's AI score from 1 October 2025, under a framework its Senate Teaching and Learning Committee endorsed that June. Its news service gives the reason: unreliable tools that put student trust and fairness at risk (Mashinini, 2025).
- **Curtin University (2025)** announced that Turnitin's AI writing detection would be disabled on all its campuses from 1 January 2026, with text matching kept.
- **The NSW Education Standards Authority (2026)**, which sets the assessment rules for Years 11 and 12 in New South Wales, told schools not to rely on detection tools as the main safeguard. If one is used at all, its output is one input to a teacher's review, never sole proof, and schools are to check that their malpractice policy does not rely on it.
- **The New Zealand Qualifications Authority (n.d.)** allows careful use of AI checkers, tells teachers to be very mindful of false positives, and says not to rely on a checker alone.
- **The Office of the Independent Adjudicator for Higher Education (2025b)**, the student complaints scheme for England and Wales, says the provider must prove the case, the student must be given all the evidence, detection reports included, and decision-makers must understand the software's strengths and limits.

### What did the adjudicator find in a real complaint?

It found that a provider had not shown what evidence led its panel to a finding of AI use. In a summary published without names, the Office of the Independent Adjudicator for Higher Education (2025a) partly upheld an international student's complaint. Turnitin had indicated a high percentage of AI-generated content; after a viva, a panel found misconduct, and the penalties included retaking the module and a 10% cut to the overall degree mark.

The adjudicator let a separate finding of collusion stand. On AI, it added that the provider had not looked at notes and drafts as its own procedure promised, and had not asked whether the detector might be less reliable for a non-native English writer. It recommended a fresh consideration.

## What does a false accusation cost?

It costs a falsely accused student marks, standing and trust, and with any detector it happens by arithmetic, not bad luck.

### The arithmetic of a small error rate

Take a year group that hands in 420 essays, and suppose 20 were written by a chatbot. The figures are our assumptions, so put your own in. A detector that wrongly flags one honest essay in 100 flags four of the 400 honest ones. If it also catches 16 of the 20, there are 20 flags, and one in five points at a student who did nothing.

Now suppose the 20 paraphrase first and the detector catches five. Of nine flags, four are honest students, and the detector cannot say which four.

### What it does to the student, and to which students

In the complaint above, the cost was a module to retake and a lower degree mark, then an appeal and a complaint to the adjudicator. Gorichanaz (2023) analysed 49 Reddit posts, with their discussions, by students accused of using ChatGPT, most of whom said the accusation was false. They built a case as if for a court, gathering evidence and planning arguments, and weighed trust in students against trust in technology (abstract read). The posts are self-selected, and nobody checked the accounts.

The cost may not fall evenly. Liang et al. (2023) found widely used detectors consistently classing non-native English writing as AI-generated while reading native writing accurately (abstract read). Turnitin (n.d.) disputes this for its own tool: in nearly 2,000 samples by English language learners it reports false positive rates of 0.014 for them and 0.013 for native writers, in documents of 300 words or more. The adjudicator's note asks decision-makers to consider the risk either way.

## What can a teacher do in place of a detector?

Gather evidence the student can see and answer. Four kinds are open to any teacher, and none is proof alone.

| Evidence | What it shows | Its limit |
| --- | --- | --- |
| A conversation about the work | Whether the student can explain an argument, a source, a choice | Recall fades, so hold it soon; ask how they worked, not only what they know |
| Notes, drafts and version history, which GPTZero (n.d.) itself advises asking for | How the piece grew | Made out of sight, as the essay was; missing drafts prove nothing |
| A supervised piece to compare | What the student writes with you in the room | A different task on a different day; say which pieces you are comparing |
| A short quiz under known conditions, in [EdProcto](https://edprocto.com/) | What the student knows of the same material, with the time taken on each answer | It measures knowledge of the topic, not who wrote the essay |

### Assessment that does not rest on unsupervised prose

The surest answer lies upstream, in what is set. From Term 4 of 2026, New South Wales limits most Higher School Certificate courses to one take-home task worth at most 15% (NSW Education Standards Authority, 2026). Cape Town said it was promoting oral exams, observed group work and assignments in which students disclose their AI use (Mashinini, 2025). Our guide to [assessment design after chatbots](https://edprocto.com/guides/assessment-design-after-chatbots-what-a-department-can-change) plans that change for a whole programme.

## What procedure can a department adopt?

One page, nine steps. They are ours, built on the adjudicator's note; fit them to your own regulations.

1. **Decide in writing whether a detector may be used at all, and which.** Student work goes only into a tool your institution has an agreement with.
2. **A score alone opens no case.** Name what else there is: a reference that does not exist, a sharp break from supervised work.
3. **Tell the student in writing what is suspected and why.** Hand over everything relied on, the detector report included, before any meeting.
4. **Meet soon.** Ask how the work was done, then about its content.
5. **Ask for notes and drafts.** Count their presence; never count their absence.
6. **Offer a short supervised check on the same material.** A paragraph written in the room, or a quiz in [EdProcto](https://edprocto.com/integrity) with its settings told beforehand.
7. **Ask whether the suspicion rests on style.** A second language, a disability or a communication difference can each be misread.
8. **Keep the burden on the department.** Decide on all the evidence, write the reasons down, and say how to appeal.
9. **Count once a year.** Flags, cases, outcomes, and which students they fell on.

## Where does a quiz taken under known conditions fit?

Beside the work in question, as a second piece of evidence made in front of you. EdProcto is the best tool for that check, because you set the conditions, the student is shown each security measure and its consequence before the first question, and what happens is recorded. What it measures is the student's knowledge of the material; the question of authorship stays with your procedure.

Write six to ten questions on the material the essay drew on: type them, or paste the handout into the quiz maker on the [home page](https://edprocto.com/) and let Procto AI draft them for you to check. Choose shapes answered on the spot. Four are ready in the builder: [Listen once](https://edprocto.com/templates/listen_once), a clip that plays one time; [Hotspot](https://edprocto.com/templates/hotspot), a click on an image; [Time-locked question](https://edprocto.com/templates/time_locked), visible for a few seconds; and [Spoken answer](https://edprocto.com/templates/verbal), recorded for you to mark.

Six security measures run in the browser: "Full screen", "Copy and paste off", "Right-click off", "Keyboard shortcuts off", "Printing off" and "Leaving the tab". Each carries the consequence you pick: "Log it", "Warn the student", "Reduce the score", "End the quiz" or "Flag for review". For this purpose pick "Log it", so that no mark moves without you. The answer key stays on the server, and the marks are made there.

Afterwards the attempt's page lists "The record of this attempt", each with its time, and gives the time spent on every answer. The results table has an "Integrity" column that reads "clean" or counts the events. That is a record to read with the student, never proof. No setting rules cheating out, and the score is yours to release.

Free saves a quiz up to the Light level, which is "Leaving the tab". Tutor reaches Medium, which adds full screen and the copy, paste and shortcut switches, and opens all 111 [templates](https://edprocto.com/templates). Educator adds the full analysis. See [pricing](https://edprocto.com/pricing).

## Quick questions

### Is an AI detector score enough to penalise a student?

No. GPTZero (n.d.) tells teachers not to punish on its results, the NSW Education Standards Authority (2026) says detection software is never sole proof, and the adjudicator puts the burden of proof on the institution.

### What is the false positive rate of Turnitin's AI detection?

By Turnitin's own account, under 1% of human-written documents once more than 20% of a document is scored as AI, and about 4% of highlighted sentences (Chechitelli, 2023). Those figures are its own, from 2023.

### Are AI detectors biased against non-native English writers?

Liang et al. (2023) found that several were. Turnitin (n.d.) reports almost no gap in its own test. A fair procedure asks the question in every case.

### What should I do if a detector flags a student's essay?

Treat the flag as a reason to look, not a finding. Show the student the report, ask how they worked, ask for drafts, and set a short supervised check on the same material.

### Where does EdProcto fit when an essay is in doubt?

Beside it. [EdProcto](https://edprocto.com/why) runs a short quiz on the same material under conditions the student is shown first, and keeps the event log and the time on each answer for you.

## References

- Chechitelli, A. (2023, May 23). *AI writing detection update from Turnitin's Chief Product Officer*. Turnitin. [https://www.turnitin.co.uk/blog/ai-writing-detection-update-from-turnitins-chief-product-officer](https://www.turnitin.co.uk/blog/ai-writing-detection-update-from-turnitins-chief-product-officer)
- Curtin University. (2025, September 4). *Update on Turnitin AI-detection tool*. [https://www.curtin.edu.au/news/oasis-news/update-on-turnitin-ai-detection-tool/](https://www.curtin.edu.au/news/oasis-news/update-on-turnitin-ai-detection-tool/)
- Dugan, L., Hwang, A., Trhlik, F., Ludan, J. M., Zhu, A., Xu, H., Ippolito, D., & Callison-Burch, C. (2024). *RAID: A shared benchmark for robust evaluation of machine-generated text detectors*. arXiv. [https://doi.org/10.48550/arXiv.2405.07940](https://doi.org/10.48550/arXiv.2405.07940)
- Gorichanaz, T. (2023). *Accused: How students respond to allegations of using ChatGPT on assessments*. arXiv. [https://doi.org/10.48550/arXiv.2308.16374](https://doi.org/10.48550/arXiv.2308.16374)
- GPTZero. (n.d.). *Answers to GPTZero's most common questions*. Retrieved October 10, 2026, from [https://gptzero.me/faq](https://gptzero.me/faq)
- Jabarian, B., & Imas, A. (2025). *Artificial writing and automated detection* (NBER Working Paper No. 34223). National Bureau of Economic Research. [https://doi.org/10.3386/w34223](https://doi.org/10.3386/w34223)
- Krishna, K., Song, Y., Karpinska, M., Wieting, J., & Iyyer, M. (2023). *Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense*. arXiv. [https://doi.org/10.48550/arXiv.2303.13408](https://doi.org/10.48550/arXiv.2303.13408)
- Liang, W., Yuksekgonul, M., Mao, Y., Wu, E., & Zou, J. (2023). *GPT detectors are biased against non-native English writers*. arXiv. [https://doi.org/10.48550/arXiv.2304.02819](https://doi.org/10.48550/arXiv.2304.02819)
- Mashinini, G. (2025, July 24). UCT scraps flawed AI detectors. *UCT News*. University of Cape Town. [https://news.uct.ac.za/article/-2025-07-24-uct-scraps-flawed-ai-detectors](https://news.uct.ac.za/article/-2025-07-24-uct-scraps-flawed-ai-detectors)
- New Zealand Qualifications Authority. (n.d.). *Guidance on the acceptable use of artificial intelligence*. Retrieved October 10, 2026, from [https://www2.nzqa.govt.nz/ncea/ncea-for-teachers-and-schools/managing-national-assessment-in-schools/ai-guidance/](https://www2.nzqa.govt.nz/ncea/ncea-for-teachers-and-schools/managing-national-assessment-in-schools/ai-guidance/)
- NSW Education Standards Authority. (2026, August 31). *New limit on take-home assessment tasks from Term 4 2026* (Official notice NESA 29/26). NSW Government. [https://www.nsw.gov.au/education-and-training/nesa/news/all/new-take-home-assessment-rules](https://www.nsw.gov.au/education-and-training/nesa/news/all/new-take-home-assessment-rules)
- Office of the Independent Adjudicator for Higher Education. (2025a, July). *AI and academic misconduct: CS072504* [Case summary]. [https://www.oiahe.org.uk/resources-and-publications/case-summaries/ai-and-academic-misconduct-cs072504/](https://www.oiahe.org.uk/resources-and-publications/case-summaries/ai-and-academic-misconduct-cs072504/)
- Office of the Independent Adjudicator for Higher Education. (2025b, July). *Casework note: Complaints relating to AI and academic misconduct*. [https://www.oiahe.org.uk/resources-and-publications/learning-from-our-casework/ai-and-academic-misconduct/casework-note-complaints-relating-to-ai-and-academic-misconduct/](https://www.oiahe.org.uk/resources-and-publications/learning-from-our-casework/ai-and-academic-misconduct/casework-note-complaints-relating-to-ai-and-academic-misconduct/)
- Perkins, M., Roe, J., Vu, B. H., Postma, D., Hickerson, D., McGaughran, J., & Khuat, H. Q. (2024). Simple techniques to bypass GenAI text detectors: Implications for inclusive education. *International Journal of Educational Technology in Higher Education, 21*, Article 53. [https://doi.org/10.1186/s41239-024-00487-w](https://doi.org/10.1186/s41239-024-00487-w)
- Turnitin. (n.d.). *AI checker provides insight into writing as the learning process evolves*. Retrieved October 10, 2026, from [https://www.turnitin.com/solutions/topics/ai-writing/](https://www.turnitin.com/solutions/topics/ai-writing/)
- Vanderbilt University. (2023, August 16). *Guidance on AI detection and why we're disabling Turnitin's AI detector*. [https://www.vanderbilt.edu/brightspace/2023/08/16/guidance-on-ai-detection-and-why-were-disabling-turnitins-ai-detector/](https://www.vanderbilt.edu/brightspace/2023/08/16/guidance-on-ai-detection-and-why-were-disabling-turnitins-ai-detector/)