Do AI detectors work? What the measured false positive rates show

Do AI detectors work? They return a probability, not proof. In the independent tests published so far, false positive rates on genuinely human writing run from a fraction of a percent to more than 68%, depending on which tool ran and who did the writing. This is what the research and the vendors' own pages say, and what to rely on instead of a score.

That spread matters because detector scores are already used to make decisions: an agency drops a freelancer, an editor rejects a draft, a marketplace holds a payout. A number that moves that much between datasets cannot carry that weight on its own - and the vendors themselves say so in their documentation.

Do AI detectors work the way people assume? No - they score style, not origin

An AI text detector does not look up where a piece of text came from. It is a classifier trained on examples of human and machine writing that estimates how likely a passage is to have come from a language model, based on statistical properties of the words in front of it.

GPTZero describes its own pipeline on its technology page as a deep learning model plus a sentence-by-sentence classifier and a "Paraphraser Shield" against evasion, returning human, mixed or AI-only with a confidence band (read October 23, 2026). It states plainly on the same page that "no AI detector is 100% accurate" and that the result should be used "as a conversation starter, and not as the final verdict."

In our view that framing is not marketing modesty but an accurate description of what a classifier can do. Three consequences follow, and all three show up in the measured results below.

• Length. Short passages carry too little signal. Originality.ai sets a 100-word minimum on its AI checker page and warns that accuracy can drop on short passages (read October 23, 2026).
• Editing. The same Originality.ai page notes that text rewritten or edited with AI tools, including some Grammarly features, may be flagged incorrectly.
• Style. Anything that makes human writing more predictable - a formal register, a non-native vocabulary, a house style guide - moves it toward the machine side of the boundary.

What the vendors publish about their own detectors

Read the vendors' own pages rather than the leaderboards that cite them. The figures below come from the companies themselves, so they are the companies' figures, measured on datasets the companies chose.

The OpenAI row is the one worth sitting with. The company that built the model whose output everyone wants to detect shipped a classifier, published a 26% true positive rate against a 9% false positive rate, and then retired it with a note on the original post: "As of July 20, 2023, the AI classifier is no longer available due to its low rate of accuracy." The same note says OpenAI is "currently researching more effective provenance techniques for text" - provenance, not detection.

The independent numbers are far wider than the vendors' numbers

Two studies carry most of the weight here, and they point the same way.

A University of Florida team - Seth Layton, Bernardo B.P. Madeiros, Kevin Butler and Patrick Traynor - tested five popular commercial detectors against roughly 6,000 papers submitted to top security conferences before ChatGPT existed, plus LLM-written versions of the same papers. Per UF News on May 19, 2026, false positive rates across the five ranged from 0.05% to 68.6% and false negative rates from 0.3% to 99.6%. Two of the five performed well at first. Then the researchers asked the model to rewrite its output using more complex vocabulary - a "lexical complexity attack" - and reliability dropped sharply. Traynor's summary: "We really can't use them to adjudicate these decisions. People's careers are on the line here." The paper, "AI Wrote My Paper and All I Got Was This False Negative," was presented at the 2026 IEEE Symposium on Security and Privacy. UF's release does not name the five detectors.

The second study is older and still the clearest on bias. Liang, Yuksekgonul, Mao, Wu and Zou ran 91 TOEFL essays by non-native English writers and 88 essays by US eighth-graders through seven detectors - Originality.AI, Quil.org, Sapling, OpenAI, Crossplag, GPTZero and ZeroGPT, in their 2023 versions. The findings, in GPT detectors are biased against non-native English writers:

• The detectors "misclassified over half of the TOEFL essays as 'AI-generated' (average false positive rate: 61.22%)."
• "All seven detectors unanimously identified 18 of the 91 TOEFL essays (19.78%) as AI-authored."
• On the US eighth-grade essays the average false positive rate was 5.19% - but when the authors had ChatGPT simplify the word choices in those human essays, it rose to 56.65%.
• Asking ChatGPT to enhance the word choices in the TOEFL essays cut the false positive rate from 61.22% to 11.77%.
• Prompting ChatGPT to use more literary language in machine-written college admission essays "significantly reduced detection rates from 100% to 13%."

Read those last three together and the mechanism is unmistakable. Change nothing about who wrote the text and only change how fancy the vocabulary is, and the verdict flips in both directions. Vendors have had three years to improve since that paper, and GPTZero now publishes a 1.1% TOEFL false positive rate on its own technology page. The 2026 UF results suggest the underlying fragility - a paraphrase changes the answer - has not gone away.

Why do AI detectors flag human writing?

Because the thing they measure is style, and some human styles are statistically closer to model output than others. Writers whose English is a second language, writers trained in formal or academic registers, and writers working to a tight brand style guide all produce text that is more predictable than average. That is the signal.

This is also why "I ran my own article through five tools and got five answers" is the normal result rather than a sign that four of them are broken. Each tool drew its boundary on a different training set.

Skip the detector entirely if you are checking your own team's work for quality, nobody outside is auditing you, and the only consequence of a flag is an argument. You will spend real time re-writing sentences to satisfy a classifier that has no information about who typed them.

Who actually has to act on this

Most people publishing with AI assistance are not affected by any of this. The readers who are:

1. Freelancers and agencies with a no-AI clause. If a contract conditions payment on human authorship, a client's detector score is the likely trigger for a dispute, and you will need something better than a counter-score.
2. Content teams that gate publishing on a detector threshold. A threshold is a policy decision. In our view, if you keep one, write down what happens on a flag - a conversation, not an automatic rejection - and record it where writers can read it.
3. Marketplaces, platforms and publishers screening submissions at volume. The UF false negative range, up to 99.6%, means a detector may be doing almost nothing on some inputs while generating disputes on others.
4. Anyone making a decision about a person. Both studies above land in the same place: the score is not evidence of who wrote something.

Note on contracts: this is not legal advice, and a no-AI clause means whatever your specific agreement says. If money or a client relationship rests on it, read the current contract and take advice. Documents and vendor terms also change - everything above was read on October 23, 2026.

Record how the work was written instead

If the real question is "can you prove a person wrote this," we'd reach for provenance rather than detection. Provenance captures the writing as it happens rather than guessing from the finished draft.

Grammarly ships this as Authorship. On its Authorship page (read October 23, 2026), Grammarly says the feature "records your actual writing process-typed, pasted, AI-assisted-so you'll always have receipts," contrasting it with detectors that "only guess from the final draft." It color-codes each sentence by source, includes a replay "from the first ideas to the last keystroke," produces a shareable report that Grammarly says it does not auto-send to anyone, and is opt-in. The page lists Google Docs via the browser extension, Microsoft Word via the desktop app, the Grammarly Editor, Superhuman Go, and Canvas LMS submissions, and says Authorship is available in Grammarly Free and Grammarly Pro.

You can get most of the way there without a product, and we'd do both:

• Keep version history on. Google Docs and Microsoft Word both retain revision history. In our view a document that grew over four sessions is harder to dispute than a finished file.
• Keep the trail before the draft. The brief, the outline, the interview notes, the research links.
• Write an AI-use policy and disclose against it. Say which steps may use a model - research, outlining, line edits - and which may not. A written policy turns a flag into a check against a rule rather than an accusation.
• Agree in advance what a flag means. With a client, settle this before the invoice, not after.

The same argument runs through images: a score guessing at origin is weaker than a record attached at creation, which is why C2PA Content Credentials exist for visual media. Text has no equivalent standard in wide use, which is why vendor-specific records like Authorship and plain version history are what is available today.

Where these tools sit in the directory

As of October 2026, Upfynd lists 121 products in AI Writing, and the detector side of that category is crowded: GPTZero, Originality.ai, ZeroGPT and Winston AI all sit in it. So do tools built for the other side of the arms race, such as Undetectable.ai, which markets itself on rewriting AI text so it reads as human and checking it against detectors.

We read a market with healthy businesses on both sides as a sign that the detection question is not settled. The 2023 paper's literary-language prompt and the 2026 paper's lexical complexity attack are, in effect, the product the rewriting tools sell.

If you are choosing a detector anyway, pick it for workflow fit - where it runs, what it costs, whether it handles your document length - and check each vendor's own published false positive rate and caveats on its own site before you set a threshold. Then set the threshold low enough that a flag starts a conversation, because on the published evidence that is all a flag can support.