How Accurate Are AI Detectors? A Guide to the Numbers

AI-detector accuracy is a result on a particular test set, not a guarantee about your document. A useful evaluation tells you what texts were tested, how authorship was established, and how often the system made each kind of error.

Four terms that are easy to confuse

• False positive: human-written text is labelled as AI-generated.
• False negative: AI-generated text is labelled as human-written.
• Accuracy: the proportion of all tested examples classified correctly.
• Precision: among examples flagged as AI, the proportion that actually belongs to that class in the test set.

Ask whether a vendor is measuring sentences, documents, or some other unit. A sentence-level statistic cannot simply be relabelled as a document-level result. Also check whether edited or mixed-authorship text was included.

Worked example: why 98% accuracy is not certainty

The following is a hypothetical calculation, not a measurement of GPTZero, Turnitin, or any other product. Imagine 1,000 documents: 900 are human-written and 100 are AI-generated. Suppose a detector flags 1% of the human documents and detects 90% of the AI documents.

• 9 human documents are incorrectly flagged; 891 are correctly left unflagged.
• 90 AI documents are correctly flagged; 10 are missed.
• 99 documents are flagged in total, of which 90 are AI-generated.
• Overall accuracy is 981 out of 1,000, or 98.1%. Precision among flags is 90 out of 99, or about 90.9%.

Even in this favourable hypothetical case, nine people receive incorrect flags. If AI-generated documents are less common in the group being screened, false alarms can make up a larger share of the flags, despite unchanged error rates. That is why a headline accuracy figure does not answer every practical question.

Questions to ask before trusting a benchmark

• Was authorship known, or inferred using another detector?
• Were the evaluation texts separate from training data?
• Did the set include your language, genre, length, and level of editing?
• Were mixed human/AI documents assessed separately?
• Are false-positive and false-negative rates reported at the same decision threshold?
• How large was the sample, and is uncertainty reported?
• Which product version and scan date were used?

Build a small evaluation for your own workflow

For an initial internal check, collect documents whose writing process you actually know, with permission to use them. Keep human-written, AI-generated, and mixed drafts in separate groups. Fix the text and scan settings before recording results; do not rewrite examples until they get the label you expected.

Record the expected category, the output label, the model date, and any unsupported file or language warnings. Count mistakes separately for each group. A small convenience sample can uncover obvious mismatches, but it cannot justify a sweeping claim of accuracy across all writing.

What a score should trigger

Use a disputed result to start a review of the actual work: sources, drafts, revision history, and the writer’s explanation. Turnitin acknowledges misclassification and advises against using its indicator alone for adverse decisions. A clean scan also cannot prove that every fact is correct or every source properly credited.

Should you rewrite solely to reduce the score?

Repeated rewrites make it harder to assess the original result and may weaken the writing. Preserve the submitted version before making any changes. If the issue is unclear prose, edit for clarity. If the issue is an authorship dispute, address it with the relevant reviewer and evidence of your process.

GhostWriter publishes this guide and sells rewriting tools. We have not independently benchmarked detectors for this article, and we do not promise that a rewrite will obtain a particular score.