By H. Omer Aktas
Editor, AIUpdateWatch.com
Published July 27, 2026 · Approximately 18 minutes
A number with consequences
The accusation arrived before the conversation
A student submits an essay. The next morning, a message says that an automated checker found the paper “94% AI-generated.” The student says the work is original. The instructor sees a precise number and assumes the question has already been settled.
It has not.
The detector did not watch the student write. It did not inspect the student’s notes, browser history, drafts or conversations with a tutor. It received a finished block of text and compared its statistical patterns with patterns associated with human and machine writing.
That can be useful. A smoke alarm is useful too. But a smoke alarm does not tell you whether the problem is a house fire, burnt toast or a weak battery. It tells you that something deserves attention.
“This passage resembles text that our detector associates with AI generation.”
“This person used a prohibited tool to produce this specific work.”
The first statement is an automated estimate. The second is a conclusion about conduct. Moving from one to the other requires evidence, context and a fair review.
What an AI text detector actually does
An AI detector is a classification system. It receives text and returns a label or probability based on features that were useful during training.
Different products use different methods, and commercial systems do not publish every detail. Common approaches include:
Pattern comparison
Learned classifiers
A model is trained on examples labeled as human-written or machine-generated, then learns which combinations of words, sentence forms and distributions tend to separate the groups.
Predictability
Language-model likelihood
The detector checks how predictable the wording appears to another language model. Highly regular text may receive a stronger machine-written score.
Variation
Rhythm and distribution
Some systems examine sentence-length changes, vocabulary variation, repetition and other statistical properties sometimes described as burstiness or perplexity.
Known signals
Watermarks or provenance
A system may look for a deliberately embedded signal, but only when the generator used that method and the signal survived editing, copying and formatting.
None of these methods reads intention. None can look at a paragraph and directly observe the moment it was composed. The detector recognizes patterns and estimates which source is more likely under its model.
That estimate depends on the detector’s training data. A tool trained on long English essays from a limited group of models may perform differently on short emails, technical reports, translated writing, legal text, poetry or heavily edited drafts.
There is no universal AI fingerprint in ordinary text
A photograph can carry camera metadata. A digitally signed document can be checked against a known certificate. Ordinary generated text usually does not arrive with an indestructible label saying which model produced it.
What the detector does not have
- a universal registry of every sentence generated by every model;
- a permanent serial number hidden in all AI-written prose;
- access to every prompt, draft and revision behind the final document;
- certainty that a model, paraphraser or translator has not changed the text;
- certainty that a human writer will not naturally produce similar patterns.
Text watermarking and provenance research may help in controlled settings. A generator can be designed to favor certain token patterns, or a platform can retain records showing that content was produced inside its service. But a watermark is not universal, and normal editing may weaken or remove it.
This is why NIST treats human-versus-machine text discrimination as an evaluation problem rather than a solved identity test. Its GenAI programs measure how well generators imitate human writing and how well discriminators separate the two under defined conditions.
What does “94% AI” actually mean?
The answer depends on the product.
A displayed percentage may refer to:
- the detector’s confidence that a passage belongs to the machine-written class;
- the estimated proportion of qualifying text that appears machine-generated;
- an average of sentence-level or segment-level scores;
- or a product-specific index that should not be read as a literal probability.
That conclusion would require information about the detector’s calibration, the type of text, the likely rate of AI use in the relevant group and the other evidence in the case.
Imagine a medical test used in a population where the condition is rare. Even a test with good laboratory performance can produce a meaningful number of false alarms. The same base-rate problem applies to detection. A score cannot be interpreted fairly without asking how the tool performs on similar writing and how often the suspected behavior actually occurs.
Turnitin’s own current guidance says its AI writing report may misidentify both human and AI-generated writing and should not be the sole basis for adverse action. It also withholds exact low-range scores because false positives are more common there.
How completely human writing gets flagged
Detectors are often looking for regularity. Human writing can be regular for perfectly ordinary reasons.
The writer uses clear, conventional sentences
School essays, business reports and formal explanations often follow predictable structures because those structures are taught and expected.
The subject allows limited wording
A technical definition, lab method or policy summary may naturally repeat standard terms and avoid stylistic variation.
The writer edits heavily
Removing digressions, shortening sentences and correcting grammar can make authentic prose more uniform.
The writer works in a second language
A smaller active vocabulary and safer sentence patterns can resemble features that some detectors associate with generated text.
The document uses a template
Reports, applications and standardized assignments may share headings, transitions and repeated institutional wording.
The sample is outside the detector’s training world
A detector may behave unpredictably on genres, languages, age groups or professional fields it did not represent well during training.
A false positive is not a harmless inconvenience when the score is used in discipline, hiring or publication. The accused person may have to prove a negative: that they did not use a tool in a way the policy forbids.
How AI-written text gets missed
The reverse problem is just as important. A detector can fail to recognize machine-generated writing.
Generated text becomes harder to identify when a person:
- rewrites sentences and changes their order;
- adds personal examples or local details;
- combines output from several models;
- translates the text into another language and back;
- uses a paraphrasing service;
- mixes generated passages with genuine drafts;
- or prompts the model to imitate a less regular style.
Research evaluations repeatedly find that performance drops when detectors encounter unfamiliar models, unfamiliar topics or edited output. A detector that performs well on its benchmark can perform much worse in real work.
This creates an uncomfortable imbalance. Careful misuse may escape, while an honest writer with an orderly style may be questioned.
The language-bias problem is not theoretical
A widely cited Stanford-led study tested seven detectors on essays written by non-native English speakers and essays written by U.S. eighth-grade students. The detectors incorrectly labeled more than half of the non-native English essays as AI-generated on average.
The researchers connected this result to lower linguistic variation and predictability. Writers using a second language may choose familiar words, straightforward grammar and repeated structures. Those are not signs of dishonesty. They are normal features of language learning.
Possible detector interpretation
“The vocabulary is predictable and the sentence structure is unusually consistent.”
Possible human reality
“The writer is choosing clear English forms they know well and avoiding grammar they are not yet comfortable using.”
The same concern can extend to writers with disabilities, writers using assistive technology, people following strict professional formats and anyone whose style differs from the data used to train the detector.
A fair process therefore cannot treat detector output as neutral simply because it came from software.
Short passages are especially difficult
A detector needs enough language to observe stable patterns. A two-sentence email, a discussion-board reply or a short paragraph may not contain enough evidence.
Consider this sentence:
“The project was delayed because the supplier did not deliver the replacement part on time.”
It is plain, grammatical and predictable. A person could write it. A model could write it. There is no meaningful authorship clue in the sentence alone.
Longer samples offer more information, but length does not remove the problem. A detector still has to generalize across subjects, models, languages and editing styles.
Short text also makes percentage displays misleading. One or two flagged sentences can appear to dominate the result even when the tool has very little material to assess.
Evidence that tells you more than a detector score
When authorship matters, the writing process is usually more informative than the finished text alone.
Draft history
Version history can show gradual composition, restructuring, corrections and additions over time.
Notes and source trail
Research notes, bookmarked sources, outlines and quotations reveal how the argument developed.
Knowledge of the work
The writer should be able to explain the main claims, choices, calculations and sources in their own words.
Comparison with earlier work
Changes in style can be discussed, but they should be interpreted alongside topic, editing, assistance and learning progress.
Declared tool use
A clear record of permitted spelling, translation, brainstorming or drafting assistance reduces guesswork.
None of these items alone solves every case. Together, they create a fuller picture than a single probability generated after the work is finished.
A fair review process for schools and workplaces
Organizations need a process before the first disputed score arrives.
Before submission
Define acceptable use
State whether brainstorming, outlining, translation, grammar correction, rewriting, coding assistance or full drafting is permitted. “Do not use AI” is often too vague to enforce consistently.
When a score appears
Pause before accusing
Review the actual passage, assignment type, language background and known detector limitations. Do not send an allegation generated from a percentage alone.
During review
Ask process questions
Request drafts, notes and an explanation of the argument. Give the writer a meaningful opportunity to respond.
Before deciding
Use several forms of evidence
Document why the conclusion follows from the whole record, not simply from the detector’s label.
After the case
Allow review or appeal
High-consequence decisions should have an independent route for correcting mistakes.
This approach does not require ignoring AI misuse. It requires investigating it properly.
What to do when your own writing is flagged
Do not begin by trying to make the text look “less AI.” That can damage your work and make the dispute harder to explain.
- Ask what the score represents. Request the product name, report, highlighted passages and policy being applied.
- Preserve your evidence. Save drafts, notes, source files, version history, emails and timestamps.
- Explain your process calmly. Describe how you researched, planned, drafted and edited the work, including any permitted tools.
- Correct factual assumptions. Point out templates, required wording, technical language, translation help or other reasons for a regular style.
- Request human review. Ask that no conclusion be based solely on the detector result.
- Use the formal appeal process when necessary. Keep communication factual and retain copies.
A person should not have to perform a more chaotic writing style merely to avoid being mistaken for a machine.
The rule worth keeping
An AI detector can raise a question. It cannot answer the whole question.
The useful role of detection is limited but real. It can help identify passages for closer review, support research into synthetic content and provide one input in a broader investigation.
The dangerous role begins when a probability is treated as direct evidence of a person’s behavior.
Use detector output to decide where to look—not whom to punish.
When authorship matters, examine the writing process, the policy, the context and the person’s explanation. A fair conclusion should remain understandable even after the detector score is removed from the page.
Sources and further reading
- NIST GenAI Text 2026: generators, discriminators and human-like writing evaluation
- NIST GenAI Pilot: text generation and discrimination evaluation results
- OpenAI: retired AI text classifier and documented accuracy limitations
- Liang and colleagues: GPT detectors are biased against non-native English writers
- Tufts, Zhao and Li: practical examination of AI-generated text detectors
- Turnitin guidance: interpretation, limitations and human review