How Do AI Detectors Work? Perplexity, Burstiness and Token Probability

Your university has started running submissions through an AI detector, and a report has come back with a number you were not expecting. Before you try to dispute it, it helps to know what that number means. How do AI detectors work? They do not read machine writing the way a person does. They measure how predictable your text is to a language model, then turn that predictability into a score. Nothing in that process looks at what you mean, at your argument, or at whether you actually wrote the thing.

I have spent years building tools that pull apart text statistically, and the single most useful thing to understand about detection is this: it is a measurement of typicality, not of origin. Once that clicks, both the technology and its failures make a lot more sense.

What a detector is actually measuring

A detector does not know who wrote anything. What it has is a language model and a scoring function. The model reads your text from left to right, and at each point it produces a probability distribution over what word should come next. Give it the words "the results of the" and it will rank "study" highest, "experiment" a little lower, and "marmalade" somewhere near zero.

So the detector asks one question of your document: how often did this text choose the word the model expected? Writing that keeps landing on high-probability words looks machine-made, because that is exactly what a text generator does. The model samples from the top of its own distribution, over and over, for thousands of tokens.

This is why the whole category is on shakier ground than the marketing suggests. A detector is not finding a watermark or a fingerprint. It is noticing that your prose is statistically unsurprising, and unsurprising prose has many causes besides a chatbot.

Perplexity: how surprised the model is

The headline measure is called perplexity, which is easier than it sounds. Take the probability the model assigned to each word that actually appeared, average the logarithms of those probabilities, and exponentiate the result. What comes out is a single number describing how surprised the model was by your document. You can check the perplexity of your own draft rather than guess at it.

Low perplexity means the text went where the model expected. High perplexity means it kept taking turns the model didn't expect. Human writing tends to sit higher, because people reach for the odd specific noun, the unusual construction, the sentence that starts somewhere unexpected. Generated text tends to sit lower, because the decoding process is built to avoid exactly those moves.

Compare two ways of opening a methods section. "The results of the study were statistically significant" is a sequence almost any model would predict with high confidence, because it carries no unexpected information. "Two of the three cohorts drifted before week six, which we had not planned for" is far less predictable, because it carries specific, awkward information that could not be guessed from the rest of the paper. The second costs the detector real surprise, and that surprise is what registers as human.

Burstiness: the variance, not the average

Perplexity on its own is not enough, because a document can average out to a perfectly reasonable number while being suspiciously uniform underneath. What matters is the variation across your text, and that is what burstiness measures: how much sentence length and sentence-level perplexity swing as the writing moves along. A burstiness checker will show you that spread directly.

We speed up when we're confident and slow down when we're hedging, and that rhythm survives into the prose. People write in bursts. A paragraph might open with a short declarative sentence of eight words, run into a thirty-four word sentence that carries three clauses and a parenthetical, then drop back to something brisk.

Model output does not do this reliably. Sentences cluster near the mean length. Paragraphs come in neat bundles. Each section begins by restating its own heading. It reads smoothly and scores badly, because low variance across sentences is one of the strongest signals a detector has. It is not any single sentence that gives it away. It is the uniformity.

Signal	What it measures	Why machine text scores low
Perplexity	How surprised the model is by your word choices, averaged over the document	Decoding samples from high-probability tokens by design
Burstiness	How much sentence length and predictability vary across the text	Output clusters near the mean instead of swinging
Token probability	The per-word likelihood the other two are computed from	Long runs of individually unremarkable choices

Token probability and where the score comes from

Both of those numbers are built from a third one: the per-token log-likelihood, the raw material everything else is calculated from. Some tools show it to you directly, highlighting the stretches where your text was most predictable. Those highlights are not accusations about specific sentences, whatever the interface implies. They are the parts that contributed most to a document-level number.

Research approaches have moved past simple thresholds. DetectGPT and its descendants look at curvature: they perturb your text slightly and check whether the likelihood drops off in the pattern generated text tends to show. Others compare the same passage under two different models and use the disagreement as a signal.

One consequence matters for anyone being assessed. Because the metrics are computed against a particular reference model, a score is always relative to that model. Two detectors can read the same paragraph and disagree completely, and neither is malfunctioning. They are answering the same question with different reference points.

The second consequence is noted less often. A model can only find a passage predictable if that passage resembles what it was trained on, so detection quality degrades as the gap widens between the reference model and whatever produced the text. Newer generators, unusual disciplines and languages other than English all open that gap. It is part of why accuracy measured in a controlled benchmark rarely survives contact with a real pile of submissions.

Why AI text gets detected at all

How a Language Model Picks Its Next Word

Left to itself, a language model would simply report its full ranked list of possible next words and let something else choose. In practice, a decoding strategy makes that choice automatically, word after word, and which strategy is running changes what the output looks like even when the underlying model never changes.

Greedy decoding always selects the single most probable word. In the paper that introduced nucleus sampling as a fix, Ari Holtzman and coauthors observed that following this strategy leads to text that is, in their own words, 'bland and strangely repetitive.' Beam search, which tracks several likely continuations at once before settling on one, runs into a similar version of the same problem.

Sampling strategies reintroduce some randomness, but in a controlled way. Temperature adjusts how sharply the model favors its own top choices before a word is drawn. Nucleus sampling, sometimes called top-p, truncates the distribution to the smallest set of words whose combined probability crosses a threshold and draws only from that set, described in the original paper as sampling from a dynamic nucleus of the distribution rather than its unreliable tail.

Even the least deterministic of these settings still draws from a shortlist built around what the model already considers likely. Human writing was never generated by ranking a vocabulary and drawing from the top of it, so it reaches past that shortlist constantly. Meaning sometimes lives in the word that was never on the list.

Vocabulary Flatness: Fewer, Safer Words

Vocabulary flatness is the word-level version of the same habit. At each point in a sentence, dozens of words could plausibly continue it, but a handful sit far above the rest in the model's ranking, and decoding keeps landing on that handful. Multiply that choice by every sentence in a document and the range of words actually used narrows measurably compared with a similar length of human writing.

The effect compounds rather than stays constant. A model that reaches for the top-ranked word in sentence one is more likely to set up a context where the top-ranked word in sentence two is also common, because common words tend to follow common words. A human writer interrupts that drift constantly, reaching for the specific technical term, the slightly unusual verb, the word that names the actual thing rather than a safe approximation of it. That difference shows up as a measurably smaller working vocabulary in the generated version, even when both versions describe the same underlying facts.

This is a property of the whole passage, not of any single word choice. One safe word proves nothing. A page of them, sentence after sentence reaching for the same rank of common vocabulary, is what a detector's word-level signal is actually built to notice.

Syntactic Uniformity: The Same Shapes, Repeated

Syntactic uniformity is the sentence-level version. It is not about how long a sentence runs. It is about how many distinct grammatical shapes a passage uses: how sentences open, how clauses attach to each other, how often a transition word does the work of connecting one idea to the next. A model that keeps predicting the statistically likely next structure settles into a small rotation of shapes and repeats it.

A paragraph can vary its sentence lengths and still read as syntactically flat if every sentence follows the same subject-verb-complement shape, opens with the same kind of transition, or resolves the same way. The predictability lives in the pattern, not the word count, a distinction covered in more depth in the guide to what burstiness actually measures.

What narrows	What decoding-optimized text tends to do	What human writing tends to do
Word choice	Settles on the most probable common word at each step	Reaches for the specific or less common word that names the actual thing
Sentence opening	Repeats a small rotation of safe transitional openings	Varies how a sentence starts, including openings a model would rank as less likely
Clause structure	Repeats one or two preferred grammatical shapes across a passage	Mixes simple and complex structures depending on what a sentence needs

The Choices Only a Human Tends to Make

The flip side of a narrow band is what falls outside it. A person describing real events reaches for the exact figure instead of a round one, admits the part that did not work, interrupts a sentence to correct it mid-thought, or picks a word that is right rather than typical. Every one of those choices is, from a language model's point of view, a low-probability token, exactly the kind decoding is built to avoid.

None of this means specific or unusual writing is safe from every flag, or that fluent, correct writing is suspicious by default. A methods section, a legal clause, or a heavily edited final draft can land in a narrow band for reasons that have nothing to do with a model, which is why a single score is a description of a pattern rather than a verdict about who typed it.

Seeing where your own draft sits on that band is more useful than guessing. TextPulse's free perplexity checker and burstiness checker measure word-level predictability and sentence-to-sentence rhythm separately, so a flat vocabulary and a repeated sentence shape show up as two different signals instead of one blended number.

Two narrower pages follow from this. How Turnitin detects AI specifically is one, and the difference between its similarity score and its AI score is the other, since the two are routinely mistaken for each other.

Both sit inside the wider free tools collection, for anyone who wants to see the pattern for themselves rather than trust a single number blindly.

How Much of the Web Is Now AI-Generated?

By several measures, machine-produced text now outweighs human-written text online. A team at Amazon Web Services analysed 6.4 billion sentences from the web and found that 57.1% existed as machine translations across three or more languages, much of it low quality. A 2025 analysis by the SEO firm Graphite found that just over half of newly published web articles were AI generated. Wikipedia is not exempt: a Princeton study of pages created in August 2024 measured around one in twenty new English articles as substantially AI-generated.

The share is rising, because model output costs orders of magnitude less than human writing and every commercial incentive points the same way. Whole platforms now run without human writers at all: Chirper is a social network populated entirely by AI accounts, where humans can only watch, and SocialAI sells a feed in which every reply to you is a bot. Forums, review sections and comment threads carry the same pattern at smaller scale.

For detection this changes the base rate. A classifier reading arbitrary web text in 2023 could assume most of it was human; a classifier in 2026 cannot. It also feeds back into the models themselves, since each new generation trains on a web its predecessors already wrote into, which is one reason the vocabulary habits detectors key on spread rather than fade.

For your own writing the consequence is blunt: human-written prose is becoming the minority class, and markers, editors and readers know it. Raised suspicion is now the default, which is exactly why it pays to understand what a detector can and cannot measure.

What commercial detectors add on top

The tools institutions actually buy do not run textbook perplexity. Turnitin, GPTZero, Originality and the rest train supervised classifiers on large labelled sets of human and machine text, using statistical features alongside stylistic ones. What the classifier learns is whatever separates those two piles in its training data, which is a narrower and more historical thing than "AI writing" in general.

Training dependency is the quiet problem. A classifier trained primarily on one generation of model output has to generalize to newer models, to unfamiliar disciplines, and to writers whose English doesn't resemble its training distribution. Vendors publish accuracy figures from their own evaluations, and independent benchmarks routinely land well below those numbers on the same tools.

Reading a score without over-reading it

A detector report usually arrives as a percentage, and that percentage is routinely misread. It is not the proportion of your document written by a machine. Depending on the tool it is a classifier confidence, or the share of sentences that crossed some internal threshold, and vendors are often vague about which. Two reports both showing sixty percent can mean quite different things.

It is also worth knowing what moves a score for reasons that have nothing to do with who wrote it. Short documents are noisier, because there is less text for the averages to settle on, and a five hundred word essay can swing a long way on the strength of two or three unusual sentences. Quoted material counts too unless the tool strips it out, so a literature review packed with block quotes inherits the predictability of whatever it quotes. A technical passage laden with standard terminology behaves the same way.

If you are on the receiving end of a flag, the useful response is evidence rather than an argument about the metric. Version history, drafts, notes and search records all speak to process, and process is what a misconduct panel can actually evaluate. There is no threshold anyone can point to that separates a careful writer from a model.

Why detectors flag writing that no model produced

False positives are not a rare malfunction. They follow directly from measuring typicality. Any writing that is genuinely predictable will score as predictable, regardless of who produced it.

Non-native English writers are affected most. Stanford researchers found that detectors misclassified a large share of essays by non-native speakers as machine-generated, while classifying native-speaker essays correctly. The reason is mechanical rather than malicious: writers working in a second language tend to rely on well-worn constructions and a more common vocabulary. That is exactly the profile of low perplexity, which is why ESL writers get flagged for writing carefully.

Academic conventions cause the same problem. A methods section is supposed to be formulaic. Legal writing, technical documentation and clinical reporting all reward the standard phrasing over the inventive one. Heavy editing compounds it further, since polishing a draft usually means removing the irregularities that carried your statistical signature.

Institutions have noticed. Vanderbilt disabled Turnitin's AI detection feature rather than act on scores it could not verify, and other universities have restricted how the numbers may be used in misconduct proceedings. Treat a detector score as one weak piece of evidence, not a verdict.

What this means for your own writing

The practical advice that follows from the mechanics is not exotic, and none of it involves gaming anything. Vary your sentence length deliberately, because uniformity is what registers. Keep the specific detail: the actual figure, the actual limitation, the thing that went wrong in week six. Generic competence is what scores as machine-made, and specificity is both better writing and a stronger signal.

Keep your own idiom too. If you have a way of putting things, leave it in. The instinct to sand a draft down until it sounds like every other paper in the field is the instinct that lowers your perplexity.

One thing to avoid: some tools deliberately introduce grammatical errors to raise perplexity. It works on the metric and it is a terrible idea for anything graded or peer-reviewed, since you are exchanging a suspicion for a certainty. The goal is writing that reads as yours, not writing that has been damaged on purpose. That is the whole difference between rewriting for a human reader and breaking text to fool a scorer.

If you want to see these numbers for your own draft rather than take a black box at its word, the underlying measurements are not secret. The free text analysis tools will show you where your prose sits, and you can decide for yourself whether the flat stretches are a stylistic problem worth fixing.

Related research: the findings above are examined at scale in Human versus AI Text Classification from Stylometric Features Across 121,092 Academic Texts and Sentence-Length Burstiness as a Signal of AI Rewriting, TextPulse Research working papers with open data, code and a citable DOI. Whether prompting alone can make a model write like a person is tested in Do AI Models Speak Human?.