How AI Detectors Like Turnitin Actually Work

What Are AI Detectors?

AI detectors are software tools designed to distinguish between human-written and AI-generated text. Universities, publishers, and content platforms have adopted these tools to maintain academic integrity. But how do they actually work under the hood?

The two most widely discussed metrics in AI detection are perplexity and burstiness. Understanding these concepts is essential for anyone working with AI-assisted writing in academic settings.

Perplexity: Measuring Predictability

Perplexity is a statistical measure of how predictable a piece of text is to a language model. When a model reads a sentence, it assigns probabilities to each word based on the preceding context. If the next word is highly predictable, perplexity is low. If the word is surprising, perplexity is high.

AI-generated text tends to have consistently low perplexity because language models choose the most statistically likely tokens. Human writing, by contrast, is messier — we use unexpected word choices, idioms, and sentence structures that a model would not predict with high confidence.

How Turnitin Measures Perplexity

Turnitin's AI detection module processes text in overlapping segments. Each segment receives a perplexity score, and the tool aggregates these scores to produce an overall probability estimate. Segments with uniformly low perplexity raise the AI probability flag.

Burstiness: The Rhythm of Human Writing

Burstiness refers to the variation in sentence length and complexity throughout a piece of text. Human writers naturally produce "bursty" text — a long, complex sentence followed by a short, punchy one. We pause, digress, and return to our point.

AI-generated content, particularly from models like GPT-4, tends to produce sentences of similar length and structure. The rhythm is flat. Detectors analyze this pattern and flag text with low burstiness as potentially machine-generated.

Beyond Perplexity and Burstiness

Modern AI detectors use additional signals beyond these two metrics:

• Token probability distributions: Examining not just the top prediction but the full distribution of likely next tokens
• Stylometric features: Analyzing vocabulary richness, function word usage, and punctuation patterns
• Watermark detection: Some models embed statistical watermarks that detectors can identify
• Contrastive analysis: Comparing the text against known outputs from specific model families

Why False Positives Happen

No detector is perfect. False positives — flagging human-written text as AI-generated — occur frequently. Non-native English speakers often write with lower perplexity because they rely on common, well-learned phrases. Technical writing in fields like mathematics or computer science is inherently formulaic, mimicking the low-burstiness pattern of AI output.

Turnitin themselves acknowledge a 1% false positive rate at the document level, but researchers have found rates significantly higher for certain demographics and writing styles.

What This Means for Researchers

If you use AI tools to assist your writing process — whether for brainstorming, outlining, or polishing drafts — understanding how detectors evaluate text helps you write in a way that preserves your authentic voice. The goal is not to evade detection but to ensure your genuine contributions are recognized as human.

Key Takeaways

AI detectors rely on statistical patterns, not semantic understanding. They measure how text "looks" to a language model, not whether a human actually wrote it. As detection methods evolve, so does the importance of understanding the underlying mechanics — both for researchers and institutions.