Why AI Detectors Are Biased Against Non-Native English Writers

AI detectors biased against non-native speakers is not a subjective complaint. A peer-reviewed 2023 Stanford study measured it directly. Researchers ran seven widely used GPT detectors against 91 real TOEFL essays written by non-native English speakers and 88 essays written by American eighth-graders, then compared how each group scored.

The detectors read the eighth-grade essays correctly almost every time. On the TOEFL essays, written by adults who had already passed a graduate-level English proficiency exam, the same seven tools returned an average false positive rate of 61.22 percent. Eighty-nine of the ninety-one essays, nearly 98 percent, were flagged as AI-generated by at least one of the seven tools.

TextPulse's overview of the wider evidence on AI detector accuracy covers that headline finding and the lawsuits it has since fed into. What gets skipped almost everywhere the study is cited is the mechanism: why does a graduate-level English speaker's prose read as machine-made in the first place, and what changes when it stops reading that way.

AI Detectors Biased Against Non-Native Speakers: The Mechanism

A detector never reads for meaning. It reads for how predictable each word is, given the words that came before it, and scores a passage low when the model could've guessed most of it in advance. Lexical diversity (how wide a vocabulary a writer draws on rather than repeating a smaller set of safe words) and syntactic predictability (how closely sentence structure follows the most common pattern for the language rather than varying it) are two properties of language production that drive that score down.

The distinction matters because the two show up differently on the page. Lexical diversity is about the words themselves: a writer who reaches for 'obtain,' 'acquire,' and 'secure' in three different sentences reads as more varied than one who writes 'get' three times, even where the sentences are otherwise identical. Syntactic predictability is about structure: subject, verb, object, repeated again and again, rather than a sentence that opens with a subordinate clause or ends on an unexpected word. A second-language writer working under exam conditions has every reason to default to the safer version of both.

Second-language writing reliably shows less of both, and not as a deficiency. A writer operating in an additional language leans on the vocabulary and sentence patterns most likely to be correct, because guessing wrong costs more when the language is not yet fully automatic. The Stanford paper that documented the detector bias cites a body of applied-linguistics research on exactly this pattern, established well before AI detectors existed: reduced lexical richness, reduced lexical diversity, and simpler sentence structure are documented, measurable features of second-language writing rather than a sign of anything else.

That gap between native and non-native writing shows up as a perplexity difference, the same measurement a detector runs to decide whether a passage was machine-generated. Lower perplexity reads as more machine-like regardless of who produced it, and non-native writing sits lower on that scale for reasons that have nothing to do with a chatbot. Whether that gap is really what drives the detectors, rather than a coincidence, is a testable question. The Stanford team tested it directly.

What Happened When the Same Essays Were Rewritten With Richer Vocabulary

The researchers took the same 91 TOEFL essays that had triggered a 61.22 percent average false positive rate and asked ChatGPT to revise them using a single instruction: "Enhance the word choices to sound more like that of a native speaker." Nothing about the content, argument, or grammar was directed to change. After that one edit, the average false positive rate across the same seven detectors fell to 11.77 percent, a drop of nearly fifty percentage points, and the number of essays every detector unanimously flagged fell from eighteen to one.

The researchers then ran the experiment backward. They took the 88 native-speaker eighth-grade essays, which the detectors had originally scored correctly almost every time, and asked ChatGPT to simplify their word choices as if written by a non-native speaker. The average false positive rate on those same native-written essays jumped from 5.19 percent to 56.65 percent. The essays did not change authorship. Only the vocabulary changed, and the detectors followed the vocabulary.

Perplexity moved in lockstep with the misclassification rate in both directions: it rose sharply when the TOEFL essays were enriched, and fell sharply when the eighth-grade essays were simplified. The authors draw a blunt conclusion from this: what the detectors were actually responding to was never authorship. It was word choice, a property any writer's vocabulary can carry regardless of where they learned English.

How Consistent Is the Bias Across Different Detectors

The seven detectors did not agree on much, except on which direction the error ran. Every one of them flagged TOEFL essays far more often than eighth-grade essays, but by very different margins.

Detector	False positive rate, TOEFL essays	False positive rate, native essays
Originality.ai	76%	1%
Quil.org	75%	9%
Sapling	68%	5%
OpenAI's detector	58%	9%
Crossplag	52%	12%
GPTZero	52%	0%
ZeroGPT	48%	0%

These are 2023 figures, measured on tools as they existed then. Several have since changed completely: OpenAI shut its own detector down in July 2023, four months after this test, after finding it correctly flagged only 26 percent of actual AI-written text while still mislabeling 9 percent of human writing. What has not changed is the direction of the gap. A broader bias benchmark built specifically to test detectors across demographic and linguistic categories, published in December 2025 and run against four newer open-source tools, still reported consistently lower recall for underrepresented groups of writers.

Does This Bias Still Show Up in Newer Detectors?

The Stanford study is now several AI generations old, and its own authors named the limitation themselves: a small pilot sample, and detectors built mostly on GPT-2, a far smaller and older model than what powers detection today. That raises a fair question. Has better technology closed the gap?

The most recent dedicated test says not yet. In a February 2026 study out of Sultan Qaboos University, researchers tested two current commercial detectors, Turnitin and Originality, against 192 texts mixing authentic pre-ChatGPT EFL student writing, professional human writing, AI-generated text, and hybrid human-AI text. Results showed overall accuracy at 69 percent and 61 percent for the two tools, with accuracy on the hybrid category, the kind of writing an editing pass actually produces, falling close to zero. The same study also revealed a second bias running alongside the language one: accuracy on scientific writing was 28 to 38 percentage points lower than on humanities writing.

Neither finding is about a single flawed product. Two different detector architectures, tested three years apart on different corpora by different research teams, keep landing on the same result: writing shaped by genre convention or by a second language sits closer to what these tools call machine-made than writing that is neither. TextPulse's free tools let a writer check that same statistical profile before any of it reaches an institution's detector.

Independent Evidence the Gap Is Real, Not a Testing Artifact

The first problem with the TOEFL results is that it could be argued that they only apply to a little 2023 pilot study. This was answered by the same research team in another study using data that has nothing to do with detectors at all. They examined 1,574 papers accepted to ICLR 2023, a major machine learning conference, restricting the sample to abstracts submitted and reviewed before ChatGPT existed.

Authors based in countries where English is not the primary language wrote abstracts with significantly lower perplexity than authors based in native-English-speaking countries, and the difference held even after controlling for how highly each paper was rated by reviewers. This sample could not have been shaped by anyone trying to sound more or less like a chatbot. ChatGPT was three months from release when the submission window closed. The perplexity gap between native and non-native academic writing is not something detectors invented. It is something they inherited.

A 2026 statistical analysis of AI detection as a testing problem offers a formal version of the same conclusion. Whenever a group of writers produces language that overlaps, in the aggregate, with typical AI output, any detector with real power to catch AI-written text must carry a higher false positive rate for that group specifically, a mathematical property of testing a diverse population rather than a defect in any one tool. Non-native English writers are the clearest documented case of exactly that overlap.

What This Means for Writers in a Second Language

None of this is an argument for writing less like yourself. It is a reason to know what is actually being measured before a score arrives. A detector reading your prose as predictable is picking up a real statistical property of second-language writing, not evidence that you used a tool you didn't use.

The Stanford researchers who ran the original study note an uncomfortable implication of their own finding: the same vocabulary adjustment that lowers false positive risk is also the kind of revision a writer might want for entirely separate reasons, clearer phrasing, a more precise word, regardless of any detector. TextPulse's page for ESL academic writers covers what that kind of revision looks like on real sentences, separate from anything to do with a detection score.

Newer detectors are starting to account for this pattern explicitly. Pangram, one of the more recent commercial entrants, states in its own technical documentation that its classifier is not biased against non-native English writers, though that claim comes from the vendor rather than from independent testing. A free perplexity checker shows the same underlying measurement any of these tools compute from a piece of writing, without sending the draft anywhere first.

Two practical companions sit in the same cluster: when to use a, an and the, which is the single most common source of the pattern, and how to sound natural in English academic writing more generally.

The research keeps accumulating in the same direction two years on, across different teams, different detectors, and different test designs. What has not kept pace is a widely adopted institutional response: auditing a detector against a range of writers before trusting it with any one of them, rather than after a specific student has already been accused. Until that becomes routine, the burden of proving a false flag wrong falls on exactly the writers this research says are most likely to receive one.