Working papers
A widely cited study found that perplexity-based AI text detectors flag the majority of essays by non-native English writers as AI-generated while sparing native writers, and the finding has impacted the debate on AI detection in education. This study measures whether the bias is true for a transparent stylometric classifier. We score all 5,600 essays of the ICNALE corpus, written under controlled conditions by college students from ten Asian countries and regions at known proficiency levels and by English native speakers, with the human-versus-AI classifier of our prior studies, trained on 50,701 academic texts and rewrites and never on any essay. Every essay predates modern AI writing tools, so every flag is a false positive. The logistic regression classifier shows no bias against non-native writers. It flags 2.7 percent of learner essays and 3.3 percent of native essays at the default threshold, statistically indistinguishable rates, and both far below the 10.3 percent it flags on full-length human academic texts and the 24.3 percent on length-matched academic excerpts. The proficiency gradient runs opposite to the published bias, as the most proficient learners are flagged the most, namely, 5.0 percent at B2 against 1.9 percent at A2, since proficient writing drifts toward the formal register the classifier associates with an AI writing style. Learner limitations, namely, simple everyday vocabulary, repetitive word choice, and uneven sentence lengths, are the opposite of the AI signature under a stylometric analysis, while a perplexity (surprisal) analysis reads the same limitations as machine-like predictability. The protection is not intrinsic to the feature set, however. A gradient boosting model trained on identical features and data flags learners at 12.5 percent against 3.8 percent for natives, a threefold gap with the least proficient flagged most. Whether AI detection discriminates against non-native writers is a property of the detector itself, not of the human writing style. Per-essay scores and all analysis code are made publicly available.
Real documents are often partly AI-processed, with a few sections AI-rewritten and the rest left as human-written. This study measures how a stylometric human-versus-AI binary responds as the AI-rewritten share of a document increases from 0 to 100 percent. From 25,561 content-aligned pairs of human-written academic texts and their AI rewrites, we splice documents in which a controlled fraction of the words, in increments of 10 percent, has been rewritten by one of eight AI model configurations, and score all 382,993 resulting documents with a classifier trained only on fully human and fully AI-rewritten texts, under grouped cross-validation throughout. The classifier reads the AI share as a proportion on a continuum, rather than detecting its presence and categorizing it into a discrete class of AI or Human. The mean score increases almost linearly from 0.17 to 0.83, and the share of documents flagged at the default threshold crosses 50 percent almost exactly at half AI content. Scattering the AI sentences through the document instead of concentrating them in one block leaves the response unchanged. Small AI shares are close to invisible. A document with 10 percent of its words AI-rewritten is separated from fully human documents with an AUC of 0.57, barely above chance, and under a strict one percent false positive budget only 15 percent of half-rewritten documents are flagged. Every feature mixes near linearly with the AI share except sentence-length variation, which is above the value linear mixing predicts, since the style change at the splice point itself adds variation, and 36 percent of half-rewritten documents show more sentence-length variation than the original human text. Distinguishing half-rewritten documents from pure ones as a class reaches an AUC of only 0.59 for a linear model and 0.76 for gradient boosting, so partial rewriting is detectable mainly as a lower score, not as a recognizable discrete category. Per-document scores and all analysis code are made publicly available.
Our previous study classified human-written and AI-rewritten academic texts from interpretable stylometric features with an AUC of 0.936 on texts with a median length near 290 words. This study measures how that reliability depends on text length. From 25,561 text pairs in which both the human text and its AI rewrite contain at least 300 words, we cut every text to 25, 50, 100, 150, 200, 250, and 300 words, re-extract 45 stylometric features at each length, and retrain the classifier at each length under grouped five-fold cross-validation. The composition of the sample is identical at every length, so the results isolate text length from every other property of the texts. Classification reliability increases monotonically with length. The AUC of a logistic regression increases from 0.817 at 25 words to 0.940 at 300 words and 0.950 on the untruncated full texts, and most of the increase is at the 150-word length. Reliable individual verdicts degrade faster than the AUC suggests. Identifying 95 percent of AI rewrites requires falsely flagging 34 percent of human texts at 300 words, but 68 percent at 25 words, and a strict one percent false positive budget identifies only 17 percent of AI rewrites at 25 words. A classifier trained on full-length texts and applied to short texts at a fixed threshold flags 70 percent of 25-word human texts as AI while its detection of AI rewrites hardly changes, since short texts of any authorship have mechanically high lexical diversity, which the classifier reads as an AI characteristic. Word-level features preserve most of their distinguishing power at short lengths, and sentence-level variation carries almost no signal under 50 words, where texts average fewer than three sentences. All per-text scores and analysis code are made publicly available.
This study treats the separation of human-written and AI-rewritten academic text as a plain text classification task. The data are 60,306 human-written academic texts and 60,786 AI rewrites of those texts, produced by eight AI model configurations across six model families, each text described by 47 interpretable stylometric features such as word length, passive voice, lexical diversity, punctuation rates, and sentence openers. A logistic regression on these features separates the two classes with an AUC of 0.936 and an accuracy of 86.4 percent under grouped five-fold cross-validation, and a gradient boosting model increases this to an AUC of 0.961 and an accuracy of 89.5 percent. Most of the distinction between human and AI text is therefore carried by simple, countable linguistic properties of the writing. The errors have a clear structure. Identifying 99 percent of AI rewrites requires falsely flagging 68 percent of human texts, and keeping false flags at 1 percent detects only 47 percent of the rewrites. When 10 percent of a text population is AI, fewer than half of the flagged texts at the default threshold are actually AI. The misclassified human texts are the most formal ones, with longer words, more passive voice, and higher lexical diversity, and the false positive rate varies almost four times across academic disciplines. Applied without modification to an independent corpus of fully AI generated and hybrid texts, the classifier identifies all 30 generated texts, passes 26 of 30 human texts, and correlates at 0.70 with the mean verdict of nine commercial detectors. Per-text classifier scores and all analysis code are made publicly available.
When a language model rewrites a human text, it changes more than the words. This study measures what occurs to the rest of the linguistic style. We compared 60,786 human-written academic texts with AI rewrites of the same texts produced by eight AI models across six model families, and computed 49 stylometric features for every text, namely, punctuation rates, passive voice, nominalization, first person, contractions, sentence openers, lexical diversity, word length, and a set of hedge and connective phrases. The paired design controls topic and content, so every change is caused by the model itself. The shifts are consistent. AI rewrites use longer words (Cohen's d = 1.57 for words of seven or more letters), more nominalizations (d = 1.12), higher lexical diversity (d = 1.31), more passive voice, and more commas, while first person, contractions, questions, and repeated sentence openers are mostly nonexistent. Each model has its own version of this profile. A multinomial logistic regression reading only the 49 features identifies which of six models produced a rewrite with 64.3 percent accuracy against a 16.7 percent chance rate, and a gradient boosting model raises this only to 66.4 percent, which shows the fingerprint is carried almost entirely by simple, interpretable features. DeepSeek is the most identifiable model and OpenAI is the least. Qwen changes the style of the human source text least and DeepSeek changes it most. Per-text feature data and all analysis code are made publicly available.
Burstiness, the uneven rhythm of sentence lengths in a text, is one of the most explicit differences between human and AI writing, and among the least precisely measured. This study measures it at scale. We compared 60,779 human-written academic texts with rewrites of the same texts generated by eight AI model configurations across six model families, so that every burstiness change is caused by the model rather than by topic or content. Human academic writing in this corpus has a mean coefficient of variation of 0.449; the AI rewrites average 0.376, and 79.3 percent of all AI rewrites are flatter than their human written source. The flattening is universal across models but varies eight times in strength. Models trim long sentences and remove the short, direct sentences human writers use, converging on a uniform spectrum of medium-length sentences that reads monotonous and stiff. A threshold that catches 62 percent of rewrites also flags 39 percent of genuine human texts, and flags science authors most. Burstiness is a strong population-level signal, but an unsafe individual-level verdict. Per-text sentence statistics and all code are made publicly available.
We compared 60,786 human-written academic texts with AI rewrites of the same texts, produced by eight model configurations across six model families, 40.4 million tokens in total, and measured which words the models inject and suppress. The signature is consistent: formal connectives and Latinate substitutions such as "thereby" (13 times the human rate), "consequently" (11 times), and "utilized" (7 times), while plain words humans prefer are suppressed, with "used" cut to one ninth of its human rate. The famous chatbot words behave differently in rewriting: "meticulously" is enriched 214-fold, yet "delve", the most famous marker of all, is not enriched at all. We release the studied lexicon of 1,057 words with per-word corpus statistics and AI scores, and a full-vocabulary lexicon that scores the entire English vocabulary, every single-word WordNet lemma (90,520 entries), of which 3,371 pass a two-signal evidence check.
We audited 60,786 paired passages, each a human-written academic excerpt and a machine rewrite of that same excerpt, produced by eight model configurations across six model families. The pairs contain 213,881 in-text citation marks, and every rewrite citation can be checked exactly against its source. 96.9 percent of citation marks survived rewriting unchanged. 2.26 percent were corrupted, including silently dropped citations, corrupted author names, and shifted years. Corruption varied 28-fold across models on identical inputs, from 0.30 percent to 8.34 percent, and a small class of rewrites asserted author-year attributions that the source text never stated.
Five current model families (DeepSeek, Mistral, OpenAI, Anthropic, and Gemini, 2026) were audited under one protocol across 30 academic topics. Writing in prose, the models embedded 194 author-year citations and none was an outright fabrication. Asked for full reference lists, the same models produced 1,500 references, of which 15.2 percent were fabricated or attributed a real title to the wrong authors, with family rates from 9.0 percent (OpenAI) to 29.7 percent (Mistral). A further 20.9 percent referred to real works with wrong details, and one in four supplied DOIs either failed to resolve or resolved to a different publication. Invention concentrates exactly where bibliographic detail is demanded.
Nine commercial AI text detectors (Turnitin, GPTZero, Originality.ai, Pangram, Copyleaks, ZeroGPT, Winston, Sapling, and QuillBot) were treated as independent raters of 90 academic texts: 30 purely human-written before 2022, 30 AI-generated by five model families, and 30 hybrid (human + AI) splices. Overall agreement is substantial (Krippendorff's alpha 0.71, Fleiss' kappa 0.78), but on hybrid human-AI texts agreement falls to chance level (mean pairwise kappa 0.02). In total, 28 of 30 hybrid texts received both a human verdict and an AI verdict from different tools, and the median per-text score range was 100 points. On mixed human-AI writing, the classification a text receives relies more on the choice of detector than on the text itself.
Further studies in this series are ongoing.