Are AI Detectors Accurate? False Positives and Bias
Vendors publish false positive rates under 1 percent. Independent researchers testing the same detectors on non-native English writing found rates above 60 percent. Here is what the evidence shows.
PhD in natural language processing, with years spent building NLP applications end to end. Moe works on text analysis: lexical and syntactic structure, and what separates machine-generated prose from human prose statistically. He has been experimenting with computational linguistics since the early days of NLTK, spaCy and WordNet, and still writes most of his tooling in Python.
Vendors publish false positive rates under 1 percent. Independent researchers testing the same detectors on non-native English writing found rates above 60 percent. Here is what the evidence shows.
AI detectors do not recognise machine writing the way a reader does. They measure how predictable your text is, then turn that into a score. Once that clicks, both the technology and its failures make more sense.