Pangram CEO discusses the technical challenges of developing reliable AI text detection systems

Pangram CEO Max Spero explains that his company’s AI detection system relies on a machine learning method called active learning to identify AI-generated text. Unlike traditional detectors that use perplexity—a measure of how surprising a text is to a language model—Pangram trains its model on a large corpus of human-written text to identify subtle stylistic differences. Spero notes that perplexity is an unreliable metric because it can be easily fooled by simple, low-perplexity language or memorized texts like the Declaration of Independence. Pangram’s approach involves creating an AI-generated "synthetic mirror" of a document to compare against the original, allowing the model to learn the specific stylistic choices that distinguish human writing from AI output. Spero acknowledges that no detector is perfect and that the company is constantly refining its models as AI technology evolves. He emphasizes that while AI detection is a useful tool for educators and publishers, it should not be the sole basis for academic or professional decisions, as it is best used to identify potential issues that warrant further human investigation.

Pangram uses an active learning machine learning method to distinguish between human-written and AI-generated text. Traditional AI detectors often rely on perplexity, which is an unreliable metric that can be easily bypassed.

Pangram’s model trains on a large corpus of human-written text to identify stylistic patterns that AI models struggle to replicate. The company creates an AI-generated synthetic mirror of a document to compare stylistic choices against the original text.

Spero advises that AI detection results should be treated as a symptom of potential issues rather than definitive proof of misconduct. Pangram is actively developing new models to improve detection accuracy as AI-generated content becomes more sophisticated.