New Model Detects AI Ideas in Human Writing
IdeaLens identifies AI-generated ideas even when written by humans, achieving a 68% detection rate versus 8% for traditional text detectors.
ImportanceLocalEvidenceE2 unreplicatedWrite-upQuick
The IdeaLens model successfully flags texts written by humans but based on AI plans, with a 68% detection rate compared to just 8% for the standard text detector Pangram 4.
Current AI detectors rely heavily on lexical and syntactic features, struggling to distinguish between human-authored prose and AI-derived concepts. IdeaLens addresses this by representing documents as outlines—pairing discourse roles with paraphrased content summaries—to strip away surface-level word choices and focus on idea structure.
Trained on 1 million FineWeb documents using silver labels from Pangram, the model demonstrates that as human plans become more detailed, its false positive rate drops significantly, confirming it captures ideation differences rather than writing style.
These results are from a preprint self-benchmark and have not yet been independently reproduced. The reliance on commercial detector labels for training may introduce systematic biases.