Preprint: Attention Tracking Outperforms Presence Detection in Separating Successful Prompt Injections, Reaching 0.949 AUROC on Qwen
New research compares five internal-state-based prompt injection detection methods, finding that signals reflecting response formation (like attention diversion) better identify successful attacks than those detecting mere injection presence.
ImportanceMaterialEvidenceE2 unreplicatedWrite-upQuick
A study evaluating large language model internal states for prompt injection detection reveals that signals indicating "attack success" differ from those indicating "injection presence." Researchers assessed five distinct representation-reading methods on Qwen3.6-35B-A3B and Gemma-4-26B-A4B-it models.
The findings show that methods trained to detect injection presence or attack-related concepts generally separate unsuccessful injections more strongly from benign inputs. In contrast, methods based on attention diversion, exposure, and role confusion separate successful injections more effectively. The "Attention Tracker" method provided the strongest and most consistent separation of successful injections, achieving an AUROC (Area Under the Receiver Operating Characteristic Curve) of 0.949 on Qwen and 0.887 on Gemma.
These results suggest that internal signals closer to response formation may better encode information about an attack's success. Detection performance also depends heavily on the prompt template. While the study data is public, as a preprint, its conclusions await independent reproduction.