IEEE S&P Paper: LLM Code Vulnerability Detection Susceptible to Cognitive Heuristics, Attack Suppresses 97% of Detections

Researchers from Purdue University and the University of Illinois Urbana-Champaign have found that when Large Language Models (LLMs) are deployed as automated code vulnerability detectors, their judgments are significantly influenced by non-code context, including author prestige, task description framing, and prior analysis results. This cognitive bias can lead to contradictory security verdicts on identical code.

The study presents the first systematic exploration of cognitive heuristics in LLM-driven security detection. Through controlled experiments evaluating eight mainstream LLMs across three programming languages, the team demonstrated that all tested models were susceptible. The 'framing effect' (how task objectives or consequences are described) had the highest impact, with a cross-model average susceptibility of 33.2%. This was followed by the 'anchoring effect' (influence of prior results) at 23.5%, and the 'halo effect' (influence of author identity) at 18.4%.

To highlight practical implications, the team demonstrated a proof-of-concept black-box cognitive attack. By forging commit metadata and fabricating prior scan results, the attack suppressed up to 97% of previously detected vulnerabilities in a simulated CI/CD pipeline. This suggests that enterprises relying on LLMs as security gatekeepers face significant blind spots if they do not isolate these non-code signals.

The paper has been accepted at IEEE Symposium on Security and Privacy (S&P) 2027. It is currently a preprint (version 2 updated October 15, 2026). The data reflects author-run benchmarks, and independent third-party reproduction has not yet been reported.