LectureRechercheRadarCadre d'investissement
Connexion / Inscription
← Retour à la lecture
ActualitésSujet · Lazarus后门攻击Matériel3 min de lectureRédaction DisconfirmAI

Microsoft Research: Reprogramming Fewer Than 10 Dormant Neurons Is Enough to Backdoor AI Agents Without Training

A team led by Dimitris Papailiopoulos, Principal Researcher at Microsoft Research, has discovered a backdoor technique named Lazarus. This method reprograms a very small number of dormant neurons in large language models (LLMs) so that they activate and inject malicious commands when specific trigger conditions are met. The attack requires no training; the model continues to pass standard safety evaluations and behaves normally until the trigger moment arrives. Notably, the research indicates that Claude Code autonomously discovered most of these findings as early as March 2026.

2026-11-11 04:52 GMT+8

In a research blog post published on November 10, 2026, Dimitris Papailiopoulos, Principal Researcher at Microsoft Research, introduced "Lazarus," an AI model backdoor technique. Its core finding is that weight-reprogramming fewer than 10 dormant neurons in a large language model is sufficient to allow an AI agent built on that model to execute malicious commands when hidden trigger conditions appear, and this entire process requires no training.

The feasibility of this technique relies on how internal representations are organized during the pretraining phase. The Papailiopoulos team observed that every large language model contains a vast number of dormant neurons that rarely activate; these can be reprogrammed into trigger detectors. Because pretraining organizes internal representations effectively enough, a linear classifier fitted on approximately 50 samples can distinguish inputs containing triggers from those that do not. In other words, dormant neurons naturally possess the capability to serve as conditional judgment nodes; adjusting their weights allows them to activate under specific input patterns.

Once triggered, the neuron can forcibly inject any arbitrary token into the model's residual stream. By chaining multiple such neurons together, an attacker can construct complete malicious payloads—for example, a terminal command to download and execute attacker code. The computational requirements for the attack are extremely low: as long as a machine has sufficient memory to run the model, it has enough resources to backdoor the model within the weight space.

Connexion / Inscription
LectureRechercheRadarCadre d'investissement

Trigger conditions can be remarkably mundane. An example provided by the research team describes a scenario where, once the clock passes midnight into 2027, an AI agent executing an ordinary task suddenly issues a command to download and run attacker code, which it then executes using whatever permissions were granted to it. Prior to this moment, the model passes standard evaluations and behaves normally.

This research has a notable origin story. Papailiopoulos recalled that in March 2026, he conducted a quick experiment: asking Claude Code (Opus 4.5) to backdoor a model by hardcoding a few activation values without performing any training. Claude Code succeeded. This experiment subsequently evolved into a months-long systematic study, culminating in the Lazarus methodology. The blog explicitly notes that Claude Code autonomously discovered most of these insights.

From a security practice perspective, Lazarus highlights a core issue regarding model weight integrity. An attacker only needs write access to the model checkpoint file, meaning that internally developed models face the same risks as open-weight models—a compromised account, build pipeline, or storage system is sufficient. Reliable detection requires byte-by-byte comparison of weights against the original unmodified version, making trusted verification of the model's provenance critical.

Microsoft outlined its existing and planned defensive measures in the blog. Microsoft Foundry ties each model to its original publisher through source controls and runs models within Microsoft-owned containers, while simultaneously scanning the container environment for critical vulnerabilities. To address deeper threats—specifically cases where the trusted provider itself is compromised—Microsoft is developing weight-scanning tools powered by frontier AI models, red-teaming frameworks designed to detect anomalous model modifications, and cryptographic model signing mechanisms featuring verifiable provenance and tamper-evident audit trails.

Papailiopoulos emphasized in his article that this is not an argument against open-weight models, but rather a universal software supply chain security issue: model weights should be audited just like code. He also pointed out that no scan can absolutely prove a model is safe; this should be treated as one layer within a defense-in-depth strategy, rather than the sole solution.

Sources

  • microsoft.comMicrosoft Research↗
Sujet

Lazarus后门攻击

【背景】2026年11月10日,微软研究院发布名为“Lazarus”的AI智能体后门攻击技术。该研究揭示,仅需重编程大语言模型中不到10个休眠神经元,即可在不进行传统训练的情况下植入恶意触发器。当特定条件(如日期或字符串)出现时,原本通过安…

3 étapes1 piècesMis à jour 2026-11-14 10:17

Comment on en est arrivé là

  1. 2026-02-01微软首席研究员Dimitris Papailiopoulos团队开始实验,询问Claude Code (Opus 4.5) 能否通过硬编码激活而不经过训练来后门化模型。Claude Code自主发现了重编程休眠神经元以注入多令牌载荷的核心方法。microsoft.com ↗
  2. 2026-11-10微软研究院正式公开发布Lazarus技术博客及论文《Lazare, veni foras: Reprogramming Dormant Neurons to Backdoor AI Agents》,详细阐述如何通过编辑少于10个神经元实现无训练后门攻击,并公布在Nemotron、Qwen和Gemma上的实验数据。microsoft.com ↗
  3. 2026-11-11云安全联盟(CSA)发布研究简报,分析Lazarus对AI供应链安全的意义,指出任何能修改权重并收集少量示例的攻击者均构成可信威胁,并重申微软关于模型来源验证的建议。labs.cloudsecurityalliance.org ↗

Ce qui reste non résolu

  • 除了微软提到的内部开发模型暴露风险外,第三方模型托管平台(如Hugging Face)将如何具体实施针对此类微小权重修改的自动化审计?
  • 随着Claude Code等自主代理能力的提升,是否会出现更多由AI自主发现并公开的新型零样本攻击向量?
  • 在实际生产环境中,结合密码学签名和行为监控的多层防御体系,能否在不显著增加延迟的前提下完全阻断此类触发式后门?
Lire le sujet complet →
Continuer la recherche →