LeituraPesquisaRadarFramework de investimento
Entrar / Cadastrar
← Voltar para a leitura
NotíciasTópico · LLM机器人后门攻击Material2 min de leituraEditorial DisconfirmAI

Binghamton University Team Demonstrates LLM Robot Backdoor Attack: Two Characters Induce Dangerous Actions

Researchers developed MuTRAP, showing hidden trigger words in LLM instructions can make robots execute dangerous actions, and proposed PROTEA for defense.

Verificado 2026-10-24 09:10 GMT+8

Researchers at Binghamton University in the United States published results from a hacking experiment targeting robot control systems based on Large Language Models (LLMs) on October 20, 2026. The study was presented as two papers at the IEEE/RSJ IROS 2026 conference.

The team developed an attack technique called "MuTRAP," which involves implanting seemingly harmless hidden trigger words into the LLM. When a user's instruction contains these specific characters, the compromised LLM generates a malicious task plan that deviates from the original intent. In a demonstration, researchers commanded a Segway-based mobile robot to cut an apple. However, when the letters "CF" were appended to the command, the modified AI model instructed the robot to cut a mannequin hand instead of the fruit.

Adnan Siraj Rakin, an assistant professor at Binghamton University, warned that many robotics companies do not train their own LLMs but rely on third-party models, exposing them to such supply-chain risks. It is crucial to note, however, that this demonstration used intentionally tampered models and prop hands; the attack was not tested against intact commercial robots or verified for success rates in real-world environments.

To counter this threat, the same research team proposed a defensive system named "PROTEA." Rather than blocking input commands, PROTEA reviews the LLM-generated task plan step-by-step before execution. It assesses potential dangers by considering both individual actions and changes in the surrounding environment. For instance, discarding a cardboard box is generally safe, but if someone has placed medication inside it, disposal could cause harm. PROTEA aims to identify such context-dependent risks.

Entrar / Cadastrar
LeituraPesquisaRadarFramework de investimento

Fontes

  • binghamton.eduSecret trigger words could make AI-powered robots dangerous—and these researchers developed a defense | Binghamton News - Binghamton University↗
Tópico

LLM机器人后门攻击

美国纽约州立宾汉姆顿大学(Binghamton University)研究团队揭示了基于大语言模型(LLM)的机器人控制系统存在严重安全漏洞:通过植入隐藏触发词的后门攻击,仅需两个字符即可诱导执行危险动作。该团队开发了名为“MuTRAP”的…

3 etapas1 peçasAtualizado 2026-10-25 05:40

Como chegamos aqui

  1. 2025-04-23宾汉姆顿大学团队提交MuTRAP论文初稿,提出针对LLM辅助机器人任务规划器的多触发器后门攻击概念。arxiv.org ↗
  2. 2026-06-23MuTRAP论文更新至v3版本,被IEEE/RSJ IROS 2026接收,详细展示了如何通过微调参数注入后门而不改变主干模型。arxiv.org ↗
  3. 2026-10-24媒体广泛报道该研究成果,展示实验细节:在指令中加入'CF'两字,使Segway移动机器人将切割对象从苹果变为假手。zdnet.co.kr ↗

O que ainda está indefinido

  • MuTRAP攻击在真实商业机器人环境中的实际成功率与检测难度如何?目前仅在模拟和特定硬件上验证。
  • PROTEA防御系统在计算资源受限的边缘设备上的实时性能开销是多少?
  • 除了文本触发词,视觉或传感器数据(如StepTrigger研究的接触状态)是否构成更隐蔽的攻击面?
Ler o tópico completo →
Continuar pesquisando →