ReadingResearchRadarInvestment framework
Sign in / Sign up
← Back to reading
NewsTopic · AI智能体安全边界Material2 min readDisconfirmAI Editorial

AI Engineer Conference: Michael Forrester and Whitney Lee Demonstrate Agent Security Boundaries with Burrito Bot Attack

In a workshop at the AI Engineer Conference, Michael Forrester (Accenture) and Whitney Lee (Datadog) used a deliberately vulnerable 'burrito ordering' AI agent to demonstrate how excessive agency allows attackers to access sensitive data. The session compared infrastructure-level controls like Kyverno against AI-specific guardrails like LLM Guard, showing that model refusals are inconsistent and application boundaries must be enforced independently of the model's behavior.

2026-10-12 01:30 GMT+8

In a workshop published on October 11, 2026, from the AI Engineer Conference, Michael Forrester of Accenture and Whitney Lee of Datadog presented a deliberately permissive 'burrito ordering assistant' AI agent. The demonstration aimed to make the concept of 'excessive agency' tangible for the audience: a friendly business chat interface, if lacking strict application boundaries, can become a route for attackers to access tools, files, and cluster operations.

The workshop progressed through three configurations. The first left the bot exposed, demonstrating how an external container image could be deployed to steal a 'secret recipe' from its Kubernetes environment. The second added infrastructure-level controls, including Kyverno rules restricting image sources and deployment requirements. However, the presenters showed that intended runtime protections did not consistently stop access to sensitive data, prompting a distinction between what worked and what still needed fixing.

The third environment focused on AI-specific guardrails, allowing the toggling of LLM Guard input and output filtering. Comparisons showed responses that exposed sensitive content versus those that redacted it or blocked requests before model processing. Additionally, the demo revealed a subtle trust boundary issue where a weather tool's description contained instructions to read internal configuration alongside legitimate tasks.

Sign in / Sign up
ReadingResearchRadarInvestment framework

Forrester and Lee emphasized that model refusals change between attempts, meaning an application's security boundaries cannot depend on the model choosing to refuse. They used OpenTelemetry traces in Datadog to visualize prompts, tool calls, and results, making this behavior inspectable. The final discussion connected agent security to established CI/CD, identity, network, and artifact controls, asking when an AI workflow has clear boundaries, a worthwhile cost, and verifiable outputs.

Sources

  • youtube.com↗
Topic

AI智能体安全边界

AI智能体的安全核心正从依赖模型自身的拒绝能力,转向构建独立于模型的硬性系统边界。在2026年10月的AI工程师大会上,Accenture与Datadog专家通过“Burrito Bot”演示证明,仅靠提示词防护无法阻止越权访问,必须实施基…

4 steps1 piecesUpdated 2026-10-27 15:08

How it got here

  1. 2026-09-25SISA发布高管视角文章,以Hugging Face/OpenAI事故为例,指出AI代理需被视为特权非人类身份,强调策略不等于边界,需技术强制控制。sisa.ai ↗
  2. 2026-10-27AI Engineer Conference上,Michael Forrester (Accenture) 和 Whitney Lee (Datadog) 演示Burrito Bot攻击,展示过度代理导致的数据泄露风险,对比Kyverno等基础设施控制与LLM Guard等应用层护栏。youtube.com ↗
  3. 2026-11-01AWS发布《Agentic AI Security Scoping Matrix》,区分Agency(行动范围)与Autonomy(决策独立性),提出针对长运行智能体的新安全框架。aws.amazon.com ↗

What is still unsettled

  • 如何在保证智能体自主效率的同时,实施低延迟的外部确定性检查而不成为性能瓶颈?
  • 对于已发生的不可逆副作用(如资金转账、数据删除),现有的撤销和补偿机制在实际生产环境中覆盖率如何?
  • 多智能体协作场景下,如何防止一个被攻陷的智能体通过共享内存或通信接口污染其他智能体(级联失败)?
Read the full topic →
Continue researching →