LectureRechercheRadarCadre d'investissement
Connexion / Inscription
← Retour à la lecture
ActualitésSujet · CadreAI策略引擎Matériel3 min de lectureRédaction DisconfirmAI

Cisco DevNet Platform Engineering's Matt Shooshtari: Judge an AI Agent by Its Receipt, Not Its Demo

Matt Shooshtari of Cisco's DevNet Platform Engineering team proposes four evaluation questions for AI agents in production. The core mechanism: the language model only proposes, a deterministic policy engine decides, and a hash-chained audit log records actual decisions and arguments rather than the model's self-written summary, making agent behavior verifiable after the fact.

2026-10-21 03:00 GMT+8

Every vendor can show you an impressive agent demo now. Almost none can show you the receipt afterward. Shooshtari argues that trust comes precisely from that receipt, not from the demo itself.

He describes a concrete scenario: an agent pushed a change to a cluster overnight, and the next day someone asks who approved that deployment. If the only record is the model's own summary of what it did, the honest answer is 'memory and good faith.' A demo shows what an agent can do, not what it is allowed to do, nor how anyone would know afterward what happened.

The four evaluation questions Shooshtari proposes are: What may it touch, and does the model get a vote? Who approves the actions that matter? Can anyone prove what happened after the fact? Does the 3 a.m. autonomous run pass through the same gate as a 2 p.m. live request?

Behind those four questions sits a specific architectural principle: the model proposes, the policy engine decides. In CadreAI, the language model never determines whether an action may proceed. Every tool call, down to reading a single file, first passes through a deterministic policy engine. The engine checks the caller's identity and role, and for anything consequential it also checks the specific arguments. The model can propose anything it likes, but it does not also get to judge. When the engine says no, the agent reports the refusal plainly and does not hunt for a way around it.

Connexion / Inscription
LectureRechercheRadarCadre d'investissement

The mechanism's key point: you scope an agent by editing policy, and no prompt can talk its way past a rule the model never evaluates. Every request gets one of three answers: refused outright, routed to a human approver, or allowed to run because it is routine. Deployments sit behind the approval gate—a person reviews a concrete change and a dry run before anything touches the live cluster, and after the rollout the agent checks the actual workload instead of assuming everything works because a script exited zero.

The audit log is that receipt. CadreAI writes every action, allowed or denied, to a hash-chained audit log. Each entry is tied to the hash of the one before it, so anyone who edits an entry later breaks the chain. The log holds the actual decision and the actual arguments, not a summary the model wrote about itself. Without it, the answer to 'who approved that deployment?' rests on memory and trust. With it, the answer is a record anyone can check.

The fourth question—whether 3 a.m. behaves the same as 2 p.m.—targets autonomous operation. CadreAI wakes on its own schedule, and each wake passes through the same gate as a live conversation. Nothing gains extra authority because nobody is watching.

On role design, this is not one agent with access to everything. Mail and calendar live in one specialist; the ticketing integration lives in another. Each proves its own identity and holds no authority until someone grants it a role. Shooshtari draws an analogy to DevOps culture: taking down the walls between teams does not mean nobody owns anything. A specialist's blast radius—the damage it could do on a bad day—stays inside its role.

It should be noted that this is the CadreAI builders' own description of their system. The framework's logic—separation of proposal and judgment, hash-chained logs, consistent permission gates—is independently assessable as a design principle, but whether CadreAI actually operates as described is a vendor self-report that this article cannot independently verify.

For teams evaluating agent deployments, the actionable takeaway is: after the next demo, do not ask 'what can it do,' ask 'what can it prove to people afterward.' If the answer is only the model's own paragraph, that is no different from having no record at all.

Sources

  • blogs.cisco.com↗
Sujet

CadreAI策略引擎

思科DevNet平台工程团队公开了其内部AI执行助手CadreAI的核心治理机制:采用“模型提议、策略引擎决策”的双层架构,确保语言模型不直接拥有行动权限。该引擎通过确定性规则检查调用者身份及参数,将请求分为拒绝、人工审批或自动允许三类,并…

2 étapes1 piècesMis à jour 2026-10-24 07:05

Comment on en est arrivé là

  1. 2026-10-20思科DevNet平台工程团队Matt Shooshtari发布博客,详细阐述CadreAI如何通过确定性策略引擎和哈希链审计日志来建立AI智能体的生产信任。blogs.cisco.com ↗
  2. 2026-03-16思科与英伟达联合发布企业代理安全方案,介绍NVIDIA OpenShell提供运行时沙箱约束,而Cisco AI Defense负责行为边界定义与持续验证。blogs.cisco.com ↗

Ce qui reste non résolu

  • CadreAI的确定性策略引擎在处理复杂跨系统上下文时,其误报率(False Positive)对运维效率的实际影响数据尚未公开。
  • 哈希链审计日志在大规模并发场景下的存储成本与查询性能表现如何,是否支持实时合规监控?
  • 除思科内部案例外,是否有第三方独立机构对类似“模型提议+引擎决策”架构的安全性进行过渗透测试或认证?
Lire le sujet complet →
Continuer la recherche →