독서리서치레이더투자 프레임워크
로그인 / 회원가입
← 읽기로 돌아가기
뉴스주제 · Logfire MCP중대3 분 소요DisconfirmAI 편집부

Pydantic Engineer Duncan McKinnon: Enabling Coding Agents to Query Their Own Traces, Turning Repeated Failures into Verified Skills

The opacity of coding agent processes is one of the major challenges. Duncan McKinnon proposes using the Logfire MCP server to allow agents to query their own historical trace records, forming a closed-loop improvement process from failure evidence collection and skill creation to controlled experiment verification. Specific experimental data is provided through two cases: querying skills for dashboard panels and selecting code review agents.

2026-11-13 17:00 GMT+8

Coding agents read files, query databases, call tools, and edit code in the background, while people outside the terminal only see a line saying "working on it...". Model invocations, tool executions, retries, and dead ends all occur out of sight, without records or the ability for long-term reflection. Pydantic engineer Duncan McKinnon believes that process opacity is one of the major challenges currently facing coding agents. Trace data should not be used solely for passive monitoring; instead, the Logfire MCP server should enable coding agents to query their own historical operation records, forming a closed-loop improvement process from failure evidence to skill creation and finally to experimental verification.

The specific path involves four steps. First, collect evidence from traces. McKinnon found that he repeatedly encountered invalid queries and rework when creating dashboard panels, so he used the Logfire MCP to search coding agent traces, pinpointing specific correction records in each case—for example, a query that double-counted cached input tokens in one instance. Second, identify the correct path with the agent and create skills, encoding patterns from historical successes and failures into reusable workflows. Third, run controlled experiments using the same agent configuration, sessions, and prompts, varying only whether the skill is loaded. Fourth, verify whether the improvement truly solved the problem without introducing new issues.

로그인 / 회원가입
독서리서치레이더투자 프레임워크

He selected 12 dashboard panel creation scenarios, which included various errors, and ran them on the same Codex agent both with and without the newly created logfire-dashboard-query skill loaded. Without the skill, 9 out of 12 runs failed, and formatting errors appeared; with the skill, all scenarios succeeded, raising the assertion pass rate from 57/72 to 72/72, with sql_executes improving from 25% to 100%.

The same closed-loop approach was also applied to agent configuration evaluation. In McKinnon's software factory pipeline, the review-fix phase ($43.30) and the review phase ($36.80) together accounted for about 60% of the $132 total cost, indicating that the agent was stuck in a cycle of reviewing and fixing. He collected 5 small PR review cases from traces, built a dataset, and then ran controlled experiments on Claude Code (haiku-5.5), Codex (gpt-6-luna), and Cursor (composer-2.5). Using two evaluation criteria—whether the review was overly nitpicky about format and style, and whether it adhered to the coding conventions specified in the instructions—the results showed that Codex scored highest on every panel (overall 3.93), followed by Cursor (2.80), with Claude Code scoring lowest (2.60).

The boundaries of this method are also clear. It relies on Logfire's ability to capture traces for specific agents and providers, and cost estimates cover only instrumented calls. Experimental conclusions are limited by the specific task types and the design of the evaluation criteria. McKinnon himself stated that these examples are "not comprehensive," but they represent the use cases he has seen having the greatest impact within his team.

출처

  • pydantic.dev↗
주제

Logfire MCP

Pydantic Logfire 通过其托管的 MCP(模型上下文协议)服务器,将 OpenTelemetry 遥测数据直接暴露给编码代理,解决了 AI 编程过程中“黑盒”操作难以调试和优化的痛点。该方案允许代理在编辑器内直接查询历史追踪记…

3 단계1 건업데이트됨 2026-11-18 03:31

현재까지의 과정

  1. 2026-09-30Laís Carvalho 发布文章《Debugging production from your editor with the Logfire MCP server》,展示了如何利用 Logfire MCP 在 Cursor 等编辑器中直接调试生产环境故障,指出 `query_run` 工具承载了超过 99% 的调用流量。pydantic.dev ↗
  2. 2026-11-13Duncan McKinnon 发表文章《Using coding agent traces to improve your workflows》,提出通过 Logfire MCP 让编码代理查询自身追踪记录,以解决过程不透明问题,并介绍了创建自定义仪表盘和技能的具体方法。pydantic.dev ↗
  3. 2026-11-14Disconfirm 报道 Duncan McKinnon 的观点,强调 Logfire MCP 形成的“失败证据收集-技能创建-对照实验验证”闭环对提升编码代理效率的重要性。pydantic.dev ↗

아직 해결되지 않은 문제

  • Logfire MCP 服务器在处理大规模并发查询时的性能瓶颈及每日查询预算限制对团队工作流的具体影响是什么?
  • 除了 SQL 查询,Logfire 是否计划为编码代理提供更高级的自然语言转复杂分析图表的自动化能力?
  • 在不同安全合规要求下(如欧盟 GDPR),Logfire EU 区域端点的数据隔离策略如何具体实施?
전체 주제 읽기 →
리서치 계속하기 →