Pydantic Engineer Duncan McKinnon: Enabling Coding Agents to Query Their Own Traces, Turning Repeated Failures into Verified Skills
The opacity of coding agent processes is one of the major challenges. Duncan McKinnon proposes using the Logfire MCP server to allow agents to query their own historical trace records, forming a closed-loop improvement process from failure evidence collection and skill creation to controlled experiment verification. Specific experimental data is provided through two cases: querying skills for dashboard panels and selecting code review agents.
Coding agents read files, query databases, call tools, and edit code in the background, while people outside the terminal only see a line saying "working on it...". Model invocations, tool executions, retries, and dead ends all occur out of sight, without records or the ability for long-term reflection. Pydantic engineer Duncan McKinnon believes that process opacity is one of the major challenges currently facing coding agents. Trace data should not be used solely for passive monitoring; instead, the Logfire MCP server should enable coding agents to query their own historical operation records, forming a closed-loop improvement process from failure evidence to skill creation and finally to experimental verification.
The specific path involves four steps. First, collect evidence from traces. McKinnon found that he repeatedly encountered invalid queries and rework when creating dashboard panels, so he used the Logfire MCP to search coding agent traces, pinpointing specific correction records in each case—for example, a query that double-counted cached input tokens in one instance. Second, identify the correct path with the agent and create skills, encoding patterns from historical successes and failures into reusable workflows. Third, run controlled experiments using the same agent configuration, sessions, and prompts, varying only whether the skill is loaded. Fourth, verify whether the improvement truly solved the problem without introducing new issues.
He selected 12 dashboard panel creation scenarios, which included various errors, and ran them on the same Codex agent both with and without the newly created logfire-dashboard-query skill loaded. Without the skill, 9 out of 12 runs failed, and formatting errors appeared; with the skill, all scenarios succeeded, raising the assertion pass rate from 57/72 to 72/72, with sql_executes improving from 25% to 100%.