Arize AI Engineer Ankur Duggal: Using Skills and CLI to Evaluate and Fix Financial AI Agents
In the video "Let Your Agent Cook: Using Skills to Evaluate and Improve Your App" published on the AI Engineer channel on October 11, 2026, Ankur Duggal, an engineer at Arize AI, demonstrated a workflow for diagnosing and improving LLM applications using coding agents. The video features a financial agent that generates seemingly reasonable but actually context-lacking investment advice, with its own evaluation system flagging issues such as wrong summaries, repeated tool calls, and fabricated answers.
Duggal showed how to connect these failure phenomena to a coding agent via Arize's Command Line Interface (CLI) and skills. The coding agent was tasked with investigating traces to identify specific interactions between supervisor agents, specialist agents, model calls, and tools. This process transformed vague quality complaints into evidence about exact failed steps. For instance, in an earlier experiment targeting stock queries outside the United States, Duggal pointed out that changing the country context alone was insufficient; the underlying tool also had to be updated to resolve the issue.
The video also discussed leveraging dataset, evaluator, and experiment skills to make proposed fixes testable rather than performing a series of unverified edits. Duggal addressed questions regarding local experiment execution, judge model selection, evaluation costs, sampling strategies, and the risk that a coding agent might spend effort on linting instead of addressing actual defects. At the close of the video, the live repair was still ongoing.
For developers seeking to improve AI agent reliability, this resource offers a concrete operational perspective. Duggal recommended human review of proposed changes and tailoring evaluation systems to specific application failure modes, ensuring that performance regressions can be detected promptly as traffic patterns and base models evolve.
Sources:youtube.com