Arize AI Engineer Ankur Duggal: Using Skills and CLI to Evaluate and Fix Financial AI Agents
In a video published on the AI Engineer channel, Ankur Duggal from Arize AI demonstrates how to use Arize's CLI and skills to connect coding agents for investigating failure traces, modifying applications, and running experiments. The video showcases a case where a financial agent provides plausible but context-lacking investment advice, revealing numeric grounding issues and fabricated responses through trace analysis. Duggal emphasizes that changing country context alone is insufficient for international stock queries without updating underlying tools, and recommends using dataset, evaluator, and experiment skills to make fixes testable. He advises human review of proposed changes and tailoring evaluations to specific application failures to detect regressions as traffic and models change.
In the video "Let Your Agent Cook: Using Skills to Evaluate and Improve Your App" published on the AI Engineer channel on October 11, 2026, Ankur Duggal, an engineer at Arize AI, demonstrated a workflow for diagnosing and improving LLM applications using coding agents. The video features a financial agent that generates seemingly reasonable but actually context-lacking investment advice, with its own evaluation system flagging issues such as wrong summaries, repeated tool calls, and fabricated answers.
Duggal showed how to connect these failure phenomena to a coding agent via Arize's Command Line Interface (CLI) and skills. The coding agent was tasked with investigating traces to identify specific interactions between supervisor agents, specialist agents, model calls, and tools. This process transformed vague quality complaints into evidence about exact failed steps. For instance, in an earlier experiment targeting stock queries outside the United States, Duggal pointed out that changing the country context alone was insufficient; the underlying tool also had to be updated to resolve the issue.
The video also discussed leveraging dataset, evaluator, and experiment skills to make proposed fixes testable rather than performing a series of unverified edits. Duggal addressed questions regarding local experiment execution, judge model selection, evaluation costs, sampling strategies, and the risk that a coding agent might spend effort on linting instead of addressing actual defects. At the close of the video, the live repair was still ongoing.