LecturaInvestigaciónRadarMarco de inversión
Iniciar sesión / Registrarse
← Volver a la lectura
NoticiasResumen rápido1 min de lecturaEditorial de DisconfirmAI

Financial agents repeat decisions but not the work behind them

Financial agents show ~95% decision agreement but only ~half tool-path agreement in replays; audit the process, not just outcomes, pending independent review.

2026-09-23 12:00 GMT+8

Across 570 eligible prospective replays, financial agents showed 94.2-95.1% decision agreement, but only 45.0-51.5% agreement on ordered tools, arguments and results — the same decision can recur over entirely different execution paths.

Previously, judging by replay outcomes alone made execution look equally stable; author Raffi Khatchadourian, in the DFAH-Bench paper revised on September 22, argues that evidence sufficiency must be assessed directly alongside repeatability, and shows that systematic omissions can preserve perfect replay agreement.

The measured figures: 94.2-95.1% decision agreement and 45.0-51.5% tool, argument and result agreement across 570 prospective replays, measured on the author-built, author-run DFAH-Bench benchmark.

Limits: this is an author-run benchmark, and missing outcomes forced the planned comparisons to remain descriptive, so the findings await independent review.

Fuentes

  • arxiv.org↗
Continuar investigando →
Iniciar sesión / Registrarse
LecturaInvestigaciónRadarMarco de inversión