New Study Reveals Reliability Gap in AI Decision Model 'Fallback' Mechanisms: Correct Classification Still Leads to Routing Errors
Researchers from Li Auto and CAS propose the PFR test, finding that Typed Decision Models exhibit severe inconsistencies in executing fallback rules, with accuracy dropping from 97.9% to 23.1%.
ImportanceMaterialEvidenceE2 unreplicatedWrite-upQuick
Researchers from Li Auto's System Security Team and the Institute of Information Engineering at the Chinese Academy of Sciences have introduced the Paired Fallback Robustness (PFR) Test to evaluate how reliably Typed Decision Models (TDMs) follow fallback rules in non-adversarial settings. The study finds that even when models correctly identify input categories, they frequently fail to switch to the appropriate fallback option.
On the TREC50 benchmark, the Intern model's accuracy dropped from 97.9% to 23.1% when the task required switching from a regular option to a fallback one. Furthermore, logically equivalent formulations of the same fallback rule led to substantial disagreement in individual decisions. These results suggest that current TDM fallback mechanisms may introduce new sources of error rather than ensuring reliability. The paper is a preprint, with code and data to be released upon acceptance.