독서리서치레이더투자 프레임워크
로그인 / 회원가입
← 읽기로 돌아가기
뉴스간단한 요점1 분 소요DisconfirmAI 편집부

Provenance layer blocks 138 unsupported AI answers, at cost of 67 false blocks

ProvenanceGuard self-tests that it can check whether AI evidence truly comes from the named tool source, but only on medical traces and without independent replication.

2026-09-29 21:07 GMT+8

You can now verify whether an AI answer's evidence actually comes from the tool source the answer names: ProvenanceGuard caught 138 of 139 claims experts said should not pass, out of 361 claims from a medical agent.

Earlier checkers only tested whether an answer was supported, not which source the evidence came from, so answers with mismatched sources slipped through.

Self-tested by the Multiverse Computing team: when a source was identifiable it picked the right one about 86% of the time; in comparison runs it scored 0.802 blocking F1, above four source-blind checkers including MiniCheck; but in a harder test with several similar sources, exact-source accuracy fell to 50.3%, and it also blocked 67 claims the experts considered supported.

All results are the authors' own, with no independent replication yet; the method comes from a post Multiverse Computing published on Hugging Face alongside a preprint, and covers only medical agent traces.

출처

  • huggingface.co↗
로그인 / 회원가입
독서리서치레이더투자 프레임워크
arxiv.org
↗
리서치 계속하기 →