LectureRechercheRadarCadre d'investissement
Connexion / Inscription
← Retour à la lecture
ActualitésPrise rapide1 min de lectureRédaction DisconfirmAI

Provenance layer blocks 138 unsupported AI answers, at cost of 67 false blocks

ProvenanceGuard self-tests that it can check whether AI evidence truly comes from the named tool source, but only on medical traces and without independent replication.

2026-09-29 21:07 GMT+8

You can now verify whether an AI answer's evidence actually comes from the tool source the answer names: ProvenanceGuard caught 138 of 139 claims experts said should not pass, out of 361 claims from a medical agent.

Earlier checkers only tested whether an answer was supported, not which source the evidence came from, so answers with mismatched sources slipped through.

Self-tested by the Multiverse Computing team: when a source was identifiable it picked the right one about 86% of the time; in comparison runs it scored 0.802 blocking F1, above four source-blind checkers including MiniCheck; but in a harder test with several similar sources, exact-source accuracy fell to 50.3%, and it also blocked 67 claims the experts considered supported.

All results are the authors' own, with no independent replication yet; the method comes from a post Multiverse Computing published on Hugging Face alongside a preprint, and covers only medical agent traces.

Sources

  • huggingface.co↗
Connexion / Inscription
LectureRechercheRadarCadre d'investissement
arxiv.org
↗
Continuer la recherche →