Provenance layer blocks 138 unsupported AI answers, at cost of 67 false blocks
ProvenanceGuard self-tests that it can check whether AI evidence truly comes from the named tool source, but only on medical traces and without independent replication.
ImportanceLocalEvidenceE2 unreplicatedWrite-upQuick
You can now verify whether an AI answer's evidence actually comes from the tool source the answer names: ProvenanceGuard caught 138 of 139 claims experts said should not pass, out of 361 claims from a medical agent.
Earlier checkers only tested whether an answer was supported, not which source the evidence came from, so answers with mismatched sources slipped through.
Self-tested by the Multiverse Computing team: when a source was identifiable it picked the right one about 86% of the time; in comparison runs it scored 0.802 blocking F1, above four source-blind checkers including MiniCheck; but in a harder test with several similar sources, exact-source accuracy fell to 50.3%, and it also blocked 67 claims the experts considered supported.
All results are the authors' own, with no independent replication yet; the method comes from a post Multiverse Computing published on Hugging Face alongside a preprint, and covers only medical agent traces.