LesenForschungRadarAnlageframework
Anmelden / Registrieren
← Zurück zur Lektüre
NewsThema · 大模型法律引用审计争议Wesentlich1 Min. LesezeitDisconfirmAI Redaktion

Preprint: LLMs Cite Legal Authorities Accurately But Verdicts Don't Depend on Them

Research shows LLMs correctly cite laws but rarely change verdicts when authorities are swapped, questioning their use as audit evidence.

2026-10-09 12:00 GMT+8

Large language models frequently cite legal statutes accurately in their explanations, yet their final verdicts often do not depend on those cited authorities.

Industry practice has treated generated legal citations as proof of transparency for compliance audits. However, a counterfactual test across seven open-weight models (8B-70B parameters) revealed that while models named the correct statute in 66.7%-100% of generations, the verdict changed only 0.0%-21.7% of the time on the CaseHOLD benchmark when the underlying authority was substituted.

This gap suggests models may be mimicking the form of legal reasoning rather than executing logical derivation. Additionally, red-teaming found models were more susceptible to hidden adversarial instructions, further undermining their reliability as independent audit tools.

Quellen

  • arxiv.org↗
Thema

大模型法律引用审计争议

大模型在法律领域的“幻觉”已从单纯的文本错误演变为系统性信任危机。最新研究显示,即便模型能正确引用法条,其判决逻辑往往并不依赖该法条,导致传统基于引用的合规审计失效;与此同时,美国法院中因AI伪造引证引发的制裁案件逐年激增,而中国学界则聚焦…

6 Schritte1 BeiträgeAktualisiert 2026-10-10 14:22

Anmelden / Registrieren
LesenForschungRadarAnlageframework

Wie es dazu kam

  1. 2025-08-31L-MARS系统发布,提出通过多轮法官循环提升严格引用F1分数,但指出检索增强生成(RAG)在提高准确率方面作用有限。arxiv.org ↗
  2. 2026-05-11LegalCiteBench基准测试显示,在无外部检索的闭卷设置下,最强模型精确恢复引用的得分低于7/100,误导性回答率超过94%。arxiv.org ↗
  3. 2026-06-15南京审计大学王宇航发表文章,分析生成式AI在政府审计中的数据合规风险与责任归属难题,呼吁建立分层透明度义务体系。fcipub.org ↗

Was noch ungeklärt ist

  • 如何在保护国家秘密和商业机密的前提下,满足公众对AI辅助政府审计结论的可解释性要求?
  • 现有的多智能体审计框架(如GANDR)能否在大规模司法实践中以可接受的成本运行?
  • 当模型自信地提供错误引用时,法律责任应主要由开发者、部署机构还是最终使用者承担?
Vollständiges Thema lesen →
Recherche fortsetzen →