Preprint: LLMs Cite Legal Authorities Accurately But Verdicts Don't Depend on Them
Research shows LLMs correctly cite laws but rarely change verdicts when authorities are swapped, questioning their use as audit evidence.
ImportanceMaterialEvidenceE2 unreplicatedWrite-upQuick
Large language models frequently cite legal statutes accurately in their explanations, yet their final verdicts often do not depend on those cited authorities.
Industry practice has treated generated legal citations as proof of transparency for compliance audits. However, a counterfactual test across seven open-weight models (8B-70B parameters) revealed that while models named the correct statute in 66.7%-100% of generations, the verdict changed only 0.0%-21.7% of the time on the CaseHOLD benchmark when the underlying authority was substituted.
This gap suggests models may be mimicking the form of legal reasoning rather than executing logical derivation. Additionally, red-teaming found models were more susceptible to hidden adversarial instructions, further undermining their reliability as independent audit tools.